[hendrikschilling]

In one sentence: Restore reusable chunk/GPU fiber tracing, conflict filtering and evaluation on main, with cold-cache reproduction of the historical zero-error chunk benchmark.

One real example: On the 91 reviewed fibers across five scrolls (182 directions), rebuilding 773 required path chunks from cached predictions reproduced every historical chunk path and stop reason, with zero retained GT-distance errors.

Before: Main lacked parts of the experimentally validated workflow. The first recovery's cold GPU run failed with illegal memory access; direct replay also lacked historical first-seed selection and cutoff behavior.

After this PR: Both direct and chunk benchmarks complete. Cold GPU construction and subsequent cached replay reproduce the historical chunk metrics and retained geometry. Direct tracing reproduces zero retained GT errors, with small documented numerical differences.

Proof: Same model, inputs and historical settings; established 512-base-voxel minimum-piece filter, no threshold relaxation:

Run	Retained GT errors	Pieces	Retained distance	Mean mm	Median mm
Historical direct	0	277	76.4336%	7.409	2.732
Recovered direct	0	276	76.3502%	7.453	2.789
Historical chunk	0	296	81.8276%	6.959	2.856
Recovered cold GPU chunk	0	296	81.8276%	6.959	2.856

The chunk result separately has 31 routing halts (29 no-overlap, two boundary), not GT-distance errors. All 773 constructed paths match exactly; all 182 retained case geometries and benchmark metrics match. Inference was reused; path construction started empty. Cold construction/evaluation/report completed in 22m56s on manager-granted GPUs 0–4.

Why / where this is useful: Makes the established tracing workflow available through main's reusable native and Python APIs without importing machine-specific experiments, model files or dataset configurations.

User motivation (verbatim): “ok then fix the cold-cche gpu test bug! and verify full reproduction!” The user reviewed the comparison and accepted the numerical drift: “yes those numerical drifts are fine.” Implementation and validation were agent-assisted. A human reviewer should complete the personal-verification checkbox below; no screenshot is attached, and the evidence here is captured benchmark/test output.

• I personally verified that the example and proof above were produced by this PR on the stated data.

Details

• Retain main's shared evaluation queue/cache architecture; recover optional chunk matching, seed fitting, normal-oriented conflict filtering, native RAM pooling, CUDA tracing/refinement, generic evaluation and cache/timing tools.
• Fix CUDA reduction scratch reuse synchronization and the optimized second-expansion invalid local-memory read by passing the immutable parameter block by const reference.
• Restore opt-in prediction-based first seeds, invalid-width seed rejection, forward candidate presence cutoff and geometric reverse checks without the forward cutoff.
• Cold-construction fixes: b15363287; final replay fixes/tests/docs: d25937624. Recovery commits: f2c71a438, a499f5606.
• Build: GCC13 RelWithDebInfo; CUDA12.8/g++12, H100 sm90, FMA disabled. CPU-only builds remain supported. macOS/arm64 and other GPU architectures were not runtime-tested.

Validation output:

Native regressions: 99 passed
Selected Python API tests: 16 passed, 1 skipped
CUDA memcheck: real-data six-chunk pilot, 0 errors
CUDA device regression: 1536 two-stage traces, 0 memcheck errors
Cold path comparison: chunks=773, exact=773, differences=[]
Final retained case comparison: cases=182, retained_differences=[]

Numerical limitations: direct cases 4 and 84 have small drift; one piece crosses the 512-voxel retention boundary. One discarded chunk terminal tail (case132, 4.3 reference voxels) has a different final vertex because main enforces its distance budget. No retained chunk geometry or score changes. This is not a claim of raw-document or CPU/GPU bit identity.

Build/test commands:

cmake --build volume-cartographer/build/fiber-evaluation --target vc_fiber_trace_metric test_fiber_chunk_trace test_fiber_reverse_consistency -j 8
volume-cartographer/build/fiber-evaluation/bin/test_fiber_chunk_trace
volume-cartographer/build/fiber-evaluation/bin/test_fiber_reverse_consistency
/home/hendrik/.venv_las/bin/python -m pytest -q volume-cartographer/python/tests/test_fiber_recovery.py volume-cartographer/python/tests/test_fiber_chunk_trace.py volume-cartographer/python/tests/test_fiber_evaluation.py --vc-build-python volume-cartographer/build/fiber-evaluation/python
# Inside a granted GPU job:
compute-sanitizer --tool memcheck --error-exitcode 9 volume-cartographer/build/fiber-evaluation/bin/benchmark_fiber_chunk_cuda --device-tests

See volume-cartographer/docs/trace_recovery.md for generic usage and cache provenance requirements. Cohort reproduction artifacts and exact launch/config records remain at /ephemeral/hendrik/fiber3d_train2/evaluations/trace_recovery_20261022_v1/report.md on the benchmark host. The raw best-restarts checkpoint is from mixed_reviewed_coarse_selected_tube_stride2_128_sd1_bs128_ema_20261018_133330 at 14,272,000 samples (SHA256 48f7cd9a7baa1437a273cc86e6a185db9870ecc45a744857407b80e0b024ce07). Experiment harnesses, planning/results files, dataset/model configs and logs are deliberately excluded from this code PR.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
scrollprize-org	Ignored	Preview	Oct 22, 2026 2:49pm UTC

[hendrikschilling]

Incorporated into #2231, including the complete history through f91e46f. The combined branch is pushed at cfd3cbc and has no merge conflicts with main. Its description separates global-layout workflows, native CPU/GPU chunk tracing, and pooling/evaluation integration. Combined validation passed: 966 Python tests plus 3 real-checkpoint checks, 24 native test cases, and 16 native Python API tests (skips documented in #2231). Closing this separate PR as incorporated; implementation and review continue in #2231. This does not merge the feature into main yet.