volume-cartographer

Volume Cartographer is a toolkit and set of cross-platform C++ libraries for virtually unwrapping volumetric datasets. It was designed to recover text from CT scans of ancient, badly damaged manuscripts, but can be applied in many volumetric analysis applications.

This repository (a.k.a. "VC3D") is a version of EduceLab's original. Via Vesuvius Challenge, it has gone through a few forks: EduceLab (original) -> Julian Schilliger (optical flow segmentation) -> Philip Allgaier (performance/usability improvements) -> Hendrik Schilling and Sean Johnson (various improvements) -> this repository (developed and maintained by the Vesuvius Challenge team). The original Volume Cartographer is a general purpose toolkit for virtual unwrapping, while this version is specialized for the frontier challenges of the Herculaneum papyri.

Installation Instructions

Debian/Ubuntu builds use scripts/install_build_deps.sh minimal-build for the required system packages. The Docker CI builder uses the ci-tooling profile, which additionally installs compiler/debugging tools and AWS CLI. The installer requires root or sudo; in agent mode (AGENTS_AGENT_MODE=1) installation also requires AGENTS_ALLOW_INSTALL=1. build_from_src_debian.sh invokes the minimal profile, then configures, builds and installs VC3D. Validate installer behavior without provisioning with python3 scripts/test_install_build_deps.py.

Due to a complex set of dependencies, it is highly recommended to use a prebuilt package: the release downloads described below (Linux, Windows, macOS) or the docker image. We host an image on the github container registry which can be pulled with a simple command :

docker pull ghcr.io/scrollprize/villa/volume-cartographer:edge

The edge and main image tags currently carry the build of 2026-05-13 (see #1588); the latest release downloads are rebuilt from main.

Linux

A VC3D-<version>-linux-x86_64.AppImage containing VC3D and the vc_* command-line tools is attached to each GitHub release. The host must provide graphics and font libraries, including for headless CLI use; see the runtime prerequisites. Make it executable and pass the program to run as the first argument:

chmod +x VC3D-*-linux-x86_64.AppImage
./VC3D-*-linux-x86_64.AppImage VC3D                      # the GUI
./VC3D-*-linux-x86_64.AppImage list                      # the bundled tools
./VC3D-*-linux-x86_64.AppImage vc_render_tifxyz --help   # one of the command-line tools

scripts/appimage/README.md lists what the host must provide and describes symlink dispatch and how the AppImage is built.

To install VC3D and all command-line tools from source on a recent Debian-family Linux distribution using APT:

./build_from_src_debian.sh

The script installs build dependencies from system packages, builds VC3D and flatboi, and installs the runtime under /usr/local so commands such as VC3D and vc_grow_seg_from_seed are available from the terminal. The distribution must provide CMake 3.28 or newer; Ubuntu 24.04+ and Debian 13+ meet that requirement. Set JOBS, BUILD_DIR, or PREFIX to override the corresponding defaults.

Python source builds

uv sync in this directory builds the Python bindings from C++ source. The monorepo's vesuvius environment also builds this local package when selecting the models or all extra (including uv sync --all-extras). Install the native build prerequisites first: uv manages Python packages, not system C++ libraries. The Python CI workflow lists the native packages used for the Python-only build on Ubuntu.

Qt6 Core development files are required even with GUI targets disabled. The shared SFTP implementation uses Qt Core for subprocess and URL handling. This does not require a display server, Qt Widgets, or Python Qt bindings. Installing pyqt6 is not a substitute for the native development package.

To add Qt to an otherwise prepared build environment:

# Ubuntu / Debian
sudo apt-get install qt6-base-dev

# macOS (Homebrew)
brew install qt

These commands install Qt, not the complete native dependency set. The existing Linux dependency setup and macOS build helper already include Qt. If CMake cannot find Qt6Config.cmake after installation, add the Qt installation prefix to CMAKE_PREFIX_PATH before running uv sync. Keep the Qt Core runtime libraries installed when using the compiled bindings. See SFTP access for the OpenSSH runtime and authentication requirements.

Windows

A self-contained Windows 10/11 installer (VC3D-<version>-win64.exe, with all dependencies bundled) is attached to each GitHub release — download, run, and launch VC3D from the Start menu. A portable .zip of the same tree is published alongside it.

To build natively from source on Windows, use an MSYS2 UCRT64 shell (see .github/workflows/vc3d-windows.yml for the package list), then:

cmake --preset ci-windows-mingw
cmake --build --preset ci-windows-mingw
cpack --config build/ci-windows-mingw/CPackConfig.cmake

Alternatively, VC3D builds with the native MSVC / Visual Studio 2022 toolchain using vcpkg for dependencies, via the windows-msvc preset. Bootstrap a vcpkg checkout, then from an x64 Native Tools Command Prompt for VS 2022 (so cl.exe/Ninja are on PATH):

set VCPKG_ROOT=C:\path\to\vcpkg
cmake --preset windows-msvc
cmake --build build\windows-msvc

The first configure builds the dependency closure (Qt, OpenCV, Ceres, ...) from source via vcpkg — this takes a while, but later configures restore it from vcpkg's binary cache. The result is build\windows-msvc\bin\VC3D.exe with the Qt plugins and dependency DLLs deployed alongside it, runnable in place. To work in the VS2022 IDE, Open Folder on volume-cartographer/. Note that the VS developer environment forces VCPKG_ROOT to the port-less bundled vcpkg, so create a CMakeUserPresets.json (gitignored) that pins CMAKE_TOOLCHAIN_FILE to your vcpkg and select the resulting local preset — otherwise VS reconfigures against the wrong vcpkg and rebuilds every dependency.

Note: the flatboi SLIM flattening tool (PaStiX-based) is not available on Windows; Docker or WSL still covers that workflow.

macOS

A self-contained VC3D.app for Apple Silicon (VC3D-<version>-macos-arm64.dmg, with all dependencies bundled) is attached to each GitHub release — drag it to Applications. The app is not notarized, so the first launch is right-click → Open (or xattr -dr com.apple.quarantine /Applications/VC3D.app). The vc_* command-line tools ship inside the bundle at VC3D.app/Contents/MacOS/.

To build from source on macOS, use the Homebrew LLVM build helper:

scripts/build_macos.sh --install-deps

After dependencies are installed, rebuilds can use:

scripts/build_macos.sh

The macOS PR and release workflows use this same helper with --install-deps --sccache, installing missing dependencies directly from Homebrew on the runner. They do not restore a vc3d-deps Homebrew snapshot. Compiler outputs are still cached; dependency versions follow the runner's Homebrew formulae.

installation instructions for docker

running docker as a non-root user

Sheet-following edge extension

In Manual Add, enable Extensions follow the sheet, choose Sheet walk (Projection or Streamline), and set Normal substep to the maximum internal step length in volume voxels. Zero means one step per output vertex. A positive value subdivides each mesh interval into equal steps no longer than that value; it does not add mesh vertices. Settings persist and apply to the next extension. While a sheet-following extension computes, its unfinished strip stays hidden, editing and surface selection are locked, and an animated progress indicator is shown. The completed strip appears when calculation finishes; failures discard the unfinished extension and release the lock. Streamline offers Midpoint (RK2), Heun (RK2), and RK4. Applying or canceling an extension ends the editing session but keeps Manual Add selected as the growth method until you choose another method.

The CLI exposes the same options:

build/bin/vc_extend_edge -i INPUT.tifxyz -o OUTPUT.tifxyz -n VOLUME.zarr \
    --sides right --cells 10 --rounds 1 \
    --normal-step 2 --walk-mode streamline --integrator rk4 --report report.json

Defaults (--normal-step 0 --walk-mode projection) preserve the original walk and its cached normals. Positive substeps or streamline mode evaluate the tensor weights at the actual query position, without rounding the query. The estimator still samples gradients on its existing 3-voxel grid within a 109³ patch; patch support changes at cache-cell boundaries, so the field is not perfectly smooth. Exact-position queries share a bounded cache of 32 filtered gradient patches (about 27 MiB per volume). Each query still evaluates its own tensor and normal; reuse does not round positions or change the estimator arithmetic.

Streamline integrates minimum-turn tangent transport, x' = v and v' = -n (v · Dn v), using unit normals and central directional differences at ±0.25 voxels. Normal signs are aligned before differencing. Each completed step reprojects and normalizes the tangent. The finite-difference scale and normal-field smoothness limit accuracy even with RK4. If an integration stage has no usable normal, that substep uses the projection walk; if its midpoint normal is also missing, it continues straight. There is no positional snapping or automatic stop rule. Smaller steps do not guarantee the correct winding. The report's legacy steps_without_normal field counts failed normal queries, including intermediate integration and derivative queries.

Build and run the extension tests with existing dependencies:

ninja -C build test_manual_add_extend vc_extend_edge
QT_QPA_PLATFORM=offscreen build/bin/test_manual_add_extend

Basic introduction to VC3D

The document below is a basic introduction to the ui and keybindings available in VC3D, more detailed documentation is available in the segmentation tutorial on the scrollprize website.

Warning

if you are using Ubuntu, the default open file limit is 1024, and you may encounter errors when running vc3d. To fix this, run ulimit -n 750000 in your terminal.

Launching VC3D

If you're using docker :

xhost +local:docker
sudo docker run -it --rm \
  -v "/path/to/data/:/path/to/data/" \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  -e DISPLAY=$DISPLAY \
  -e QT_QPA_PLATFORM=xcb \
  -e QT_X11_NO_MITSHM=1 \
ghcr.io/scrollprize/villa/volume-cartographer:edge

From source :

/path/to/build/bin/VC3D

Data

VC3D requires a few changes to the data you may already have downloaded. All data must be in OME-Zarr format, of dtype uint8, and contain a meta.json file. To check if your zarr is in uint8 already, open a resolution group zarray file (located at /path/to.zarr/0/.zarray) look at the dtype field. "|u1" is uint8, and "|u2" is uint16.

Remote virtual-volume locators

VC3D can expose a public remote pyramid group as logical level 0 without copying or renumbering the source Zarr. Append the portable fragment selector #vc-base-scale=N, where N is an integer from 0 through 5:

https://example.org/volume.zarr#vc-base-scale=2
s3://bucket/volume.zarr#vc-base-scale=2

Level 0 canonicalizes to the ordinary selector-free locator. For a base-2 view, source groups /2, /3, ... become logical levels 0, 1, ...; voxel coordinates and voxel size are therefore those of physical /2. The source must be a contiguous numeric, isotropic dyadic ZYX pyramid with zero coordinate translations. Query parameters remain part of the network URL and must precede the fragment, for example volume.zarr?token=abc#vc-base-scale=2.

The selector is the volume's working level, not part of its identity: a project keeps one entry per volume, VC3D changes the level with the L spinbox beside the volume selector, and the volume keeps its ID at every level. It is never sent in HTTP requests. Tools that support remote volumes receive the complete portable locator, and store-mode tools also receive --voxel-scale 2^-N with --volume-uuid, so the surfaces they read and create are in level N's voxels of the volume. Local-only tools reject the selector instead of silently discarding it.

The meta.json contains the following information. The only real change from a standard VC meta.json is the inclusion of the format:"zarr" key.

{"height":3550,"max":65535.0,"min":0.0,"name":"PHerc0332, 7.91 - zarr - s3_raw","slices":9778,"type":"vol","uuid":"20231117143551-zarr-s3_raw","voxelsize":7.91,"width":3400, "format":"zarr"}

Your folder structure should resemble this:

.
└── scroll1.volpkg/
    ├── volumes/
    │   ├── s1_uint8_ome.zarr -- this is your volume data/
    │   │   └── meta.json - REQUIRED!
    │   └── 050_entire_scroll1_ome.zarr -- this is your surface prediction/
    │       └── meta.json - REQUIRED!
    └── config.json - REQUIRED!

A VC3D project is an object store (.vcobj, File -> New Project...): it references the volumes above and holds the surfaces (segments), fibers and point collections.

Working on remote volumes from the command line

vc_grow_seg_from_seed and vc_render_tifxyz both take the remote locators described above as -v and stream them, so a segment can be grown and rendered on a public OME-Zarr volume without downloading it or building a volpkg. The tools fetch only the chunks they touch, over anonymous HTTPS or S3. The GUI equivalent is File -> Open Data Catalog…, covered in the VC3D tutorial.

Finding a volume URL

The catalog of the Vesuvius Challenge open-data bucket lives at https://vesuvius-challenge-open-data.s3.amazonaws.com/metadata.json and describes the samples with their scans, volumes and segments. It is served gzip-encoded, hence --compressed. This lists the volumes of one sample with their level-0 shape and voxel size in micrometers:

curl -s --compressed https://vesuvius-challenge-open-data.s3.amazonaws.com/metadata.json |
  python3 -c 'import json, sys
for v in json.load(sys.stdin)["samples"]["PHercParis4"]["volumes"].values():
    print(v["long_id"], v["properties"]["shape"], v["properties"]["pixel_size_um"])'

A volume sits at <sample>/volumes/<long_id> in the bucket and can be addressed either way:

https://vesuvius-challenge-open-data.s3.amazonaws.com/PHercParis4/volumes/20260310173927-45.532um-11.0m-110keV-masked.zarr
s3://vesuvius-challenge-open-data/PHercParis4/volumes/20260310173927-45.532um-11.0m-110keV-masked.zarr

The examples below use this 45.532 µm PHercParis4 volume: level 0 is 4066×2264×2264 voxels stored in 128³ chunks, and the pyramid has levels 0 to 5.

Growing a segment

URL=https://vesuvius-challenge-open-data.s3.amazonaws.com/PHercParis4/volumes/20260310173927-45.532um-11.0m-110keV-masked.zarr
echo '{"mode": "seed", "cache_size": 2000000000, "generations": 8, "thread_limit": 4}' > params.json
vc_grow_seg_from_seed -v "$URL" -t out/ -p params.json --seed 1408 1408 1024

The seed is given as x y z in level-0 voxel coordinates. cache_size caps the decoded chunk cache in bytes (default 1e9) and thread_limit the number of OpenMP threads. The tracer reads the voxel size from the volume's metadata.json (voxelsize: 45.532 in the log) and writes the segment to out/auto_grown_<timestamp>/ as x.tif, y.tif, z.tif, generations.tif and meta.json. It also keeps a copy of every chunk it fetches under VC3D's remote cache root (~/.VC3D/remote_cache unless cache.json or remote_cache_dir under [viewer] in ~/.VC3D/VC3D.ini says otherwise), and later runs on the same volume read from there. On this volume the eight generations take ten to twenty seconds on a 14-core workstation and fetch about 2 MiB the first time.

To grow the segment further, pass --resume out/auto_grown_<timestamp> and raise generations in params.json: it is the total generation count the segment stops at, not a number of additional generations, so resuming with the unchanged generations: 8 reports Resuming from generation 7 with 196 points, adds nothing and writes the same 196 points again. With generations: 16 the same command continues from generation 8 to 15 (1.41 cm² to 7.02 cm² here) and writes the result to a new auto_grown_<timestamp>/ under the target directory, reading the remote volume as before.

Rendering it

vc_render_tifxyz -v "$URL" -s out/auto_grown_* \
  --scale 1 -g 0 --voxel-size 45.532 --voxel-unit micrometer --tif-output render/

This writes render/00.tif (360×360 pixels for the segment above, about 17 MiB fetched). Things to know:

• -v takes the URL itself, and nothing is written under a local directory. Since #1657 the renderer opens the remote volume the same way the tracer does, so the chunks it fetches persist in the shared remote cache root; --cache-gb (default 16) sizes the volume's decoded-chunk memory cache. The renderer fetches only the chunks its samples read, in parallel.
• The renderer reads the voxel size from the remote volume's metadata, as the tracer does: without --voxel-size the log says Voxel size (from volume metadata): 45.532 and the TIFF carries the matching resolution tag. Metadata values count as micrometers, while --voxel-unit applies to --voxel-size and defaults to nanometer, so pass both flags when overriding; --voxel-size 45.532 alone yields a TIFF resolution 1000× too large.
• -g selects the pyramid level: -g 3 renders the same segment at 1/8 scale (45×45 pixels, about 4 MiB fetched). A level the remote pyramid does not have fails immediately with Error opening remote zarr: group index 6: ....
• The #vc-base-scale=N selector described above applies to the tracer as well: with #vc-base-scale=2 it reports zarr dataset size for scale group 0 [1017, 566, 566] and voxelsize: 182.128, the seed is expected in level-2 coordinates, and the segment's meta.json keeps the selector in target_volume.

Opening a volume package and navigating the UI

First, click File -> Open Project... and select the project (.vcobj) you wish to work with.

On the left side of the UI, you have a few dock widgets. The first being the Volume Package / Surface List. The most important things to note here are the Volume dropdown list, which selects the currently displayed volume, and the surface list, which lists the surfaces of the project (an Open Data sample's project includes its catalog surfaces) and selects the currently displayed segment. Selecting a surface in the list should populate the Surface volume view in the top left, and initialize the rest the volume viewers.

There are a number of widgets, each with their own purpose. From top to bottom, and left to right :

Volume Package

Main UI widget for interacting with the volume package and surface list

• Filters : Apply a single (or multiple) filters to show or hide specific surface ids from the current set
  • the most important one to remember here is Current Segment Only, which will hide the intersection overlays of all other segmentations in the volume viewers (this can greatly speed up ui interaction)
• Tags : filter by the tags of the surface's object store record (Approved, Defective, Reviewed, Inspect)

Viewer Controls

Main UI widget for adjusting the ROI and display of the volume viewers

• Focus : Displays the current location of the focus point (in XYZ) -- also can be used to set the focus point by modifying the values in the text box
• Zoom : Can be used in place of scroll wheel based zooming, mostly exists to alleviate touchy track pad zooming
• Use axis-aligned slice planes (Ctrl+J) : by default, vc3d attempts to align your XZ and YZ planes orthogonal to the normal of the current selected surface, this checkbox will instead render them on fixed axis-aligned slices
• Show axis overlays in XY view : if you do not wish to view the axis overlays in the XY view, this checkbox will disable them

The opacity of the plane slice axis overlays is set by Axis overlay opacity in the settings dialog.

Overlay

Overlays a second volume onto the base volume

• Overlay volume : Selects the volume to overlay onto the base volume
• Overlay opacity : Modifies the opacity of the overlay volume
• Overlay window : The overlay intensity range to draw; values outside it are transparent (raise its low end to remove background)
• Overlay colormap : The colormap to use for the overlay volume
• Overlay interpolation : Selects nearest-neighbor (default) or trilinear sampling independently of the base volume

Keybinds:

• Right mouse + drag : pans the current viewer
• Scroll wheel : zooms the current viewer
• Ctrl + left mouse button : centers the focus point on the cursor position
• Middle mouse + drag : with axis-aligned slice planes on, rotates the slicing plane in the XZ or YZ volume viewers
• Spacebar : toggles the overlay on / off
• C : toggle a composite view of the current surface in the segmentation window (parameters of which can be located in the Composite View dock, under the View menu)

Segmentation

Primary entry point for interacting with the segmentation, more info in extending and modifying segmentations

• Enable editing (Shift+T) : Must be checked to enable any of the actions in this widget. Edits are applied to the surface and saved to the attached object store as new revisions
• Surface Growth - grouping of settings mostly pertaining to the Grow action
  • Steps : The number of "generations" the growth action will undertake
  • Allowed directions : Limits the directions of growth relative to the flattened 2d quad surface! (i.e if you look in the segmentation window in the top left, and you set 'up' it will grow towards the top of the window)
  • Limit Z range : Constrains the growth to a selected Z range
  • Volume : the volume to pass to the tracer should be a surface prediction volume!
• Editing - grouping of settings which mostly apply to drag/push/pull actions for mesh deformation. All mesh deformation-type actions are performed on a gaussian-like area which has a circular shape centered at the current mouse location and whose strength is reduced as we reach the edges. Radius is in quad vertices in the 2d flattened surface. Drag Brush and Push/Pull each have their own:
  • Radius : the radius of the area to be affected by the action
  • Sigma : the strength of the action on affected vertices other the original one (aka how quickly the influence tapers as we step away from the original point)
  • Step (Push/Pull only) : The step size (in world units) to take along the surface normal for each push/pull action
• Corrections : group which controls / manages the "correction points" for the current growth session
  • Active set : chooses an existing correction set to apply
  • New correction set : creates a new point collection containing a single "correction"

Keybinds:

• A or D (hold) : push/pull the current surface inward or outward along the surface normal (Ctrl for alpha mode)
• 1, 2, 3, 4, 5 : grow left, up, down, right, all, respectively (requires Enable growth keybinds)
• T : toggle correction annotation mode (click adds a correction point, right-click removes one)

Seeding

Ctrl+right-click in a volume viewer and choose Create Segment (GrowPatch) to select default GrowPatch, projection, or streamline seed growth. See seed growth modes and parameters.

Widget for creating initial segmentations which can be used for traces or later growth actions

• Draw paths : while on, a left-drag in a viewer draws a path
• Source Collection : the source point collection to use as seed points (Analyze Paths writes its peaks to seeding_peaks; select it here to seed from them)
• Parallel Processes : the number of processes to run seeding with
• OMP Threads : limits the amount of threads each process can use (recommend to set this to 1)
• Intensity Threshold : minimum intensity for a peak along a path
• Expansion Iterations : how many iterations to run expansion mode on after initial seeding
• Clear Peaks : clears the detected peaks
• Analyze Paths : after drawing paths, detect peaks along the line segment on which to place seed points
• Run Seeding : creates segmentations from the current seed points
• Expand Seeds : expands the current seed points to the given number of iterations
• Clear All Paths / Reset Points : clears the drawn paths and the detected peaks