reading.md

Reading

Reads take one of three shapes: record lookups (current or at a numbered version), geometry reads through a snapshot (bounded, typed), and spatial queries over the persisted index. Everything here works on a read-only open. Retired objects are omitted from listings, traversal, and spatial results unless include_retired=True; direct UUID and version lookups always find them.

Opening a store

from vc_object_store import ObjectStore

store = ObjectStore.create("objects.vcobj", author="reviewer")   # new store
store = ObjectStore.open("objects.vcobj", author="reader", read_only=True)

Form	Notes
Python	ObjectStore.create(path, author=..., catalog=None), ObjectStore.open(path, author=None, read_only=False, catalog=None); catalog is a vc_reader.Catalog (or its dict form) supplying registrations the store has not recorded — see transforms
C++	ObjectStore::create / ::open with OpenOptions (author, readOnly, catalogTransforms, serviceOwner)
Hosted	Python HostedStore(endpoint); C++ vc::objects::HostedStore or a session attachment — see hosted

A writable open takes a host-local ownership lock on <path>-owner, shared between standalone writers and exclusive for the server's service owner, so a store served by vc_object_server rejects other writable opens (read-only ones still work). A store with an unsupported schema is rejected untouched; convert it with vc_object_convert (see Converting old stores).

Closing a store with live snapshots is an error in Python. Use the store as a context manager, or call store.close() after closing snapshots and graphs.

Listing and selecting records

store.object(uuid)                       # current record, or None
store.objects_by_kind("surface")         # live records of a kind
store.objects_by_tag("initial_import")
store.object_count("fiber", include_retired=True)
store.object_tags()                      # distinct tags of live objects
rows = store.select_objects(object_kind="track", fields=["name", "track"])

select_objects is the general query: kind, tag, bounding box, length filters, coordinate declaration, volume target, connection, limits and cursors, with optional fields projection. Each row carries uuid, kind, and the version and version_generation it was read at, so a listing pins exactly what it returned (read snapshots only). select_edges does the same for relationships. On HostedStore these are paged; select_all_objects / select_all_edges return the complete result exact as of one commit and can raise StoreBusy under sustained concurrent writes. In C++ the same queries take an ObjectSelection / EdgeSelection in a read transaction.

Volume-target listings (volume=..., the volume's id) match every frame declared in that volume, and their bbox, length filters, and length ordering use volume voxels; include_unbound=True appends the rest in UUID order. volume and an exact coordinates declaration are mutually exclusive. See transforms.

Geometry: snapshots and bounded reads

A geometry snapshot pins one consistent store state. Views read only the ranges or rectangles asked for; full materialization is explicit (load()).

with store.geometry_snapshot() as snapshot:
    surface = snapshot.surface(uuid)
    tile = surface.read_tile(row=30, column=31, rows=4, columns=4)
    collection = snapshot.point_collection(pcl_uuid)
    xyz = collection.point(42)                    # by stable point ID
    ids = collection.ids(0, len(collection))
    fiber = snapshot.fiber(fiber_uuid)
    controls = fiber.control_points(start=0, count=2)
    owned = snapshot.track(track_uuid).load()     # -> vc_object_store.Track

• Factories: snapshot.surface/fiber/track/point_collection/umbilicus(uuid), or snapshot.geometry(uuid) for the matching view. A kind mismatch or an object without stored geometry raises.
• Views expose counts and dimensions from the pinned geometry revision (not import summaries), plus view.version — the numbered object version read, for checked edits.
• A tile holds points and validity; other surface channels are read with read_geometry_rectangle. Point lookup loads the ID array once per view, not the coordinates.
• Views keep their snapshot alive; closing the snapshot invalidates them. Snapshots are thread-confined and hold one reader slot — close them promptly. load() results stay valid after closure.
• Any view can be read in another volume's frame with view.view_transformed_at(target_volume) — see transforms.

C++ mirrors this with store.geometrySnapshot() (store/GeometryViews.hpp).

Raw array reads

For byte-level access by geometry revision UUID (store.current_geometry(uuid) returns the current descriptor including its uuid):

revision = store.current_geometry(uuid)["uuid"]
raw   = store.read_geometry_array(revision, "points")
piece = store.read_geometry_ranges(revision, "points", [[24, 24]])
rect  = store.read_geometry_rectangle(revision, "points", 0, 0, 64, 64)
packed = store.read_geometry_arrays([rev_a, rev_b], "points")  # .values/.offsets
notes = store.geometry_point_annotations(revision, "points")

Packed results expose dtype, shape, data, and row offsets; trailing dimensions and dtypes must match, and request order/duplicates are preserved. Storage dtypes are |u1, <u2, <i4, <i8, <u8, <f4, <f8. Reads are checksum-verified: damaged payloads raise instead of returning bytes. geometry_point_annotations returns per-point annotations aligned with that revision's sequence (by stable ID, not array order).

History reads

store.current_version(uuid)              # numbered head version
store.current_object_revision(uuid)      # record + selected geometry revision
store.object_history(uuid)               # every published object revision
store.geometry_history(uuid)             # every geometry revision
store.geometry_revision(revision_uuid)   # one geometry descriptor
refs = store.versions_at(when_us)        # [{store, generation, object, version}]
records = store.objects_at_versions(refs)

Object revisions capture metadata, geometry identity, and relationship membership; a metadata-only edit creates no geometry revision. versions_at returns the numbered version of every object live at when_us (objects created later, or physically deleted by then, excluded; retired objects omitted unless include_retired=True). Restoring historical content publishes a new checked action — see retire and restore. Relationships at a selection are read with relationships_at_versions(refs), and the whole graph with graph_read — see graph edges.

Spatial queries

Both queries use the persisted R-tree/tile index; neither quantizes bounds.

nearest_surfaces(point, tolerance, uuids=None, coordinates=None, volume=None) (C++ ObjectStore::nearestSurfaces) returns the nearest point of each surface within tolerance, ordered by distance: object and geometry revision, fractional grid [column, row], surface point, distance. It reads only tiles near the point and tests their exact valid triangles — invalid vertices are holes. uuids=[] matches nothing; volume= poses the query in that volume's voxels.

geometry_region(slice, options) browses an oriented slab. It returns original unclipped XYZ and zero-based segment indices in contiguous runs for one polyline kind (fiber, track, or surface — a surface contributes the cut of its exact triangle mesh by the slab's mid-plane):

from vc_object_store import RegionSlice, RegionOptions

view = RegionSlice()
view.origin = [100, 200, 300]
view.u, view.v, view.normal = [1, 0, 0], [0, 1, 0], [0, 0, 1]
view.local_bounds = [-50, -50, -0.5, 50, 50, 0.5]   # u0, v0, n0, u1, v1, n1
options = RegionOptions()
options.kind = "fiber"
options.maximum_objects = 100          # 0 = unlimited
result = store.geometry_region(view, options)

Axes must be finite and orthonormal (tolerance 1e-12); zero slab thickness is allowed. A segment counts when it crosses the slab, even with both endpoints outside. Fibers use line_points when nonempty, else controls; tracks widen exactly to float64. options.uuids restricts candidates (None unrestricted, empty = nothing), selected is attempted first under a limit, and only intersecting objects consume the limit. Family and minimum-length filters use import annotations, not recomputed geometry; surfaces reject them. With options.volume, run points come back in that volume's voxels.

Region work is budgeted; a stopped result is valid but incomplete. Only stop == "exhausted" means complete:

Budget	Default	Charged work
output_byte_budget	8 MiB	24 bytes per appended XYZ
processing_byte_budget	64 MiB	Payload, verification, metadata, reconstruction
candidate_work_budget	1,000,000	Candidate visits and segment tests
sql_step_budget	10,000,000	SQLite VM instructions
max_unit_bytes	1 MiB	Largest single processing unit

RegionCancellation.cancel() may run from another Python thread (the query releases the GIL). C++ callbacks inspect stats, must not re-enter the store, and propagate exceptions. Budgets bound work, not wall-clock latency or allocator memory; there is no continuation cache — rerun with a bigger budget.

store.spatial_regions(...) returns the stored bounds themselves (per object or per tile), without geometry reconstruction.

Selection records

A selection record freezes the exact result of an object selection at one commit, so a long job or training set can read its inputs back later at the same versions while the store moves on. It holds the query, store, generation, commit sequence, every match in listing order at its numbered version, and — for volume/coordinate selections — the versions of the recorded transforms in force. record_selection(query) captures one (hosted stores too); encode_selection_record / decode_selection_record give the binary form (a JSON header, then 24 bytes per object), meant to be kept as an artifact. Read the objects back with objects_at_versions. Explicit transforms and limits are refused.

Converting old stores

The runtime opens schema 20 only. Stop every client, then convert into a new file (the source is never changed):

vc_object_convert old.vcobj new.vcobj   # writes new.vcobj.mapping.json and new.vcobj.report.json

The mapping gives every old object/relationship revision UUID its numbered reference, for converting pins in exports and job configs. The report lists gaps rather than guessing (untimed baselines, geometry revisions selecting no object version, metadata still holding old revision UUIDs). Converted commits have author unrecorded. Referenced artifact blobs are copied and verified; a missing or corrupt blob fails conversion and removes the partial destination.