Skip to content

Refactor native handles for safe worker sharing #48

Description

@ronag

Context

#47 introduced RocksStatistics using a type-tagged napi_external. This is a good environment-local representation: the tag validates the native resource type before casting, while copied shared_ptr instances provide native lifetime ownership.

A type tag is not, however, a transferable instance handle. Worker messages use structured clone: BigInts are cloneable, but arbitrary native externals are not. A RocksStatistics wrapper sent to another worker loses its prototype and Symbol-held native context, while sending the external directly fails with DataCloneError.

Current limitations

  • RocksStatistics can be shared by databases in one JavaScript environment, but not across worker_threads.
  • Cache, write-buffer-manager, and database handles expose native addresses as BigInts.
  • Checking that a BigInt converts losslessly to int64_t does not prove that the pointer is live or has the expected native type.
  • Cache/WBM imports depend on the originating external remaining alive until the receiving environment copies the shared_ptr.
  • A stale, forged, or wrong-resource BigInt can cause invalid dereferencing or type confusion.
  • The database handle path additionally has known cross-environment ownership, close-race, cleanup-hook, and napi_ref hazards documented in binding.cc.

Proposed direction

Use two distinct layers:

  1. Environment-local representation

    • Give every addon-owned opaque external a unique type tag per native resource type.
    • Check the tag before every unwrap or napi_get_value_external.
    • Let each local wrapper own an appropriate shared_ptr or environment-specific state.
  2. Cross-worker transport

    • Use an opaque, cloneable token rather than a raw pointer.
    • Resolve tokens through a process-wide, synchronized, type-aware registry.
    • Have each worker import the token into a fresh tagged external owning its own shared_ptr.
    • Reject unknown, stale, expired, and wrong-kind tokens without dereferencing arbitrary memory.
    • Define explicit export/import/release and concurrent-use semantics.

Cache, write-buffer-manager, and statistics resources are the likely first candidates. Database sharing should remain a separate redesign because type tags alone cannot fix its environment-specific references and close semantics. Iterators, batches, updates, and column handles should likely remain environment-local.

Acceptance criteria

  • No public BigInt handle is interpreted directly as a native address.
  • All opaque native externals are type-tagged and validated before casting.
  • Worker imports create independently owned local wrappers.
  • Invalid, stale, expired, and wrong-resource tokens fail with JavaScript errors rather than undefined behavior.
  • Tests cover cross-worker use, source-wrapper GC before/after import, worker exit, wrong token kind, stale tokens, and concurrent cleanup.
  • Documentation distinguishes same-environment sharing from worker sharing.

Scheduling

This is future hardening/design work and is not planned for immediate implementation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions