Skip to content

feat(zero-rust): Rust client - #130

Merged
tmm merged 30 commits into
mainfrom
rust-client
Nov 11, 2026
Merged

tmm merged 30 commits into
mainfrom
rust-client

Conversation

@tmm

@tmm tmm commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

The Rust Zero client, zero-client, through milestone M5. The goal is parity with the JS client, proven by conformance against the JS vectors in packages/zero-conformance. The milestones and design are in rust/DESIGN.md, and what's ported and every documented difference is in rust/PARITY.md. The zbugs apps (M6) and hardening (M7) follow in their own PRs.

What's in it

  • M0, scaffolding: the rust/ workspace, and a conformance runner that reports every vector file and generated corpus. A file it doesn't replay is listed as not implemented.
  • M1, primitives and wire: JSON with JS semantics, JS number formatting and parsing, string orders, hashes, row keys, value comparison, LIKE, TTL, the AST and its normalization, query hashes, the query builder, name mapping, protocol messages, and binary poke chunks.
  • M1.5, walking skeleton: speaks the protocol to a real zero-cache. CI runs it end to end against Postgres.
  • M2, IVM: the memory source, every client operator, views and change application, and the pipeline builder.
  • M3, codegen: zero-rust codegen (npm zero-rust, packages/zero-rust-cli, on the shared packages/zerogen) generates row structs, the schema, typed queries, and typed CRUD for mutators (mutate::issue(tx).update(…)).
  • M4, client engine: connection, query manager, pokes, mutators with rebase and replay, auth, recovery, and the client lifecycle, ported after zero-swift's engine. An opt-in tokio WebSocket transport (tokio-transport, tls with rustls on ring).
  • M5, storage: public StorageBackend and StoredDatabase traits; ZeroStorage::Memory (the default, which keeps nothing across restarts), Sqlite (sqlite or sqlite-bundled, zero-swift's file layout), and Custom; memory fallback, corrupt-store recovery, and the database GC; Zero::delete.

Conformance: 63,453 of 63,453 cases replayed, 0 failures.

  • 61,198 match JS exactly.
  • 2,255 match under the exceptions documented in PARITY.md: 2,251 lone surrogates, 1 nesting-depth case, and 3 JSON path indexes Rust's u32 index type can't hold (each checked against JS's error message).
  • Every client scenario file passes on memory and again on SQLite, as its own (sqlite) row: client, client-harden-connection, client-harden-mutations, harden-queries, bughunt-client, and the seeded client-aggregates-fuzz.
  • No vector is skipped or weakened.

End to end (CI, against Postgres and zero-cache from this repo): the walking skeleton, and the whole client on the tokio transport: sync, optimistic mutations the server confirms, a relaunch on memory and on SQLite, two clients of one group, and synced aggregates.

Dependencies and toolchain

  • zero-client's default build has no third-party dependencies; CI fails if one appears outside an opt-in feature. tokio, tokio-tungstenite, rustls and rusqlite are opt-in.
  • MSRV is 1.88, checked with the default and with every feature (rusqlite is held at 0.40 for it).

Outside rust/

  • rust.yml: CI on every PR. rust-nightly.yml: fresh IVM and client-scenario seeds every night; it starts once this merges, since GitHub runs scheduled workflows only from the default branch.
  • rust-release.yml, packages/zero-rust-cli (npm zero-rust, also the Changesets version carrier), packages/zerogen (the Rust target), and the changeset docs.
  • swift-release.yml: its Version Packages PR now versions both clients and is retitled chore: version packages.
  • Comment-only edits in packages/zero-conformance/src/e2e/.

Merging releases nothing. The crate and CLI are at 0.0.0, which rust-release.yml never publishes, and there's no zero-rust changeset. The first crates.io and npm publishes are tmm's, by hand. Seeding wevm/zero-rust's main after the merge waits for his go-ahead.

Amp thread: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c

ampagent and others added 23 commits November 10, 2026 18:44
…d CI

Adds rust/, a Cargo workspace with two crates:

- zero-client, the client, starting with the JSON layer: JS property
  order, JSON.parse and JSON.stringify, Number#toString and
  Number(string), UTF-8 and UTF-16 string orders, and xxHash32 with
  h64 and h128.
- zero-conformance, which replays packages/zero-conformance's vectors
  and seeded fuzz corpora and reports, per file, the cases that match
  JS, the ones that match only under a documented exception, and the
  ones that fail. Files with no suite yet are listed as not implemented.

Replayed so far: string-compare, hash, number-format, number-parse,
json-parse, json-stringify-strings, sweep-json, harden-json, and the
generated harden-json-fuzz corpus. All 23,757 cases match JS: 2,251 only
under the lone-surrogate exception and one under the nesting limit,
both as in the Swift client (rust/DESIGN.md).

The Rust workflow checks the vectors are current, then runs rustfmt,
clippy, the tests, and the conformance report on rust-client.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
compareValues and valuesEqual, toPrimaryKeyString and the aggregate row
keys, LIKE and ILIKE on UTF-16 code units as JS's non-Unicode RegExp
matches them, and parseTTL, normalizeTTL, clampTTL, and compareTTL.

compare-values, primary-key-string, aggregate-row-key, like, and ttl
replay with all 655 cases matching JS exactly.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
Sets up the Rust client's release the way the Swift client's works:

- packages/zero-rust carries the crates' version and changelog for
  Changesets; it isn't published to npm.
- Swift Release's Version Packages PR now versions both clients: it
  also copies zero-rust's version into rust/Cargo.toml and Cargo.lock
  (rust/scripts/set-version.sh). A second Changesets job would race it
  for the same branch, so the PR is now titled "chore: version
  packages".
- rust-release.yml, on main, releases a version that isn't out yet
  after approval in the rust-release environment: a subtree split of
  rust/ to wevm/zero-rust with a tag and a GitHub release from the
  changelog, and cargo publish of zero-client, with trusted publishing
  once the crate exists. 0.0.0 is never released.
- rust/.github/sts/mirror.sts.yaml authorizes the mirror token.
- rust.yml adds rustdoc and cargo package checks, and an API-break job
  (cargo semver-checks) that starts once zero-client is on crates.io.

Also adds rust/PARITY.md, the crate README, and a release section in
rust/DESIGN.md. The conformance test skips in the mirror, which has no
vectors; CI's conformance step still fails without them.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
…mapping

The AST and its JSON form (astSchema), strictASTSchema's depth limit and
unknown-key rejection, normalizeAST with V8's Array#sort (the comparators
are inconsistent, so the sort's exact comparisons decide the hash),
hashOfAST, hashOfQueryInternals, and hashOfNameAndArgs, the view Format,
and the client-to-server NameMapper over a schema's tables and columns.

ast, ast-features, query-internals-hash, name-args-hash,
sweep-name-args-hash, and the generated ast-fuzz corpus replay with all
2,288 cases matching JS exactly. ast-features maps names with the JS test
schema as JS wrote it into ivm.json, and compares rejections by verdict,
not by message.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
…tocols header

Upstream and Downstream with every message body: initConnection and the
Sec-WebSocket-Protocol header (base64 and decodeURIComponent as atob and
the browser do them), query patches, pokes and row patches, pushes,
mutation results, error bodies with their kinds, origins, and reasons,
the client schema and its normalization, and the inspect messages,
checked and stripped by a small interpreter of valita schemas. Binary
poke chunks are joined as bytes before strict UTF-8 decoding.
PROTOCOL_VERSION is 57, inside meta.json's window.

All replayed vectors match JS: protocol-upstream, protocol-downstream,
sweep-downstream, protocol-poke-chunks, protocol-sec-protocols,
protocol-client-schema, meta, and the generated protocol fuzz,
harden-downstream-text, and harden-poke-bytes corpora, 28,139 cases.
Loosening one field's decoding fails 26 of them, so the suites do catch
a wrong decoder.

Also fixes rust/PARITY.md, whose ported table had missed the AST rows.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
AnyQuery and ExpressionBuilder, ported from QueryImpl and expression.ts
by way of Swift's AnyQuery, and schema relationships. sweep-query-hash
replays all 347 programs, matching JS's AST and hash. This finishes M1.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
The manifest said 1.85, but the library uses let-chains, stable since
1.88; it fails to build on 1.85. The higher MSRV also enables a clippy
lint, so xxhash32 uses as_chunks.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
The connect URL (createConnectionURL) and the client schema
(clientSchemaFrom), and tests/walking_skeleton.rs: it connects with the
initConnection message in Sec-WebSocket-Protocol, asks for issues.open,
decodes the pokes, text and binary, and checks the fixture's rows. CI
runs it in a new end-to-end job with Postgres and zero-cache, and checks
the build on the minimum Rust version. This finishes M1.5.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
The rust-release environment, like swift-release, limits the release jobs
to main and has no required reviewers.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
Nodes, changes, lazy streams, constraints, row comparison, the operator
and source traits, pipeline error capture, and row predicates, ported
from packages/zql/src/ivm by way of Swift's IVM. The start of M2. Also
records tmm's transport decision: a Transport trait in the core and an
opt-in tokio-tungstenite reference transport.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
…cided

Records the dependency review: tokio-tungstenite without default
features, TLS as its own feature so no crypto provider is pulled in by
default, and rusqlite not required until tmm decides how storage is
opted into. Also records the crate name, zero-client.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
MemorySource and its connections, ported from memory-source.ts by way
of Swift: sorted indexes made on demand, constrained and reverse scans
from a bound, the overlay that shows a push in flight to the fetches it
triggers, split edits, multi-constraint fetches, and filterPush.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
JSON objects and schemas use map::OrderedMap, an insertion-ordered map
in the crate: a linear scan for small maps, such as rows, and a hash
index of positions above eight keys. Object's iterators are the crate's
own types, so no indexmap type is in the public API. CI now fails when
the default build of zero-client has a normal or build dependency.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
The storage design for M5, after zero-swift's: the core keeps the mirror,
the decisions, and the lifecycle; a backend trait loads at open and
applies an ordered write stream; memory is in the core and is the
fallback; SQLite comes behind sqlite and sqlite-bundled features, with
writes on a std thread. Whether the default build is memory-only is
still open.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
tmm's decision. The README and the crate docs say a default build
persists nothing across restarts, and the quick start adds the sqlite
feature. The storage options are named as in Swift: memory, SQLite, and
a custom store.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
tmm makes the first publish of zero-client himself, with his own cargo
login. Until the crate exists the release run skips the crates.io job
with a notice, still pushing the mirror and its release; afterwards it
publishes with trusted publishing only. The CARGO_REGISTRY_TOKEN
fallback is gone.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
Ports Filter, FilterStart, FilterEnd, Exists, FanOut, FanIn, UnionFanOut,
UnionFanIn, Join, TakeGate, JoinIndexTap, and the join index and overlay
helpers from the Swift client. Not yet exercised by conformance.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
…perators

Ports FlippedJoin, Take, Skip, Cap, LiftField, Aggregate, GroupBy,
GroupProjection, and DeferredInput from the Swift client. Not yet
exercised by conformance.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
Ports ArrayView and view change application, the pipeline builder in
compute mode with correlated predicate pushdown, and Take and Cap's
early-return hydration error. The conformance runner now replays ivm,
ivm-features, bughunt-ivm, and the ivm, sweep-ivm and harden-ivm corpora:
7,321 of 7,321 cases, 3 under the new unwritable-path exception. Adds a
nightly differential run on fresh seeds.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
Formats zero-client's Cargo.toml, which Public JS's oxfmt check flagged
on the pull request.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
zero_client::typed wraps AnyQuery with row structs, typed column and
relationship constants, and result types read from views. zerogen gets a
Rust target and a zero-rust CLI (packages/zero-rust-cli, private for now)
that share schema loading with zero-swift. rust/codegen-tests holds the
generated Rust for the zql test schema, the codegen fixture, zbugs, and
every sweep schema, compiled with clippy's warnings denied, and tests that
typed queries build the untyped queries' ASTs and that their views read as
generated rows.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
@socket-security

socket-security Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedcargo/​tokio@​1.54.15510093100100
Addedcargo/​rustls@​0.23.468110093100100
Addedcargo/​futures-util@​0.3.3410010093100100
Addedcargo/​rusqlite@​0.40.210010093100100
Addedcargo/​tokio-tungstenite@​0.30.09910093100100

View full report

ampagent and others added 6 commits November 10, 2026 23:17
…e CLIs

syncpack failed on zero-rust-cli's typescript 6.0.3. Like zero-swift and
zerogen, it reads schemas with the TypeScript compiler API, which
TypeScript 7 doesn't have.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
… version

As zero-swift does for Swift, the CLI's npm package now carries the Rust
client's version and changelog: packages/zero-rust-cli is the zero-rust
package, and the version carrier packages/zero-rust is gone. It is bundled
with esbuild like zero-swift, with a packed-package smoke test in CI.
rust-release.yml reads the version and changelog from it and publishes it
to npm with trusted publishing once the package exists; the first publish
is made by hand, as for crates.io.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
Port the JS client engine to Rust, after the Swift client's Engine: the
connection manager, query manager, poke handler with rebase and replay,
mutation tracker, query context and views, CRUD transactions with
optimistic aggregates, auth, recovery, and the client and client-group
lifecycle, on a thread-local memory database.

The socket is behind SocketFactory, and zero_client::testing has a manual
scheduler and mock sockets. The conformance runner replays the client
scenarios: client 183/183, client-harden-connection 62/62,
client-harden-mutations 25/25, harden-queries 41/41, bughunt-client 12/12.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
Add zero_client::tokio_transport behind the tokio-transport feature: a
Scheduler on a tokio LocalSet and a SocketFactory on tokio-tungstenite,
with no tokio or tungstenite type in the public API. The tls feature adds
wss:// with rustls on ring and webpki roots. The default build still has
no dependencies.

tests/tokio_transport.rs checks frames, closes, and timers against a local
WebSocket server; tests/client_e2e.rs runs the whole client against
zero-cache, as Swift's ClientE2ETests and AggregateE2ETests do. CI lints
and tests with every feature and runs the client end-to-end tests.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
Generate, per table, <Row>Update (the primary key, an Option per other
column, and a ColumnUpdate for one that can be null) and <Row>Key, and a
mutate module with a function per table returning a TypedTableMutator:
mutate::issue(tx).update(&IssueUpdate { closed: Some(true),
..IssueUpdate::new(id) }), as Swift's tx.mutate.<table>. The names are
claimed after the schema's own types, so existing generated names don't
change.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
The client suite replays generated/client-aggregates-fuzz, the seeded
aggregate scenarios of client/bughunt.test.ts, which the runner skipped
without listing: it now lists every generated corpus no suite replays as
not implemented, as it does checked-in files. Rust Nightly generates them
on fresh seeds, as Swift Nightly does, and replays them with the IVM
corpora. 150/150 on the PR seed; 3,000/3,000 on four fresh seeds locally.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
Put the client's database behind public StorageBackend and StoredDatabase
traits, as zero-swift's ReplicaDatabase: the core keeps the client groups
and clients, makes every decision, and writes an ordered StorageOp stream.
ZeroOptions gains storage (Memory, the default; Sqlite behind the sqlite
feature; Custom) and on_storage_failure. A store that can't open falls
back to memory, one that doesn't decode is dropped as lost client state,
and the database GC runs five minutes in and every 12 hours. Zero gains
delete and flush_storage.

SQLite (rusqlite 0.40, the newest on Rust 1.88; sqlite-bundled compiles it
in) uses zero-swift's file layout, a writer thread per open database, WAL,
and one transaction per write. The client scenarios run again on SQLite,
each on a directory of its own: 473/473 match JS. tests/sqlite_storage.rs
checks the file itself, and client_e2e relaunches on SQLite against
zero-cache.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Co-authored-by: tmm <tmm@tmm.dev>
@tmm tmm changed the title feat(zero-rust): Rust client through M2 (primitives, wire, IVM) feat(zero-rust): Rust client Nov 11, 2026
@tmm
tmm merged commit 49cb280 into main Nov 11, 2026
27 of 28 checks passed
@tmm
tmm deleted the rust-client branch November 11, 2026 02:28
tmm added a commit that referenced this pull request Nov 11, 2026
…the apps' lockfiles (#145)

changesets/action runs version-script through @actions/exec without a shell, so the && chain was passed as arguments and the Version PR job failed on every push to main since #130. The script now runs under sh -c. rust/scripts/set-version.sh also bumps zero-client in the zbugs apps' lockfiles so their --locked builds keep working after a release.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c6b9-fc99-73a1-bf70-e71fd484042c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants