Skip to content

test(zql): run the snapshot-only restored zql tests as re-baselined copies - #195

Merged
tmm merged 3 commits into
mainfrom
test/rebaseline-restored-snapshots
Nov 13, 2026
Merged

tmm merged 3 commits into
mainfrom
test/rebaseline-restored-snapshots

Conversation

@tmm

@tmm tmm commented Nov 13, 2026

Copy link
Copy Markdown
Member

Runs ten more of the zql tests restored from rocicorp/mono 7ebeb2172, as re-baselined copies. Each original fails against today's zql only on inline snapshots of internals: operator traces, operator storage and AST shape. Test-only, so there's no changeset.

No restored file is edited. The originals stay out, as before. Each runs as <original>.rebaselined.test.ts in the original's directory, with a header naming the original, the source commit and the upstream commits that changed the internals. zql's suite and zqlite-zql-test (both modes) pick them up.

Copy of (packages/zql/src/) Snapshots changed Upstream commits
ivm/cap.push 19 of 41: storage, trace fd30e64 (storage keys), 8702aea (countOnly fetch)
ivm/take.fetch, ivm/take.push 9 of 63, 17 of 197: storage fd30e64
ivm/flipped-join.push, ivm/flipped-join.more-fetch 39 of 173, 1 of 3: trace, storage 5c2a12e (:join-index steps; the index holds only the result)
query/complete-ordering, named, query-impl.interning, query-impl.query-kitchen-sink, test/query-gen 14, 1, 1, 1, 1: AST c9d1b15, effd2da (aggregate, groupBy keys)

Commits

  1. Adds the copies exactly as published, below the header, plus query-gen's helpers query-gen.ts and zql-integration-tests/src/helpers/data-gen.ts. The helpers are restored unchanged and pinned in verbatim.txt (now 174 files). The copies' snapshots fail at this commit by design.
  2. Snapshot updates only: 103 snapshots, +1,437 and −91 lines. A script that parses both versions confirmed that nothing outside toMatchInlineSnapshot bodies changed. It also undid the five upstream changes above on each new snapshot, and the result matched the old snapshot. The two exceptions are two actualStorage snapshots in flipped-join.push. There, the in-memory join index from 5c2a12e no longer keeps a stale entry. No view, push or other assertion changed. Vitest re-indented 6 snapshots, which I put back. flipped-join.* contain NUL bytes in storage-key snapshots, as upstream's do, so GitHub shows them as binary; git diff --text shows them.
  3. Updates the README: the copies, how much they duplicate, how to re-baseline again, and what this means for upstream syncs. It also fixes the AST row, which named 5493ba5 where effd2da added groupBy.

Left out

These still fail on real results, fail tsc, or depend on removed code:

  • query-impl.ast: 7 assertions compare AST keys.
  • join-utils, join.push: they call removed helpers.
  • connection-index: its module is gone.
  • exists.fetch, yield.fetch, join.fetch: only snapshots fail, but they also fail tsc.
  • The 7 tsc-only zql files.
  • The zqlite and zero-protocol files already listed in the README.

Duplication

The copies hold 28,553 lines (710 KB): 78 header lines, 1,437 added by the re-baseline, and 27,038 as published. They duplicate only mono's published files, which aren't in this repo.

Upstream syncs

  • Rehearsal: merging rocicorp/mono main at 13bbb08 (15 commits ahead) gives no conflicts. align-manifests.mjs has nothing to change, pnpm install --frozen-lockfile passes, verbatim.txt matches, zql passes 1,313 tests, zqlite-zql-test passes 2,434, and tsc passes.
  • Conflicts: mono never had the *.rebaselined.test.ts paths, so a merge can't conflict on them unless mono adds a file at the same path.
  • When internals change again: the copies' snapshots go stale and Restored Tests fails on the sync PR. The sync only reports it; re-baselining again needs tmm's OK. These ten files broke on five upstream commits in seven weeks, so expect that every few weeks.

Checks run locally

  • zql (CI=true): 1,313 pass, 2 skipped upstream.
  • zqlite-zql-test: 2,434 pass, 4 skipped.
  • 3,000 random seeds for query-gen's random-seed case: none throws.
  • check-types for zql, zql-integration-tests, zqlite-zql-test and zero-conformance.
  • pnpm run lint, oxfmt --check and syncpack lint.
  • zero-conformance generate leaves the vectors byte-identical.

ampagent and others added 3 commits November 13, 2026 16:05
…pshots, as mono published them

Ten zql test files from rocicorp/mono 7ebeb21 fail against today's zql
only on inline snapshots of internals: operator traces, operator storage
and AST shape. The originals stay out; these copies are named
*.rebaselined.test.ts and carry a header naming the original, the source
commit and the upstream commits that changed the internals. Below the
header each copy is byte-identical to the original, so the next commit's
diff is exactly the re-baseline.

query-gen's copy needs query-gen.ts and zql-integration-tests'
helpers/data-gen.ts, restored unchanged and pinned in verbatim.txt.

Co-authored-by: tmm <tmm@tmm.dev>
Regenerated with vitest --update=all against today's zql, then put back at
each snapshot's original indentation (vitest ignores it). Only
toMatchInlineSnapshot bodies change, and only those of operator traces,
operator storage and ASTs; every view, push and other assertion is as
mono published it. The changes come from upstream:

- fd30e64: Cap and Take storage keys are cap:s<value> and take:s<value>.
- 8702aea: Cap's fetch passes countOnly.
- 5c2a12e: :join-index trace steps; the join index holds only the result.
- c9d1b15, effd2da: ASTs carry aggregate and groupBy keys.

Co-authored-by: tmm <tmm@tmm.dev>
The restored-tests README lists the ten copies, the snapshots each
re-baseline changed and the upstream commits behind them, how much they
duplicate, how to re-baseline one again, and what they mean for upstream
syncs. Files that now run as copies, and query-gen's restored helpers, come
out of the left-out table; the AST row names effd2da, which added the
groupBy key, instead of 5493ba5.

Amp-Thread-ID: https://ampcode.com/threads/T-472dde83-243c-421c-b41d-4e7aaf375188
Co-authored-by: tmm <tmm@tmm.dev>
@tmm
tmm merged commit cc28515 into main Nov 13, 2026
17 checks passed
@tmm
tmm deleted the test/rebaseline-restored-snapshots branch November 13, 2026 16:19
tmm added a commit that referenced this pull request Nov 13, 2026
…sh tests (#198)

restored-ivm.test.ts now also runs the re-baselined copies of cap.push,
flipped-join.push, flipped-join.more-fetch and take.push from #195, with
their own assertions, and records the cases they build. Their 125 cases,
less one the chunked flipped-join test builds twice, go to the new
vectors/restored-ivm-rebaselined.json: 124 cases, none of which
restored-ivm.json has. restored-ivm.json and every other vector file are
unchanged.

Swift's RestoredIVMTests replays the new file case by case, like
restored-ivm.json, and so does Rust's restored suite.

Amp-Thread-ID: https://ampcode.com/threads/T-472dde83-243c-421c-b41d-4e7aaf375188

Co-authored-by: Amp <amp@ampcode.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants