Skip to content

fix(zero-rust): take JSON nested as deep as zero-cache writes - #205

Merged
tmm merged 1 commit into
mainfrom
fix/zero-rust-json-depth
Nov 14, 2026
Merged

tmm merged 1 commit into
mainfrom
fix/zero-rust-json-depth

Conversation

@tmm

@tmm tmm commented Nov 14, 2026

Copy link
Copy Markdown
Member

Part of the Rust parity push: clearing the conformance runner's documented exceptions.

Before. json::parse rejected anything nested deeper than 1,500 levels. Cloning, comparing, and writing a value recursed, and a debug build overflowed a 2 MB thread stack at these depths:

Operation Overflowed at
clone (arrays) ~2,400 levels
== ~3,000 levels
write ~3,500 levels
drop ~6,700 levels (objects), ~10,600 (arrays)

zero-cache writes up to about 4,100 levels, and JS accepts all of them. So a row whose JSON column was nested deeper than 1,500 levels got its poke rejected as InvalidMessage, and the client then disconnected and failed again after every reconnect. Swift has the same 1,500 limit.

Now.

  • Clone, PartialEq, and json::write recurse 128 levels deep, then continue over a heap stack. Shallow values keep the recursive code.
  • MAX_DEPTH is 5,000.
  • Drop already fits at that depth, so it is unchanged.
  • Debug formatting still recurses.
  • The conformance runner no longer has a nesting exception. harden-json's 1,502-level case matches JS exactly, and any rejection for depth now counts as a failure.
  • rust/PARITY.md records why the two remaining exceptions stay: lone surrogates (a Rust String can't hold one) and unwritable JSON paths (a negative or fractional index doesn't type-check).

Checks.

  • New unit test: JSON 5,000 levels deep, alternating arrays and objects with siblings at every level, on a 2 MB thread. It round-trips text through parse and write, clone() == original, it compares unequal to a copy whose innermost value differs, and one more level is rejected.
  • Full conformance with the generated corpora: 66,167 cases, 0 failures. Exceptions went from 2,255 to 2,254: 2,251 lone surrogates and 3 unwritable paths.
  • cargo test --workspace, cargo clippy --workspace --all-targets --all-features -D warnings, and cargo doc -D warnings pass.
  • Benchmarks: three rounds of the full suite in turns against main. Everything is within noise or faster (pokes −6 to −8%, first sync −2 to −4%). The 100,000-row materialize timings varied 13–22 ms run to run even on main. Under callgrind, materializing 100,000 rows three times costs 4,858M instructions, against 4,865M on main.

The parser rejected JSON nested deeper than 1,500 levels, because cloning,
comparing, and writing a value recursed and a debug build overflowed a
2 MB thread stack by about 2,400 to 3,500 levels. zero-cache writes up to
about 4,100, and JS takes them, so a row with a JSON column nested deeper
than 1,500 made the client reject the poke, disconnect, and fail the same
way after every reconnect.

Clone, PartialEq, and json::write now recurse 128 levels and then go on
over a heap stack, and MAX_DEPTH is 5,000. Dropping fits in a 2 MB debug
stack at that depth without changes. The conformance runner no longer has
a nesting exception: harden-json's 1,502-level case matches JS exactly,
and a rejection for depth now counts as a failure. PARITY.md records why
the two remaining exceptions (lone surrogates, unwritable JSON paths)
stay.

Amp-Thread-ID: https://ampcode.com/threads/T-db04f816-97ed-4bc9-b19e-9ee68611a0ab
Co-authored-by: tmm <tmm@tmm.dev>
@tmm
tmm merged commit c3cb36c into main Nov 14, 2026
19 checks passed
@tmm
tmm deleted the fix/zero-rust-json-depth branch November 14, 2026 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants