Prototype #602: replace Vector[Any] with cons-cell RuntimeDataStore - #949
Merged
MateuszKubuszok merged 6 commits intoSep 2, 2026
Merged
Conversation
Replace the Vector[Any]-backed RuntimeDataStore with a custom cons-cell linked list that materializes to an Array on first indexed access. This eliminates the O(n) array copy on every .withFieldX DSL call, reducing chain construction from O(n^2) to O(n) total allocations. - New RuntimeDataStore class: O(1) prepend via cons cells, O(1) random access after lazy materialization to Array on first .apply() call - Updated type aliases in TransformerDefinitionCommons and PatcherDefinitionCommons (both Scala 2 and 3) to use the new class - Changed all addOverride implementations from `overrideData +: runtimeData` to `runtimeData.prepended(overrideData)` - Fixed DslMacros cross-quote splices for the same pattern - Added RuntimeDataStoreSpec with unit tests for core operations
Cover TransformerDefinition, TransformerInto, PartialTransformerDefinition, PartialTransformerInto, PatcherDefinition, and PatcherUsing with tests for: - 0, 1, and multiple data-carrying modifiers - data-carrying interleaved with type-only modifiers (withFieldRenamed, enableMethodAccessors, etc.) - val-reference branching (shared base, two independent continuations) - deep chains after branching
- Fix broken [[prepend]] scaladoc link → [[prepended]] - Fix @SInCE 1.7.0 → @SInCE 2.0.0 - Tone down overstated perf justification in scaladoc - Update DESIGN.md, index.md, under-the-hood.md: Vector[Any] → RuntimeDataStore - Add RuntimeDataStore.toString for debuggability - Add tests: repeated apply (materialized fast path), materialize-then-branch, toString, built transformer used multiple times - Fix scalafmt formatting in test scaladocs
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #949 +/- ##
==========================================
- Coverage 83.46% 82.74% -0.73%
==========================================
Files 176 183 +7
Lines 7011 7331 +320
Branches 509 529 +20
==========================================
+ Hits 5852 6066 +214
- Misses 1159 1265 +106 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Replace @volatile with benign-race pattern (removes 25% regression on pre-built transformer hot path). Add RuntimeDataStore.wrap(Array) factory for pre-materialized stores with System.arraycopy support for branching. Introduce RuntimeDataStoreFlattening on both Scala 2 and Scala 3: the terminal macro walks the prefix tree to detect linear WithRuntimeDataStore .update chains, and when found emits RuntimeDataStore.wrap(Array(...)) instead of N cons-cell allocations. Falls back to the standard prefix.runtimeData access for unrecognized tree shapes (val captures, forAll-produced constructors with embedded data, etc.). Guard the New-constructor base case with hasEmptyRDS to avoid incorrectly flattening forAll chains where runtime data is embedded in constructor args rather than delivered via WithRuntimeDataStore.update.
…ppers The walk function now returns a tri-state (EmptyBase/OpaqueBase) instead of Boolean, enabling three optimization tiers: - Full linear chain → RuntimeDataStore.wrap(Array(...)) - Partial chain (val capture) → base.runtimeData.prependedAll(Array(...)) - Flag wrappers → transparent recursion via hasEmptyOverrides check JMH shows 54-66% throughput improvement on partial chain benchmarks (largeConstChimneyIntoSplit, largeConstChimneyDefinedSplit) with no regression on existing linear chain benchmarks.
MateuszKubuszok
deleted the
prototype-602-dsl-allocation-optimization
branch
September 2, 2026 10:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Vector[Any]runtime data store with a custom cons-cell linked list (RuntimeDataStore) that materializes into a flatArray[Any]on first indexed accessval-captured definitions share the immutable cons chain; each branch gets its own materialized arrayDESIGN.md,index.md,under-the-hood.md) to reflect the new backing storetoStringfor debuggabilityCompile-time DSL chain flattening
The terminal macros (
.transform,.buildTransformer,.patch) now analyze the prefix tree at compile time and emit optimizedRuntimeDataStoreconstruction:define.withFieldConst(...).buildTransformer)RuntimeDataStore.wrap(Array(...))val b = define.withFieldConst(...); b.withFieldConst(...).buildTransformer)b.runtimeData.prependedAll(Array(...)).enableDefaultValues,.withTargetFlag, etc.)prefix.runtimeDataThe walk uses
hasEmptyOverrides(checks forTransformerOverrides.Empty/PatcherOverrides.Emptytype args) to distinguish extension methods (valid chain bases) from flag wrappers (must recurse through). Works on both Scala 2.13 and Scala 3.Benchmark results (JMH, Scala 2.13, 22-field case class)
Addresses #602.
Test plan
RuntimeDataStoreSpectests (including 5prependedAlltests)