Investigate x86_64 register spilling in generated mappings for large records

[stalep]

Background

Franz Nigro (Red Hat, Quarkus/Jackson performance) discovered a register spilling problem on x86_64 with per-class generated serializers in Quarkus Jackson. The finding applies to any code generation approach that produces per-class methods with many inlined field operations.

The Problem

When C2 inlines N field accessor calls (e.g., writeString, getField().asString()) into a single generated method, x86_64's 16 GP registers are insufficient for all the live values. C2 inserts register spills (push/pop to stack frame) before and after each inlined call. With N fields, the spilling overhead accumulates and dominates execution time.

ARM (Apple M2/M3) is unaffected — ARM has 31 GP registers, enough to hold all live values without spilling. This makes the problem invisible on Mac benchmarks.

Current jjq-mapper-processor Architecture

The processor generates one class per @JqMapped record. For a 5-field record:

public User fromJqValue(JqValue input, JqMapper mapper) {
    return new User(
        P_NAME.apply(input).asString(null),     // inlined: get + asString
        (int) P_AGE.apply(input).asLong(0),     // inlined: get + asLong
        P_SCORE.apply(input).asDouble(0.0),     // inlined: get + asDouble
        P_ACTIVE.apply(input).asBoolean(false), // inlined: get + asBoolean
        P_EMAIL.apply(input).asString(null)      // inlined: get + asString
    );
}

Each P_*.apply(input).asXxx() chain may be inlined by C2, creating N copies of the extraction logic in the caller frame. For 5 fields this is fine (22 ns on x86_64). For 20+ fields, register pressure could cause spilling.

Franz's Solution: Unified Accessor

Instead of per-class generated methods, generate a single class with type-dispatched accessors:

public final class UnifiedJqMapping {
    // Single method for string extraction — ONE inline site
    public static String extractString(JqValue input, String fieldName) {
        JqValue v = input.getField(fieldName);
        return v.asString(null);
    }
    
    public static int extractInt(JqValue input, String fieldName) {
        return (int) input.getField(fieldName).asLong(0);
    }
    
    public static double extractDouble(JqValue input, String fieldName) {
        return input.getField(fieldName).asDouble(0.0);
    }
}

C2 sees ONE extractString → inlines it once. The switch on field name is handled by the JIT's branch predictor. No N-way register pressure.

Trade-offs:

• Pro: Predictable register usage regardless of field count
• Pro: Smaller code size (1 class instead of N)
• Pro: Faster JIT warmup (fewer classes to compile)
• Con: Extra switch/dispatch overhead (~1-2 ns per field)
• Con: Bytecode constant pool limit risk with many types in one class

Action Items

1. Benchmark with large records (10, 20, 50 fields) on x86_64 to determine if spilling occurs with the current per-class approach
2. Compare x86_64 vs ARM numbers to detect the spilling signal (if ARM is significantly faster, spilling is likely)
3. If spilling is detected: prototype the unified accessor approach and benchmark
4. If no spilling: document the field count at which it becomes a problem (or confirm it's a non-issue for jjq due to simpler extraction logic vs Jackson's writeString)

Why jjq May Be Less Affected

jjq's field extraction (JqObject.get() + asString()) is simpler than Jackson's serialization path (writeString involves encoding, buffer management, flush checks). The simpler inlined body may use fewer registers per field, pushing the spilling threshold higher. But this needs measurement, not assumption.

References

• Franz Nigro's Quarkus Jackson UnifiedAccessor pattern
• Jackson Afterburner module (does similar per-class bytecode generation)
• x86_64: 16 GP registers (RAX, RBX, RCX, RDX, RSI, RDI, RBP, RSP, R8-R15)
• ARM64: 31 GP registers (X0-X30)
• C2 register allocation: linear scan with fixed register budget per method

[stalep]

Benchmark results (x86_64) confirm register spilling signal

Per-field cost increases with field count for generated mappings:

Implementation	5 fields	10 fields	20 fields
Generated (ns/field)	6.7	10.3	12.9
Reflection (ns/field)	26.8	21.4	17.8
Jackson (ns/field)	22.0	17.0	15.3

Generated per-field cost nearly doubles from 5 to 20 fields (6.7 → 12.9 ns/f). Reflection and Jackson show the opposite trend — decreasing per-field cost as fixed overhead amortizes.

Despite the spilling, generated is still fastest at all field counts:

• 5 fields: 33 ns vs Jackson 110 ns (3.3x faster)
• 10 fields: 103 ns vs Jackson 170 ns (1.6x faster)
• 20 fields: 259 ns vs Jackson 305 ns (1.2x faster)

Crossover point (where Jackson would beat generated) is estimated at ~25-30 fields.

Allocation is unaffected by spilling — generated uses 48-112 B/op (just the record instance) vs 384-504 B for Jackson.

Next steps:

1. Run with -prof perfasm to confirm stack spill instructions in the generated code
2. Prototype the unified accessor approach for comparison
3. Optionally test on ARM to verify the per-field cost stays flat there

[stalep]

Franz Nigro's blog confirms the root cause — see Reflection-free Jackson in Quarkus REST

Key findings that match our benchmark data:

1. Register spilling is an x86_64 C2 behavior, not a code bug. Jackson's writeString loop compiles to 21 instructions standalone, 32 instructions with 12 stack operands when inlined. Same mechanism applies to our JqObject.get() + asString() chain.

2. Moving extractions to helper methods does not help. C2 inlines the small helpers back into the caller. Our three approaches (inlined, per-field helpers, _extractAll) all show identical ~13 ns/field at 20 fields.

3. The correct shape is the unified accessor pattern — a single dispatched method per type operation (stringGetter, intGetter, etc.) reached by (classId, fieldIndex). This is what Franz implemented for Quarkus Jackson and what Afterburner/Micronaut-Serde use.

Current status for jjq:

The generated mapper is still the fastest option for records up to ~15 fields (the common case for h5m DTOs). The per-field cost increase (6.7→12.9 ns/f at 5→20 fields) narrows the advantage over reflection and Jackson but doesn't eliminate it until ~25-30 fields.

The reflection-based ClassMapping with forEach fast path shows decreasing per-field cost and would be the better choice for 25+ field records. JqMapper already falls back to it when no generated mapping exists, so users can simply omit @JqMapped on large records.

Reverted the _extractAll approach (commit d106e55) since it provides no improvement. Kept the HELPER_METHOD_THRESHOLD constant for documentation. A future unified accessor implementation (Franz's pattern adapted to jjq) would be the proper fix for large records if needed.