Skip to content

perf(rsema1d): precompute coefficient logarithms for RLC computation - #1005

Open
citizen-stig wants to merge 3 commits into
mainfrom
perf/rsema1d-rlc-precomputed-logs
Open

perf(rsema1d): precompute coefficient logarithms for RLC computation#1005
citizen-stig wants to merge 3 commits into
mainfrom
perf/rsema1d-rlc-precomputed-logs

Conversation

@citizen-stig

@citizen-stig citizen-stig commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

compute_rlc multiplied every 2-byte symbol of a row by a GF(2^128) coefficient through GF128::scalar_mul, which for each of the 8 limbs looked up: 1.log(symbol), 2. log(limb) and 3. exp(sum): 24 dependent table loads per symbol, with log(symbol) recomputed 8 times.
It was 51% of a single-core encode of a 128 MiB blob and the largest CPU consumer once parity encoding is parallel.

Encoding and context-based verification precompute the GF(2^16) logarithm of each coefficient limb. For a 32 KiB row, this lookup table uses 256 KiB.
ExtendedData builds it once for all rows, and VerificationContext caches it after the first successful verification. The original coefficients are not retained.

The public compute_rlc(row, coeffs) function keeps its existing signature and direct multiplication path. This avoids building a lookup table for one-off calls and remains efficient for sparse or zero-filled rows.

Simplified explanation:

Before:

coefficient_value = coefficients[symbol_index].limbs[limb]
coefficient_log   = GLOBAL_LOG_TABLE[coefficient_value]

For every row, the code:

1. Reads the coefficient value.
2. Uses that value as an index into the global logarithm table.
3. Reads the logarithm from that second location.

After precomputation:

coefficient_log = precomputed_logs[symbol_index][limb]

Tests compare the optimized path with the direct implementation using random rows, zero symbols, zero coefficient limbs, and exponent wraparound. All 254 Go compatibility vectors, including K=4096/N=12288 cases, also pass.

criterion, target-cpu=native, Ryzen 9 3950X
RUSTFLAGS="-C target-cpu=native" cargo bench -p rsema1d --bench codec_bench -- \
      --save-baseline main 'encode_in_place/(8MB|128MB_k4096)|^verification'
      
# Main
verification/8MB_k4096_n12288
                        time:   [19.164 µs 19.187 µs 19.214 µs]
                        time:   [19.273 µs 19.290 µs 19.308 µs]
                        time:   [19.299 µs 19.331 µs 19.369 µs]
verification/128MB_k4096_n12288
                        time:   [290.04 µs 290.76 µs 291.41 µs]
                        time:   [291.13 µs 291.46 µs 291.88 µs]
                        time:   [291.11 µs 291.30 µs 291.49 µs]
encode_in_place/128MB_k4096_n12288
encode_in_place/128MB_k4096_n12288
                        time:   [809.18 ms 810.48 ms 811.79 ms]
                        thrpt:  [157.68 MiB/s 157.93 MiB/s 158.18 MiB/s]
                        time:   [812.55 ms 813.86 ms 815.18 ms]
                        thrpt:  [157.02 MiB/s 157.28 MiB/s 157.53 MiB/s]
                        time:   [809.66 ms 810.90 ms 812.19 ms]
                        thrpt:  [157.60 MiB/s 157.85 MiB/s 158.09 MiB/s]
                                                                                                                                                       
## Branch
verification/8MB_k4096_n12288
                        time:   [9.3636 µs 9.3779 µs 9.3918 µs]
                        time:   [9.3008 µs 9.3135 µs 9.3272 µs]
                        time:   [9.2832 µs 9.2931 µs 9.3051 µs]
verification/128MB_k4096_n12288
                        time:   [126.85 µs 126.94 µs 127.04 µs]
                        time:   [130.57 µs 130.71 µs 130.87 µs]
                        time:   [126.83 µs 126.97 µs 127.16 µs]
encode_in_place/128MB_k4096_n12288
                        time:   [765.18 ms 766.44 ms 767.72 ms]
                        time:   [765.47 ms 766.64 ms 767.79 ms]
                        time:   [765.82 ms 767.14 ms 768.44 ms]
                        thrpt:  [166.73 MiB/s 167.01 MiB/s 167.28 MiB/s]
                        thrpt:  [166.71 MiB/s 166.96 MiB/s 167.22 MiB/s]
                        thrpt:  [166.57 MiB/s 166.85 MiB/s 167.14 MiB/s]

## Short summary from LLM run
  verification/128MB_k4096_n12288 (one row)   305 us -> 133 us
  verification/8MB_k4096_n12288               20.7 us -> 9.7 us
  encode_in_place/128MB_k4096_n12288           821 ms -> 763 ms
    (32 threads; the RLC pass itself went 79 ms -> 30 ms, the rest is the
    still single-threaded parity encoder on main)
  single core RLC pass over a 128 MiB blob    1.26 s -> 0.44 s (probe)

@citizen-stig
citizen-stig marked this pull request as ready for review September 2, 2026 14:37
@citizen-stig
citizen-stig requested a review from a team as a code owner September 2, 2026 14:37
@citizen-stig
citizen-stig requested review from mcrakhman and vgonkivs and removed request for a team and vgonkivs September 2, 2026 14:37
citizen-stig and others added 3 commits September 2, 2026 16:53
compute_rlc multiplied every 2-byte symbol of a row by a GF(2^128)
coefficient through GF128::scalar_mul, which for each of the 8 limbs looked
up log(symbol), log(limb) and exp(sum): 24 dependent table loads per
symbol, with log(symbol) recomputed 8 times. It was 51% of a single-core
encode of a 128 MiB blob and the largest CPU consumer once parity encoding
is parallel.

RlcCoefficients holds the coefficients together with the logarithm of
every limb (256 KiB for 32 KiB rows), computed once per blob. Per symbol
the work is now one log lookup plus one exp lookup per limb. ExtendedData
builds the table once for all K rows and VerificationContext caches it
alongside the coefficients. compute_rlc(row, coeffs) keeps its signature
and builds a table per call. Results are identical: a test compares against
the previous scalar_mul formulation on random rows with zero symbols and
zero limbs, and the Go cross-implementation vectors (254 cases including
K=4096/N=12288) pass.

criterion, target-cpu=native, Ryzen 9 3950X:
  verification/128MB_k4096_n12288 (one row)   305 us -> 133 us
  verification/8MB_k4096_n12288               20.7 us -> 9.7 us
  encode_in_place/128MB_k4096_n12288           821 ms -> 763 ms
    (32 threads; the RLC pass itself went 79 ms -> 30 ms, the rest is the
    still single-threaded parity encoder on main)
  single core RLC pass over a 128 MiB blob    1.26 s -> 0.44 s (probe)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VPtihoV1GJ9VcNkd7oMRUf
@citizen-stig
citizen-stig force-pushed the perf/rsema1d-rlc-precomputed-logs branch from 245fa38 to e3c6f12 Compare September 2, 2026 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants