Snappy benchmarks + introduce `aircompressor` by zilm13 · Pull Request #10861 · Consensys/teku

zilm13 · 2026-06-19T11:51:19Z

PR Description

Speed (µs/op, lower is better)

Payload	Op	snappy-java	Netty	aircompressor-native	aircompressor-java
Attestation	compress	2.183	3.993	2.036	2.999
Attestation	decompress	0.648	1.516	0.478	0.513
AggregateAndProof	compress	2.073	3.470	1.946	3.147
AggregateAndProof	decompress	0.658	1.462	0.471	0.561
SyncCommitteeMessage	compress	0.205	0.188	0.089	0.528
SyncCommitteeMessage	decompress	0.201	0.062	0.027	0.010
BeaconBlock	compress	3.552	20.487	3.425	9.693
BeaconBlock	decompress	1.222	5.260	~1.0–1.4 †	2.145
DataColumnSidecar	compress	2.152	4.014	2.006	2.992
DataColumnSidecar	decompress	0.681	1.581	0.489	0.567

Compression ratio

Payload	raw bytes	snappy-java	aircompressor-native	Netty	aircompressor-java
Attestation	16,621	16,623	16,623	16,622	16,622
AggregateAndProof	16,829	16,835	16,835	16,835	16,835
SyncCommitteeMessage	144	148	148	148	148
BeaconBlock	57,639	57,583	57,583	56,552	56,556
DataColumnSidecar	17,508	17,509	17,509	17,509	17,509
Synthetic repeating 64 KB	65,536	3,328	3,328	3,323	3,323
Synthetic mixed 64 KB	65,536	65,542	65,542	36,320	36,320
BeaconBlock chain generated — 32 KB chunks	32,768	14,964	14,964	14,895	14,895

TLDR; snappy-java (JNI) and aircompressor-native are both Google's C libsnappy underneath, they always have pretty close numbers, netty is always way slower but it's better on Synthetic mixed 64 KB benchmark in compression ratio (only one time when it wins), but the reason is it's very synthetic benchmark having 32 KB random + 32 KB repeating, which is very uncommon for real data, so we shouldn't care on this.

Fixed Issue(s)

Documentation

I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

I thought about adding a changelog entry, and added one if I deemed necessary.

Note

Low Risk
Benchmark-only changes in eth-benchmark-tests with no runtime networking or codec behavior modified in production modules.

Overview
Adds JMH microbenchmarks in eth-benchmark-tests to compare Snappy block compression for gossip-style workloads: snappy-java (current gossip), Netty (current RPC), and aircompressor-v3 (FFM native + pure Java). The goal is data to support consolidating on a single Snappy library later.

SnappyCompressionBenchmark parametrizes payload type (SSZ messages, 32 KB beacon-block/data-column chunks, synthetic compressible data) and implementation. It measures compress/decompress throughput and reports compression ratio via a new CompressionMetrics aux counter. On startup (snappy-java trials), verifyCrossCompatibility fails fast if any codec cannot round-trip the Snappy block wire format with snappy-java.

Fixture support: BeaconBlockFixtureGenerator plus BlockIO.readResourceBytes load gzipped Deneb beacon-block SSZ from /snappy/beacon-blocks-deneb.ssz.gz (regenerate via :eth-benchmark-tests:generateBeaconBlockFixture). The benchmarks README documents example ./gradlew :eth-benchmark-tests:jmh invocations with -prof gc and payload/impl filters.

Dependencies are scoped to JMH only: netty-handler, snappy-java, and aircompressor-v3 on the benchmark classpath—no production gossip/RPC code is switched in this diff.

^{Reviewed by Cursor Bugbot for commit e32aa2e. Bugbot is set up for automated code reviews on this repo. Configure here.}

tbenr · 2026-06-19T15:41:13Z

Snappy defines the compressed format, but not the encoder heuristics. Two valid implementations may produce different compressed sizes for the same input, so comparing encode speed alone is incomplete. A fair benchmark should also report compression ratio / compressed payload size.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit 433d276. Configure here.}

cursor · 2026-06-19T18:39:06Z

+
+    public double rawToEncodedRatio() {
+      return rawToEncodedRatio;
+    }


Wrong JMH compression ratio

Medium Severity

CompressionMetrics uses @AuxCounters(Type.EVENTS) with a double rawToEncodedRatio() getter and divides the ratio by measurementCount on every benchmark invocation. JMH treats floating-point aux counters as VALUES (last value per iteration), and EVENTS sums per-invocation deltas for integral counters—not once per measurement iteration—so the reported compression ratio is wrong or missing.

Additional Locations (1)

eth-benchmark-tests/src/jmh/java/tech/pegasys/teku/benchmarks/SnappyCodecBenchmark.java#L84-L108

^{Reviewed by Cursor Bugbot for commit 433d276. Configure here.}

zilm13 · 2026-06-19T18:41:09Z

Nice catch. I've added compression ratio to the benchmark. Though we have definitely different implementations, I think they behave similar:

SnappyCodecBenchmark.nettyEncode:rawToEncodedRatio                 RANDOM           1024  avgt    5    0.995              #
SnappyCodecBenchmark.nettyEncode:rawToEncodedRatio                 RANDOM          65536  avgt    5    1.000              #
SnappyCodecBenchmark.nettyEncode:rawToEncodedRatio                 RANDOM        1048576  avgt    5    1.000              #
SnappyCodecBenchmark.snappyJavaEncode:rawToEncodedRatio            RANDOM           1024  avgt    5    0.995              #
SnappyCodecBenchmark.snappyJavaEncode:rawToEncodedRatio            RANDOM          65536  avgt    5    1.000              #
SnappyCodecBenchmark.snappyJavaEncode:rawToEncodedRatio            RANDOM        1048576  avgt    5    1.000              #

SnappyCodecBenchmark.nettyEncode:rawToEncodedRatio              REPEATING           1024  avgt    5    3.436              #
SnappyCodecBenchmark.nettyEncode:rawToEncodedRatio              REPEATING          65536  avgt    5   19.722              #
SnappyCodecBenchmark.nettyEncode:rawToEncodedRatio              REPEATING        1048576  avgt    5   21.225              #
SnappyCodecBenchmark.snappyJavaEncode:rawToEncodedRatio         REPEATING           1024  avgt    5    3.380              #
SnappyCodecBenchmark.snappyJavaEncode:rawToEncodedRatio         REPEATING          65536  avgt    5   19.692              #
SnappyCodecBenchmark.snappyJavaEncode:rawToEncodedRatio         REPEATING        1048576  avgt    5   19.709              #

SnappyCodecBenchmark.nettyEncode:rawToEncodedRatio                  MIXED           1024  avgt    5    1.277              #
SnappyCodecBenchmark.nettyEncode:rawToEncodedRatio                  MIXED          65536  avgt    5    1.804              #
SnappyCodecBenchmark.nettyEncode:rawToEncodedRatio                  MIXED        1048576  avgt    5    1.000              #
SnappyCodecBenchmark.snappyJavaEncode:rawToEncodedRatio             MIXED           1024  avgt    5    0.995              #
SnappyCodecBenchmark.snappyJavaEncode:rawToEncodedRatio             MIXED          65536  avgt    5    1.000              #
SnappyCodecBenchmark.snappyJavaEncode:rawToEncodedRatio             MIXED        1048576  avgt    5    1.903              #

So both codecs are useless on RANDOM data and both are pretty similar on REPEATING. The difference is only on MIXED: snappyJava doesn't compress small chunks 64kb and less, netty is opposite, it compresses small chunks but not big. Maybe there is a way to configure it, not compressing big chunks looks like optimization, but the speed is not good for netty anyway.

lucassaldanha · 2026-06-20T04:45:18Z

I have a set of benchmarks with an alternative library that seems a promising way forward. Unfortunately, didn't have time to document it yet. Will do on Monday.

…mark

…le payloads to Snappy benchmark

…ssionBenchmark

zilm13 · 2026-06-25T16:54:05Z

reworked over Lucas's benchmarks from https://github.com/lucassaldanha/teku/blob/feature/snappy-benchmark/eth-benchmark-tests/src/jmh/java/tech/pegasys/teku/benchmarks/networking/SnappyCompressionBenchmark.java

Added generated blocks chain closer to real data, not random data, BEACON_BLOCKS_FIXTURE_CHUNK_32K
Added chunked sidecars benchmark (on RPC we use chunks)
Added compression ratio to jmh output

tbenr · 2026-06-26T07:39:26Z

reworked over Lucas's benchmarks from https://github.com/lucassaldanha/teku/blob/feature/snappy-benchmark/eth-benchmark-tests/src/jmh/java/tech/pegasys/teku/benchmarks/networking/SnappyCompressionBenchmark.java

Added generated blocks chain closer to real data, not random data, BEACON_BLOCKS_FIXTURE_CHUNK_32K

Added chunked sidecars benchmark (on RPC we use chunks)

Added compression ratio to jmh output

is the pr description updated with latest results?

Snappy benchmarks

02a248f

Add compression ratio

433d276

cursor Bot reviewed Jun 19, 2026

View reviewed changes

zilm13 marked this pull request as draft June 22, 2026 13:08

lucassaldanha and others added 12 commits June 25, 2026 14:02

Add SnappyCompressionBenchmark comparing snappy-java and Netty Snappy

a337d18

Apply review fixes: explicit JMH modes and guard cross-compat check

517b39c

Document how to run SnappyCompressionBenchmark with GC profiling

7785718

Add aircompressor-v3 native and pure-Java Snappy to compression bench…

5ef13bc

…mark

Record aircompressor-v3 benchmark results and revised recommendation

f81c524

Add compression-ratio reporting, shared decode buffer, and compressib…

23201b3

…le payloads to Snappy benchmark

add fixture reading blocks cas and 32k sidecars cases to SnappyCompre…

b3a2d3f

…ssionBenchmark

Add block generator

1f12287

move compression metrics to aux, remove old benchmark

93c2f95

fix spotless

3e94b84

fix spotless

abb0c44

Merge branch 'master' into snappy-benchmark

7c18a7f

zilm13 marked this pull request as ready for review June 25, 2026 16:54

zilm13 changed the title ~~Snappy benchmarks~~ Snappy benchmarks + introduce aircompressor Jun 26, 2026

zilm13 and others added 2 commits June 30, 2026 17:47

Merge branch 'master' into snappy-benchmark

22faa97

fix bad merge

bd25d1d

tbenr mentioned this pull request Jul 7, 2026

Add Aircompressor to RPC. Off by default. #10918

Merged

2 tasks

Merge branch 'master' into snappy-benchmark

e32aa2e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Snappy benchmarks + introduce `aircompressor`#10861

Snappy benchmarks + introduce `aircompressor`#10861
zilm13 wants to merge 17 commits into
Consensys:masterfrom
zilm13:snappy-benchmark

zilm13 commented Jun 19, 2026 •

edited by cursor Bot

Loading

Uh oh!

tbenr commented Jun 19, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot Jun 19, 2026

Uh oh!

zilm13 commented Jun 19, 2026

Uh oh!

lucassaldanha commented Jun 20, 2026

Uh oh!

zilm13 commented Jun 25, 2026

Uh oh!

tbenr commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

zilm13 commented Jun 19, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Description

Fixed Issue(s)

Documentation

Changelog

Uh oh!

tbenr commented Jun 19, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jun 19, 2026

Choose a reason for hiding this comment

Wrong JMH compression ratio

Uh oh!

zilm13 commented Jun 19, 2026

Uh oh!

lucassaldanha commented Jun 20, 2026

Uh oh!

zilm13 commented Jun 25, 2026

Uh oh!

tbenr commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zilm13 commented Jun 19, 2026 •

edited by cursor Bot

Loading