Skip to content

Add ConcurrentHashtable (perf toolbox) - #11675

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 44 commits into
masterfrom
feat/concurrent-hashtable
Sep 1, 2026
Merged

Add ConcurrentHashtable (perf toolbox)#11675
gh-worker-dd-mergequeue-cf854d[bot] merged 44 commits into
masterfrom
feat/concurrent-hashtable

Conversation

@dougqh

@dougqh dougqh commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

What Does This Do?

Adds a concurrent version of Hashtable. ConcurrentHashtable (like Hashtable) is parameterized on its entry type.

Motivation

Parameterizing on entry type allows the dd-trace-java Hashtable-s to excel in use cases where regular Map-s don't fit or have high overhead including...

  • high arity keys - Hashtable-s don't require constructing a composite key to perform a lookup
  • primitive values - custom Entry type can include one or more primitive fields
  • volatile / atomic values - custom Entry type can use volatile fields, atomic updaters, etc as appropriate
  • metadata - Entry can carry extra information used to drive eviction policies, etc

Additional Notes

  • Adds ConcurrentHashtable with D1 (single-key) and D2 (composite-key) inner classes, mirroring the Hashtable API with concurrent access guarantees
  • Lock-free get / getOrCreate fast path via AtomicReferenceArray volatile bucket reads plus a volatile chain-next pointer; synchronized only on miss, with a double-checked re-read under the lock
  • Shared mechanics (sizeFor, bucketIndex, bucket, unlink, removeIf, drain, clear, forEach) are public static building blocks over a caller-owned AtomicReferenceArray — the same "static functions over a caller-owned array" shape as Hashtable (see how AggregateTable uses Hashtable). Callers that need primitive/higher-arity keys or their own lock strategy subclass Entry directly and drive the table with these; D1/D2 are the batteries-included wrappers over that spine
  • Entry-typed factories createFixedBuckets(Class<TEntry> entryClass, int capacity) on all three: ConcurrentHashtable.createFixedBuckets hands back the raw AtomicReferenceArray<TEntry> for the caller-owned path, while D1/D2.createFixedBuckets return a D1/D2 instance. The entry class anchors K/K1/K2/TEntry inference and keeps the family symmetric with Hashtable/FlatHashtable — though the AtomicReferenceArray spine is type-erased, so (unlike FlatHashtable) the class isn't consumed for allocation
  • D2.get(K1, K2) and D2.getOrCreate(K1, K2, creator) accept key parts directly — no composite key object allocated for the lookup, unlike ConcurrentHashMap<Pair<K1,K2>, V> where EA must conservatively treat the key as escaping even on hits (ownership-transfer contract)
  • Adds ConcurrentHashtableD1Test and ConcurrentHashtableD2Test (JUnit 5), including a concurrency correctness test verifying exactly one entry is created under 16 racing threads
  • Adds three JMH benchmarks (@Threads(8), all threads hitting a shared table): ThreadSafeMapD1Benchmark (D1 vs ConcurrentHashMap vs ConcurrentSkipListMap), ThreadSafeMapD2Benchmark (D2 and a raw caller-owned-array arm vs ConcurrentHashMap with a Key2 wrapper vs ConcurrentSkipListMap), and ThreadSafeMapCounterBenchmark (D1 + AtomicLongFieldUpdater inline counter vs ConcurrentHashMap + AtomicLong/LongAdder)

Test plan

  • ./gradlew :internal-api:test --tests "datadog.trace.util.ConcurrentHashtable*" — all tests pass
  • ./gradlew :internal-api:jmhCompileGeneratedClasses — benchmarks compile clean
  • Run the ThreadSafeMap* benchmarks locally to validate get/getOrCreate throughput advantage over CHM and CSLM

🤖 Generated with Claude Code

dougqh and others added 2 commits June 18, 2026 11:44
…y tables

Mirrors Hashtable's D1/D2 API with concurrent access guarantees: lock-free
get via AtomicReferenceArray volatile reads, synchronized getOrCreate with
double-checked re-read on miss. Eliminates composite key object allocation
on hot read paths — the same structural advantage Hashtable.D2 has over
HashMap<Pair<K1,K2>,V>, but thread-safe.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…t class; add D2 benchmark

Extract bucketIndex and forEach into ConcurrentHashtable.Support, mirroring the
Hashtable.Support pattern. Add ConcurrentHashtableD2Benchmark comparing get and
getOrCreate throughput against ConcurrentHashMap and ConcurrentSkipListMap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@datadog-datadog-prod-us1

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.81 s 14.60 s [+0.6%; +2.2%] (maybe worse)
startup:insecure-bank:tracing:Agent 13.66 s 13.68 s [-1.0%; +0.7%] (no difference)
startup:petclinic:appsec:Agent 16.93 s 16.79 s [-0.1%; +1.8%] (no difference)
startup:petclinic:iast:Agent 16.93 s 16.91 s [-0.7%; +0.9%] (no difference)
startup:petclinic:profiling:Agent 16.63 s 16.67 s [-1.3%; +0.7%] (no difference)
startup:petclinic:sca:Agent 16.85 s 16.74 s [-0.2%; +1.5%] (no difference)
startup:petclinic:tracing:Agent 16.08 s 16.14 s [-1.3%; +0.5%] (no difference)

Commit: 0b5a1409 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

… ConcurrentHashtable

Two gaps filled per-dimension (D1 and D2):
- Chain collision: force multiple entries into the same bucket (CollidingKey
  with fixed hashCode for D1; pigeonhole via 2-bucket table for D2) and verify
  all entries are reachable after concurrent inserts.
- Concurrent distinct keys: 16 threads each insert a unique key simultaneously,
  verifying final size and that every key is retrievable — exercises concurrent
  inserts to different buckets, which the single-shared-key test does not cover.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
dougqh and others added 15 commits June 22, 2026 22:13
…htable

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ThreadSafeCounterBenchmarks

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…enchmark

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ectly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…adSafeMapD2Benchmark

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaced by the ThreadSafeMap{D1,D2,Counter}Benchmark split.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Give ConcurrentHashtable its own entry hierarchy (Entry / D1.Entry / D2.Entry)
with a volatile next pointer, independent of the single-threaded Hashtable. The
volatile chain pointer lets a chain splice under the write lock be observed by
lock-free readers, which makes removal safe:

  - remove(key)        unlink a single entry
  - removeIf(predicate) sweep the whole table under one lock
  - drain(sink)        read-and-reset: remove every entry, handing each to a
                       caller-supplied accumulator (Consumer + context-passing
                       BiConsumer overload) -- the flush/publish primitive
  - clear()            empty the table

Removed entries keep their own next pointer intact so an in-flight reader can
still traverse forward. Migrates the ThreadSafeMap* benchmarks to the new
entry base. Adds single-threaded and concurrent removal tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lookups reuse the interned KEYS/SOURCE_* instances used to populate the table,
so they exercise the == identity fast path — deliberate and realistic for the
tracer (keys are typically interned tag-name constants), not an oversight.
Clarifies so it isn't misread against the equals()-path numbers elsewhere.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… shape

Flatten the nested Support class onto the ConcurrentHashtable namespace
(static fns over a caller-owned AtomicReferenceArray, mirroring FlatHashtable)
and type the bucket arrays AtomicReferenceArray<TEntry> so the unchecked casts
on the bucket read paths disappear.

- createFixedBuckets(entryClass, capacity) factories on ConcurrentHashtable
  (returns the raw spine), D1, and D2 (return a D1/D2); D1(int)/D2(int) ctors
  are now private. entryClass is a symmetry + type-inference anchor here (the
  AtomicReferenceArray spine is erased, so it isn't consumed for allocation the
  way FlatHashtable's E[] is).
- key()/key1()/key2() accessors on D1.Entry/D2.Entry to match Hashtable
  post-#12044.
- Context-passing forEach/drain overloads use <C> for the context type param.
- Double-checked-locking + lock-striping recipes moved to the class Javadoc.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move D1/D2 tests and the ThreadSafeMap{Counter,D1,D2} benchmarks off the
removed public ctors / Support class onto createFixedBuckets and the flattened
ConcurrentHashtable.* static fns. The D2 benchmark's raw-array custom-entry arm
now drives a typed AtomicReferenceArray<SupportEntry>.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dougqh

dougqh commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5829238fde

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/jmh/java/datadog/trace/util/ThreadSafeMapD2Benchmark.java Outdated
@dougqh dougqh changed the title feat(util): add ConcurrentHashtable with lock-free D1/D2 composite-key tables Add ConcurrentHashtable with lock-free D1/D2 composite-key tables (perf toolbox) Jul 29, 2026
The bucket AtomicReferenceArray is the per-table write monitor, obtained
via getWriteLock(buckets) (opaque accessor, single source of truth) so
callers never hardcode what to synchronize on. Reads (bucket/forEach) stay
lock-free; whole-table mutators (removeIf/drain/clear) self-lock; the
single-slot write primitives (insertHeadEntry/unlink) are caller-locked and
assert Thread.holdsLock(getWriteLock(buckets)) under -ea.

insertHeadEntry mirrors Hashtable's insert helper so custom tables publish
entries without touching the chain pointer directly; Entry.setNext is
demoted to package-private accordingly while next() stays public for
lock-free chain walks.

Adapts ThreadSafeMapD2Benchmark call sites to the new API.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
dougqh and others added 8 commits August 26, 2026 10:22
…htable

# Conflicts:
#	internal-api/src/jmh/java/datadog/trace/util/ThreadSafeMapBenchmark.java
Mirrors the same guard added to Hashtable.insertHeadEntryAt. Here it
also catches reinserting an already-unlinked entry: unlink()
deliberately leaves next intact so in-flight lock-free readers can
keep traversing, so overwriting it via a reinsert would corrupt that
traversal.
Bundles buckets + a cursor-based SizeManager into a State<TEntry>,
threaded through D1/D2 as tryGetOrCreateOrEvict(OrNull) so callers can
cap table size and evict on overflow. Renames createFixedBuckets ->
createCapped and getOrCreate -> tryGetOrCreate(OrNull) to reflect the
capacity-aware contract. Adds unit test coverage for SizeManager's
reserve/evict/reset behavior and the D1/D2 eviction paths.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mirrors Hashtable.insertReserved: splices a fully-built entry into an
already-reserved slot (from tryReserve()/tryReserveOrEvict) without
double-counting. Not used by D1/D2, whose creator is fallible and so
increments only after a successful link; documented as the contrast.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@bric3 bric3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current head. I found one blocking concurrency issue and added notes for the three SpotBugs findings currently failing check_base.


Disclaimer: it's Codex points.

Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
dougqh and others added 3 commits August 31, 2026 08:25
tryReserveOrEvict is self-locking, so pairing it with insertReserved
across two critical sections lets a drain or clear land in the gap,
reset the SizeManager while the reservation is outstanding, and leave
the insert linking an entry the count never learns about -- a capped
table then drifts silently past its cap.

Document the enclosing lock as part of the contract (class level, both
tryReserveOrEvict javadocs, and insertReserved's example), fix the test
that encoded the racy shape, and add a deterministic test that a
concurrent clear cannot interleave.

Also give evictOneInRange the @GuardedBy the other cursor writers carry,
and suppress AT_STALE_THREAD_WRITE_OF_PRIMITIVE where SpotBugs cannot
model the dynamic getWriteLock(buckets) guard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
getWriteLock(buckets) had exactly one answer, so every caller asked for
the whole table whether or not that was what it needed. That makes the
locking granularity part of the API: a striped implementation would have
no object to return.

Replace it with three accessors that name a scope -- getWriteLock(state,
keyHash), getWriteLockAt(state, bucketIndex), and getTableWriteLock(state)
-- and point every @GuardedBy, assert, and call site at the one it
actually needs. All three still return the same monitor, so behavior is
unchanged; only the question each caller asks is different.

Two consequences worth having: a caller that holds one key's monitor and
mutates another is now visibly wrong rather than accidentally right, and
every getTableWriteLock use marks a spot where striping would cost
something. The class javadoc records what those spots are -- table-wide
capacity accounting and a whole-table eviction scan -- so the analysis
does not have to be redone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
SizeManager treated the entry cap as strict, and paid for it twice: a
check-then-increment reserve that needed the table write lock, and drain
and clear zeroing the count so a concurrent reservation was silently
discarded. The second of those was the P1: an undercount no later
eviction repairs, since eviction decrements too.

An approximate cap is fine here -- the bucket array is fixed-size with
load-factor headroom and never rehashes, so overshoot lengthens chains
and nothing else. Taking that latitude turns out to buy exactness where
it is cheap and delete the locking where it is not:

- tryReserve claims a slot and refunds on overshoot. Atomic on its own,
  so it needs no lock, and concurrent reservers still cannot both pass
  the cap.
- drain and clear subtract what they actually removed (release(int),
  replacing reset()), so a reservation survives a sweep landing in the
  gap between reserve and insert.

That removes the reason reserve-then-insert had to share one critical
section, so insertReserved now documents the single-bucket lock instead.
What remains lock-dependent is D1/D2's isFull-then-increment ordering,
which exists so a fallible creator cannot leak a slot and can tolerate
admitting slightly over the cap.

Counting makes clear O(entries) rather than O(buckets); clear is a rare
whole-table operation, so an honest count is worth the walk.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@bric3 bric3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good to me, assuming I got the concurrent part well.

However, aong the thing that I noticed some lower level APIs, have a public API in the ConcurrentHashtable "namespace", which I belive confuses its basic usage. Maybe using a Support class like elsewhere could help distinguish public API, versus lower level APIs. That could come as a follow-up.

Additionaly, I wonder if tests on mixed clear/drain/eviction invocations could still be useful here.

I think the javadoc could have some hman refinement, but nothing blocking.

Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated
Comment thread internal-api/src/main/java/datadog/trace/util/ConcurrentHashtable.java Outdated

@bric3 bric3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested reworded comments without the verbiage of claude's inference.

dougqh and others added 2 commits September 1, 2026 11:33
… comment

Condenses the class/method-level Javadoc across ConcurrentHashtable and the
ThreadSafeMap* benchmarks down to the load-bearing points, and reworks the
isFull()-before-creator comment in D1/D2.tryGetOrCreateOrNull to spell out
the leaked-reservation failure mode instead of a terse arrow-notation summary
(per bric3's PR review nitpick).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@bric3 bric3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK to move forward with this. But I'm not convinced by the exposure of lower level API on the "main" class. I understand it has to be done in some way though to achieve the multi key mechanism.

@dougqh

dougqh commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

OK to move forward with this. But I'm not convinced by the exposure of lower level API on the "main" class. I understand it has to be done in some way though to achieve the multi key mechanism.

@bric3 I understand. As I said, I've gone back and forth myself. I think I like this better than a Support class, but it is a close call.

Inlines the power-of-two rounding and its MAX_BUCKETS cap directly into
ConcurrentHashtable instead of delegating to Hashtable.Support.sizeFor,
which is being removed as part of the Hashtable/ConcurrentHashtable API
unification. Some duplication with Hashtable.sizeFor is accepted in
exchange for removing the cross-PR coupling.
@dougqh
dougqh enabled auto-merge September 1, 2026 19:34
…vadoc

Hashtable and FlatHashtable both carry a "Choosing between the three
tables" section; ConcurrentHashtable lacked the reciprocal reference.
@dd-octo-sts

dd-octo-sts Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Sep 1, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-09-01 20:30:16 UTC ℹ️ Start processing command /merge


2026-09-01 20:30:21 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-09-01 21:22:29 UTC ℹ️ MergeQueue: This merge request was merged

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 1, 2026
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 495add0 into master Sep 1, 2026
601 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the feat/concurrent-hashtable branch September 1, 2026 21:22
@github-actions github-actions Bot added this to the 1.66.0 milestone Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants