You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#12697 replays dynamically attached subscription filters from a persisted admission cursor: replay_batch iterates the ADMISSION_SEQ column from the cursor up to the registration watermark, point-reads and decodes every admitted statement, and matches it against the filter. The existing on-disk indexes (INDEX_BY_TOPIC, INDEX_BY_DEC_KEY) are not used on this path.
Problem
Replay cost is proportional to store size, not filter selectivity. From the benchmarks in the PR thread (store at max capacity, 4,194,304 statements / ~1.66 GiB of bodies, ~525k distinct topics):
Scenario
Time to ReplayDone
Narrow filter, 8 matches out of 4.2M
2.5–2.9 s
Any (full-store delivery, 4.19M statements in 415 chunks)
2.7–2.9 s
A filter matching 8 statements pays the same full-scan cost (~0.7 µs per scanned statement: point-get + decode + match) as delivering the entire store. The scan is bounded-memory and doesn't block submit, but for selective filters nearly all of that work is wasted.
Proposal
Evaluate whether selective filters can replay through INDEX_BY_TOPIC / INDEX_BY_DEC_KEY instead of scanning the full admission log, making replay cost O(matches) rather than O(store size).
Open questions to resolve:
Ordering and cursor semantics. Index enumeration yields hashes in no particular order, while ADMISSION_SEQ provides a stable, totally ordered, resumable cursor. Options: sort candidate hashes by admission_seq (cheap for narrow filters, unbounded for broad ones), or relax the replay ordering guarantee for the index path.
Exactly-once replay/live handoff. The watermark split (replay < watermark <= live) must be preserved regardless of which path enumerates candidates.
Path selection heuristic.Any and broad topic prefixes still require the scan path; an index path only pays off for selective filters. Needs a selectivity estimate (or a cheap index-cardinality probe) to choose between paths. Multi-topic filters additionally require index intersection.
Context
Follow-up to #12697 (see review discussion and the approval comment).
#12697 replays dynamically attached subscription filters from a persisted admission cursor:
replay_batchiterates theADMISSION_SEQcolumn from the cursor up to the registration watermark, point-reads and decodes every admitted statement, and matches it against the filter. The existing on-disk indexes (INDEX_BY_TOPIC,INDEX_BY_DEC_KEY) are not used on this path.Problem
Replay cost is proportional to store size, not filter selectivity. From the benchmarks in the PR thread (store at max capacity, 4,194,304 statements / ~1.66 GiB of bodies, ~525k distinct topics):
ReplayDoneAny(full-store delivery, 4.19M statements in 415 chunks)A filter matching 8 statements pays the same full-scan cost (~0.7 µs per scanned statement: point-get + decode + match) as delivering the entire store. The scan is bounded-memory and doesn't block
submit, but for selective filters nearly all of that work is wasted.Proposal
Evaluate whether selective filters can replay through
INDEX_BY_TOPIC/INDEX_BY_DEC_KEYinstead of scanning the full admission log, making replay cost O(matches) rather than O(store size).Open questions to resolve:
ADMISSION_SEQprovides a stable, totally ordered, resumable cursor. Options: sort candidate hashes byadmission_seq(cheap for narrow filters, unbounded for broad ones), or relax the replay ordering guarantee for the index path.replay < watermark <= live) must be preserved regardless of which path enumerates candidates.Anyand broad topic prefixes still require the scan path; an index path only pays off for selective filters. Needs a selectivity estimate (or a cheap index-cardinality probe) to choose between paths. Multi-topic filters additionally require index intersection.benches/add_filter.rsacross a selectivity sweep.