Skip to content

store: Fix stale loadRelated results while blocks are queued (<> all and exclusion of unrelated queued writes) - #6727

Open
madumas wants to merge 2 commits into
graphprotocol:masterfrom
ellipfra:fix-get-derived-queued
Open

madumas wants to merge 2 commits into
graphprotocol:masterfrom
ellipfra:fix-get-derived-queued

Conversation

@madumas

@madumas madumas commented Oct 4, 2026

Copy link
Copy Markdown

Fixes #6726.

Queue::get_derived combines queued changes with a database read at the block before the queue, and hides the queued keys from that read through excluded_keys. Two defects let older database versions through, so loadRelated could return children that a queued block had removed or moved to another parent. Whether that happened depended on writer timing.

Two commits, which can be reviewed separately:

  1. store: Exclude every listed key in FindDerivedQuery: id != any($excluded) → id <> all($excluded). With two or more keys, the old predicate excluded nothing.
  2. store: Exclude queued keys no longer related in get_derived: in effective_ops, a queued write that is no longer related to the parent now excludes its key instead of being dropped. A newer queued version always supersedes the database row, whether or not it is still related.

Tests: four store tests in store/test-store/tests/postgres/writable.rs (get_derived_pending_*). They hold the writer, call get_derived while a block is queued, then flush. Without the fixes:

  • two_removals and removal_and_creation fail without commit 1;
  • parent_change fails without commit 2;
  • one_removal is a control that passed before.

The existing get_derived_batch and get_derived_nobatch still pass.

Notes:

  • The empty array is still guarded (if !self.excluded_keys.is_empty()).
  • <> all(array) has the same planner characteristics as != any(array).
  • Commit 2 excludes every queued key of the queried entity type, so the bind array is somewhat larger on big batches.
  • Behaviour change: for subgraphs that call loadRelated on children moved or removed while blocks are queued, results and therefore PoIs can differ from released versions. The current behaviour is timing-dependent, so this probably deserves a release note.

Checks run locally on this branch: cargo fmt --all -- --check, cargo clippy --all-targets (no warnings), cargo check --release, cargo test -p graph-store-postgres --lib, cargo test -p test-store --test postgres, and the workspace unit tests.

`id != any($excluded)` holds as soon as the id differs from one element
of the array, so with two or more excluded keys no row was excluded.
`Queue::get_derived` reads the database at the block before the queue
and relies on this list to hide children that queued blocks remove or
rewrite, so `loadRelated` could return entities that a queued block had
already removed. Use `id <> all($excluded)`.

Add store tests that hold the writer and call `get_derived` while the
block is queued: one removal (control), two removals, and a removal plus
a creation.
In `Queue::get_derived`, a queued write whose `@derivedFrom` field no
longer points to the parent was dropped instead of excluding its key.
The database, read at the block before the queue, then still returned
the previous version of that child under the old parent, so
`loadRelated` could return a child that a queued block had moved to
another parent. A newer queued version always supersedes the database
row, related or not: exclude its key.

Add a store test that moves a child to another parent in a queued block.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

loadRelated can return stale children while blocks are queued for writing (!= any in FindDerivedQuery; unrelated queued writes not excluded)

1 participant