Skip to content

fix(scenarios): apply overrides only when the surfnet produces their slot - #38

Merged
92Infinitus92 merged 7 commits into
feat/scenarios/override/scenario-playback-supportfrom
feat/scenarios/apply-on-block
Nov 2, 2026
Merged

92Infinitus92 merged 7 commits into
feat/scenarios/override/scenario-playback-supportfrom
feat/scenarios/apply-on-block

Conversation

@92Infinitus92

@92Infinitus92 92Infinitus92 commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

What

surfnet_registerScenario used to apply the overrides of the base slot inside the call and schedule the rest for later slots. It now only schedules: the base slot defaults to the next slot the surfnet produces, and every override, the first one included, lands when confirm_current_block produces its slot. A base slot that is already produced is rejected with InvalidParams, since nothing would ever apply it.

Why

The in-call application existed only so that Studio's Play (pause, then register) showed the first column as applied. It made the first column behave differently from every other one and let a registration race the clock: the base slot was read under a read lock and stored under another, so a block produced in between stranded the scenario at a slot that had already passed.

Changes

  • crates/core/src/surfnet/locker.rs: register_scenario computes the base slot under the write lock (current + 1 by default, checked_add) and rejects base <= current.
  • crates/core/src/surfnet/svm.rs: SurfnetSvm::register_scenario checks every override's slot (base + relative, checked_add) before it stores any, so an overflow schedules nothing and returns InvalidParams.
  • crates/core/src/rpc/surfnet_cheatcodes.rs: the handler delegates to the locker and no longer materializes anything; docs and the example request updated; three tests (test_register_scenario_applies_overrides_when_the_next_slot_is_produced, test_register_scenario_rejects_a_produced_base_slot, test_register_scenario_rejects_a_relative_slot_that_overflows).
  • crates/core/src/tests/helpers.rs, crates/core/src/surfnet/svm.rs: scheduled_override_fixture moved to the shared test helpers so the RPC tests reuse it; the four existing callers adapted.
  • crates/core/src/tests/pump/mod.rs: two tests that schedule at earlier slots on purpose and play them by hand use the SVM method directly.
  • crates/core/src/scenarios/README.md, crates/types/src/rpc_endpoints.json, crates/types/src/scenarios.rs: wording ("applied when the surfnet produces its slot", "wait for that slot before reading the accounts", relative slot = offset from the base slot), and a note that a time travel over several slots skips the overrides of the slots it jumps over.

Behaviour change for RPC/SDK callers

  • Register, then read the account in the same slot: the read sees the old state until the next block (up to one slot time). Wait for getSlot with commitment: processed above the response's context.slot; a plain getSlot answers at finalized, which trails it.
  • An explicit slot at or below the current slot is now an error instead of a silent no-op.
  • In --block-production-mode transaction the next slot is produced by the next transaction, so that transaction executes before the overrides land.

Studio

Studio's Play needs one time travel after registering so the first column lands while the clock is paused; that is the companion PR LimeChain/surfpool-web-ui#31 (feat/studio/play-modes). Today's Studio against this engine shows the first column as applied one step early.

Verification

On upstream/main 6acd394 (v1.7.0 and later, merged into this branch):

  • cargo +nightly fmt --all -- --check
  • cargo test -p surfpool-types: 90 passed, 0 failed, 2 ignored
  • cargo test -p surfpool-core --features ignore_tests_ci: 851 passed, 0 failed, 64 ignored
  • cargo clippy -p surfpool-core --all-targets --features integration-tests: no errors, no warning on a line this PR adds
  • Live on a release build (before the upstream merge): registering with no slot changes nothing in the call and the next block applies it (watcher on the Whirlpool pool); registering with the current slot returns -32602 base slot N is already produced; the next slot is N+1; Studio Play/Step with the companion branch lands one column per produced slot.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CkgCK6kr7c138a1U8kmaN4

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What the PR does: surfnet_registerScenario no longer applies anything during the call. When no slot is given, the base slot is now the next slot (current + 1), and that slot is computed under the same write lock that stores the overrides, which fixes the old race between the read lock and the write lock. A base slot that has already been produced is rejected with InvalidParams. Every override, the first one included, is applied by confirm_current_block. The diff is small, the tests cover both the happy path and the rejection, and the docs and README are updated to match.

Assessment: The change looks correct to me. Notes:

  • Overrides can still be stranded when time travel skips slots. The PR rejects past slots because "nothing would ever apply it". But time travel (UpdateInternalClock* in runloops/mod.rs) runs confirm_current_block once, which applies current+1, and then jumps the slot counter straight to the target. Any override scheduled for a slot in between stays in scheduled_overrides forever. This already happened to later columns, but now the first column depends on it too. Studio's companion change ("one time travel after registering") works only if that jump is exactly one slot. Consider applying every scheduled slot <= new_slot on time travel, or at least documenting this.
  • Unrelated commit. ae0483e ci: add the Claude Code pull request review adds .github/workflows/claude-review.yml, which has nothing to do with scenarios and isn't mentioned in the PR body. Split it out or call it out in the description.
  • Behaviour change. Callers that register and then read in the same slot will now see the old state, so this is a breaking change for SDK users. Make sure it's flagged in the release notes and changelog.

"base slot {base_slot} is already produced; the next slot is {next_slot}"
)));
}
svm_writer.register_scenario(scenario, Some(base_slot))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The base-slot check validates input up front, but svm.register_scenario still stores overrides one by one, and a later base + relative overflow fails with SurfpoolError::internal. When that happens:

  1. The scenario is left partly scheduled, because the overrides before the failing one are already stored.
  2. A bad caller input comes back as InternalError instead of InvalidParams.

Now that this function owns RPC validation, it's cheap to check every base_slot.checked_add(o.scenario_relative_slot) before calling into the SVM and return invalid_params on failure.

.checked_add(1)
.ok_or_else(|| SurfpoolError::internal("the slot counter overflowed"))?;
let base_slot = slot.unwrap_or(next_slot);
if base_slot <= current_slot {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rejects slots that are already produced, but nothing guards against a slot the clock will skip. Time travel runs confirm_current_block once, which applies current+1, and then calls set_latest_absolute_slot(target). Anything scheduled in (current+1, target] is never applied and stays in scheduled_overrides. Since the first column now depends on slot production too, consider applying every pending slot <= new_slot during time travel, or document this limitation next to the rejection message.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The PR makes surfnet_registerScenario schedule-only. The base slot is now computed under the SVM write lock and defaults to current + 1. A base slot that has already been produced is rejected with InvalidParams. Every override, the first one included, is applied by confirm_current_block. This removes the read-lock/write-lock race and the special case for the first slot. The docs and tests are updated to match.

The core change looks correct. confirm_current_block advances the slot and then calls materialize_overrides under a single write lock, so an override scheduled at current + 1 lands on the next block. The new overflow pre-check in the locker also keeps registration all-or-nothing: SurfnetSvm::register_scenario stores overrides one at a time, so before this check a failure partway through left the earlier overrides scheduled.

Issues / suggestions

  • Unrelated file in the diff: .github/workflows/claude-review.yml (+141) is not mentioned in the description and has nothing to do with scenarios. Please split it into its own PR so it gets reviewed separately. In that workflow, the PR title and body are interpolated straight into the prompt, which allows prompt injection. The read-only token, the restricted --allowedTools and the skip for fork PRs limit the damage, but the review output can still be steered.
  • Validation lives in two places: the overflow check is now in both SurfnetSvmLocker::register_scenario (returns InvalidParams) and SurfnetSvm::register_scenario (returns internal). The SVM method also still defaults to the current slot, while the locker defaults to the next one, and its // Use provided slot or current slot comment is now misleading. Consider moving the up-front validation into the SVM method, so that direct SVM callers (as in the pump tests) get atomic registration too, and either make slot required there or document why its default differs.
  • Base slot not returned: the response's context.slot is read after the write lock is released. If a block is produced in between, it no longer equals base - 1. Clients must then follow the "wait for getSlot above context.slot" guidance without knowing which slot the scenario actually landed on. Consider returning the resolved base slot, for example as the response value or in the log. That helps both SDK callers and Studio.
  • Description out of date: it lists two new RPC tests, but the diff adds three. test_register_scenario_rejects_a_relative_slot_that_overflows is missing from the list.

Comment thread crates/core/src/surfnet/locker.rs Outdated
"base slot {base_slot} is already produced; the next slot is {next_slot}"
)));
}
for override_instance in &scenario.overrides {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This duplicates the overflow check in SurfnetSvm::register_scenario (svm.rs), which returns internal instead of invalid_params. Suggest validating every override before any store inside the SVM method and dropping the check here. That keeps one source of truth, and direct SVM callers (the pump tests, state_builder.rs) also get all-or-nothing registration.

svm_locker.register_scenario(scenario, slot)?;

Ok(RpcResponse {
context: RpcResponseContext::new(svm_locker.get_latest_absolute_slot()),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_latest_absolute_slot() is read after register_scenario has released the write lock. If a block is produced in between, context.slot already equals the base slot, so clients following the documented "wait for getSlot above context.slot" wait an extra slot without knowing where the scenario landed. Consider having register_scenario return the resolved base slot and surfacing it in the response, or at least using it for context.slot - 1.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The PR changes surfnet_registerScenario so it only schedules overrides. Nothing is applied during the call. The base slot is now computed under the write lock, defaulting to current + 1, and a base slot that has already been produced is rejected with InvalidParams. The overflow check now runs over every override before any are stored, so the schedule is all-or-nothing. That fixes the old race, where the slot was read under one lock and stored under another. Tests cover the next-slot path, rejection of produced slots, and atomic overflow rejection.

Overall the change is clean and the reasoning holds up. Points to consider:

  • Callers can't see which slot the scenario landed on. The response returns () plus context.slot, and context.slot is read under a different lock than the base-slot calculation (see inline comment). The docs tell callers to wait for getSlot to go above context.slot, which is only a rough guide. Returning the base slot, or the slots the overrides were scheduled at, would let SDK and Studio callers wait on the exact slot.
  • The two layers disagree on the default slot. SurfnetSvm::register_scenario still uses slot.unwrap_or(self.latest_epoch_info.absolute_slot), the current slot, while the locker defaults to the next slot. The only non-test caller now always passes Some, so consider making the SVM parameter a plain Slot. That would remove the outdated default and stop a future caller from getting the old behaviour by accident.
  • Overrides in skipped slots are never cleaned up. The docs now say surfnet_timeTravel skips overrides in intermediate slots, but those entries stay in scheduled_overrides forever. With in-call application gone, this is the more likely outcome after a time travel, so pruning entries below the new current slot (or at least logging them) would help. The underlying problem predates this PR.
  • Unrelated file in the PR. .github/workflows/claude-review.yml is outside the scope the description gives. Consider moving it to its own PR.

svm_locker.register_scenario(scenario, slot)?;

Ok(RpcResponse {
context: RpcResponseContext::new(svm_locker.get_latest_absolute_slot()),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

context.slot is read after register_scenario releases the write lock, so a block can be produced between the two reads. When that happens, context.slot equals the base slot and the overrides are already applied, so the documented "wait until getSlot > context.slot" makes the caller wait one slot longer than needed. That's harmless but imprecise. Consider having the locker return (current_slot, base_slot) from inside the lock, and putting base_slot in the response so callers know exactly which slot to wait for.

"base slot {base_slot} is already produced; the next slot is {next_slot}"
)));
}
svm_writer.register_scenario(scenario, Some(base_slot))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The locker always passes Some(base_slot) here, but SurfnetSvm::register_scenario still defaults None to the current slot (svm.rs:4688), which is the old behaviour. Consider making the SVM parameter non-optional (base_slot: Slot) so the two layers can't drift apart. The pump and phoenix tests already pass explicit slots.

# Conflicts:
#	crates/core/src/scenarios/protocols/phoenix-eternal/v1/state_builder.rs
#	crates/core/src/tests/helpers.rs

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

register_scenario now only schedules overrides. The base slot is chosen under the SVM write lock (current + 1 by default), and a base slot that has already been produced is rejected with InvalidParams. confirm_current_block applies each override when it produces that override's slot. It holds a single write lock for the slot bump and the materialization, so the race the PR describes is gone. The SVM-level registration also validates every slot before storing anything, which avoids partial plans when an addition overflows. The tests cover the main path, the rejection and the overflow cases.

Overall the change looks correct and is a clear improvement. A few points:

  • Unrelated file in the diff: .github/workflows/claude-review.yml (141 new lines) is part of this PR's diff, but neither the title nor the description mentions it. Either split it out or call it out in the description.
  • Base slot isn't returned: the response only carries context.slot (the current slot), so clients have to work out which slot the scenario starts at. If the response returned the base slot, SDK callers and Studio would know exactly which slot to wait for. Today they have to assume it is context.slot + 1.
  • Two different defaults: the PR description lists a test named test_register_scenario_rejects_a_relative_slot_that_overflows that it doesn't mention by name. More importantly, SurfnetSvm::register_scenario still defaults to the current slot, while the locker now defaults to current + 1. See the inline comment.
  • Overrides orphaned by time travel: after surfnet_timeTravel, overrides scheduled in the skipped slots stay in scheduled_overrides forever and are never applied or removed. The docs now mention this, but the code could prune them, or at least warn, when time travel jumps past them.

"base slot {base_slot} is already produced; the next slot is {next_slot}"
)));
}
svm_writer.register_scenario(scenario, Some(base_slot))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SurfnetSvm::register_scenario (svm.rs:4989) still defaults None to the current slot. This locker always passes Some, so the RPC path is fine. But the two layers now have different defaults, and any direct SVM caller (like the pump tests) gets the old "already produced" behaviour without being told. Make the SVM method take a Slot instead of Option<Slot>, or give both layers the same default.

message: format!("Failed to materialize initial overrides: {}", e),
data: None,
})?;
svm_locker.register_scenario(scenario, slot)?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The response only reports get_latest_absolute_slot(), so callers can't tell which base slot was chosen, and the docs tell them to wait for that slot. Return the base slot from register_scenario, e.g. SurfpoolResult<Slot>, and include it in the response (as the value or in the context). That spares SDK and Studio callers from assuming context.slot + 1, which a block produced between registration and response would make wrong.

@92Infinitus92
92Infinitus92 merged commit 3c12d40 into feat/scenarios/override/scenario-playback-support Nov 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant