Repository navigation
fix(scenarios): apply overrides only when the surfnet produces their slot #38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
8434ebc
ae0483e
0d0566a
1aa6440
ea1d829
bd5dc8b
ea41ad9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| name: Perform a code review when a pull request is created. | ||
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened, ready_for_review] | ||
|
|
||
| jobs: | ||
| claude: | ||
| # Skip drafts, and skip fork PRs: on `pull_request` they don't get the | ||
| # ANTHROPIC_API_KEY secret, so the review would only fail. | ||
| if: >- | ||
| !github.event.pull_request.draft && | ||
| github.event.pull_request.head.repo.full_name == github.repository | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| outputs: | ||
| structured_output: ${{ steps.run_claude.outputs.structured_output }} | ||
| steps: | ||
| - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0 | ||
| with: | ||
| # Explicitly check out the PR's merge commit. | ||
| ref: refs/pull/${{ github.event.pull_request.number }}/merge | ||
| persist-credentials: false | ||
|
|
||
| - name: Pre-fetch base and head refs for the PR | ||
| env: | ||
| PR_BASE_REF: ${{ github.event.pull_request.base.ref }} | ||
| PR_NUMBER: ${{ github.event.pull_request.number }} | ||
| run: | | ||
| # Pass GitHub expressions through env and quote shell expansions. | ||
| git fetch --no-tags origin \ | ||
| "$PR_BASE_REF" \ | ||
| "+refs/pull/$PR_NUMBER/head" | ||
|
|
||
| # If you want Claude to build and run code, install any dependencies that | ||
| # need to be downloaded before the "Run Claude" step, and add the commands | ||
| # it may run to --allowedTools below (e.g. "Bash(npm test:*)"). Any tool | ||
| # not listed there is denied. | ||
|
|
||
| - name: Run Claude | ||
| id: run_claude | ||
| uses: anthropics/claude-code-action@86d88e619d8e6caf07b5c3944dd14f441c533718 # v1.0.243 | ||
| with: | ||
| anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} | ||
| # Use this job's read-only token instead of the Claude GitHub App, | ||
| # which needs `id-token: write` and would get write access. | ||
| github_token: ${{ github.token }} | ||
| prompt: | | ||
| This is PR #${{ github.event.pull_request.number }} for ${{ github.repository }}. | ||
|
|
||
| Review ONLY the changes introduced by the PR, so consider: | ||
| git log --oneline ${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }} | ||
| git diff ${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }} | ||
|
|
||
| Suggest any improvements, potential bugs, or issues. | ||
| Be concise and specific in your feedback. | ||
|
|
||
| Return your review in two parts: | ||
| - `summary`: a short Markdown overview of the PR and your overall assessment. | ||
| - `comments`: one entry per specific issue, anchored to the line it concerns. | ||
| `path` is relative to the repository root. `line` is the line number in | ||
| the PR head version of the file (${{ github.event.pull_request.head.sha }}), | ||
| and must be a line shown in the diff above. The working tree is the merge | ||
| commit, so its line numbers can differ; check them with | ||
| `git show ${{ github.event.pull_request.head.sha }}:<path>`. | ||
| Put issues that don't belong to a single line in `summary` instead. | ||
|
|
||
| Pull request title and body: | ||
| ---- | ||
| ${{ github.event.pull_request.title }} | ||
| ${{ github.event.pull_request.body }} | ||
| claude_args: | | ||
| --allowedTools "Read,Grep,Glob,Bash(git log:*),Bash(git diff:*),Bash(git show:*)" | ||
| --json-schema '{ | ||
| "type": "object", | ||
| "properties": { | ||
| "summary": { | ||
| "type": "string", | ||
| "description": "Overall review as Markdown, posted as the review body." | ||
| }, | ||
| "comments": { | ||
| "type": "array", | ||
| "description": "Inline comments, each anchored to one line of the PR diff.", | ||
| "items": { | ||
| "type": "object", | ||
| "properties": { | ||
| "path": { "type": "string", "description": "File path relative to the repository root." }, | ||
| "line": { "type": "integer", "minimum": 1, "description": "Line number in the PR head version of the file. Must be within the diff." }, | ||
| "body": { "type": "string", "description": "The comment, as Markdown." } | ||
| }, | ||
| "required": ["path", "line", "body"] | ||
| } | ||
| } | ||
| }, | ||
| "required": ["summary", "comments"] | ||
| }' | ||
|
|
||
| post_feedback: | ||
| runs-on: ubuntu-latest | ||
| needs: claude | ||
| if: needs.claude.outputs.structured_output != '' | ||
| permissions: | ||
| issues: write | ||
| pull-requests: write | ||
| steps: | ||
| - name: Report Claude feedback | ||
| uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0 | ||
| env: | ||
| CLAUDE_STRUCTURED_OUTPUT: ${{ needs.claude.outputs.structured_output }} | ||
| with: | ||
| github-token: ${{ github.token }} | ||
| script: | | ||
| const { summary = '', comments = [] } = JSON.parse(process.env.CLAUDE_STRUCTURED_OUTPUT); | ||
| const pr = context.payload.pull_request; | ||
| if (!summary.trim() && comments.length === 0) { | ||
| core.info('Claude returned an empty review; nothing to post.'); | ||
| return; | ||
| } | ||
|
|
||
| try { | ||
| await github.rest.pulls.createReview({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| pull_number: pr.number, | ||
| commit_id: pr.head.sha, | ||
| event: 'COMMENT', | ||
| body: summary.trim() || 'See inline comments.', | ||
| comments: comments.map(({ path, line, body }) => ({ path, line, side: 'RIGHT', body })), | ||
| }); | ||
| } catch (error) { | ||
| // GitHub rejects the whole review if any comment points outside the | ||
| // diff. Fall back to a single PR comment so no feedback is lost. | ||
| core.warning(`Could not post inline review (${error.message}); posting a plain comment instead.`); | ||
| const details = comments.map(({ path, line, body }) => `**\`${path}:${line}\`**\n\n${body}`); | ||
| await github.rest.issues.createComment({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| issue_number: pr.number, | ||
| body: [summary.trim(), ...details].filter(Boolean).join('\n\n---\n\n'), | ||
| }); | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1424,11 +1424,13 @@ pub trait SurfnetCheatcodes { | |
| /// - `fetchBeforeUse`: If true, fetch fresh account data just before transaction execution (useful for price feeds, oracle updates, and dynamic balances) | ||
| /// - `account`: Account address (either `{ "pubkey": "..." }` or `{ "pda": { "programId": "...", "seeds": [...] } }`) | ||
| /// - `tags`: Array of tags for categorization | ||
| /// - `slot` (optional): The base slot from which relative slot offsets are calculated. If omitted, uses the current slot. | ||
| /// - `slot` (optional): The base slot from which relative slot offsets are calculated. If omitted, the next slot the surfnet produces. A base slot that is already produced is rejected with InvalidParams. | ||
| /// | ||
| /// ## Returns | ||
| /// A `RpcResponse<()>` indicating whether the Scenario registration was successful. | ||
| /// | ||
| /// Overrides are applied when the surfnet produces their slot; nothing is applied during this call. | ||
| /// | ||
| /// ## Example Request (with slot) | ||
| /// ```json | ||
| /// { | ||
|
|
@@ -1460,7 +1462,7 @@ pub trait SurfnetCheatcodes { | |
| /// ], | ||
| /// "tags": ["defi", "price-feed"] | ||
| /// }, | ||
| /// 355684457 | ||
| /// 355684458 | ||
| /// ] | ||
| /// } | ||
| /// ``` | ||
|
|
@@ -2542,37 +2544,14 @@ impl SurfnetCheatcodes for SurfnetCheatcodesRpc { | |
| scenario: Scenario, | ||
| slot: Option<Slot>, | ||
| ) -> BoxFuture<Result<RpcResponse<()>>> { | ||
| let SurfnetRpcContext { | ||
| svm_locker, | ||
| remote_ctx, | ||
| } = match meta.get_rpc_context(CommitmentConfig::confirmed()) { | ||
| Ok(res) => res, | ||
| Err(e) => return e.into(), | ||
| }; | ||
| let SurfnetRpcContext { svm_locker, .. } = | ||
| match meta.get_rpc_context(CommitmentConfig::confirmed()) { | ||
| Ok(res) => res, | ||
| Err(e) => return e.into(), | ||
| }; | ||
|
|
||
| Box::pin(async move { | ||
| // Get the base slot for registration (either provided or current) | ||
| let base_slot = slot.unwrap_or_else(|| svm_locker.get_latest_absolute_slot()); | ||
|
|
||
| // Register the scenario with explicit base slot | ||
| svm_locker | ||
| .register_scenario(scenario, Some(base_slot)) | ||
| .map_err(|e| jsonrpc_core::Error { | ||
| code: jsonrpc_core::ErrorCode::InternalError, | ||
| message: format!("Failed to register scenario: {}", e), | ||
| data: None, | ||
| })?; | ||
|
|
||
| // Immediately materialize overrides for the BASE slot (not current slot) | ||
| // This ensures slot 0's override is applied right away | ||
| svm_locker | ||
| .materialize_overrides_for_slot(&remote_ctx, base_slot) | ||
| .await | ||
| .map_err(|e| jsonrpc_core::Error { | ||
| code: jsonrpc_core::ErrorCode::InternalError, | ||
| message: format!("Failed to materialize initial overrides: {}", e), | ||
| data: None, | ||
| })?; | ||
| svm_locker.register_scenario(scenario, slot)?; | ||
|
|
||
| Ok(RpcResponse { | ||
| context: RpcResponseContext::new(svm_locker.get_latest_absolute_slot()), | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
|
@@ -2608,7 +2587,8 @@ mod tests { | |
|
|
||
| use super::*; | ||
| use crate::{ | ||
| rpc::surfnet_cheatcodes::SurfnetCheatcodesRpc, tests::helpers::TestSetup, | ||
| rpc::surfnet_cheatcodes::SurfnetCheatcodesRpc, | ||
| tests::helpers::{TestSetup, UNHEALTHY_OFFSET, scheduled_override_fixture}, | ||
| types::confidential_key_signature, | ||
| }; | ||
|
|
||
|
|
@@ -5941,4 +5921,86 @@ mod tests { | |
| "the wrong aesKey should fail rather than decrypt" | ||
| ); | ||
| } | ||
|
|
||
| #[tokio::test(flavor = "multi_thread")] | ||
| async fn test_register_scenario_applies_overrides_when_the_next_slot_is_produced() { | ||
| let client = TestSetup::new(SurfnetCheatcodesRpc::empty()); | ||
| let (account_pubkey, instance) = client | ||
| .context | ||
| .svm_locker | ||
| .with_svm_writer(scheduled_override_fixture); | ||
| let account = || { | ||
| client.context.svm_locker.with_svm_reader(|svm_reader| { | ||
| svm_reader | ||
| .inner | ||
| .get_account(&account_pubkey) | ||
| .unwrap() | ||
| .unwrap() | ||
| }) | ||
| }; | ||
| let scheduled_slots = || { | ||
| client | ||
| .context | ||
| .svm_locker | ||
| .with_svm_reader(|svm_reader| svm_reader.scheduled_overrides.keys().unwrap()) | ||
| }; | ||
| let before = account(); | ||
| let mut scenario = Scenario::new("next slot".to_string(), String::new()); | ||
| scenario.add_override(instance); | ||
|
|
||
| let response = client | ||
| .rpc | ||
| .register_scenario(Some(client.context.clone()), scenario, None) | ||
| .await | ||
| .expect("registering without a slot should succeed"); | ||
|
|
||
| assert_eq!(response.context.slot, 123); | ||
| assert_eq!(account(), before, "registering must not apply anything"); | ||
| assert_eq!(scheduled_slots(), vec![124]); | ||
|
|
||
| client | ||
| .context | ||
| .svm_locker | ||
| .confirm_current_block(&None) | ||
| .await | ||
| .unwrap(); | ||
|
|
||
| assert_eq!( | ||
| &account().data[UNHEALTHY_OFFSET..UNHEALTHY_OFFSET + 16], | ||
| &1_234u128.to_le_bytes(), | ||
| "producing slot 124 applies its override" | ||
| ); | ||
| assert!(scheduled_slots().is_empty()); | ||
| } | ||
|
|
||
| #[tokio::test(flavor = "multi_thread")] | ||
| async fn test_register_scenario_rejects_a_produced_base_slot() { | ||
| let client = TestSetup::new(SurfnetCheatcodesRpc::empty()); | ||
| let (_, instance) = client | ||
| .context | ||
| .svm_locker | ||
| .with_svm_writer(scheduled_override_fixture); | ||
|
|
||
| for produced_slot in [123, 100] { | ||
| let mut scenario = Scenario::new("produced slot".to_string(), String::new()); | ||
| scenario.add_override(instance.clone()); | ||
|
|
||
| let error = client | ||
| .rpc | ||
| .register_scenario(Some(client.context.clone()), scenario, Some(produced_slot)) | ||
| .await | ||
| .expect_err("a base slot that is already produced must be rejected"); | ||
|
|
||
| assert_eq!(error.code, jsonrpc_core::ErrorCode::InvalidParams); | ||
| assert_eq!( | ||
| error.message, | ||
| format!("base slot {produced_slot} is already produced; the next slot is 124") | ||
| ); | ||
| } | ||
| let scheduled_slots = client | ||
| .context | ||
| .svm_locker | ||
| .with_svm_reader(|svm_reader| svm_reader.scheduled_overrides.keys().unwrap()); | ||
| assert!(scheduled_slots.is_empty()); | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2811,13 +2811,26 @@ impl SurfnetSvmLocker { | |
| self.with_svm_reader(Self::offline_account_owners) | ||
| } | ||
|
|
||
| /// Registers a scenario for execution | ||
| /// Schedules a scenario's overrides from `slot`, or from the next slot when omitted. | ||
| /// A base slot that is already produced is rejected: nothing would ever apply it. | ||
| pub fn register_scenario( | ||
| &self, | ||
| scenario: surfpool_types::Scenario, | ||
| slot: Option<Slot>, | ||
| ) -> SurfpoolResult<()> { | ||
| self.with_svm_writer(move |svm_writer| svm_writer.register_scenario(scenario, slot)) | ||
| self.with_svm_writer(move |svm_writer| { | ||
| let current_slot = svm_writer.get_latest_absolute_slot(); | ||
| let next_slot = current_slot | ||
| .checked_add(1) | ||
| .ok_or_else(|| SurfpoolError::internal("the slot counter overflowed"))?; | ||
| let base_slot = slot.unwrap_or(next_slot); | ||
| if base_slot <= current_slot { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This rejects slots that are already produced, but nothing guards against a slot the clock will skip. Time travel runs |
||
| return Err(SurfpoolError::invalid_params(format!( | ||
| "base slot {base_slot} is already produced; the next slot is {next_slot}" | ||
| ))); | ||
| } | ||
| svm_writer.register_scenario(scenario, Some(base_slot)) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The base-slot check validates input up front, but
Now that this function owns RPC validation, it's cheap to check every There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The locker always passes There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| }) | ||
| } | ||
|
|
||
| /// Materializes overrides for a specific slot (not necessarily the current slot) | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The response only reports
get_latest_absolute_slot(), so callers can't tell which base slot was chosen, and the docs tell them to wait for that slot. Return the base slot fromregister_scenario, e.g.SurfpoolResult<Slot>, and include it in the response (as the value or in the context). That spares SDK and Studio callers from assumingcontext.slot + 1, which a block produced between registration and response would make wrong.