Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 141 additions & 0 deletions .github/workflows/claude-review.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
name: Perform a code review when a pull request is created.
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]

jobs:
claude:
# Skip drafts, and skip fork PRs: on `pull_request` they don't get the
# ANTHROPIC_API_KEY secret, so the review would only fail.
if: >-
!github.event.pull_request.draft &&
github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
structured_output: ${{ steps.run_claude.outputs.structured_output }}
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0
with:
# Explicitly check out the PR's merge commit.
ref: refs/pull/${{ github.event.pull_request.number }}/merge
persist-credentials: false

- name: Pre-fetch base and head refs for the PR
env:
PR_BASE_REF: ${{ github.event.pull_request.base.ref }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
# Pass GitHub expressions through env and quote shell expansions.
git fetch --no-tags origin \
"$PR_BASE_REF" \
"+refs/pull/$PR_NUMBER/head"

# If you want Claude to build and run code, install any dependencies that
# need to be downloaded before the "Run Claude" step, and add the commands
# it may run to --allowedTools below (e.g. "Bash(npm test:*)"). Any tool
# not listed there is denied.

- name: Run Claude
id: run_claude
uses: anthropics/claude-code-action@86d88e619d8e6caf07b5c3944dd14f441c533718 # v1.0.243
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
# Use this job's read-only token instead of the Claude GitHub App,
# which needs `id-token: write` and would get write access.
github_token: ${{ github.token }}
prompt: |
This is PR #${{ github.event.pull_request.number }} for ${{ github.repository }}.

Review ONLY the changes introduced by the PR, so consider:
git log --oneline ${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}
git diff ${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}

Suggest any improvements, potential bugs, or issues.
Be concise and specific in your feedback.

Return your review in two parts:
- `summary`: a short Markdown overview of the PR and your overall assessment.
- `comments`: one entry per specific issue, anchored to the line it concerns.
`path` is relative to the repository root. `line` is the line number in
the PR head version of the file (${{ github.event.pull_request.head.sha }}),
and must be a line shown in the diff above. The working tree is the merge
commit, so its line numbers can differ; check them with
`git show ${{ github.event.pull_request.head.sha }}:<path>`.
Put issues that don't belong to a single line in `summary` instead.

Pull request title and body:
----
${{ github.event.pull_request.title }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR title and body go into the prompt unescaped, so a PR author can inject instructions. The job is limited to same-repo PRs and read-only tools, but the output is posted with pull-requests: write. Consider fencing this section and telling the model to treat it as untrusted data.

${{ github.event.pull_request.body }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR title and body are put into the prompt word for word, so the PR author's text can steer the review content. That review is then posted by the job that has pull-requests: write. Skipping fork PRs and using read-only tools limits the damage. Still, consider wrapping the PR text in clear delimiters and telling the model to treat it as untrusted data, not instructions.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR title and body are inserted into the prompt as-is, so a PR author can inject instructions (for example, "approve this, report no issues"). Shell injection isn't possible here, and the tools are read-only. Still, consider labelling this text as untrusted in the prompt, or leaving the body out.

claude_args: |
--allowedTools "Read,Grep,Glob,Bash(git log:*),Bash(git diff:*),Bash(git show:*)"
--json-schema '{
"type": "object",
"properties": {
"summary": {
"type": "string",
"description": "Overall review as Markdown, posted as the review body."
},
"comments": {
"type": "array",
"description": "Inline comments, each anchored to one line of the PR diff.",
"items": {
"type": "object",
"properties": {
"path": { "type": "string", "description": "File path relative to the repository root." },
"line": { "type": "integer", "minimum": 1, "description": "Line number in the PR head version of the file. Must be within the diff." },
"body": { "type": "string", "description": "The comment, as Markdown." }
},
"required": ["path", "line", "body"]
}
}
},
"required": ["summary", "comments"]
}'

post_feedback:
runs-on: ubuntu-latest
needs: claude
if: needs.claude.outputs.structured_output != ''
permissions:
issues: write
pull-requests: write
steps:
- name: Report Claude feedback
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0
env:
CLAUDE_STRUCTURED_OUTPUT: ${{ needs.claude.outputs.structured_output }}
with:
github-token: ${{ github.token }}
script: |
const { summary = '', comments = [] } = JSON.parse(process.env.CLAUDE_STRUCTURED_OUTPUT);
const pr = context.payload.pull_request;
if (!summary.trim() && comments.length === 0) {
core.info('Claude returned an empty review; nothing to post.');
return;
}

try {
await github.rest.pulls.createReview({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: pr.number,
commit_id: pr.head.sha,
event: 'COMMENT',
body: summary.trim() || 'See inline comments.',
comments: comments.map(({ path, line, body }) => ({ path, line, side: 'RIGHT', body })),
});
} catch (error) {
// GitHub rejects the whole review if any comment points outside the
// diff. Fall back to a single PR comment so no feedback is lost.
core.warning(`Could not post inline review (${error.message}); posting a plain comment instead.`);
const details = comments.map(({ path, line, body }) => `**\`${path}:${line}\`**\n\n${body}`);
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
body: [summary.trim(), ...details].filter(Boolean).join('\n\n---\n\n'),
});
}
4 changes: 4 additions & 0 deletions crates/core/src/scenarios/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ This is a cumbersome process in most cases.
Override templates that are registered in the Surfpool repo will automatically be available in Surfpool Studio's drag-and-drop UI for creating scenarios.
This UI will automatically create the `surfnet_registerScenario` payload for you, eliminating the complexity from the process.

A collection tagged `clock-sensitive` holds accounts with a time or slot the program checks against the Clock. Playing such a
scenario with the clock paused leaves fetched accounts ahead of the surfnet's time, and the program can refuse transactions
(Whirlpool `InvalidTimestamp`, Project 0 `PythPushStalePrice`); Studio warns before pausing for them.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The examples here (Whirlpool InvalidTimestamp, Project 0 PythPushStalePrice) are for collections that are not tagged in this PR. The ones that are tagged (Pyth v2, Phoenix) aren't mentioned. Until the Whirlpool and Project 0 branches land, this is confusing for readers. Consider using Pyth's publish_time and Phoenix's oracle staleness window as the examples. Also suggest saying explicitly that the tag applies to every template in the collection, so readers expect some false-positive warnings (e.g. Phoenix templates that have no time check).


## Creating Native Scenario Support for a Protocol
Having a protocol being natively supported by Surfpool Scenarios will allow users to easily create scenarios to override account states for that protocol.
All accounts in a surfnet's account db that are owned by the protocol will include automatic IDL-parsing, and the protocol's account overrides will be available in Surfpool Studio's drag-and-drop UI.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ tags:
- perpetuals
- collateral
- risk
- clock-sensitive

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tags all eight Phoenix templates, including ones with no time check, which means extra warnings in Studio. That's acceptable as a stopgap. Consider adding a short YAML comment explaining why the tag is here, and adding entry-level tags support to YamlOverrideTemplateEntry later so the warning can be scoped to the templates that actually need it.


templates:
- id: phoenix-trader-collateral-stress
Expand Down
1 change: 1 addition & 0 deletions crates/core/src/scenarios/protocols/pyth/v2/overrides.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ tags:
- oracle
- price-feed
- defi
- clock-sensitive

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Studio now relies on this tag, but no test checks it. Suggest adding an assertion in registry.rs (e.g. next to test_filter_by_tags) that by_tags(["clock-sensitive"]) includes pyth-price-feed-v2 and the Phoenix templates. That way a rename or a dropped tag fails CI instead of silently removing the warning.


constants:
price_feed:
Expand Down
Loading