Add project config, CLAUDE.md rewrite, user-facing auditor README#41
Conversation
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughUpdates Python invocation commands from Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related PRs
Suggested labels
Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 warning, 2 inconclusive)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Comment |
Greptile SummaryThis PR makes three coordinated improvements: a Key changes and observations:
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| .claude/settings.json | Good security improvements: .env deny entries extended to nested paths and Edit/Write operations; Bash(ls *) removed from allow list; `Edit |
| CLAUDE.md | Clean improvements: added cross-repo boundary-change rule for starknet-agentic/starkclaw, updated all python → python3 invocations, and clarified "Skill Markdown" scope for frontmatter requirement. No issues found. |
| cairo-auditor/README.md | Major user-facing rewrite: adds install instructions for Claude Code CLI, Cursor (clearly labelled as reference-only), and Plugin Marketplace; adds example finding output, pipeline overview, and known limitations section. Cursor clarification correctly addresses previous thread feedback. Deterministic scan command updated to python3. |
| .claude/rules/eval-changes.md | Straightforward python → python3 consistency fix across three command examples. No issues. |
| .claude/rules/skill-editing.md | Single-line python → python3 fix in the validation command reminder. No issues. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Incoming Tool Call] --> B{Tool Type?}
B -->|Bash| C[Bash PreToolUse Hooks]
B -->|Read| D[Read PreToolUse Hook]
B -->|Edit or Write| E[Edit-Write PreToolUse Hook]
C --> C1{Recursive rm?}
C1 -->|yes| BLOCK1[BLOCKED]
C1 -->|no| C2{Push to main or master?}
C2 -->|yes| BLOCK2[BLOCKED]
C2 -->|no| C3{git reset --hard?}
C3 -->|yes| BLOCK3[BLOCKED]
C3 -->|no| C4{Force push?}
C4 -->|yes| BLOCK4[BLOCKED]
C4 -->|no| C5{Credential path?}
C5 -->|yes| BLOCK5[BLOCKED]
C5 -->|no| ALLOW_BASH[Allowed]
D --> D1{.env / ssh / aws / gnupg / git-creds / docker / kube / npmrc / pypirc / Keychains?}
D1 -->|yes| BLOCK6[BLOCKED]
D1 -->|no| ALLOW_READ[Allowed]
E --> E1{.bashrc / .zshrc / ssh / aws / gnupg / git-creds / docker / kube / npmrc / pypirc / gh / .env?}
E1 -->|yes| BLOCK7[BLOCKED - NEW in this PR for .env]
E1 -->|no| ALLOW_EDIT[Allowed]
style BLOCK1 fill:#ef4444,color:#fff
style BLOCK2 fill:#ef4444,color:#fff
style BLOCK3 fill:#ef4444,color:#fff
style BLOCK4 fill:#ef4444,color:#fff
style BLOCK5 fill:#ef4444,color:#fff
style BLOCK6 fill:#ef4444,color:#fff
style BLOCK7 fill:#f97316,color:#fff
style ALLOW_BASH fill:#22c55e,color:#fff
style ALLOW_READ fill:#22c55e,color:#fff
style ALLOW_EDIT fill:#22c55e,color:#fff
Comments Outside Diff (3)
-
.claude/settings.json, line 115-120 (link)/Keychains/protection missing fromEdit|WritehookThis PR correctly adds
|\\.env$|\\.env\\.to theEdit|Writehook, but theReadhook (line 110) already includes/Keychains/to protect macOS Keychain files — and that protection was never ported to theEdit|Writehook. A Claude session can nowEditorWritea file inside the macOS Keychain directory without being blocked.Since this PR already touches this hook, consider keeping all three hooks consistent:
-
.claude/settings.json, line 65-66 (link)Legacy
pythonallow entry remains alongsidepython3All script invocations in
CLAUDE.md,eval-changes.md,skill-editing.md, and the README have been updated topython3. TheBash(python scripts/quality/*)allow entry on line 65 is now dead code — all actual invocations usepython3. Keeping both entries could confuse contributors about which interpreter to use and may inadvertently permitpython(Python 2) execution of quality scripts if Python 2 is still installed on an older machine. -
.claude/settings.json, line 75 (link)Bash(wc *)permits size probing of credential filesAfter correctly removing
Bash(ls *)(which could enumerate key filenames),Bash(wc *)remains unconditionally allowed. A Claude session can still runwc -c ~/.ssh/id_rsaorwc -l ~/.aws/credentials, which reveals file sizes and line counts — metadata that can indicate key type/format and which credential profiles exist.This is lower severity than full content disclosure, but it's inconsistent with the intent to lock down credential directories. The Bash hook at line 101 does not catch
wcsince it only checks for path patterns in the command text, andwc ~/.ssh/id_rsacontains/.ssh/which actually would match the hook regex. Let me clarify: looking at the hook pattern,wc ~/.ssh/id_rsa→ the command contains/\.ssh/which is matched by the hook's pattern(~|\\$HOME|...)/(\\.ssh|...). Sowc ~/.ssh/...would actually be blocked by the Bash hook at line 101.However,
wc -c /home/user/.aws/credentialswith an explicit absolute path using/home/user/would be caught by the/home/[^/]+pattern in the hook. This appears to be covered already. Consider adding a note in code comments to document this reliance on the hook forwcprotection, since the allow list itself provides no restriction.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: .claude/settings.json
Line: 115-120
Comment:
**`/Keychains/` protection missing from `Edit|Write` hook**
This PR correctly adds `|\\.env$|\\.env\\.` to the `Edit|Write` hook, but the `Read` hook (line 110) already includes `/Keychains/` to protect macOS Keychain files — and that protection was never ported to the `Edit|Write` hook. A Claude session can now `Edit` or `Write` a file inside the macOS Keychain directory without being blocked.
Since this PR already touches this hook, consider keeping all three hooks consistent:
```suggestion
"command": "command -v jq >/dev/null 2>&1 || { echo 'BLOCKED: jq is required for safety hooks but is not installed.' >&2; exit 2; }; FILE=$(jq -r '.tool_input.file_path // empty'); if echo \"$FILE\" | grep -qiE '(\\.bashrc|\\.zshrc|/\\.ssh/|/\\.aws/|/\\.gnupg/|/\\.git-credentials|/\\.docker/config|/\\.kube/|/\\.npmrc|/\\.pypirc|/\\.config/gh/|/Keychains/|\\.env$|\\.env\\.)'; then echo \"BLOCKED: Editing shell config or credentials ($FILE) is not allowed.\" >&2; exit 2; fi"
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: .claude/settings.json
Line: 65-66
Comment:
**Legacy `python` allow entry remains alongside `python3`**
All script invocations in `CLAUDE.md`, `eval-changes.md`, `skill-editing.md`, and the README have been updated to `python3`. The `Bash(python scripts/quality/*)` allow entry on line 65 is now dead code — all actual invocations use `python3`. Keeping both entries could confuse contributors about which interpreter to use and may inadvertently permit `python` (Python 2) execution of quality scripts if Python 2 is still installed on an older machine.
```suggestion
"Bash(python3 scripts/quality/*)",
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: .claude/settings.json
Line: 75
Comment:
**`Bash(wc *)` permits size probing of credential files**
After correctly removing `Bash(ls *)` (which could enumerate key filenames), `Bash(wc *)` remains unconditionally allowed. A Claude session can still run `wc -c ~/.ssh/id_rsa` or `wc -l ~/.aws/credentials`, which reveals file sizes and line counts — metadata that can indicate key type/format and which credential profiles exist.
This is lower severity than full content disclosure, but it's inconsistent with the intent to lock down credential directories. The Bash hook at line 101 does not catch `wc` since it only checks for path patterns in the command text, and `wc ~/.ssh/id_rsa` contains `/.ssh/` which actually *would* match the hook regex. Let me clarify: looking at the hook pattern, `wc ~/.ssh/id_rsa` → the command contains `/\.ssh/` which is matched by the hook's pattern `(~|\\$HOME|...)/(\\.ssh|...)`. So `wc ~/.ssh/...` would actually be blocked by the Bash hook at line 101.
However, `wc -c /home/user/.aws/credentials` with an explicit absolute path using `/home/user/` would be caught by the `/home/[^/]+` pattern in the hook. This appears to be covered already. Consider adding a note in code comments to document this reliance on the hook for `wc` protection, since the allow list itself provides no restriction.
How can I resolve this? If you propose a fix, please make it concise.Last reviewed commit: b5f6b03
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 12
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.claude/skills/benchmark/SKILL.md (1)
8-15:⚠️ Potential issue | 🔴 CriticalMatch the SKILL validator contract.
The pipeline is already failing here: this file is missing
## When to Use,## When NOT to Use, and at least one local markdown link for progressive disclosure. Add those sections before merge.Based on learnings, refer to SKILL.md as the canonical router for project guidelines and skill definitions.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.claude/skills/benchmark/SKILL.md around lines 8 - 15, The SKILL.md is missing required sections and a local link: open .claude/skills/benchmark/SKILL.md and add the headings "## When to Use" and "## When NOT to Use" with concise guidance (one or two short bullets each) and insert at least one local markdown link (e.g., `[More details](`#when-to-use`)` or a link to another local section/file) for progressive disclosure; ensure the new sections follow the existing structure/ordering of SKILL.md and update any nearby summary or index lines to reference the new headings so validators recognize them.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.claude/rules/eval-changes.md:
- Around line 7-12: Update the rule text to point to the correct evaluation/data
validator or explicitly state none exists: replace the reference to
scripts/quality/validate_skills.py (which validates skill markdown) and the
non-existent datasets/schemas/ with the actual eval/dataset validator tool (or a
note like “no automated schema validator available — validate JSONL against the
eval/data schema manually”). Also add guidance to prefer held-out regressions
from datasets/audits/ for evaluation validation and ensure new findings include
confidence, provenance, severity and cases updated under evals/cases/.
In @.claude/rules/python-scripts.md:
- Around line 6-10: The Markdown file violates MD041 because the first content
line is plain text; insert a top-level heading immediately after the frontmatter
by replacing the plain text "Python script conventions:" with an actual H1
heading "# Python script conventions" and leave the following bullet list intact
so the bullets remain under that heading (ensure the frontmatter boundary is
preserved and the new H1 is the first content line).
In @.claude/rules/skill-editing.md:
- Around line 8-14: Update the rule text in .claude/rules/skill-editing.md to
require that when content is moved into references/ or workflows/, the SKILL.md
must include a local markdown link pointing to those files (so SKILL.md remains
the canonical router); mention SKILL.md, references/, workflows/, and
validate_skills.py in the rule and add a line like “If details are moved to
references/ or workflows/, SKILL.md must include local markdown links to those
files” so the validator (`validate_skills.py`) can detect progressive-disclosure
links.
In @.claude/settings.json:
- Around line 36-42: The allow-list currently permits generic file readers
("Bash(head *)", "Bash(tail *)", "Bash(jq *)") which bypass the secret-read hook
that only blocks "cat"; either remove these entries from the Bash allow-list or
extend the secret-read hook to detect and block sensitive path patterns for all
Bash readers (inspect command arguments from matchers like "Bash(head *)",
"Bash(tail *)", "Bash(jq *)" and deny access to paths matched by the Read-tool
sensitive list used by the Read hook); update the secret-read hook logic so it
checks the same sensitive paths/regexes as the Read tool hook and rejects the
Bash invocation when a sensitive path is present.
In @.claude/skills/audit-local/SKILL.md:
- Around line 8-14: The SKILL.md for the audit-local skill is missing required
router sections; update .claude/skills/audit-local/SKILL.md to start the body
with the top-level heading "# audit-local", then add the required sections "##
When to Use", "## When NOT to Use", and "## Rationalizations to Reject", and
include at least one local markdown link (e.g., linking to another SKILL.md or a
project guideline) within the body; ensure the new sections briefly describe
usage, non-usage scenarios, and acceptable rationales for rejecting this skill
so the file conforms to the project's skill-editing rules.
- Line 11: The command in SKILL.md containing "python
scripts/quality/audit_local_repo.py --repo-root $ARGUMENTS --scan-id
local-$(date +%s)" should quote $ARGUMENTS to prevent word-splitting for paths
with spaces; update that line to use "--repo-root \"$ARGUMENTS\"" (i.e., wrap
$ARGUMENTS in double quotes) so the audit_local_repo.py receives the full path
even when it contains spaces.
In @.claude/skills/validate/SKILL.md:
- Around line 7-13: Update the SKILL.md content so the main body starts with a
top-level heading "# validate", then include the required subsections "## When
to Use" and "## When NOT to Use" (each with a brief description), and add at
least one local markdown link to deeper guidance (for example linking to another
repo doc or README like "./README.md" or "./docs/usage.md") so the file matches
the router shape expected by the validator; edit SKILL.md to place these
headings and the local link near the top so validate_skills.py will recognize
the skill format.
In `@cairo-auditor/README.md`:
- Around line 90-106: The fenced code block in README.md is missing a language
tag which triggers markdownlint MD040; update the opening fence from ``` to
```text so the block is explicitly marked as text (the content showing the vuln
for fn upgrade(ref self: ContractState, new_class: ClassHash) and the suggested
self.ownable.assert_only_owner() fix remains unchanged).
- Around line 141-156: The fenced code block showing the project tree is missing
a language tag and triggers markdownlint MD040; update the opening fence from
``` to ```text so the block is treated as plain text and the linter passes,
i.e., find the README.md fenced block that starts with ``` and replace it with
```text for the cairo-auditor project tree snippet.
- Around line 33-43: The cp -r usage in the README causes nested installations
when the destination directory already exists; update the two install commands
that target ~/.claude/commands and ~/.cursor/skills so they either remove the
old installation first (e.g., rm -rf ~/.claude/commands/cairo-auditor before
copying) or copy into the parent directory (copy into ~/.claude/commands/ and
~/.cursor/skills/ rather than specifying the final cairo-auditor directory) to
avoid creating ~/.claude/commands/cairo-auditor/cairo-auditor; change both
occurrences accordingly in the README.
In `@CLAUDE.md`:
- Around line 12-18: Restore the dropped cross-repo boundary rule in the "Hard
Rules" section: add an explicit bullet requiring any changes that affect
downstream repos to call out impacted repositories, specifically naming
keep-starknet-strange/starknet-agentic and keep-starknet-strange/starkclaw;
update the Hard Rules list (the block starting with "Hard Rules") to include
this requirement and a brief note that boundary edits must acknowledge those two
repos.
- Around line 32-36: The Markdown frontmatter rule in the Style section is too
broad — it currently mandates YAML frontmatter (`name`, `description`) for all
Markdown files (e.g., CLAUDE.md and cairo-auditor/README.md) but should apply
only to skill/marketplace-facing docs; update the bullet "Markdown: YAML
frontmatter required (`name`, `description`)" to scope it to SKILL.md or
marketplace-facing documentation (e.g., change wording to "Markdown (SKILL.md /
marketplace-facing): YAML frontmatter required (`name`, `description`)") so the
guidance is accurate and does not imply every .md needs frontmatter.
---
Outside diff comments:
In @.claude/skills/benchmark/SKILL.md:
- Around line 8-15: The SKILL.md is missing required sections and a local link:
open .claude/skills/benchmark/SKILL.md and add the headings "## When to Use" and
"## When NOT to Use" with concise guidance (one or two short bullets each) and
insert at least one local markdown link (e.g., `[More details](`#when-to-use`)` or
a link to another local section/file) for progressive disclosure; ensure the new
sections follow the existing structure/ordering of SKILL.md and update any
nearby summary or index lines to reference the new headings so validators
recognize them.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 10497944-2085-4ae8-bdc4-e5f2cfa3df0b
📒 Files selected for processing (10)
.claude/rules/eval-changes.md.claude/rules/python-scripts.md.claude/rules/skill-editing.md.claude/settings.json.claude/skills/audit-local/SKILL.md.claude/skills/benchmark/SKILL.md.claude/skills/validate/SKILL.md.claudeignoreCLAUDE.mdcairo-auditor/README.md
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Greptile Review
🧰 Additional context used
🧠 Learnings (10)
📓 Common learnings
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-03-08T13:12:42.932Z
Learning: Refer to SKILL.md as the canonical router for project guidelines and skill definitions
📚 Learning: 2026-03-08T13:12:42.932Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-03-08T13:12:42.932Z
Learning: Refer to SKILL.md as the canonical router for project guidelines and skill definitions
Applied to files:
.claude/skills/validate/SKILL.md.claude/rules/skill-editing.mdCLAUDE.md.claude/skills/audit-local/SKILL.md
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Applies to **/*eval*.{py,cairo,txt} : Detection quality changes must add or update evaluation cases
Applied to files:
.claude/rules/eval-changes.md.claude/skills/benchmark/SKILL.md
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Applies to **/*eval*.{py,cairo,txt} : Prefer held-out regressions from `datasets/audits/` for evaluation validation
Applied to files:
.claude/rules/eval-changes.md.claude/skills/benchmark/SKILL.md.claudeignore
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Move large details to `references/` and `workflows/` directories
Applied to files:
.claude/rules/skill-editing.mdcairo-auditor/README.md
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Boundary changes must acknowledge impacted repos: keep-starknet-strange/starknet-agentic and keep-starknet-strange/starkclaw
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Keep this repo focused on correctness, security patterns, and evaluation-backed guidance
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Keep root skill modules concise
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Keep operational SDK/protocol usage content out of this repo
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Applies to contracts/**/*.cairo : Fail if unsafe patterns are detected in Cairo contracts without mitigation documentation
Applied to files:
cairo-auditor/README.md
🪛 GitHub Actions: Full Evals
.claude/skills/validate/SKILL.md
[error] 1-1: missing '## When to Use' section
[error] 1-1: missing '## When NOT to Use' section
[error] 1-1: no local markdown links for progressive disclosure
.claude/skills/benchmark/SKILL.md
[error] 1-1: missing '## When to Use' section
[error] 1-1: missing '## When NOT to Use' section
[error] 1-1: no local markdown links for progressive disclosure
.claude/skills/audit-local/SKILL.md
[error] 1-1: missing '## When to Use' section
[error] 1-1: missing '## When NOT to Use' section
[error] 1-1: no local markdown links for progressive disclosure
🪛 GitHub Actions: Quality Gate
.claude/skills/validate/SKILL.md
[error] 1-1: SKILL validation failed: missing '## When to Use' section
[error] 1-1: SKILL validation failed: missing '## When NOT to Use' section
.claude/skills/benchmark/SKILL.md
[error] 1-1: SKILL validation failed: missing '## When to Use' section
[error] 1-1: SKILL validation failed: missing '## When NOT to Use' section
.claude/skills/audit-local/SKILL.md
[error] 1-1: SKILL validation failed: missing '## When to Use' section
[error] 1-1: SKILL validation failed: missing '## When NOT to Use' section
🪛 markdownlint-cli2 (0.21.0)
.claude/skills/validate/SKILL.md
[warning] 8-8: First line in a file should be a top-level heading
(MD041, first-line-heading, first-line-h1)
.claude/rules/eval-changes.md
[warning] 7-7: First line in a file should be a top-level heading
(MD041, first-line-heading, first-line-h1)
.claude/rules/skill-editing.md
[warning] 7-7: First line in a file should be a top-level heading
(MD041, first-line-heading, first-line-h1)
.claude/skills/audit-local/SKILL.md
[warning] 8-8: First line in a file should be a top-level heading
(MD041, first-line-heading, first-line-h1)
.claude/rules/python-scripts.md
[warning] 6-6: First line in a file should be a top-level heading
(MD041, first-line-heading, first-line-h1)
cairo-auditor/README.md
[warning] 90-90: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
[warning] 141-141: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
There was a problem hiding this comment.
♻️ Duplicate comments (1)
.claude/rules/eval-changes.md (1)
9-17: 🧹 Nitpick | 🔵 TrivialConsider adding held-out regression guidance.
The rule correctly directs users to the appropriate validators and clarifies the separation from SKILL validation. However, it would benefit from explicit guidance about preferring held-out regressions from
datasets/audits/for evaluation validation, which aligns with best practices for this repository.📝 Suggested addition
- Detection quality changes must add/update cases in evals/cases/ +- Prefer held-out regressions from `datasets/audits/` for evaluation validation - `python3 scripts/quality/validate_skills.py` is only for SKILL/link checks, not dataset schema validationBased on learnings: Prefer held-out regressions from
datasets/audits/for evaluation validation.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.claude/rules/eval-changes.md around lines 9 - 17, Add a sentence to the evaluation changes guidance that explicitly recommends preferring held-out regressions from datasets/audits/ when creating/validating evaluation data; update .claude/rules/eval-changes.md near the validators list to mention that held-out regressions (from datasets/audits/) should be used for evaluation validation and regression tests, and clarify this complements the existing notes about JSONL/JSON validation and the separation from SKILL checks (validate_jsonl.py, validate_json.py, and scripts/quality/validate_skills.py).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In @.claude/rules/eval-changes.md:
- Around line 9-17: Add a sentence to the evaluation changes guidance that
explicitly recommends preferring held-out regressions from datasets/audits/ when
creating/validating evaluation data; update .claude/rules/eval-changes.md near
the validators list to mention that held-out regressions (from datasets/audits/)
should be used for evaluation validation and regression tests, and clarify this
complements the existing notes about JSONL/JSON validation and the separation
from SKILL checks (validate_jsonl.py, validate_json.py, and
scripts/quality/validate_skills.py).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 3ff4929d-4ad3-417e-8819-5c2211d5a4c2
📒 Files selected for processing (15)
.claude/rules/eval-changes.md.claude/rules/python-scripts.md.claude/rules/skill-editing.md.claude/settings.json.claude/skills/audit-local/SKILL.md.claude/skills/audit-local/workflow.md.claude/skills/benchmark/SKILL.md.claude/skills/benchmark/workflow.md.claude/skills/validate/SKILL.md.claude/skills/validate/workflow.md.claudeignoreCLAUDE.mdcairo-auditor/README.mdscripts/quality/benchmark_cairo_auditor.pyscripts/quality/benchmark_contract_skills.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Greptile Review
- GitHub Check: full-evals
🧰 Additional context used
📓 Path-based instructions (1)
scripts/**
⚙️ CodeRabbit configuration file
scripts/**: Flag any script that runs with elevated privileges without justification. Verify input sanitization in deployment scripts. Flag hardcoded credentials, URLs, or environment-specific assumptions.
Files:
scripts/quality/benchmark_contract_skills.pyscripts/quality/benchmark_cairo_auditor.py
🧠 Learnings (10)
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Applies to .github/workflows/** : For changes in CI/CD workflows, flag secrets referenced without need-to-know scope
Applied to files:
.claude/settings.json
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Applies to **/*eval*.{py,cairo,txt} : Prefer held-out regressions from `datasets/audits/` for evaluation validation
Applied to files:
.claudeignore.claude/rules/eval-changes.md
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Boundary changes must acknowledge impacted repos: keep-starknet-strange/starknet-agentic and keep-starknet-strange/starkclaw
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: If contracts/** or packages/** are changed, the PR must declare cross-repo impact or explicitly state 'no cross-repo impact'
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-08T13:12:42.932Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-03-08T13:12:42.932Z
Learning: Refer to SKILL.md as the canonical router for project guidelines and skill definitions
Applied to files:
CLAUDE.md.claude/skills/audit-local/SKILL.md.claude/rules/skill-editing.md.claude/skills/benchmark/SKILL.mdcairo-auditor/README.md.claude/skills/validate/SKILL.md
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Keep this repo focused on correctness, security patterns, and evaluation-backed guidance
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Keep root skill modules concise
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-08T13:12:39.745Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-08T13:12:39.745Z
Learning: Applies to **/*eval*.{py,cairo,txt} : Detection quality changes must add or update evaluation cases
Applied to files:
.claude/rules/eval-changes.mdscripts/quality/benchmark_cairo_auditor.py
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Applies to contracts/**/*.cairo : Fail if unsafe patterns are detected in Cairo contracts without mitigation documentation
Applied to files:
cairo-auditor/README.md
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Applies to contracts/**/*.cairo : For changes in Cairo contracts, verify that all external/public functions have explicit access control
Applied to files:
cairo-auditor/README.md
🔇 Additional comments (20)
.claude/rules/python-scripts.md (1)
1-11: LGTM!The file structure is correct with proper YAML frontmatter and an H1 heading. The Python conventions are clear and actionable, with explicit version pinning examples.
.claude/skills/audit-local/workflow.md (1)
1-7: LGTM!The workflow is clear and well-structured, with appropriate emphasis on input sanitization and proper artifact handling. The triage ordering (severity first, then confidence) aligns with the evaluation governance rules.
.claude/skills/validate/SKILL.md (1)
1-28: LGTM!The SKILL.md now follows the required router shape with proper YAML frontmatter, an H1 heading,
## Quick Start,## When to Use,## When NOT to Usesections, and a local markdown link to the workflow. This addresses the concerns from the previous review.scripts/quality/benchmark_contract_skills.py (2)
579-583: LGTM!The
--saveflag addition is clean and follows the same pattern as the siblingbenchmark_cairo_auditor.pyscript.
636-641: LGTM!The save logic is implemented safely:
- Directory creation with
mkdir(parents=True, exist_ok=True)is safe- Using
output_path.name(basename only) for the target prevents path traversal- The resolved path comparison avoids self-copy when output is already in the target directory
.claudeignore (1)
1-29: LGTM!The ignore patterns are well-organized and appropriate. Notably,
evals/scorecards/is correctly not ignored, allowing the benchmark--savefunctionality to persist results for comparison..claude/skills/validate/workflow.md (1)
1-9: LGTM!The workflow is clear and comprehensive, covering skill validation, marketplace validation, linting, and conditional schema validation for eval/dataset changes. The iterative fix-and-verify approach in steps 4-5 is good practice.
scripts/quality/benchmark_cairo_auditor.py (4)
8-8: LGTM!The
shutilimport is correctly added to support the new copy functionality.
942-946: LGTM!The
--saveflag is consistent with the siblingbenchmark_contract_skills.pyscript.
976-984: LGTM!The save logic is implemented safely and mirrors the pattern in
benchmark_contract_skills.py:
- Uses
output_path.name(basename only) to construct the target, preventing path traversal- Resolves paths before comparison to avoid self-copy
- Records the saved path for inclusion in the JSON output
993-993: LGTM!Adding
saved_outputto the JSON output enables downstream consumers to track where the scorecard was persisted..claude/skills/benchmark/workflow.md (1)
1-9: LGTM!The workflow documentation is clear and provides complete command examples with appropriate threshold values. The three-phase approach (run without save → optionally persist with
--save→ compare against prior scorecards) is well-structured for regression detection..claude/skills/benchmark/SKILL.md (1)
1-28: LGTM!The skill definition follows the required structure: YAML frontmatter with
nameanddescription, Quick Start section, usage guidance sections, and a local markdown link to the workflow reference. The--saveflag behavior aligns with the script changes described in the PR..claude/skills/audit-local/SKILL.md (1)
1-34: LGTM!The skill definition is well-structured with all required sections for a security skill, including "Rationalizations to Reject". The path validation on line 13 intentionally restricts to safe characters, and the
$REPOvariable is properly quoted in line 15. The workflow link provides progressive disclosure as required..claude/settings.json (1)
1-108: Well-designed security guardrails.The configuration properly addresses the previously identified bypass issue—the Bash secret guard (line 73) now comprehensively covers
cat|head|tail|jqcommands accessing sensitive paths, and the allow-list no longer includes generic file readers with wildcards. The deny/allow permissions and hooks provide defense-in-depth for preventing destructive operations and credential exfiltration.Minor note: The notification hook (line 102) uses
osascript, which is macOS-specific, but fails gracefully with|| true..claude/rules/skill-editing.md (1)
1-19: LGTM!The skill-editing rules are comprehensive and now correctly require local Markdown links for progressive disclosure (line 16), addressing the previous concern. The rules align with the canonical router pattern where SKILL.md serves as the entry point with links to detailed references and workflows. Based on learnings, refer to SKILL.md as the canonical router for project guidelines and skill definitions.
CLAUDE.md (1)
1-41: LGTM!The CLAUDE.md rewrite addresses all previously identified issues: the cross-repo boundary rule is restored (line 15), and the frontmatter requirement is now correctly scoped to "Skill Markdown" (line 36). The document provides clear contributor onboarding with key paths, hard rules, commands, stack info, and style conventions. Based on learnings, boundary changes must acknowledge impacted repos: keep-starknet-strange/starknet-agentic and keep-starknet-strange/starkclaw.
cairo-auditor/README.md (3)
33-36: Install commands correctly handle updates.The
cp -R source/. dest/pattern copies the contents of the source directory into the destination, avoiding the nested directory issue on subsequent runs. This is the correct approach for idempotent installations.
38-47: Clear Cursor guidance disclaimer.Good clarification that Cursor does not execute the package as a runnable command and that users should treat the files as reference guidance only. This prevents confusion about functionality expectations.
111-127: Excellent transparency on limitations.The "How it works" and "Known limitations" sections provide valuable context about the pipeline architecture and honest guidance about where AI-based auditing falls short (multi-transaction state, specification bugs, cross-protocol composability, game-theoretic attacks). This sets appropriate expectations.
Resolved by follow-up commits; all threads addressed.
|
All review threads are resolved, required checks are passing/queued, and stale bot change requests were dismissed after fixes. This PR is ready for final human approval to satisfy branch protection and merge. |
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (1)
.claude/skills/audit-local/SKILL.md (1)
8-8:⚠️ Potential issue | 🟡 MinorUse the canonical router heading
# audit-local.Line 8 still uses a display title instead of the canonical skill-name heading expected by the skill router, so this can keep failing the same validation path that was flagged earlier.
Suggested fix
-# Audit Local Skill +# audit-local🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.claude/skills/audit-local/SKILL.md at line 8, Replace the non-canonical display heading "# Audit Local Skill" with the canonical router heading "# audit-local" so the skill router validation passes; locate the top-level markdown heading that currently reads "Audit Local Skill" and change it to the exact lowercase, hyphenated heading "audit-local".
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.claude/settings.json:
- Around line 97-107: The Notification hook currently uses the macOS-only
command string ("command": "osascript -e 'display notification \"Claude Code
needs your attention\" with title \"Claude Code\"' 2>/dev/null || true") inside
the "Notification" array, so add cross-platform fallbacks: update the hook to
try notify-send on Linux and a PowerShell Toast/ToastNotification on Windows (or
replace the single command with a small wrapper script that detects OS and
invokes osascript, notify-send, or PowerShell accordingly). Ensure the wrapper
or expanded hooks preserve silent failure behavior but attempt each
platform-specific notifier in order and include a descriptive message consistent
with the existing one.
- Around line 87-95: The Edit|Write hook (matcher "Edit|Write" command hook
using FILE=$(jq -r '.tool_input.file_path // empty')) only blocks shell and key
dirs and must be expanded to match the Read hook's blocked paths; update the
command string in that hook to include the same grep -qiE pattern used by the
Read hook so it also checks for '\.aws', '\.git-credentials', '\.docker/config',
'\.kube', '\.npmrc', and '\.pypirc' (in addition to '\.bashrc', '\.zshrc',
'/\.ssh/', '/\.gnupg/') and exit with the same error behavior when matched.
- Around line 21-28: The deny list is missing consistent Edit/Write entries for
some credential dirs; update .claude/settings.json to add explicit Edit and
Write deny patterns for "~/.gnupg/**", and add both Edit and Write deny patterns
for "~/.aws/**" and "~/.config/gh/**" so each of those three directories has
Read, Write, and Edit entries (match the style of the existing entries like
"Edit(~/.ssh/**)" and "Write(~/.ssh/**)"); ensure entries use the same glob
pattern form and ordering consistent with the other deny rules.
In @.claude/skills/audit-local/SKILL.md:
- Around line 12-26: Add a blank line immediately before the opening fenced code
block (```bash) and a blank line immediately after the closing fence (```) in
the SKILL.md content so the fenced block is separated by empty lines (fixing
MD031); update the surrounding text so the blank lines sit between the numbered
list item text and the code fence and between the closing fence and the next
numbered item, leaving the code contents intact.
- Around line 16-18: The current allowlist on RAW_REPO uses grep -qE
'^[A-Za-z0-9_./-]+$' which wrongly rejects valid paths with spaces; replace that
single restrictive check with two checks: (1) ensure RAW_REPO contains no
control characters by detecting POSIX cntrl class (e.g. grep -qE '[[:cntrl:]]'
and fail if matched), and (2) prevent path traversal by rejecting any '..' path
segments (e.g. grep -qE '(^|/)\.\.($|/)' and fail if matched). Keep the variable
RAW_REPO and the same error messages but perform these safer, separate
validations instead of banning spaces.
---
Duplicate comments:
In @.claude/skills/audit-local/SKILL.md:
- Line 8: Replace the non-canonical display heading "# Audit Local Skill" with
the canonical router heading "# audit-local" so the skill router validation
passes; locate the top-level markdown heading that currently reads "Audit Local
Skill" and change it to the exact lowercase, hyphenated heading "audit-local".
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 1bad9cb7-1df0-405c-971b-46a04844aa6f
📒 Files selected for processing (2)
.claude/settings.json.claude/skills/audit-local/SKILL.md
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Greptile Review
- GitHub Check: full-evals
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Applies to .github/workflows/** : For changes in CI/CD workflows, flag secrets referenced without need-to-know scope
Applied to files:
.claude/settings.json
📚 Learning: 2026-03-08T13:12:42.932Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-03-08T13:12:42.932Z
Learning: Refer to SKILL.md as the canonical router for project guidelines and skill definitions
Applied to files:
.claude/skills/audit-local/SKILL.md
🪛 markdownlint-cli2 (0.21.0)
.claude/skills/audit-local/SKILL.md
[warning] 13-13: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
[warning] 26-26: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
🔇 Additional comments (2)
.claude/settings.json (2)
34-49: Appropriate tightening of the allow list.The previous concern about generic file readers (
head,tail,jq) in the Bash allow list has been addressed. The current list is appropriately restrictive, permitting only specific quality scripts, linters, and safe read-only git commands.
51-76: Bash safety hooks are well-structured.The hooks appropriately block destructive operations (recursive rm, force push, hard reset, push to main/master) and secret exfiltration via common readers (cat, head, tail, jq). The regex patterns correctly handle various home directory representations (
~,$HOME,/home/user,/Users/user,/root).Commands not in the allow list (like
less,more) would require user approval, providing adequate coverage.
Replace sparse repo context with actionable sections: key paths, hard rules, validation commands, stack, style conventions, and compact instructions for context management. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace internal structure dump with Pashov-style README: - "Built for" pitch with clear audience - Install instructions for Claude Code CLI, Cursor, and marketplace - Usage examples with all modes and flags - Example finding output so users know what to expect - "How it works" pipeline overview - Known limitations (codebase size, what AI misses) - Demo GIF placeholder for future recording - Structure section moved to bottom Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
0c287ab to
6a84ed3
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
.claude/rules/eval-changes.md (1)
1-5:⚠️ Potential issue | 🟡 MinorMissing required
nameanddescriptionfields in YAML frontmatter.The frontmatter contains only
pathsbut lacks the requirednameanddescriptionfields. As per coding guidelines, Markdown files must include YAML frontmatter with these required fields.🛠️ Proposed fix
--- +name: Eval/Data Change Rules +description: Guidelines for modifying evaluation data and datasets paths: - "evals/**" - "datasets/**" ---🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.claude/rules/eval-changes.md around lines 1 - 5, The YAML frontmatter at the top of .claude/rules/eval-changes.md is missing the required name and description fields; update the frontmatter block (the lines around the existing paths key) to include a descriptive name (e.g., name: "Eval changes rule") and a short description (e.g., description: "Rule that applies to evals and datasets paths"), ensuring both keys are present and filled with meaningful text alongside the existing paths entry..claude/rules/skill-editing.md (1)
1-6:⚠️ Potential issue | 🟡 MinorMissing required
nameanddescriptionfields in YAML frontmatter.The frontmatter contains only
pathsbut lacks the requirednameanddescriptionfields. As per coding guidelines, Markdown files must include YAML frontmatter with these required fields.🛠️ Proposed fix
--- +name: Skill Editing Rules +description: Guidelines for editing SKILL.md and related skill files paths: - "**/SKILL.md" - "**/references/**" - "**/workflows/**" - ".claude/skills/**/workflow.md" ---🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.claude/rules/skill-editing.md around lines 1 - 6, The YAML frontmatter currently only defines the "paths" key and is missing the required "name" and "description" fields; update the frontmatter to include a top-level "name" (short identifier/title) and "description" (one-line summary) alongside the existing "paths" entry so that the frontmatter contains "name", "description", and "paths" keys; ensure the values are strings and follow the project's frontmatter conventions so SKILL.md and related rule consumers can parse them.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.claude/settings.json:
- Around line 48-51: Add explicit Edit(...) and Write(...) deny entries matching
the new Read patterns for ".env" files so Edit/Write protections mirror Read
protections (i.e., add Edit(./.env), Edit(./.env.*), Edit(./**/.env),
Edit(./**/.env.*) and the same for Write). Also update the Edit|Write hook
configuration that filters paths to include those ".env" patterns so the hook
enforces the denylist consistently; ensure the same glob strings used for Read
are reused for Edit and Write to maintain defense-in-depth.
---
Outside diff comments:
In @.claude/rules/eval-changes.md:
- Around line 1-5: The YAML frontmatter at the top of
.claude/rules/eval-changes.md is missing the required name and description
fields; update the frontmatter block (the lines around the existing paths key)
to include a descriptive name (e.g., name: "Eval changes rule") and a short
description (e.g., description: "Rule that applies to evals and datasets
paths"), ensuring both keys are present and filled with meaningful text
alongside the existing paths entry.
In @.claude/rules/skill-editing.md:
- Around line 1-6: The YAML frontmatter currently only defines the "paths" key
and is missing the required "name" and "description" fields; update the
frontmatter to include a top-level "name" (short identifier/title) and
"description" (one-line summary) alongside the existing "paths" entry so that
the frontmatter contains "name", "description", and "paths" keys; ensure the
values are strings and follow the project's frontmatter conventions so SKILL.md
and related rule consumers can parse them.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: b0caee7f-ca6e-495a-af8b-4dc9564b2c68
📒 Files selected for processing (5)
.claude/rules/eval-changes.md.claude/rules/skill-editing.md.claude/settings.jsonCLAUDE.mdcairo-auditor/README.md
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Greptile Review
🧰 Additional context used
📓 Path-based instructions (1)
**/*.md
📄 CodeRabbit inference engine (CLAUDE.md)
Markdown files must include YAML frontmatter with required fields:
nameanddescription
Files:
CLAUDE.mdcairo-auditor/README.md
🧠 Learnings (15)
📚 Learning: 2026-03-12T18:23:36.582Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-12T18:23:36.582Z
Learning: Run `python scripts/quality/validate_skills.py` to perform structure validation of skills
Applied to files:
.claude/rules/skill-editing.mdCLAUDE.md.claude/rules/eval-changes.md
📚 Learning: 2026-03-12T18:23:36.582Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-12T18:23:36.582Z
Learning: Applies to **/cairo-*/SKILL.md : Skill files require: 'When to Use', 'When NOT to Use' sections; security skills also require: 'Rationalizations to Reject' section
Applied to files:
.claude/rules/skill-editing.mdCLAUDE.mdcairo-auditor/README.md
📚 Learning: 2026-03-12T18:23:36.582Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-12T18:23:36.582Z
Learning: Applies to **/cairo-*/SKILL.md : Keep SKILL.md concise; details go in `references/` and `workflows/` subdirectories
Applied to files:
.claude/rules/skill-editing.mdCLAUDE.mdcairo-auditor/README.md
📚 Learning: 2026-03-08T13:12:42.932Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-03-08T13:12:42.932Z
Learning: Refer to SKILL.md as the canonical router for project guidelines and skill definitions
Applied to files:
.claude/rules/skill-editing.mdCLAUDE.mdcairo-auditor/README.md
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Applies to .github/workflows/** : For changes in CI/CD workflows, flag secrets referenced without need-to-know scope
Applied to files:
.claude/settings.json
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Boundary changes must acknowledge impacted repos: keep-starknet-strange/starknet-agentic and keep-starknet-strange/starkclaw
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: If contracts/** or packages/** are changed, the PR must declare cross-repo impact or explicitly state 'no cross-repo impact'
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-12T18:23:36.582Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-12T18:23:36.582Z
Learning: Applies to **/*.md : Markdown files must include YAML frontmatter with required fields: `name` and `description`
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-12T18:23:36.582Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-12T18:23:36.582Z
Learning: No operational SDK content — only provide correctness, security, and eval-backed guidance
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-12T18:23:36.582Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-12T18:23:36.582Z
Learning: Run `python scripts/quality/validate_marketplace.py` to validate marketplace metadata
Applied to files:
CLAUDE.md.claude/rules/eval-changes.md
📚 Learning: 2026-03-12T18:23:36.582Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-12T18:23:36.582Z
Learning: Run `ruff check scripts/` to lint all Python scripts
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-12T18:23:36.582Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-12T18:23:36.582Z
Learning: Detection/remediation changes MUST include eval case updates
Applied to files:
CLAUDE.md.claude/rules/eval-changes.md
📚 Learning: 2026-03-12T18:23:36.582Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-12T18:23:36.582Z
Learning: Applies to **/evals/**/*.jsonl : JSONL evaluation files must have one JSON object per line and validate against schemas in `datasets/manifests/` and `datasets/normalized/`
Applied to files:
CLAUDE.md.claude/rules/eval-changes.md
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Applies to contracts/**/*.cairo : For changes in Cairo contracts, verify that all external/public functions have explicit access control
Applied to files:
cairo-auditor/README.md
📚 Learning: 2026-03-08T21:08:11.403Z
Learnt from: CR
Repo: keep-starknet-strange/starknet-skills PR: 0
File: coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt:0-0
Timestamp: 2026-03-08T21:08:11.403Z
Learning: Applies to contracts/**/*.cairo : Fail if unsafe patterns are detected in Cairo contracts without mitigation documentation
Applied to files:
cairo-auditor/README.md
🔇 Additional comments (13)
.claude/rules/skill-editing.md (1)
19-19: LGTM!The change from
pythontopython3aligns with the Python 3 migration across validation scripts in this PR and ensures explicit Python 3 invocation. Based on learnings,python scripts/quality/validate_skills.pyis used for structure validation of skills..claude/settings.json (1)
116-126: LGTM!The notification hook now includes cross-platform support with
osascriptfor macOS andnotify-sendfor Linux, addressing the previous platform-specific limitation..claude/rules/eval-changes.md (1)
11-17: LGTM!The changes correctly:
- Update all validation commands to use
python3for consistency- Reference the appropriate validator scripts (
validate_jsonl.pyandvalidate_json.pyfor data validation)- Clarify that
validate_skills.pyis only for SKILL/link checks, not dataset schema validationThis addresses the previous feedback about pointing to the correct validators. Based on learnings, detection/remediation changes MUST include eval case updates.
CLAUDE.md (3)
15-15: LGTM!The cross-repo boundary rule has been restored, ensuring that changes impacting
keep-starknet-strange/starknet-agenticandkeep-starknet-strange/starkclaware explicitly called out. Based on learnings, boundary changes must acknowledge these impacted repos.
23-25: LGTM!All validation commands now use
python3consistently, aligning with the Python 3 migration across the repository.
36-36: LGTM!Correctly scoped from "Markdown" to "Skill Markdown" to clarify that YAML frontmatter is required only for SKILL.md and marketplace-facing docs, not all Markdown files. This addresses the previous feedback and aligns with the actual project requirements.
cairo-auditor/README.md (7)
7-15: LGTM!The new "Built for" section clearly communicates the target audience and sets appropriate expectations with the disclaimer that it's "Not a substitute for a formal audit."
33-35: LGTM!The
cp -R source/. target/syntax correctly avoids the nesting issue by copying the contents of the source directory into the target, rather than the directory itself. This addresses the previous feedback aboutcp -rcreating nested directories on update.
38-47: Clear Cursor guidance with appropriate caveats.The documentation correctly clarifies that Cursor cannot execute the package as a runnable command and should only use the files as reference guidance. This prevents user confusion about expected functionality.
86-89: LGTM!Consistent use of
python3for the local audit script, aligning with the Python 3 migration across the repository.
93-109: LGTM!Both fenced code blocks now include the
textlanguage specifier, addressing the previous markdownlint MD040 warnings.Also applies to: 144-159
111-121: LGTM!The "How it works" section provides a clear, concise explanation of the 4-turn pipeline (Discover, Prepare, Spawn, Report) with appropriate technical detail about parallel agents and attack-vector partitions.
122-126: LGTM!The "Known limitations" section sets realistic expectations about codebase size limits (~5,000 lines) and clearly articulates what AI catches vs. what humans catch. The guidance to run per-module for large codebases is practical.
1 participant
Summary
.claude/settings.json(permission guardrails + safety hooks),.claude/rules/(path-scoped editing rules),.claude/skills/(audit-local, benchmark, validate shortcuts),.claudeignore(exclude large data dirs)Before → After (README)
Before: Starts with badges and jumps to
/cairo-auditorcommands. No install instructions beyond marketplace. No example output. No limitations. Structure-focused.After: Starts with "findings in minutes, not weeks" pitch and "Built for" audience. Shows install for Claude Code CLI, Cursor, and marketplace. Usage with commented examples. Example finding with severity, exploit path, fix diff. "How it works" pipeline overview. Honest known limitations section. Demo GIF placeholder ready.
Commit breakdown
fa25e4aCLAUDE.md75251d3.claude/*,.claudeignore42e980dcairo-auditor/README.mdEach commit is reviewable independently.
Test plan
python3 scripts/quality/validate_marketplace.py— marketplace metadata still valid.claude/settings.jsonhooks — verify deny/allow lists make sense for the project.claudeignoreexcludes match actual large directories🤖 Generated with Claude Code
Summary by CodeRabbit
Documentation
Chores