diff --git a/.claude/rules/eval-changes.md b/.claude/rules/eval-changes.md index 05be9c2b..8d15331c 100644 --- a/.claude/rules/eval-changes.md +++ b/.claude/rules/eval-changes.md @@ -8,10 +8,10 @@ paths: When modifying evaluation data or datasets: - JSONL files: one JSON object per line, validate with: - `python scripts/audit-pipeline/validate_jsonl.py --schema datasets/normalized/finding.schema.json --jsonl ` + `python3 scripts/audit-pipeline/validate_jsonl.py --schema datasets/normalized/finding.schema.json --jsonl ` - JSON files: validate with: - `python scripts/audit-pipeline/validate_json.py --schema datasets/normalized/audit.schema.json --glob 'datasets/normalized/audits/*.json'` + `python3 scripts/audit-pipeline/validate_json.py --schema datasets/normalized/audit.schema.json --glob 'datasets/normalized/audits/*.json'` - New findings must include: confidence tag, provenance, severity - Redact client-identifying material from audit data - Detection quality changes must add/update cases in evals/cases/ -- `python scripts/quality/validate_skills.py` is only for SKILL/link checks, not dataset schema validation +- `python3 scripts/quality/validate_skills.py` is only for SKILL/link checks, not dataset schema validation diff --git a/.claude/rules/skill-editing.md b/.claude/rules/skill-editing.md index 5a68ace8..038e492d 100644 --- a/.claude/rules/skill-editing.md +++ b/.claude/rules/skill-editing.md @@ -16,4 +16,4 @@ When editing skill files: - Must include at least one local Markdown link for progressive disclosure (for example `./workflow.md` or `./reference.md`) - Keep SKILL.md under 500 lines; keep deep details in `references/` and `workflows/` - One level of linking depth only (no chained references) -- Run `python scripts/quality/validate_skills.py` after changes +- Run `python3 scripts/quality/validate_skills.py` after changes diff --git a/.claude/settings.json b/.claude/settings.json index ee40f88c..9bca6279 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -46,7 +46,17 @@ "Read(~/.pypirc)", "Write(~/.pypirc)", "Read(./.env)", - "Read(./.env.*)" + "Read(./.env.*)", + "Read(./**/.env)", + "Read(./**/.env.*)", + "Edit(./.env)", + "Edit(./.env.*)", + "Edit(./**/.env)", + "Edit(./**/.env.*)", + "Write(./.env)", + "Write(./.env.*)", + "Write(./**/.env)", + "Write(./**/.env.*)" ], "allow": [ "Read", @@ -60,7 +70,6 @@ "Bash(git diff*)", "Bash(git log*)", "Bash(git branch*)", - "Bash(ls *)", "Bash(pwd)", "Bash(which *)", "Bash(wc *)" @@ -107,7 +116,7 @@ "hooks": [ { "type": "command", - "command": "command -v jq >/dev/null 2>&1 || { echo 'BLOCKED: jq is required for safety hooks but is not installed.' >&2; exit 2; }; FILE=$(jq -r '.tool_input.file_path // empty'); if echo \"$FILE\" | grep -qiE '(\\.bashrc|\\.zshrc|/\\.ssh/|/\\.aws/|/\\.gnupg/|/\\.git-credentials|/\\.docker/config|/\\.kube/|/\\.npmrc|/\\.pypirc|/\\.config/gh/)'; then echo \"BLOCKED: Editing shell config or credentials ($FILE) is not allowed.\" >&2; exit 2; fi" + "command": "command -v jq >/dev/null 2>&1 || { echo 'BLOCKED: jq is required for safety hooks but is not installed.' >&2; exit 2; }; FILE=$(jq -r '.tool_input.file_path // empty'); if echo \"$FILE\" | grep -qiE '(\\.bashrc|\\.zshrc|/\\.ssh/|/\\.aws/|/\\.gnupg/|/\\.git-credentials|/\\.docker/config|/\\.kube/|/\\.npmrc|/\\.pypirc|/\\.config/gh/|\\.env$|\\.env\\.)'; then echo \"BLOCKED: Editing shell config or credentials ($FILE) is not allowed.\" >&2; exit 2; fi" } ] } diff --git a/CLAUDE.md b/CLAUDE.md index ee23e3e0..6a2f39da 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,6 +12,7 @@ Cairo/Starknet reasoning skills for agents. 7 modules, evaluation-backed. ## Hard Rules - No operational SDK content — only correctness, security, eval-backed guidance +- Boundary changes MUST call out impacted repos: `keep-starknet-strange/starknet-agentic` and `keep-starknet-strange/starkclaw` - Detection/remediation changes MUST include eval case updates - Skill files require: "When to Use", "When NOT to Use"; security skills also: "Rationalizations to Reject" - Keep SKILL.md concise; details go in `references/` and `workflows/` @@ -19,9 +20,9 @@ Cairo/Starknet reasoning skills for agents. 7 modules, evaluation-backed. ## Commands ```bash -python scripts/quality/validate_skills.py # structure check -python scripts/quality/validate_marketplace.py # marketplace metadata -python scripts/quality/parity_check.py # optional, needs snforge+sncast +python3 scripts/quality/validate_skills.py # structure check +python3 scripts/quality/validate_marketplace.py # marketplace metadata +python3 scripts/quality/parity_check.py # optional, needs snforge+sncast ruff check scripts/ # Python lint ``` @@ -32,7 +33,7 @@ Python 3.12 · Scarb 2.16.1 · Starknet Foundry 0.57.0 · ruff · shellcheck ## Style - Python: ruff defaults -- Markdown: YAML frontmatter required (`name`, `description`) +- Skill Markdown: YAML frontmatter required (`name`, `description`) - JSONL: one JSON object per line, validate against `datasets/manifests/` and `datasets/normalized/` schemas ## Compact instructions diff --git a/cairo-auditor/README.md b/cairo-auditor/README.md index 5cce033e..7f76ae9f 100644 --- a/cairo-auditor/README.md +++ b/cairo-auditor/README.md @@ -4,61 +4,126 @@ # cairo-auditor -Flagship Cairo/Starknet audit skill. +A security agent for Cairo/Starknet — findings in minutes, not weeks. + +Built for: + +- **Cairo devs** who want a security check before every commit +- **Security researchers** looking for fast wins before a manual review +- **Anyone** deploying on Starknet who wants an extra pair of eyes + +Not a substitute for a formal audit — but the check you should never skip.

mode default mode deep fp gate - deterministic smoke - agent eval + deterministic smoke

-## Usage + + + + +## Install + +**Claude Code CLI:** ```bash -/cairo-auditor -/cairo-auditor deep -/cairo-auditor contracts/account.cairo +git clone https://github.com/keep-starknet-strange/starknet-skills.git \ + && mkdir -p ~/.claude/commands/cairo-auditor \ + && cp -R starknet-skills/cairo-auditor/. ~/.claude/commands/cairo-auditor/ ``` -Install only this module from the plugin marketplace: +**Cursor (manual guidance only):** + +```bash +git clone https://github.com/keep-starknet-strange/starknet-skills.git \ + && mkdir -p docs/cairo-auditor \ + && cp -R starknet-skills/cairo-auditor/references/. docs/cairo-auditor/ +``` + +Cursor does not execute this package as a runnable `/cairo-auditor` command. Use Claude Code CLI or Plugin Marketplace for the executable orchestrator flow. In Cursor, treat these files as reference guidance only. +There is no official global `~/.cursor/skills` install path for this package. + +**Claude Code Plugin Marketplace:** ```bash /plugin marketplace add keep-starknet-strange/starknet-skills /plugin install cairo-auditor@starknet-skills ``` -Deterministic local repo audit entrypoint: +**Update to latest:** ```bash -python scripts/quality/audit_local_repo.py \ - --repo-root /path/to/your/cairo-repo \ - --scan-id local-cairo-audit \ - --output-json /tmp/local-cairo-audit.json \ - --output-md /tmp/local-cairo-audit.md +cd starknet-skills && git pull +# Claude Code CLI: +cp -R cairo-auditor/. ~/.claude/commands/cairo-auditor/ +# Cursor docs refresh (manual guidance only): +cp -R cairo-auditor/references/. docs/cairo-auditor/ +``` + +## Usage + +```bash +# Scan the full repo (default — 4 parallel agents) +/cairo-auditor + +# Full repo + adversarial reasoning agent (slower, more thorough) +/cairo-auditor deep + +# Review specific file(s) +/cairo-auditor src/contracts/account.cairo +/cairo-auditor src/contracts/account.cairo src/contracts/factory.cairo + +# Write report to a markdown file (terminal-only by default) +/cairo-auditor --file-output ``` -With Sierra confirmation (build mode): +### Deterministic local scan (no AI) ```bash -python scripts/quality/audit_local_repo.py \ +python3 scripts/quality/audit_local_repo.py \ --repo-root /path/to/your/cairo-repo \ - --scan-id local-cairo-audit-sierra \ - --sierra-confirm \ - --allow-build \ - --output-json /tmp/local-cairo-audit-sierra.json \ - --output-md /tmp/local-cairo-audit-sierra.md + --scan-id my-audit ``` -## Structure +## Example output + +```text +[P0] 1. Ungated Upgrade Path + NO_ACCESS_CONTROL_MUTATION · src/contracts/account.cairo:42 · Confidence: 92 + + Description + External upgrade() calls replace_class_syscall without caller gate. + Any account can replace the contract class, leading to full takeover. -- `SKILL.md`: 4-turn orchestration contract. -- `agents/`: vector + adversarial specialist instructions. -- `references/attack-vectors/`: partitioned D:/FP: vectors (120 total). -- `references/judging.md`: strict FP gate and confidence rules. -- `references/vulnerability-db/`: canonical class docs. -- `references/semgrep/`: optional Semgrep auxiliary rules. + Fix + - fn upgrade(ref self: ContractState, new_class: ClassHash) { + + fn upgrade(ref self: ContractState, new_class: ClassHash) { + + self.ownable.assert_only_owner(); + + Required Tests + - Unauthorized caller reverts on upgrade + - Owner successfully upgrades and new class hash persists +``` + +## How it works + +The skill orchestrates a **4-turn pipeline**: + +1. **Discover** — find in-scope `.cairo` files, run deterministic preflight +2. **Prepare** — build 4 code bundles, each with a different attack-vector partition +3. **Spawn** — 4 parallel vector specialists (`model: sonnet`), optionally + 1 adversarial (`model: opus` in deep mode) +4. **Report** — merge, deduplicate by root cause, sort by confidence, emit findings + +Each agent scans the full codebase against 30 attack vectors from its partition (120 total), applies a strict false-positive gate, and formats findings with exploit paths and fix diffs. + +## Known limitations + +**Codebase size.** Works best under ~5,000 lines of Cairo. Past that, triage accuracy and mid-bundle recall degrade. For large codebases, run per-module rather than everything at once. + +**What AI misses.** AI catches pattern-based vulnerabilities reliably: missing access controls, CEI violations, unsafe upgrades, zero-address initialization. It struggles with: multi-transaction state setups, specification/invariant bugs, cross-protocol composability, game-theoretic attacks, and off-chain oracle assumptions. AI catches what humans forget to check. Humans catch what AI cannot reason about. You need both. ## Benchmarks @@ -73,13 +138,22 @@ Additional quality signals: - External triage: [v0.2.0-cairo-auditor-external-triage.md](../evals/scorecards/v0.2.0-cairo-auditor-external-triage.md) - Manual gold: [v0.2.0-cairo-auditor-manual-19-gold-recall.md](../evals/scorecards/v0.2.0-cairo-auditor-manual-19-gold-recall.md) -- Sierra auxiliary: [sierra-parallel-low-profile-2026-03-09-v2.md](../evals/reports/sierra-parallel-low-profile-2026-03-09-v2.md) -- Caracal + Semgrep adapters: see [../scripts/quality/README.md](../scripts/quality/README.md) -## References +## Structure -- Skill policy: [SKILL.md](SKILL.md) -- Workflow docs: [workflows/default.md](workflows/default.md), [workflows/deep.md](workflows/deep.md) -- Vector index: [references/attack-vectors/](references/attack-vectors) -- FP gate: [references/judging.md](references/judging.md) -- Class docs: [references/vulnerability-db/README.md](references/vulnerability-db/README.md) +```text +cairo-auditor/ + SKILL.md # 4-turn orchestration contract + agents/ + vector-scan.md # vector specialist instructions + adversarial.md # adversarial specialist instructions + references/ + attack-vectors/ # 120 vectors in 4 partitions + vulnerability-db/ # 13 canonical vulnerability classes + judging.md # FP gate + confidence scoring + report-formatting.md # finding template + priority mapping + semgrep/ # optional Semgrep auxiliary rules + workflows/ + default.md # 4-agent pipeline reference + deep.md # + adversarial agent details +```