Skip to content
6 changes: 3 additions & 3 deletions .claude/rules/eval-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ paths:

When modifying evaluation data or datasets:
- JSONL files: one JSON object per line, validate with:
`python scripts/audit-pipeline/validate_jsonl.py --schema datasets/normalized/finding.schema.json --jsonl <path>`
`python3 scripts/audit-pipeline/validate_jsonl.py --schema datasets/normalized/finding.schema.json --jsonl <path>`
- JSON files: validate with:
`python scripts/audit-pipeline/validate_json.py --schema datasets/normalized/audit.schema.json --glob 'datasets/normalized/audits/*.json'`
`python3 scripts/audit-pipeline/validate_json.py --schema datasets/normalized/audit.schema.json --glob 'datasets/normalized/audits/*.json'`
- New findings must include: confidence tag, provenance, severity
- Redact client-identifying material from audit data
- Detection quality changes must add/update cases in evals/cases/
- `python scripts/quality/validate_skills.py` is only for SKILL/link checks, not dataset schema validation
- `python3 scripts/quality/validate_skills.py` is only for SKILL/link checks, not dataset schema validation
2 changes: 1 addition & 1 deletion .claude/rules/skill-editing.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ When editing skill files:
- Must include at least one local Markdown link for progressive disclosure (for example `./workflow.md` or `./reference.md`)
- Keep SKILL.md under 500 lines; keep deep details in `references/` and `workflows/`
- One level of linking depth only (no chained references)
- Run `python scripts/quality/validate_skills.py` after changes
- Run `python3 scripts/quality/validate_skills.py` after changes
15 changes: 12 additions & 3 deletions .claude/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,17 @@
"Read(~/.pypirc)",
"Write(~/.pypirc)",
"Read(./.env)",
"Read(./.env.*)"
"Read(./.env.*)",
"Read(./**/.env)",
"Read(./**/.env.*)",
"Edit(./.env)",
"Edit(./.env.*)",
"Edit(./**/.env)",
"Edit(./**/.env.*)",
"Write(./.env)",
"Write(./.env.*)",
"Write(./**/.env)",
"Write(./**/.env.*)"
],
"allow": [
"Read",
Expand All @@ -60,7 +70,6 @@
"Bash(git diff*)",
"Bash(git log*)",
"Bash(git branch*)",
"Bash(ls *)",
"Bash(pwd)",
"Bash(which *)",
"Bash(wc *)"
Comment thread
omarespejel marked this conversation as resolved.
Expand Down Expand Up @@ -107,7 +116,7 @@
"hooks": [
{
"type": "command",
"command": "command -v jq >/dev/null 2>&1 || { echo 'BLOCKED: jq is required for safety hooks but is not installed.' >&2; exit 2; }; FILE=$(jq -r '.tool_input.file_path // empty'); if echo \"$FILE\" | grep -qiE '(\\.bashrc|\\.zshrc|/\\.ssh/|/\\.aws/|/\\.gnupg/|/\\.git-credentials|/\\.docker/config|/\\.kube/|/\\.npmrc|/\\.pypirc|/\\.config/gh/)'; then echo \"BLOCKED: Editing shell config or credentials ($FILE) is not allowed.\" >&2; exit 2; fi"
"command": "command -v jq >/dev/null 2>&1 || { echo 'BLOCKED: jq is required for safety hooks but is not installed.' >&2; exit 2; }; FILE=$(jq -r '.tool_input.file_path // empty'); if echo \"$FILE\" | grep -qiE '(\\.bashrc|\\.zshrc|/\\.ssh/|/\\.aws/|/\\.gnupg/|/\\.git-credentials|/\\.docker/config|/\\.kube/|/\\.npmrc|/\\.pypirc|/\\.config/gh/|\\.env$|\\.env\\.)'; then echo \"BLOCKED: Editing shell config or credentials ($FILE) is not allowed.\" >&2; exit 2; fi"
}
]
}
Expand Down
9 changes: 5 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,17 @@ Cairo/Starknet reasoning skills for agents. 7 modules, evaluation-backed.
## Hard Rules

- No operational SDK content — only correctness, security, eval-backed guidance
- Boundary changes MUST call out impacted repos: `keep-starknet-strange/starknet-agentic` and `keep-starknet-strange/starkclaw`
- Detection/remediation changes MUST include eval case updates
- Skill files require: "When to Use", "When NOT to Use"; security skills also: "Rationalizations to Reject"
- Keep SKILL.md concise; details go in `references/` and `workflows/`

Comment thread
coderabbitai[bot] marked this conversation as resolved.
## Commands

```bash
python scripts/quality/validate_skills.py # structure check
python scripts/quality/validate_marketplace.py # marketplace metadata
python scripts/quality/parity_check.py # optional, needs snforge+sncast
python3 scripts/quality/validate_skills.py # structure check
python3 scripts/quality/validate_marketplace.py # marketplace metadata
python3 scripts/quality/parity_check.py # optional, needs snforge+sncast
ruff check scripts/ # Python lint
```

Expand All @@ -32,7 +33,7 @@ Python 3.12 · Scarb 2.16.1 · Starknet Foundry 0.57.0 · ruff · shellcheck
## Style

- Python: ruff defaults
- Markdown: YAML frontmatter required (`name`, `description`)
- Skill Markdown: YAML frontmatter required (`name`, `description`)
- JSONL: one JSON object per line, validate against `datasets/manifests/` and `datasets/normalized/` schemas

## Compact instructions
Expand Down
146 changes: 110 additions & 36 deletions cairo-auditor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,61 +4,126 @@

# cairo-auditor

Flagship Cairo/Starknet audit skill.
A security agent for Cairo/Starknet — findings in minutes, not weeks.

Built for:

- **Cairo devs** who want a security check before every commit
- **Security researchers** looking for fast wins before a manual review
- **Anyone** deploying on Starknet who wants an extra pair of eyes

Not a substitute for a formal audit — but the check you should never skip.

<p>
<img alt="mode default" src="https://img.shields.io/badge/mode-default-0969da" />
<img alt="mode deep" src="https://img.shields.io/badge/mode-deep-7c3aed" />
<img alt="fp gate" src="https://img.shields.io/badge/false--positive-gated-2ea043" />
<img alt="deterministic smoke" src="https://img.shields.io/badge/deterministic%20smoke-pass-f59e0b" />
<img alt="agent eval" src="https://img.shields.io/badge/agent%20eval-in%20progress-f59e0b" />
<img alt="deterministic smoke" src="https://img.shields.io/badge/deterministic%20smoke-pass-2ea043" />
</p>

## Usage
<!-- TODO: add demo GIF once recorded -->
<!-- ## Demo -->
<!-- ![Running cairo-auditor in terminal](assets/demo.gif) -->

## Install

**Claude Code CLI:**

```bash
/cairo-auditor
/cairo-auditor deep
/cairo-auditor contracts/account.cairo
git clone https://github.com/keep-starknet-strange/starknet-skills.git \
&& mkdir -p ~/.claude/commands/cairo-auditor \
Comment thread
omarespejel marked this conversation as resolved.
&& cp -R starknet-skills/cairo-auditor/. ~/.claude/commands/cairo-auditor/
```

Install only this module from the plugin marketplace:
**Cursor (manual guidance only):**

```bash
git clone https://github.com/keep-starknet-strange/starknet-skills.git \
&& mkdir -p docs/cairo-auditor \
&& cp -R starknet-skills/cairo-auditor/references/. docs/cairo-auditor/
```

Cursor does not execute this package as a runnable `/cairo-auditor` command. Use Claude Code CLI or Plugin Marketplace for the executable orchestrator flow. In Cursor, treat these files as reference guidance only.
There is no official global `~/.cursor/skills` install path for this package.

**Claude Code Plugin Marketplace:**

```bash
/plugin marketplace add keep-starknet-strange/starknet-skills
/plugin install cairo-auditor@starknet-skills
```

Deterministic local repo audit entrypoint:
**Update to latest:**

```bash
python scripts/quality/audit_local_repo.py \
--repo-root /path/to/your/cairo-repo \
--scan-id local-cairo-audit \
--output-json /tmp/local-cairo-audit.json \
--output-md /tmp/local-cairo-audit.md
cd starknet-skills && git pull
# Claude Code CLI:
cp -R cairo-auditor/. ~/.claude/commands/cairo-auditor/
# Cursor docs refresh (manual guidance only):
cp -R cairo-auditor/references/. docs/cairo-auditor/
```

## Usage

```bash
# Scan the full repo (default — 4 parallel agents)
/cairo-auditor

# Full repo + adversarial reasoning agent (slower, more thorough)
/cairo-auditor deep

# Review specific file(s)
/cairo-auditor src/contracts/account.cairo
/cairo-auditor src/contracts/account.cairo src/contracts/factory.cairo

# Write report to a markdown file (terminal-only by default)
/cairo-auditor --file-output
```

With Sierra confirmation (build mode):
### Deterministic local scan (no AI)

```bash
python scripts/quality/audit_local_repo.py \
python3 scripts/quality/audit_local_repo.py \
--repo-root /path/to/your/cairo-repo \
--scan-id local-cairo-audit-sierra \
--sierra-confirm \
--allow-build \
--output-json /tmp/local-cairo-audit-sierra.json \
--output-md /tmp/local-cairo-audit-sierra.md
--scan-id my-audit
```

## Structure
## Example output

```text
[P0] 1. Ungated Upgrade Path
NO_ACCESS_CONTROL_MUTATION · src/contracts/account.cairo:42 · Confidence: 92

Description
External upgrade() calls replace_class_syscall without caller gate.
Any account can replace the contract class, leading to full takeover.

- `SKILL.md`: 4-turn orchestration contract.
- `agents/`: vector + adversarial specialist instructions.
- `references/attack-vectors/`: partitioned D:/FP: vectors (120 total).
- `references/judging.md`: strict FP gate and confidence rules.
- `references/vulnerability-db/`: canonical class docs.
- `references/semgrep/`: optional Semgrep auxiliary rules.
Fix
- fn upgrade(ref self: ContractState, new_class: ClassHash) {
+ fn upgrade(ref self: ContractState, new_class: ClassHash) {
+ self.ownable.assert_only_owner();

Required Tests
- Unauthorized caller reverts on upgrade
- Owner successfully upgrades and new class hash persists
```

## How it works

The skill orchestrates a **4-turn pipeline**:

1. **Discover** — find in-scope `.cairo` files, run deterministic preflight
2. **Prepare** — build 4 code bundles, each with a different attack-vector partition
3. **Spawn** — 4 parallel vector specialists (`model: sonnet`), optionally + 1 adversarial (`model: opus` in deep mode)
4. **Report** — merge, deduplicate by root cause, sort by confidence, emit findings

Each agent scans the full codebase against 30 attack vectors from its partition (120 total), applies a strict false-positive gate, and formats findings with exploit paths and fix diffs.

## Known limitations

**Codebase size.** Works best under ~5,000 lines of Cairo. Past that, triage accuracy and mid-bundle recall degrade. For large codebases, run per-module rather than everything at once.

**What AI misses.** AI catches pattern-based vulnerabilities reliably: missing access controls, CEI violations, unsafe upgrades, zero-address initialization. It struggles with: multi-transaction state setups, specification/invariant bugs, cross-protocol composability, game-theoretic attacks, and off-chain oracle assumptions. AI catches what humans forget to check. Humans catch what AI cannot reason about. You need both.

## Benchmarks

Expand All @@ -73,13 +138,22 @@ Additional quality signals:

- External triage: [v0.2.0-cairo-auditor-external-triage.md](../evals/scorecards/v0.2.0-cairo-auditor-external-triage.md)
- Manual gold: [v0.2.0-cairo-auditor-manual-19-gold-recall.md](../evals/scorecards/v0.2.0-cairo-auditor-manual-19-gold-recall.md)
- Sierra auxiliary: [sierra-parallel-low-profile-2026-03-09-v2.md](../evals/reports/sierra-parallel-low-profile-2026-03-09-v2.md)
- Caracal + Semgrep adapters: see [../scripts/quality/README.md](../scripts/quality/README.md)

## References
## Structure

- Skill policy: [SKILL.md](SKILL.md)
- Workflow docs: [workflows/default.md](workflows/default.md), [workflows/deep.md](workflows/deep.md)
- Vector index: [references/attack-vectors/](references/attack-vectors)
- FP gate: [references/judging.md](references/judging.md)
- Class docs: [references/vulnerability-db/README.md](references/vulnerability-db/README.md)
```text
cairo-auditor/
SKILL.md # 4-turn orchestration contract
agents/
vector-scan.md # vector specialist instructions
adversarial.md # adversarial specialist instructions
references/
attack-vectors/ # 120 vectors in 4 partitions
vulnerability-db/ # 13 canonical vulnerability classes
judging.md # FP gate + confidence scoring
report-formatting.md # finding template + priority mapping
semgrep/ # optional Semgrep auxiliary rules
workflows/
default.md # 4-agent pipeline reference
deep.md # + adversarial agent details
```
Loading