Community wave6: 5-repo auditor sweep, reproducibility report, and UX improvements

[omarespejel]

Community Audit Sweep (5 repos) - 2026-03-09

This is a first-time-user reproducibility sweep using the deterministic cairo-auditor external-scan path.

Repro command

python3 scripts/quality/scan_external_repos.py \
  --scan-id community-wave6-2026-03-09 \
  --repos-file /tmp/community_repos_wave6.txt \
  --exclude 'test,tests,mock,mocks,example,examples,preset,presets,fixture,fixtures,vendor,vendors,artifacts' \
  --output-json evals/reports/data/external-repo-scan-community-wave6-2026-03-09.json \
  --output-md evals/reports/data/external-repo-scan-community-wave6-2026-03-09.md \
  --output-findings-jsonl evals/reports/data/external-repo-scan-community-wave6-2026-03-09.findings.jsonl

Repos scanned:

• argentlabs/argent-contracts-starknet
• 10k-swap/10k_swap-contracts
• Carbonable/carbon-protocol
• EkuboProtocol/governance
• snapshot-labs/sx-starknet

High-level outcome

• Repos scanned: 5/5
• Failures: 0
• Total findings: 6
• Classes hit:
  • CRITICAL_ADDRESS_INIT_WITHOUT_NONZERO_GUARD: 3
  • IMMEDIATE_UPGRADE_WITHOUT_TIMELOCK: 2
  • IRREVOCABLE_ADMIN: 1

Repo summary CSV

repo,ref,all_cairo_files,prod_cairo_files,prod_hits
argentlabs/argent-contracts-starknet,c30f61c5b43e12fb88ea64dcdad9649033685d83,67,35,0
10k-swap/10k_swap-contracts,c47f2e158687a757d0834893d5b01ba4131a55ec,22,18,0
Carbonable/carbon-protocol,be727aa87095527de4c9928cb4dbe182fa32cf17,38,23,3
EkuboProtocol/governance,799840d4a37f62ba48015296cb27ecd07026019b,19,18,1
snapshot-labs/sx-starknet,af131eb27748671dae5700f6c13c08d622e58383,75,49,2

Findings CSV (manual audit input)

repo,ref,file,class_id,scope
Carbonable/carbon-protocol,be727aa87095527de4c9928cb4dbe182fa32cf17,src/contracts/offseter.cairo,CRITICAL_ADDRESS_INIT_WITHOUT_NONZERO_GUARD,prod_scan
Carbonable/carbon-protocol,be727aa87095527de4c9928cb4dbe182fa32cf17,src/contracts/project.cairo,CRITICAL_ADDRESS_INIT_WITHOUT_NONZERO_GUARD,prod_scan
Carbonable/carbon-protocol,be727aa87095527de4c9928cb4dbe182fa32cf17,src/contracts/yielder.cairo,CRITICAL_ADDRESS_INIT_WITHOUT_NONZERO_GUARD,prod_scan
EkuboProtocol/governance,799840d4a37f62ba48015296cb27ecd07026019b,src/governor.cairo,IMMEDIATE_UPGRADE_WITHOUT_TIMELOCK,prod_scan
snapshot-labs/sx-starknet,af131eb27748671dae5700f6c13c08d622e58383,starknet/src/execution_strategies/timelock.cairo,IRREVOCABLE_ADMIN,prod_scan
snapshot-labs/sx-starknet,af131eb27748671dae5700f6c13c08d622e58383,starknet/src/space/space.cairo,IMMEDIATE_UPGRADE_WITHOUT_TIMELOCK,prod_scan

Manual triage CSV template

finding_id,repo,ref,file,class_id,scope,detector_label,manual_verdict,manual_notes
CW6-001,Carbonable/carbon-protocol,be727aa87095527de4c9928cb4dbe182fa32cf17,src/contracts/offseter.cairo,CRITICAL_ADDRESS_INIT_WITHOUT_NONZERO_GUARD,prod_scan,tp,,
CW6-002,Carbonable/carbon-protocol,be727aa87095527de4c9928cb4dbe182fa32cf17,src/contracts/project.cairo,CRITICAL_ADDRESS_INIT_WITHOUT_NONZERO_GUARD,prod_scan,tp,,
CW6-003,Carbonable/carbon-protocol,be727aa87095527de4c9928cb4dbe182fa32cf17,src/contracts/yielder.cairo,CRITICAL_ADDRESS_INIT_WITHOUT_NONZERO_GUARD,prod_scan,tp,,
CW6-004,EkuboProtocol/governance,799840d4a37f62ba48015296cb27ecd07026019b,src/governor.cairo,IMMEDIATE_UPGRADE_WITHOUT_TIMELOCK,prod_scan,tp,,
CW6-005,snapshot-labs/sx-starknet,af131eb27748671dae5700f6c13c08d622e58383,starknet/src/execution_strategies/timelock.cairo,IRREVOCABLE_ADMIN,prod_scan,tp,,
CW6-006,snapshot-labs/sx-starknet,af131eb27748671dae5700f6c13c08d622e58383,starknet/src/space/space.cairo,IMMEDIATE_UPGRADE_WITHOUT_TIMELOCK,prod_scan,tp,,

First-time reproducibility notes

What worked:

• Single command produced JSON, Markdown, and JSONL outputs with deterministic detectors.
• Repo-level failures are isolated, so one bad repo does not kill the sweep.

Friction observed:

• A repo containing a directory named *.cairo originally failed (fixed operationally by adding artifacts to exclude markers for this run).
• There is no built-in CSV export for manual triage; had to post-process JSON.
• No single “one-shot” CLI for first-timers (they still compose flags manually).

Proposed improvements (ease + speed), aligned with Pashov-style usability

Reference baseline: https://github.com/pashov/skills/tree/main/solidity-auditor

1. Add one-shot external sweep wrapper:

   • python scripts/quality/audit_external_pack.py --pack community-top5 --scan-id ...
   • Built-in repo packs + safe default excludes (artifacts, build, target, etc.).

2. Add native CSV outputs to scan_external_repos.py:

   • --output-findings-csv
   • --output-manual-triage-csv

3. Harden file discovery:

   • In iter_cairo_files, include only files (path.is_file()) to avoid *.cairo directory edge cases.

4. Add confidence/triage stage to external sweep output:

   • Deterministic hit -> confidence bucket (high/med/low) with rationale hooks.
   • Mirror Pashov’s practical triage flow (quick first pass, then deep pass).

5. Add first-time UX docs section:

   • Copy-paste 3 commands: run sweep, open CSV, run manual triage scoring.

6. Add optional deep orchestrator step after deterministic pass:

   • Stage 1: current deterministic detectors (fast prefilter)
   • Stage 2: parallel vector-specialist agent pass (Pashov-style)
   • Stage 3: merged report with FP gate.

If useful, next wave can be run on another 5 repos and appended as community-wave7-* with trend deltas.

[omarespejel]

Per-repo runtime (clone + deterministic scan)

Measured with one-repo-per-run invocations of scan_external_repos.py (same exclude profile as wave6).

Repo	Seconds (wall)	Prod Cairo files	Findings
argentlabs/argent-contracts-starknet	7.940	35	0
10k-swap/10k_swap-contracts	1.056	18	0
Carbonable/carbon-protocol	1.175	23	3
EkuboProtocol/governance	1.529	18	1
snapshot-labs/sx-starknet	1.262	49	2

• Total: 12.962s
• Average: 2.592s/repo
• CSV: evals/reports/data/timing-wave6/community-wave6-per-repo-timing.csv