Skip to content

benchmarks.md

Latest commit

 

History

History
41 lines (30 loc) · 1.51 KB

benchmarks.md

File metadata and controls

41 lines (30 loc) · 1.51 KB
Raw

Benchmarks

clawscan benchmark <benchmark-id> runs a supported benchmark through the selected scanners and optional judge harness:

clawscan benchmark list

clawscan benchmark SkillTrustBench \
  --profile clawhub \
  --output ./artifacts/skilltrustbench-clawhub.json

Use --ids <path-or-url> with SkillTrustBench to run a fixed subset from a plain text ID list or JSONL rows with an id field.

Available benchmarks

Benchmark ID Source
ClawHub Security Signals clawhub-security-signals Hugging Face
SkillTrustBench SkillTrustBench Hugging Face

Submitting a patch to the clawhub profile

If you are a security researcher who found malicious skills live on ClawHub and want to improve the production scanner so it catches them, use GitHub private vulnerability reporting for the sensitive details and open a PR containing only a candidate proposals/<GHSA-ID>/clawscan.yml config. For a guided walkthrough, ask Codex:

Use $report-clawhub-malicious-skill to walk me through reporting a malicious ClawHub skill.

ClawHub Profile Baseline

Maintainers validate accepted clawhub profile proposals against the public SkillTrustBench leaderboard subset. The maintainer gate writes compact dated baselines under benchmarks/skilltrustbench-leaderboard-10pct/; the latest YYYY-MM-DD.json file is the current accepted baseline.