Repository navigation
Gitleaks: narrow the fixture allowlist and run the guard in CI #2342
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: gates | |
| # Enforce the standards this repo documents. The skill it ships now flags | |
| # "standards documented but no gate enforces them" as a finding — so this repo | |
| # wires its own documented gates (CLAUDE.md pre-commit block + definition of done) | |
| # into CI rather than leaving them advisory. The gate logic lives in one place, | |
| # scripts/ci-gates.sh, exercised by scripts/test-ci-gates.sh; CI runs that same | |
| # helper against the real repo so the enforced check and its tests never diverge. | |
| # PR-only: branch protection requires `gates` with strict up-to-date branches, | |
| # so a PR's run already tested the exact tree that lands on main; a second run | |
| # on the post-merge push would repeat it and burn runner minutes. Superseded | |
| # runs on the same PR are cancelled. | |
| on: | |
| pull_request: | |
| branches: [main] | |
| workflow_dispatch: | |
| concurrency: | |
| group: gates-${{ github.event.pull_request.number || github.ref }} | |
| cancel-in-progress: true | |
| permissions: | |
| contents: read # least privilege (OWASP Scorecard: Token-Permissions) | |
| jobs: | |
| # Heavy steps run as parallel jobs (wall time = slowest job, not the sum); the aggregate `gates` job below keeps the | |
| # required check name. scripts/test-ci-gates.sh shards its cases by SHARD=i/N; `perpr` shards scripts/ci-tests.d with the same SHARD=i/N spelling. | |
| selftests: | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| shard: [1, 2, 3] | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 35 | |
| steps: | |
| # SHA-pinned, not a mutable tag (the skill's own K rule for CI actions). | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| with: | |
| fetch-depth: 0 # fix-class gate diffs base..head; a depth-1 clone cannot resolve the range | |
| persist-credentials: false # no later step pushes or fetches a private ref; GH_TOKEN env covers gh | |
| # Derived files (SHA256SUMS, size-budgets.tsv rows, INDEX.md, INDEX-scripts.md) are not committed by lane PRs, so two | |
| # PRs never conflict on them. Rebuild them here, before any gate reads them; a land commit (VERSION | |
| # moved) is skipped, so there the committed files are checked byte-for-byte. | |
| - name: Regenerate derived files (unlanded PR) | |
| if: github.event_name == 'pull_request' | |
| run: bash scripts/regen-unlanded.sh origin/main | |
| # Prove the gate helper against its own fixtures (routing/version/privacy | |
| # + the real install path) before trusting it on this repo. | |
| - name: Gate helper self-tests | |
| env: | |
| SHARD: ${{ matrix.shard }}/3 | |
| run: bash scripts/test-ci-gates.sh | |
| enforce: | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 35 | |
| steps: | |
| # SHA-pinned, not a mutable tag (the skill's own K rule for CI actions). | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| with: | |
| fetch-depth: 0 # fix-class gate diffs base..head; a depth-1 clone cannot resolve the range | |
| persist-credentials: false # no later step pushes or fetches a private ref; GH_TOKEN env covers gh | |
| # Derived files (SHA256SUMS, size-budgets.tsv rows, INDEX.md, INDEX-scripts.md) are not committed by lane PRs, so two | |
| # PRs never conflict on them. Rebuild them here, before any gate reads them; a land commit (VERSION | |
| # moved) is skipped, so there the committed files are checked byte-for-byte. | |
| - name: Regenerate derived files (unlanded PR) | |
| if: github.event_name == 'pull_request' | |
| run: bash scripts/regen-unlanded.sh origin/main | |
| # Enforce the documented gates on the real repo via the same helper the | |
| # self-tests pin — one authoritative implementation, no inline copies. | |
| - name: Enforce documented gates | |
| run: | | |
| set -euo pipefail | |
| # Routing runs on every shipped skill dir — a reference under an overlay | |
| # cannot ship unrouted (dogfoods "documented but unenforced is a finding"). | |
| # Globbed, not enumerated: a new skill adds no line here, so PRs on different skills never conflict. | |
| for d in .claude/skills/*/; do | |
| bash scripts/ci-gates.sh routing --max-bytes 24000 "$d" | |
| done | |
| bash scripts/ci-gates.sh version . | |
| bash scripts/ci-gates.sh privacy --banlist .banlist.txt . | |
| bash scripts/ci-gates.sh enumeration . | |
| bash scripts/check-leader-doc.sh --selftest && bash scripts/check-leader-doc.sh | |
| # Frozen per-file byte-count budgets (scripts/size-budgets.tsv): | |
| # every shipped SKILL.md/references/*.md must stay within its | |
| # budget, which only ratchets down via consolidation, never up. | |
| # A raise is enforced by the "Size budgets ratchet down" step below. | |
| bash scripts/ci-gates.sh size --config scripts/size-budgets.tsv . | |
| # No git-tracked file at a banned image/media/archive/build-output | |
| # extension (scripts/binaries-allowlist.tsv): stops the | |
| # committed-screenshot/artifact bloat class; extension-scoped, not | |
| # size-scoped, so a legitimately large text fixture never trips it. | |
| bash scripts/ci-gates.sh binaries . | |
| # Frozen per-archetype MUST-LOAD token-est ceilings | |
| # (scripts/mustload-budgets.tsv), parsed from SKILL.md's own | |
| # "Archetype -> load map" table: makes the always-paid must-load | |
| # read cost per archetype visible and ratchets it down, never up. | |
| bash scripts/ci-gates.sh mustload --config scripts/mustload-budgets.tsv . | |
| # Rule-enforcement lint: a MUST/NEVER/ALWAYS rule in a SKILL.md must | |
| # name a mechanism or be marked advisory. Existing debt is frozen in | |
| # scripts/rule-enforcement-baseline.tsv; only NEW unenforced rules | |
| # fail. Baseline growth vs the PR base is the next step below. | |
| bash scripts/ci-gates.sh rules --baseline scripts/rule-enforcement-baseline.tsv . | |
| # Four legs, SHARD=i/4 (the i/N spelling selftests uses): each runs only the scripts/ci-tests.d registrations that scripts/ci-tests-shard.sh lists | |
| # for it, so the job takes as long as its slowest slice, not the sum. `gates` needs `perpr`, so it fails unless every leg passes. | |
| perpr: | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| shard: [1, 2, 3, 4] | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 35 | |
| steps: | |
| # SHA-pinned, not a mutable tag (the skill's own K rule for CI actions). | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| with: | |
| fetch-depth: 0 # fix-class gate diffs base..head; a depth-1 clone cannot resolve the range | |
| persist-credentials: false # no later step pushes or fetches a private ref; GH_TOKEN env covers gh | |
| # Derived files (SHA256SUMS, size-budgets.tsv rows, INDEX.md, INDEX-scripts.md) are not committed by lane PRs, so two | |
| # PRs never conflict on them. Rebuild them here, before any gate reads them; a land commit (VERSION | |
| # moved) is skipped, so there the committed files are checked byte-for-byte. | |
| - name: Regenerate derived files (unlanded PR) | |
| if: github.event_name == 'pull_request' | |
| run: bash scripts/regen-unlanded.sh origin/main | |
| # A slow per-PR test registers as scripts/ci-tests.d/<slug>.bg.sh: it starts here, runs alongside this leg's serial | |
| # tests, and the "Per-PR tests" step waits for it and fails on its exit code (a hang is killed after 15 minutes), so it adds no serial time. | |
| # This leg's registrations (serial and background) are listed once, here, into $RUNNER_TEMP/bg/leg; both steps read that list. | |
| - name: Start background per-PR tests (scripts/ci-tests.d/*.bg.sh) | |
| env: | |
| SHARD: ${{ matrix.shard }}/4 | |
| run: | | |
| mkdir -p "$RUNNER_TEMP/bg" | |
| date +%s >"$RUNNER_TEMP/bg/started" | |
| bash scripts/ci-tests-shard.sh >"$RUNNER_TEMP/bg/leg" | |
| unset SHARD # the tests must not inherit it: scripts/test-ci-gates.sh reads SHARD as its own slice | |
| for t in scripts/ci-tests.d/*.bg.sh; do | |
| [ -e "$t" ] || continue | |
| grep -qxF "$t" "$RUNNER_TEMP/bg/leg" || continue | |
| f="$RUNNER_TEMP/bg/$(basename "$t")" | |
| : >"$f.log" | |
| # shellcheck disable=SC2016 # $1 and $2 expand in the child shell, on purpose | |
| nohup bash -c 'timeout 900 bash -e "$1" >"$2.log" 2>&1; echo $? >"$2.rc.tmp"; date +%s >"$2.end"; mv "$2.rc.tmp" "$2.rc"' _ "$t" "$f" >/dev/null 2>&1 & | |
| done | |
| # Per-PR tests: a PR registers its test by adding scripts/ci-tests.d/<slug>.sh (one command line), | |
| # not by appending a line to the step above, so PRs never conflict on this file. | |
| # pyyaml: scripts/test_workflow_persist_credentials.py fails closed without it. | |
| - name: Install hypothesis for the property tests | |
| run: python3 -m pip install hypothesis==6.170.12 pyyaml==6.0.3 | |
| - name: Per-PR tests (scripts/ci-tests.d) | |
| run: | | |
| set -euo pipefail | |
| [ -s "$RUNNER_TEMP/bg/leg" ] || { echo "FAIL: the start step wrote no test list for this leg"; exit 1; } # else grep fails and every test is skipped | |
| for t in scripts/ci-tests.d/*.sh; do | |
| case "$t" in *.bg.sh) continue ;; esac | |
| grep -qxF "$t" "$RUNNER_TEMP/bg/leg" || continue # another leg runs it | |
| echo "== $t" | |
| bash -e "$t" | |
| done | |
| for t in scripts/ci-tests.d/*.bg.sh; do | |
| [ -e "$t" ] || continue | |
| grep -qxF "$t" "$RUNNER_TEMP/bg/leg" || continue | |
| f="$RUNNER_TEMP/bg/$(basename "$t")" | |
| echo "== $t (started in the background)" | |
| [ -f "$f.log" ] || { echo "FAIL: $t was never started"; exit 1; } | |
| end=$(($(cat "$RUNNER_TEMP/bg/started") + ${BG_WAIT_S:-960})) # counted from the background start, not from here | |
| while [ ! -f "$f.rc" ] && [ "$(date +%s)" -lt "$end" ]; do sleep 1; done | |
| cat "$f.log" | |
| took='?'; [ -f "$f.end" ] && took=$(($(cat "$f.end") - $(cat "$RUNNER_TEMP/bg/started"))) | |
| echo "bg elapsed: ${f##*/} $took s" | |
| [ -f "$f.rc" ] || { echo "FAIL: $t did not finish in time"; exit 1; } | |
| [ "$(cat "$f.rc")" = 0 ] | |
| done | |
| evals: | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 35 | |
| steps: | |
| # SHA-pinned, not a mutable tag (the skill's own K rule for CI actions). | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| with: | |
| fetch-depth: 0 # fix-class gate diffs base..head; a depth-1 clone cannot resolve the range | |
| persist-credentials: false # no later step pushes or fetches a private ref; GH_TOKEN env covers gh | |
| # Derived files (SHA256SUMS, size-budgets.tsv rows, INDEX.md, INDEX-scripts.md) are not committed by lane PRs, so two | |
| # PRs never conflict on them. Rebuild them here, before any gate reads them; a land commit (VERSION | |
| # moved) is skipped, so there the committed files are checked byte-for-byte. | |
| - name: Regenerate derived files (unlanded PR) | |
| if: github.event_name == 'pull_request' | |
| run: bash scripts/regen-unlanded.sh origin/main | |
| # Live harness (stub HTTP server, no key, no spend) and the review-scoring | |
| # instrument (canaries, FP budget, fixture integrity). | |
| - name: Live eval harness and review scorer tests (offline) | |
| run: | | |
| python3 scripts/test_live_evals.py | |
| python3 scripts/test_score_review.py | |
| python3 scripts/test_issue1418.py | |
| python3 scripts/test_install_commands.py | |
| python3 scripts/test_spend_report.py | |
| python3 scripts/test_field_learnings.py | |
| python3 scripts/test_terse_scope_checks.py | |
| python3 scripts/test_reply_budget_prevention.py | |
| python3 scripts/test_handback_copypaste_exempt.py | |
| python3 scripts/test_lane_guard_banlist_seed.py | |
| python3 scripts/test_lane_guard_hotfiles.py | |
| python3 scripts/test_sandbox_preflight.py | |
| python3 scripts/test_learning_to_pr.py | |
| python3 scripts/test_wire_defaults.py | |
| python3 scripts/test_weekly_receipt.py | |
| python3 scripts/test_token_ratchet.py | |
| python3 scripts/test_destructive_fp.py | |
| python3 scripts/test_repeat_poll_and_brief_lint.py | |
| python3 scripts/test_token_report_rollup.py | |
| python3 scripts/test_tokens_per_pr_parity.py | |
| python3 scripts/test_bench_corpus.py | |
| python3 scripts/test_bench_run_isolation.py | |
| python3 scripts/test_resource_policy.py | |
| python3 scripts/test_council.py | |
| python3 scripts/test_veche.py | |
| python3 scripts/test_veche_timebox.py | |
| python3 scripts/test_roadmap_rank.py | |
| python3 scripts/test_policy_ux.py | |
| bash scripts/test-resource-policy-train.sh | |
| python3 scripts/test_review_feedback.py | |
| python3 scripts/test_learned_rules.py | |
| python3 scripts/test_review_lean.py | |
| bash scripts/test-post-review.sh | |
| python3 scripts/test_eval_predicates_defended.py | |
| python3 scripts/test_perun_doctor.py | |
| python3 scripts/test_install_ux.py | |
| python3 scripts/test_extra_hosts_install.py | |
| python3 scripts/test_hosts_parity.py | |
| python3 scripts/test_host_safety_rows.py | |
| python3 scripts/test_auto_update.py | |
| python3 scripts/test_auto_update_global.py | |
| python3 scripts/test_auto_update_pr.py | |
| python3 scripts/test_fleet_hooks.py | |
| python3 scripts/test_field_mos_1009.py | |
| bash scripts/test-land-release-tag.sh | |
| bash scripts/test-host-manifests.sh | |
| bash scripts/test-land-release-publish.sh | |
| python3 scripts/test_scan_tracked_only.py | |
| python3 scripts/test_review_gate.py | |
| python3 scripts/test_max_pace_no_waste.py | |
| cheap: | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 35 | |
| steps: | |
| # SHA-pinned, not a mutable tag (the skill's own K rule for CI actions). | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| with: | |
| fetch-depth: 0 # fix-class gate diffs base..head; a depth-1 clone cannot resolve the range | |
| persist-credentials: false # no later step pushes or fetches a private ref; GH_TOKEN env covers gh | |
| # Derived files (SHA256SUMS, size-budgets.tsv rows, INDEX.md, INDEX-scripts.md) are not committed by lane PRs, so two | |
| # PRs never conflict on them. Rebuild them here, before any gate reads them; a land commit (VERSION | |
| # moved) is skipped, so there the committed files are checked byte-for-byte. | |
| - name: Regenerate derived files (unlanded PR) | |
| if: github.event_name == 'pull_request' | |
| run: bash scripts/regen-unlanded.sh origin/main | |
| - name: review_checks self-test | |
| run: bash scripts/test-review-checks.sh | |
| - name: mktemp TMPDIR regression | |
| run: bash scripts/test-mktemp-tmpdir.sh | |
| # Shipped gate scripts prove they FIRE on planted violations before any | |
| # result of theirs is trusted (independent oracle per script). | |
| # Strict: a new script nothing calls, or that no registered CI test runs, was built but never wired (field learning, #1380). | |
| - name: New scripts are wired and run | |
| env: | |
| BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }} | |
| HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} | |
| run: | | |
| cat scripts/ci-tests.d/*.sh scripts/test_*.py scripts/test-*.sh > "$RUNNER_TEMP/runs.txt" | |
| python3 .claude/skills/agentic-delivery/scripts/wired_check.py "$BASE_SHA" "$HEAD_SHA" --strict --runs "$RUNNER_TEMP/runs.txt" | |
| - name: Shipped script self-tests | |
| run: | | |
| set -euo pipefail | |
| python3 .claude/skills/deep-code-review/scripts/parity_differ.py --selftest | |
| python3 .claude/skills/deep-code-review/scripts/token_differ.py --selftest | |
| python3 .claude/skills/deep-code-review/scripts/fix_class_gate.py --selftest | |
| python3 .claude/skills/deep-code-review/scripts/closes_lint.py --selftest | |
| python3 .claude/skills/deep-code-review/scripts/binaries_gate.py --selftest | |
| python3 .claude/skills/deep-code-review/scripts/reaper_lint.py --selftest | |
| python3 .claude/skills/deep-code-review/scripts/source_scan_tests.py --selftest | |
| python3 .claude/skills/deep-code-review/scripts/ci_cost_lint.py --selftest | |
| bash .claude/skills/deep-code-review/templates/integrator-gate.sh --selftest | |
| python3 scripts/eval_citation_lint.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/serial_gate.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/handback_cap.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/lane_guard.py --selftest | |
| python3 scripts/test_done_means_verified.py | |
| python3 .claude/skills/agentic-delivery/scripts/clock_lint.py --selftest | |
| python3 scripts/test_clock_lint.py | |
| python3 scripts/test_clock_lint_bombs.py | |
| python3 .claude/skills/agentic-delivery/scripts/env_lint.py --selftest | |
| python3 scripts/test_env_lint.py | |
| python3 scripts/test_vacuous_static.py | |
| python3 scripts/test_mutation_changed.py | |
| bash scripts/test-peer-pains-4.sh | |
| python3 .claude/skills/agentic-delivery/scripts/board_common.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/board_sync.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/board_post.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/board_state.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/claim_probe.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/refix_gate.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/priority_gate.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/queue_guard.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/focus_gate.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/surface_check.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/restore_ledger.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/lane_liveness.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/review_digest.py --selftest | |
| python3 .claude/skills/agentic-ceo/scripts/task_ledger.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/union_overlap.py --selftest | |
| python3 .claude/skills/agentic-ceo/scripts/stop_reminder.py --selftest | |
| python3 .claude/skills/agentic-ceo/scripts/token_report.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/feedback_ledger.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/merge_train.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/tier_gate.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/lane_cap.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/escaped_defects.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/destructive_guard.py --selftest | |
| python3 .claude/skills/agentic-delivery/scripts/outcome_claim_guard.py --selftest | |
| python3 scripts/trigger_lint.py --selftest | |
| # scripts/skill_index.py generates the per-skill and repo-level INDEX.md | |
| # lookup tables (which file answers which trigger, without opening every | |
| # references/*.md or scripts/*.py blindly). Self-test proves the parser | |
| # on synthetic fixtures (never the real repo's prose); --check proves the | |
| # committed INDEX.md files are byte-identical to what the generator | |
| # would write today, so a routed reference added without regenerating | |
| # goes stale loudly instead of silently. | |
| - name: Skill index (INDEX.md lookup tables) selftest and freshness | |
| run: | | |
| set -euo pipefail | |
| python3 scripts/skill_index.py --selftest | |
| python3 scripts/skill_index.py --check | |
| # A "`<file>.md` **<Section>**" pointer must resolve in that file, so a split or | |
| # rename of a reference cannot silently orphan the pointers to it. | |
| - name: Skill section pointers resolve | |
| run: | | |
| set -euo pipefail | |
| python3 scripts/check_section_pointers.py --selftest | |
| python3 scripts/check_section_pointers.py | |
| # A doc number bound to a source command with `<!-- derived: CMD -->N` | |
| # must equal the command's output, so a stale count fails here. | |
| - name: Derived doc numbers match their source commands | |
| run: | | |
| set -euo pipefail | |
| python3 scripts/test_derived_numbers.py | |
| python3 .claude/skills/deep-code-review/scripts/derived_numbers.py . | |
| # Every fix(...) commit in the pushed/PR range must touch a test surface | |
| # or carry a non-empty No-Test-Reason: trailer. Fails closed on an | |
| # unresolvable range (e.g. the all-zeros before-SHA of a new branch). | |
| # A shipped script is deliberately NOT its own test surface: editing the | |
| # script under test would satisfy the gate by itself (fail-open). Known | |
| # limitation: touching ANY skill's evals/evals.json satisfies it, not | |
| # only the fixed skill's. | |
| - name: Fix commits carry a pinned test | |
| env: | |
| BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }} | |
| HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} | |
| run: | | |
| set -euo pipefail | |
| python3 .claude/skills/deep-code-review/scripts/fix_class_gate.py \ | |
| --base "$BASE_SHA" --head "$HEAD_SHA" \ | |
| --test-glob '.claude/skills/*/evals/evals.json' \ | |
| --test-glob 'scripts/test-ci-gates.sh' \ | |
| --test-glob 'scripts/test_*.py' \ | |
| --test-glob 'scripts/test-mktemp-tmpdir.sh' | |
| # Review tiers (agentic-delivery/templates/review-tiers.tsv): a tier-1 | |
| # commit needs `Model: opus` plus an independent, receipted | |
| # `Reviewed-By: <id> receipt:<sha|path|url>` trailer. ADVISORY here: it | |
| # prints every finding (merge commits on the first-parent line | |
| # included) and never fails the build, because this repo's history | |
| # predates the trailers and one shared account cannot prove reviewer | |
| # independence. Release squash commits carry both trailers. | |
| - name: Review tiers (advisory) | |
| env: | |
| BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }} | |
| HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} | |
| run: python3 .claude/skills/agentic-delivery/scripts/tier_gate.py --advisory "$BASE_SHA..$HEAD_SHA" | |
| # Lesson -> mechanism: a commit that edits skill prose (a SKILL.md or a | |
| # references/*.md) must also land an executable mechanism (an eval, a | |
| # shipped script, or a gate self-test case) or carry a non-empty | |
| # No-Mechanism-Reason: trailer, so a filed lesson cannot stay prose that | |
| # never fires. Same gate implementation, range, and fail-closed exits as | |
| # the fix-class step above, except that merge commits are judged by their | |
| # own changes and a SKILL.md version-stamp-only bump is not a trigger. | |
| # Here a shipped script DOES count: in this mode the script is the | |
| # mechanism being added, not the code under test. | |
| # Known limitation: it proves a mechanism path was touched, not that the | |
| # mechanism pins the lesson (review judges that). | |
| - name: Prose lessons carry a mechanism | |
| env: | |
| BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }} | |
| HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} | |
| run: | | |
| set -euo pipefail | |
| python3 .claude/skills/deep-code-review/scripts/fix_class_gate.py \ | |
| --base "$BASE_SHA" --head "$HEAD_SHA" \ | |
| --trigger-glob '.claude/skills/*/references/*.md' \ | |
| --trigger-glob '.claude/skills/*/SKILL.md' \ | |
| --test-glob '.claude/skills/*/evals/evals.json' \ | |
| --test-glob '.claude/skills/*/scripts/*' \ | |
| --test-glob 'scripts/test-ci-gates.sh' \ | |
| --test-glob 'scripts/test-*.sh' \ | |
| --test-glob 'scripts/test_*.py' | |
| # Lesson replay (ADVISORY signal, not proof): the step above proves a | |
| # mechanism path was touched; this one looks for a tie between the | |
| # mechanism and the lesson. Each prose-lesson commit prints REPLAYS (a | |
| # changed eval case names a term the lesson newly added, or reverting a | |
| # changed script hunk fails an assertion in its selftest), VACUOUS (no | |
| # tie found), or COULD_NOT_CHECK with a reason. Token overlap is | |
| # gameable; a reviewer judges every line. It exits 0 on VACUOUS and | |
| # fails closed (exit 2) only on an unresolvable range, like the gate | |
| # above. Adding `--gate` makes VACUOUS (only VACUOUS) exit 1; | |
| # COULD_NOT_CHECK never blocks. | |
| - name: Lesson mechanisms replay (advisory) | |
| timeout-minutes: 15 | |
| env: | |
| BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }} | |
| HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} | |
| run: | | |
| set -euo pipefail | |
| python3 scripts/lesson_replay.py --selftest | |
| python3 scripts/lesson_replay.py --base "$BASE_SHA" --head "$HEAD_SHA" | |
| # A size-budget row may only rise with a `size-budget-raise:` marker | |
| # (commit range or CHANGELOG) — budgets ratchet down by default. | |
| - name: Size budgets ratchet down | |
| env: | |
| BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }} | |
| run: bash scripts/ci-gates.sh size-ratchet --base "$BASE_SHA" --config scripts/size-budgets.tsv . | |
| # The rule-enforcement baseline may only grow with a | |
| # `rule-baseline-raise: <reason>` commit-message line. | |
| - name: Rule baseline ratchets down | |
| env: | |
| BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }} | |
| run: bash scripts/ci-gates.sh rules --baseline scripts/rule-enforcement-baseline.tsv --base "$BASE_SHA" . | |
| - name: SKILL.md name matches its directory | |
| run: | | |
| set -euo pipefail | |
| # Every shipped skill's frontmatter name must equal its directory (not | |
| # just deep-code-review) — an overlay cannot ship with a mismatched | |
| # name that the routing/version gates would miss. | |
| for d in .claude/skills/*/; do | |
| skill="$(basename "$d")" | |
| grep -qE "^name: ${skill}\$" "${d}SKILL.md" \ | |
| || { echo "name: does not match directory for ${skill}"; exit 1; } | |
| done | |
| - name: install.sh parses | |
| run: bash -n install.sh | |
| # Execute the fabrication-refusal evals offline: each deterministic | |
| # predicate must separate a fabricated answer (red fixture) from a refusal | |
| # (good fixture). No model, no network, no spend — the discrimination is | |
| # the property under test. The same gate the self-tests pin above. | |
| - name: Fabrication-refusal predicates discriminate (offline, no model) | |
| run: python3 scripts/eval_predicates.py --selftest | |
| # Prove the split-rubric eval RUNNER's own guards, offline: dry-run | |
| # enumerates + classifies every eval (hard = a deterministic predicate is | |
| # bound; soft = needs the live judge), the decorrelation guard aborts on an | |
| # equal-or-missing model id, and the live path refuses without configuration | |
| # (its model-call site is an un-built stub). No model, no network, no spend — | |
| # the guards are the property under test. The live judge itself is owner-gated. | |
| - name: Eval runner guards hold (offline, no model) | |
| run: python3 scripts/run-evals.py --selftest | |
| # Guards .gitleaks.toml: the fixture allowlist must not blind the scanner (full history needed for the history scan). | |
| gitleaks: | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 10 | |
| permissions: | |
| contents: read | |
| steps: | |
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | |
| with: | |
| fetch-depth: 0 | |
| persist-credentials: false | |
| - name: Install gitleaks 8.30.1 (pinned sha256) | |
| run: | | |
| set -euo pipefail | |
| cd "$RUNNER_TEMP" | |
| curl -fsSL -O https://github.com/gitleaks/gitleaks/releases/download/v8.30.1/gitleaks_8.30.1_linux_x64.tar.gz | |
| echo "551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb gitleaks_8.30.1_linux_x64.tar.gz" | sha256sum -c - | |
| mkdir -p "$RUNNER_TEMP/bin" | |
| tar -xzf gitleaks_8.30.1_linux_x64.tar.gz -C "$RUNNER_TEMP/bin" gitleaks | |
| echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH" | |
| - name: Allowlist guard | |
| run: REQUIRE_GITLEAKS=1 bash scripts/test_gitleaks_allowlist.sh | |
| # Branch protection requires a check named exactly `gates`: it passes only when every shard job succeeded. | |
| gates: | |
| needs: [selftests, enforce, perpr, evals, cheap, gitleaks] | |
| if: always() | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 5 | |
| steps: | |
| - run: | | |
| [ "${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') || contains(needs.*.result, 'skipped') }}" = false ] |