Skip to content

Commit de41a5a

Browse files
remigiusz-antczakRemigiusz Antczakclaude
authored
feat(skill): 1.91.0 — wave 12c evidence freshness (#206, #209, #217) (#225)
Reproduce a finding against the build and SHA it came from, and validate the receipt. Three dogfood lessons, each referencing existing canon (the reproduce-with-detector family, principle 2, the pinned start_sha) rather than restating it. method.md: - Reproduce a built-artifact finding against the build it audits, not the dev server (#206). The environment axis of reproduction fidelity: a production-build audit finding (a control obscured under a sticky header, a WCAG focus-not-obscured failure) often will not appear on the dev server -- minification, CSS order, hydration, asset paths differ. A dev-server "can't reproduce" does not refute it. - Re-validate a carried-forward finding before repeating it (#217). A finding at an earlier start_sha is a hypothesis until re-checked: confirm the file:line exists at HEAD and re-run the surfacing gate. Repeating an already-fixed finding is a false positive; reporting a worsened one as unchanged over-claims a trust-critical status (rate the claim). "Still open" holds only at the current SHA. report-format.md: - The report records its capture SHA (#217) -- a "Reviewed at (start_sha)" ground-truth line, matching the machine report's field, so a follow-up knows what to re-verify each finding against. product-ux-quality.md: - The UI receipt must be a valid non-empty image (#209) -- a proxy/504-wiped stub passes a bare existence check but proves nothing (existence is not content, principle 2); capture from a clean or separate tree (a shots script that stashes discards the diff under review). Two new evals (88 total). Lockstep bump to 1.91.0; SHA256SUMS regenerated. Independent dogfood reviewer: PASS. Closes #206, #209, #217. Co-authored-by: Remigiusz Antczak <remigiusz@antczak.io> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 7252335 commit de41a5a

13 files changed

Lines changed: 95 additions & 19 deletions

File tree

‎.claude-plugin/plugin.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"name": "deep-code-review",
33
"description": "Perun — bring the thunder to your codebase. A portable, evidence-grounded quality bar for building with AI agents: the flagship deep-code-review skill (default install is review-only), plus opt-in overlays — gated delivery, an idea critic, a suite conductor, and stage-aware product, growth, positioning, business-ops, and output-safety specialists.",
44
"skills": "./.claude/skills",
5-
"version": "1.90.0",
5+
"version": "1.91.0",
66
"author": {
77
"name": "deep-code-review contributors"
88
},

‎.claude/skills/agentic-delivery/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ description: >-
1010
license: MIT
1111
metadata:
1212
author: deep-code-review contributors
13-
version: "1.90.0"
13+
version: "1.91.0"
1414
---
1515

1616
# Agentic delivery
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.90.0
1+
1.91.0

‎.claude/skills/deep-code-review/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ description: >-
1111
license: MIT
1212
metadata:
1313
author: deep-code-review contributors
14-
version: "1.90.0"
14+
version: "1.91.0"
1515
---
1616

1717
# Deep Code Review
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.90.0
1+
1.91.0

‎.claude/skills/deep-code-review/evals/evals.json‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -908,6 +908,28 @@
908908
"Requires the second PR to rebase onto the merged first and rebuild from combined source (superset fold), not layer a partial build",
909909
"Flags generated-artifact PRs as superset-fold-required while a sibling source PR is open"
910910
]
911+
},
912+
{
913+
"id": "revalidate-carried-forward-finding-before-repeating",
914+
"prompt": "A follow-up review session. The prior report (captured at SHA a1b2c3) lists a Blocker: a missing authz check at api/orders.ts:88. The branch has since advanced 20 commits. Can I carry that Blocker into the new report as still-open?",
915+
"expected_output": "Not without re-validating it -- a finding carried forward from an earlier SHA without a re-run is a hypothesis, not a confirmed defect. The prior report resolves at its START_SHA (a1b2c3); the tree is now 20 commits ahead, so before repeating the Blocker: (1) re-check that api/orders.ts:88 still exists and still has the gap at HEAD (the line may have moved or the code may be gone), and (2) re-run the gate or probe that surfaced it. Carried forward unchecked it is unverified, not still-open. Two things go wrong if you skip this: the finding may already be FIXED in the 20 commits (repeating it is a false positive that spends the owner's trust), or it may have CHANGED or worsened while you report it unchanged -- reporting a worsened finding as still-open over-claims a trust-critical status; rate that claim, not the staleness. A 'still open' status is only valid at the current SHA -- so record the new report's START_SHA and re-verify each carried finding against it. (method.md.)",
916+
"expectations": [
917+
"Refuses to carry the finding forward as still-open without re-running it; a carried finding without a re-run is a hypothesis (unverified)",
918+
"Requires re-checking the file:line exists at HEAD AND re-running the surfacing gate before repeating",
919+
"Names both failure modes: repeating an already-fixed finding (false positive) and reporting a worsened one as unchanged (trust-critical over-claim)",
920+
"Grounds validity at the current SHA and records the report's start_sha for the next re-verification"
921+
]
922+
},
923+
{
924+
"id": "reproduce-audit-finding-against-production-build",
925+
"prompt": "A layout-audit gate that runs against our production build flags a focused control as fully obscured under the sticky header (WCAG focus-not-obscured), at several viewports. I ran the dev server and the control looks fully visible every time -- I can't reproduce it. Is the gate finding a false alarm I can dismiss?",
926+
"expected_output": "No -- don't dismiss a production-build finding because the dev server can't reproduce it. The gate scored the PRODUCTION build (bundled, minified, deployed-style); the dev server is a different artifact -- minification, CSS ordering, hydration timing, and asset paths differ, so a sticky-header overlap real in the built output often does not appear in dev. This is repro fidelity at the ENVIRONMENT axis (the sibling of reproducing with the gate's own detector, and of the local-vs-CI divergence rule): reproduce against the same artifact the gate scored -- build the production bundle and inspect it, or hit the deployed/preview URL -- not the dev server. A dev-server 'looks fine' is unverified against the built surface, not a refutation. Chasing it on the dev server either fixes the wrong tree or dismisses a live accessibility defect. (method.md.)",
927+
"expectations": [
928+
"Refuses to dismiss the production-build finding on a dev-server non-repro; the dev server is a different artifact",
929+
"Names why they diverge (minification, CSS order, hydration, asset paths) -- the environment axis of reproduction fidelity",
930+
"Requires reproducing against the same artifact the gate scored (production build or deployed/preview URL)",
931+
"Treats the dev-server 'looks fine' as unverified against the built surface, not a refutation"
932+
]
911933
}
912934
]
913935
}

‎.claude/skills/deep-code-review/references/method.md‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,29 @@ and any wired security/dependency scanners. Then, before trusting "green":
183183
non-empty result is not proof the layer ran"), from repro-fidelity w.r.t.
184184
**conditions** (`parallel-audit.md`'s reproduce-at-low-concurrency), and from
185185
`report-format.md`'s `mechanism-unproven` (a fix you *could not* reproduce;
186-
this is a repro you *did* run, with the wrong mechanism).
186+
this is a repro you *did* run, with the wrong mechanism), and from repro-fidelity
187+
w.r.t. the **build environment** (the production-build bullet next).
188+
- **Reproduce a built-artifact finding against the build it audits, not the dev
189+
server.** The same rule at the environment axis: an audit gate that scores the
190+
**production build** (a bundled/minified deploy artifact) can surface a real defect
191+
— a control obscured by a sticky header, a WCAG focus-not-obscured failure — that
192+
the **dev server never shows**, because minification, CSS ordering, hydration
193+
timing, and asset paths differ between them. A dev-server "I can't reproduce it"
194+
does **not** refute a production-build finding; reproduce against the same artifact
195+
the gate scored (build it, or hit the deployed/preview URL), the way the local≠CI
196+
rule names the divergent axis (below). Chasing it on the dev server fixes the wrong
197+
tree, or dismisses a live defect.
198+
- **Re-validate a carried-forward finding before repeating it — a finding without a
199+
re-run is a hypothesis.** A finding captured at one SHA (the report's `START_SHA`,
200+
`report-format.md`) is current only for that tree; before repeating it in a later
201+
session, re-check the `file:line` still exists at HEAD **and** re-run the gate or
202+
probe that surfaced it. Carried forward unchecked it is `unverified`, not
203+
still-open, and the report's `START_SHA` is what tells a follow-up what to re-check
204+
against. It prevents two failures: repeating a finding **already fixed** in the
205+
intervening commits (a false positive that spends the owner's trust), and — worse —
206+
repeating one that **changed or worsened** as if unchanged, which over-claims a
207+
**trust-critical** status (rate that *claim*, not the staleness). A "still open"
208+
status holds only at the current SHA.
187209
- **A green gate clears only the surface it enumerated — not one it never
188210
visited.** A gate that ran and passed proves something about the routes,
189211
states, and inputs its coverage set actually reached; a surface it never

‎.claude/skills/deep-code-review/references/product-ux-quality.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -351,7 +351,7 @@ confidence number published as precision (confidence tier).
351351
- [ ] Told "not the same" → **asked which axis** before acting (after one wrong guess, asked not guessed), and compared the **reference itself** at the element × breakpoint × theme, not from memory?
352352
- [ ] Parity target expressed in **measured device-pixels at the actual render scale** (not user-space units — equal user-units ≠ equal pixels), and same-axis oscillation treated as a **duplicate-implementation-at-different-scale** signal (measure the ratio, don't tune)?
353353
- [ ] No status is green-with-a-caveat — a status the author can immediately qualify is **downgraded**, not asserted beside a hedge (`report-format.md`)?
354-
- [ ] UI change: headed-browser receipt on the exact route after the action (screenshot or equivalent)? Unit tests alone are not this box.
354+
- [ ] UI change: headed-browser receipt on the exact route after the action (screenshot or equivalent) — and the receipt is a **valid non-empty image**, not a proxy/504-wiped stub that passes a bare existence check (existence is not content — `SKILL.md` principle 2), captured from a **clean or separate tree** (a shots script that stashes uncommitted changes discards the very diff under review)? Unit tests alone are not this box.
355355

356356
## Parity claims: the default state is the canonical surface
357357

‎.claude/skills/deep-code-review/references/report-format.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,10 @@ communication defect: unlike a hedged ⚠️ that tells the owner to keep checki
2626
Counts: Blocker N · Critical N · High N · Medium N · Low N · Nit N
2727
2828
## Ground truth
29+
- Reviewed at (`START_SHA`): <the SHA every finding resolves at — the same pin the
30+
invariant rows use (`file:line`@`START_SHA` below); a later session re-checks each
31+
finding against this before repeating it (`method.md`). A machine report records
32+
the same pin as its `start_sha` field.>
2933
- Build: <ok / failed: …>
3034
- Tests: <X/Y pass, Z skipped, coverage — scoped per subtree if any gate
3135
excludes code, e.g. `root: N (excludes X)` + `X: M (separate gate)`>

‎.claude/skills/idea-critic/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ description: >-
99
license: MIT
1010
metadata:
1111
author: deep-code-review contributors
12-
version: "1.90.0"
12+
version: "1.91.0"
1313
---
1414

1515
# Idea critic

0 commit comments

Comments
 (0)