Skip to content

Commit 80d2e12

Browse files
remigiusz-antczakRemigiusz Antczakclaude
authored
feat(skill): 1.73.0 — wave 2 data-quality lenses (#149, #151, #152, #153, #161, #162) (#174)
Wave 2 of the #143–#164 dogfooding batch: six data-quality review lenses in deep-code-review/references/data-quality.md, each pre-distinguished from the section it extends. - Non-regression gate: a fanout/uniqueness gate false-blocks legitimate coverage expansion — hard-block only a value absorbed from a now-departed distinct record, not a newly-shared standing value among related entities (#149). - Entity resolution: the resolution order when no stable id exists, and surfacing the unresolved/ambiguous count as a first-class output (#153). - Scoring & config: never sum heterogeneous constructs into one composite score (#162); test every enum/config map against the source's real value distribution, not the literals a unit test feeds it (#153). - Measuring honestly: backtest a proxy-derived metric against ground truth before shipping (#151); match the validation metric to the claim — an ordinal rank validated by concordance / a C-index, not MAE (#161). - Cost discipline: measure existing-source coverage before scoping new enrichment or scrapers, and scope to the measured residual (#152). - New red-flag detectors; fixed a Cyrillic-to-Latin typo in Entity resolution. +6 evals (deep-code-review 45 -> 51). Lockstep bump to 1.73.0. Gates green: test-ci 59/59, routing / version / privacy / enumeration, eval-predicates, checksums (82 files). Ran the loop: the advisor (step 4) genericized unsourced figures and dropped a borrowed-authority claim; the independent dogfood reviewer (step 6) caught a cross-section duplication of the never-lower-a-baseline rule — both fixed before merge. Closes #149 Closes #151 Closes #152 Closes #153 Closes #161 Closes #162 Co-authored-by: Remigiusz Antczak <remigiusz@antczak.io> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent d55df1d commit 80d2e12

11 files changed

Lines changed: 162 additions & 17 deletions

File tree

‎.claude-plugin/plugin.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"name": "deep-code-review",
33
"description": "Perun — bring the thunder to your codebase. A portable, evidence-grounded quality bar for building with AI agents: the flagship deep-code-review skill (default install is review-only), plus opt-in overlays — gated delivery, an idea critic, a suite conductor, and stage-aware product, growth, positioning, business-ops, and output-safety specialists.",
44
"skills": "./.claude/skills",
5-
"version": "1.72.0",
5+
"version": "1.73.0",
66
"author": {
77
"name": "deep-code-review contributors"
88
},

‎.claude/skills/agentic-delivery/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ description: >-
1010
license: MIT
1111
metadata:
1212
author: deep-code-review contributors
13-
version: "1.72.0"
13+
version: "1.73.0"
1414
---
1515

1616
# Agentic delivery
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.72.0
1+
1.73.0

‎.claude/skills/deep-code-review/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ description: >-
1111
license: MIT
1212
metadata:
1313
author: deep-code-review contributors
14-
version: "1.72.0"
14+
version: "1.73.0"
1515
---
1616

1717
# Deep Code Review
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.72.0
1+
1.73.0

‎.claude/skills/deep-code-review/evals/evals.json‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -465,6 +465,70 @@
465465
"Explains that a stale or partial reference yields confidently-wrong output that still passes its own gates",
466466
"Frames the check as the build-time analog of verify-before-you-report"
467467
]
468+
},
469+
{
470+
"id": "fanout-gate-distinguishes-corruption-from-shared-value",
471+
"prompt": "Our field-integrity gate fails closed when a unique-per-company handle appears on two records. This refresh it flagged 33 violations, about 32 of them sub-teams that legitimately now share their parent org's handle, and blocked the publish. Lower the gate's strictness?",
472+
"expected_output": "Don't lower or disable it -- refine what it hard-blocks. The gate is right that a unique-to-one value on two entities CAN be corruption, but it false-positives during legitimate coverage expansion: many related entities newly and correctly sharing one value (sub-teams adopting a parent org's handle -- co-branding) each trip the fanout arm. Distinguish the shapes: hard-block only a value ABSORBED from a now-departed distinct record (two records collapsed into one); for a newly-shared STANDING value among related entities, keep a human at the gate or a small reviewed allowlist, and stop re-refusing an unchanged standing shared value every run (that just gets the gate switched off). Never 'green' the run by lowering the baseline -- acknowledge the intended shared values explicitly, per field, with a reason. (data-quality.md.)",
473+
"expectations": [
474+
"Refuses to lower or disable the gate or its baseline to pass the run",
475+
"Distinguishes corruption (a value absorbed from a now-departed distinct record) from a legitimately newly-shared standing value among related entities",
476+
"Hard-blocks only the corruption shape; routes legitimate shared values to a human or a reviewed allowlist",
477+
"Flags re-refusing an unchanged standing shared value every run as what gets the gate disabled"
478+
]
479+
},
480+
{
481+
"id": "measure-existing-coverage-before-new-enrichment",
482+
"prompt": "We're scoping a set of paid scrapers and enrichment APIs to hit a coverage target for our company list. The plan is ready -- greenlight the spend?",
483+
"expected_output": "Measure existing-source coverage for that exact list first -- re-querying your internal warehouse is frequently the largest, cheapest lever and can match a from-scratch enrichment plan's target at zero marginal cost, often surfacing already-reconciled fields (a stable entity id, dated events, location) the working copy lacks. Scope the external, paid acquisition to the measured RESIDUAL only. A plan that adds scrapers before measuring what you already have over-scopes -- and paid calls sit behind free/keyless pre-gates and the usual spend caps regardless. (data-quality.md.)",
484+
"expectations": [
485+
"Requires measuring existing-source (warehouse) coverage for the target set before scoping new acquisition",
486+
"Identifies re-querying existing data as the largest and cheapest coverage lever",
487+
"Scopes external or paid enrichment to the measured residual only",
488+
"Treats the pre-measurement plan as over-scoped"
489+
]
490+
},
491+
{
492+
"id": "no-name-join-and-test-config-maps-vs-real-values",
493+
"prompt": "Two data-integrity things in this pipeline: (a) we resolve companies by matching on the name string, and (b) we have a geography-to-multiplier lookup. Unit tests for both pass. Ship?",
494+
"expected_output": "Both are traps that pass unit tests while failing in production. (a) Never resolve identity on a bare name string -- namesake collisions merge distinct records, and a wrong merge is worse than a missed one. Resolve by stable id, then normalized domain/handle, then a name only as a last resort behind a collision (namesake) guard, and surface the unresolved/ambiguous count as a first-class output. (b) Test the geography-to-multiplier map against the source's REAL value distribution: if it's keyed on region names while the source emits ISO-3166 alpha-2 codes (plus lowercase and placeholders), it silently no-ops -- carrying its cost, delivering nothing -- and unit tests pass because they feed it the literals it expects. SELECT DISTINCT the real values and test against those. (data-quality.md.)",
495+
"expectations": [
496+
"Rejects name-string identity joins; requires stable id then normalized domain then name-behind-a-collision-guard, with the unresolved count surfaced",
497+
"Requires testing the enum/config map against the source's real value distribution (SELECT DISTINCT), not chosen literals",
498+
"Explains that a mis-keyed map silently no-ops while unit tests still pass",
499+
"Treats a passing unit test as insufficient for either trap"
500+
]
501+
},
502+
{
503+
"id": "backtest-proxy-metric-vs-ground-truth",
504+
"prompt": "We built a runway estimate from public proxies (last-round size / headcount x burn). Unit tests pass and the formula looks sound. Ship it as a ranking signal?",
505+
"expected_output": "Not until you backtest it against ground truth -- a plausible proxy formula that passes unit tests can be near-useless for its actual job. For any derived or scored value from indirect proxies, require a validation step in review that reports MAE / correlation / base-rate against real self-reported actuals. Unit tests prove the math; only a backtest proves the value. If it can't rank (large MAE, weak correlation, a base rate far off the real one), demote or gate it -- a coarse band plus 'corroboration-required' -- rather than shipping it as a ranker. (data-quality.md.)",
506+
"expectations": [
507+
"Requires a ground-truth backtest (MAE / correlation / base-rate vs actuals) before shipping a proxy-derived metric",
508+
"Distinguishes unit tests proving the math from a backtest proving the value",
509+
"Demotes or gates a metric that fails the backtest instead of shipping it as a ranker"
510+
]
511+
},
512+
{
513+
"id": "ordinal-rank-validated-by-concordance-not-mae",
514+
"prompt": "Our score correlates with the target at about r=0.34 -- weak but real. MAE is too big to show a point estimate on the 0-24 month runway we're predicting, so we'll publish the estimate with a +/-22 month band. Validate it with MAE and ship?",
515+
"expected_output": "Don't publish a point estimate or a +/-22-month band -- on a 0-24-month decision range that band is self-refuting noise on screen. A weak-but-nonzero predictor can still be useful as an ORDINAL RANK. Validate the ranking with concordance / a C-index against an OBSERVABLE binary event ('raised within 6 months', 'shut down within 6 months') -- not MAE on the noisy latent quantity, which measures the wrong thing. Emit an ordinal tier, not a number, when MAE is large relative to the decision range. Match the validation metric to the claim the score actually makes. (data-quality.md.)",
516+
"expectations": [
517+
"Rejects the point estimate and the band when the band is wide relative to the decision range",
518+
"Requires validating rank quality with concordance / a C-index against an observable binary event, not MAE",
519+
"Outputs an ordinal tier rather than a point estimate when MAE is large relative to the decision range",
520+
"Frames it as matching the validation metric to the claim the score makes"
521+
]
522+
},
523+
{
524+
"id": "split-composite-score-never-sum-heterogeneous",
525+
"prompt": "We're rolling raise-timing and financial-distress into one 'urgency' score by summing them, so a high number means 'act now.' Good?",
526+
"expected_output": "Don't sum them -- they're heterogeneous constructs that imply OPPOSITE actions, so one blended number manufactures false positives and hides which action is warranted. Raise-timing overdue means 'introduce to investors'; distress/contraction means 'triage or support.' A record high on one and low on the other averages to a middling 'urgency' that tells you nothing. Score each construct separately and derive the action from the combination -- a 2x2 or tiering -- never from one summed number. (data-quality.md.)",
527+
"expectations": [
528+
"Refuses to sum heterogeneous constructs into one composite score",
529+
"Explains the constructs imply opposite actions, so a blended number hides which is warranted",
530+
"Requires scoring each construct separately and deriving the action from the combination (a 2x2 or tiering)"
531+
]
468532
}
469533
]
470534
}

‎.claude/skills/deep-code-review/references/data-quality.md‎

Lines changed: 59 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,18 @@ Also run a **run-over-run drift guard** on derived-field population counts:
5050
enrichment is additive, so any populated-column count that falls beyond known
5151
record attrition is a regression.
5252

53+
**A uniqueness violation is corruption only if the value was *absorbed from a
54+
now-departed distinct record* — not when it is newly and legitimately *shared*.**
55+
The within-dataset invariant above false-positives during legitimate coverage
56+
expansion: many related entities newly and correctly sharing one value (sub-teams
57+
adopting a parent org's handle — co-branding) each trip the fanout arm. Hard-block
58+
only the collapse-of-two-distinct-records shape; for a newly-shared *standing*
59+
value among related entities keep a human at the gate or a small reviewed
60+
allowlist, and never re-refuse an unchanged standing shared value every run (that
61+
just gets the gate switched off) — and don't relax the gate's baseline to pass,
62+
which the never-lower-a-baseline rule under *Scoring & config discipline* already
63+
governs.
64+
5365
## 2. No fabrication in the data itself
5466

5567
- **Skip a field rather than guess it.** An empty cell beats a confident-looking
@@ -68,9 +80,13 @@ record attrition is a regression.
6880

6981
## 3. Entity resolution — bias false-exclude over false-merge
7082

71-
- **Identity requires a stable id or same-person/же-entity proof — never a
83+
- **Identity requires a stable id or same-person/same-entity proof — never a
7284
name-only match.** Two distinct records wrongly merged is worse than two left
7385
separate.
86+
- **Resolution order when no stable id exists:** normalized domain/handle first,
87+
then a name **only as a last resort behind a collision (namesake) guard** — and
88+
surface the **unresolved/ambiguous count as a first-class output**, never a
89+
silent drop.
7490
- Matches must clear a threshold on **multiple independent signals**; ambiguous
7591
or conflicting matches are **flagged for review, never auto-merged**.
7692
- Prefer revealed-preference, hard-to-game, multi-signal evidence over a single
@@ -170,6 +186,19 @@ rate), validity (schema/format/range). For each:
170186
build pass.** An intended drop is acknowledged explicitly, per field, with a
171187
stated reason. (This is the monotonic invariant applied to the *gates
172188
themselves*.)
189+
- **Never sum heterogeneous constructs into one composite score.** A blended
190+
"urgency"/"risk" number often merges constructs that imply *opposite* actions —
191+
raise-timing overdue (introduce to investors) vs distress/contraction (triage) —
192+
so summing them manufactures false positives and hides which action is warranted.
193+
Score each construct separately and derive the action from the combination (a
194+
2×2 / tiering), never from one blended number.
195+
- **Test every enum/config mapping against the source's *real* value
196+
distribution.** A lookup keyed on the wrong domain — a geography→multiplier map
197+
keyed on region names while the source emits ISO-3166 alpha-2 codes (plus
198+
lowercase and placeholder values) — silently no-ops: it carries its cost and
199+
delivers nothing, and unit tests pass because they feed the map the literals it
200+
expects. `SELECT DISTINCT` the real values and test against them before trusting
201+
the map.
173202

174203
## 8. Measuring the outcome honestly
175204

@@ -179,6 +208,22 @@ rate), validity (schema/format/range). For each:
179208
quality as *unmeasured* until an expert rates a frozen, labeled cohort; don't
180209
stack features on an unvalidated base. See `testing-and-evals.md` for the
181210
eval-harness pattern.
211+
- **Backtest a proxy-derived metric against ground truth before shipping it — a
212+
plausible formula that passes unit tests can be near-useless.** For any
213+
derived/scored value built from indirect proxies (estimating runway from
214+
last-round size ÷ headcount × burn, say), require a ground-truth validation step
215+
in review — report MAE / correlation / base-rate against real actuals. Unit
216+
tests prove the math; only a backtest proves the *value*. On failure, **demote
217+
or gate** it (a coarse band + "corroboration-required"), never ship it as a
218+
ranker.
219+
- **Match the validation metric to the claim the score makes.** A predictor whose
220+
correlation is weak but nonzero, with MAE too large to publish a point estimate,
221+
can still rank usefully — but validate ranking with **concordance / a C-index
222+
against an observable binary event** ("raised within 6 months", "shut down within
223+
6 months"), *not* MAE on the noisy latent quantity. Emit an ordinal tier, not a
224+
point estimate, when MAE is large relative to the decision range, and reject a
225+
self-refuting "±N" band — a band wider than the decision range is noise on
226+
screen.
182227

183228
## 9. The model's role in a data pipeline (if any)
184229

@@ -215,6 +260,13 @@ rate), validity (schema/format/range). For each:
215260
- **Data-specific precision floor:** a gate that **drops/deletes** records needs
216261
far higher measured precision than one that merely enriches — refuse a
217262
whole-dataset apply unless it is scoped to a verified subset.
263+
- **Measure existing-source coverage for the target set before scoping new
264+
enrichment or scrapers.** Re-querying the internal warehouse is frequently the
265+
largest, cheapest coverage lever — it can match a from-scratch enrichment plan's
266+
target at zero marginal cost and surface already-reconciled fields (a stable
267+
entity id, dated events, location) the working copy lacks. Scope external
268+
acquisition to the **measured residual** only; a plan that adds scrapers before
269+
measuring over-scopes.
218270

219271
---
220272

@@ -227,4 +279,9 @@ call that returns a score or a boolean gate; weights inlined in code with no
227279
snapshot; a consumer/export that re-queries raw instead of the filtered set;
228280
written artifact with no reader; mass status-change on an upstream error; live
229281
counts hard-coded into docs; a coverage threshold lowered in the same diff that
230-
would otherwise fail it.
282+
would otherwise fail it; a composite score that **sums** heterogeneous
283+
constructs; a ranker validated with **MAE** instead of concordance, or a "±N"
284+
band wider than the decision range; a config/enum map never tested against a
285+
`SELECT DISTINCT` of real source values; a fanout/uniqueness gate that
286+
blanket-blocks a newly-shared standing value; new enrichment scoped before
287+
existing-source coverage was measured.

‎.claude/skills/idea-critic/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ description: >-
99
license: MIT
1010
metadata:
1111
author: deep-code-review contributors
12-
version: "1.72.0"
12+
version: "1.73.0"
1313
---
1414

1515
# Idea critic

‎.claude/skills/idea-critic/VERSION‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.72.0
1+
1.73.0

‎CHANGELOG.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,30 @@
33
All notable changes to this repository are documented here. Format loosely
44
follows Keep a Changelog; versioning follows Semantic Versioning.
55

6+
## [1.73.0] — 2026-09-16
7+
8+
Wave 2 of the #143–#164 dogfooding batch: six data-quality review lenses, all in
9+
`deep-code-review/references/data-quality.md`, each pre-distinguished from the
10+
section it extends.
11+
12+
### Added — deep-code-review
13+
- **`data-quality.md`** — six lenses:
14+
- a fanout/uniqueness gate false-blocks legitimate coverage expansion — hard-block
15+
only a value *absorbed from a now-departed distinct record*, not a newly-shared
16+
*standing* value among related entities (#149);
17+
- the resolution order when no stable id exists, and surfacing the unresolved
18+
count as a first-class output (#153a);
19+
- never sum heterogeneous constructs into one composite score (#162);
20+
- test every enum/config mapping against the source's real value distribution,
21+
not the literals a unit test feeds it (#153b);
22+
- backtest a proxy-derived metric against ground truth before shipping, and match
23+
the validation metric to the claim — an ordinal rank validated by concordance /
24+
a C-index, not MAE (#151, #161);
25+
- measure existing-source coverage before scoping new enrichment/scrapers, and
26+
scope to the measured residual (#152).
27+
- Plus new 🚩 red-flag detectors and a Cyrillic-to-Latin typo fix.
28+
- Six new evals (deep-code-review 45 → 51).
29+
630
## [1.72.0] — 2026-09-15
731

832
Wave 1 of a new dogfooding batch (#143–#164): five field lenses from peer

0 commit comments

Comments
 (0)