Skip to content

Commit 285af52

Browse files
G6: personal-data leads and report-contract lint fixes (#2228)
Adds pii_data_map.py, a report-only, stdlib-only lead finder for personal data kept or sent in the clear (JS/TS, Python, Go, Java, Kotlin source; test paths skipped). Rules: plaintext-national-id (quoted SSN-shaped or national-id-keyed literals, placeholders skipped), unencrypted-national-id-column (Python model fields with no encrypt/hash/vault/mask), and raw-pii-to-third-party (an email or phone passed to an analytics or tracking host without hashing or masking). Rows never print the value. Wired into review_checks.sh and the security-review mode as a lead source that still needs a code read. report_contract_lint.py: - chunked drafts (--part N / --done) so long reports avoid one giant heredoc; the side files are opened O_NOFOLLOW, must be owned by the caller and private, parts must arrive in order, and --done removes them; - the object-level authorization label maps to the frozen scorer's class; - citation rule: a token counts as a file citation when it has an extension or line anchor, or its first path segment exists under the root; empty and ".." prefixes are rejected. route_auth_map.py: --all now cites a Django route at its resolved view's def line instead of the urls.py line. The size budget for references/security-review-mode.md is raised in the changelog fragment with the reason. Tests: 70/70 (pii_data_map 15, report_contract_lint 41, route_auth_map 8, csrf_cookie_map 6) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 6673494 commit 285af52

19 files changed

Lines changed: 525 additions & 35 deletions

‎.claude/skills/INDEX.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,7 @@ Run `python3 <skill>/scripts/<name>` (`bash` for `.sh`). `name: purpose — argu
149149
- outline.py: cheap symbol map (line ranges) so a lane reads only the ranges… — FILE...
150150
- parity_differ.py: the two-sided design<->app parity gate: sections first, then… — --design <file> --app <file> [--accept…
151151
- partition_diff.py: split a unified diff into whole-file review chunks… — [--diff FILE | --base REF]…
152+
- pii_data_map.py: heuristic lead-finder for personal data kept or sent in the… — DIR
152153
- post_review.sh: post review findings to a PR (opt-in). — <PR-number> <findings.json> [--post]
153154
- provision_tools.sh: make review_checks.sh scanners actually run. — [--print] [DIR]
154155
- report_contract_lint.py: Lint a whole-project security audit report against the G0… — REPORT.md --root REPO (REPORT - reads…

‎.claude/skills/deep-code-review/INDEX-scripts.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,11 @@ Generated by `scripts/skill_index.py` (do not hand-edit). The script half of `IN
3434
| `scripts/parity_differ.py` | the two-sided design<->app parity gate: sections first, then the element inventory of every matched… | parity_differ.py --design <file> --app <file> [--accept <tsv> [--accept-rev REV]] [--min-pairs N] [--style [--style-min… |
3535
| `scripts/partition_diff.py` | split a unified diff into whole-file review chunks, deterministically. | partition_diff.py [--diff FILE \| --base REF] [--max-lines N] [--split-hunks] [--skip-glob GLOB ...] [--out-dir DIR] par… |
3636
| `scripts/perf_compare.sh` | perf_compare.sh [--runs N] [--base REF] [--json FILE] -- CMD...: hyperfine A/B of CMD in a detached… | — |
37+
| `scripts/pii_data_map.py` | heuristic lead-finder for personal data kept or sent in the clear (CWE-312/359/200). | python3 -I pii_data_map.py DIR |
3738
| `scripts/post_review.sh` | post review findings to a PR (opt-in). | post_review.sh <PR-number> <findings.json> [--post] |
3839
| `scripts/provision_tools.sh` | make review_checks.sh scanners actually run. | provision_tools.sh [--print] [DIR] |
3940
| `scripts/reaper_lint.py` | a heuristic, opt-in lint that flags a bulk/fleet-wide process-reaper pattern in a cleanup script be… | — |
40-
| `scripts/report_contract_lint.py` | Lint a whole-project security audit report against the G0 report contract. | report_contract_lint.py REPORT.md --root REPO (REPORT - reads stdin) Template and rules: references/whole-project-repor… |
41+
| `scripts/report_contract_lint.py` | Lint a whole-project security audit report against the G0 report contract. | report_contract_lint.py REPORT.md --root REPO (REPORT - reads stdin) report_contract_lint.py --part N --root REPO <chun… |
4142
| `scripts/review_checks.sh` | run local, read-only analyzers over the files changed since REF (default origin/main, compared as R… | review_checks.sh [--base REF] [--tests] [--history] [--full] [--hygiene] [--timeout SECS] |
4243
| `scripts/review_feedback.py` | repo-local accept/dismiss ledger for review findings. | — |
4344
| `scripts/route_auth_map.py` | heuristic lead-finder for HTTP handlers with no authentication evidence (CWE-306/862). | route_auth_map.py [--paths \| --all] DIR |

‎.claude/skills/deep-code-review/INDEX.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ Generated by `scripts/skill_index.py` (do not hand-edit; rerun it after changing
126126
| `references/security-ai-agents.md` | the code calls an LLM, embeds/retrieves (RAG), or runs an agent that plans, calls tools, executes c… | 9786 |
127127
| `references/security-api.md` | the target serves its own HTTP, GraphQL, gRPC, or WebSocket API (always for api / service) | 4361 |
128128
| `references/security-appsec.md` | Read when the target exposes any network surface (web app, API, service, CLI touching untrusted inp… | 7061 |
129-
| `references/security-review-mode.md` | the user asks for a security audit or security review of a whole app (not a diff). | 3038 |
129+
| `references/security-review-mode.md` | the user asks for a security audit or security review of a whole app (not a diff). | 3354 |
130130
| `references/skill-authoring-and-size.md` | the target repository ships or installs agent skills (/SKILL.md, .claude/skills/, .cursor/skills/,… | 991 |
131131
| `references/small-diff-read-budget.md` | reviewing a small DIFF (the fast path in SKILL.md), before the first read or command. | 131 |
132132
| `references/standards-index.md` | citing a standard URL or verification date from an installed copy of the skill. | 117 |
@@ -149,8 +149,8 @@ Generated by `scripts/skill_index.py` (do not hand-edit; rerun it after changing
149149
| `references/ux-writes.md` | the target or diff writes from the UI: an optimistic mutation whose failure path rolls back, a load… | 2949 |
150150
| `references/web-fetch.md` | the target or diff fetches data for a view: dependent requests on the render-critical path, a share… | 5710 |
151151
| `references/web-render.md` | the target or diff renders a list or grid of rows / tiles, adds or removes React.memo / useMemo / u… | 4731 |
152-
| `references/whole-project-report.md` | writing a whole-project security audit report, or when scripts/report_contract_lint.py rejects one. | 1041 |
152+
| `references/whole-project-report.md` | writing a whole-project security audit report, or when scripts/report_contract_lint.py rejects one. | 1193 |
153153

154154
## Scripts
155155

156-
45 scripts. Read `INDEX-scripts.md` when you need a script (a trigger and CLI per script). `../INDEX.md`, `### deep-code-review`, has only each one-line purpose and arguments.
156+
46 scripts. Read `INDEX-scripts.md` when you need a script (a trigger and CLI per script). `../INDEX.md`, `### deep-code-review`, has only each one-line purpose and arguments.

‎.claude/skills/deep-code-review/SKILL.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,8 @@ You are a senior reviewer hunting defects a change introduces. For a small DIFF
2121
4. One finding per distinct mechanism, most severe first, at most 8.
2222
5. Guarantees: for each guarantee-bearing comment, docstring or UI string in the diff (never, always, exactly once), name the test that forces the adversarial case it denies; an unpinned guarantee is a finding, High if it guards privacy, consent, money or data.
2323

24-
Whole-app security audit (not a diff): read only `references/security-review-mode.md`; it routes the rest. Security review of a small DIFF: read only `references/appsec-verify.md` (section 0 first), not full-map.
24+
Whole-app security audit: read only `references/security-review-mode.md`. Security review of a small DIFF: read only `references/appsec-verify.md` (section 0 first), not full-map. Paths are in this skill's base dir, not the repo.
2525

2626
BUDGET (small DIFF): read `references/small-diff-read-budget.md` first.
2727

28-
OUTPUT. The severity-ranked findings, each as `file:line`, severity, mechanism and failing input in one or two sentences. Report template: `references/full-map-report.md`.
28+
OUTPUT. Severity-ranked findings, each `file:line`, severity, mechanism and failing input in 1-2 sentences. Template: `references/full-map-report.md`.

‎.claude/skills/deep-code-review/references/security-review-mode.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,11 @@ Read this when the user asks for a security audit or security review of a whole
66

77
## 1. Whole-app mode: inventory first, then chains
88

9-
1. **Tool pass, then entry inventory.** First run `bash <skill dir>/scripts/review_checks.sh --full` from the repo root (`<skill dir>` is the parent of this `references/` folder) (`tool-hub.md` explains its rows); each `not run:` tool goes under NOT RUN with its reason, and a status it backs cites `tool:<name>`. Then build the entry-point table from `full-audit.md` section 1 (routes, actions, jobs, consumers, webhooks), then add the entries it omits for a security review: CLI and admin scripts, CI workflows and their triggers, agent tools and MCP servers, file-upload and import paths, third-party callbacks, and any host-platform integration that can deploy or read config. List every route from `python3 -I <skill dir>/scripts/route_auth_map.py --all <repo>` (Go, Spring, Django, Next, Express, Flask, FastAPI) in the route inventory, one line each. The list is heuristic: unresolved constants (`{?}`) and includes can leave routes out, so cross-check with a grep of the framework's route annotations or registrations.
9+
1. **Tool pass, then entry inventory.** Run every command in this mode alone, with no `;`, `&&`, `for` loop, `cd X &&` chain or `VAR=` prefix, because restricted permission modes deny compound commands; a denied command is retried as single commands and is never a reason to skip the tool pass. First run `bash <skill dir>/scripts/review_checks.sh --full` from the repo root (`<skill dir>` is the parent of this `references/` folder) (`tool-hub.md` explains its rows); each `not run:` tool goes under NOT RUN with its reason, and a status it backs cites `tool:<name>`. When `review_checks.sh` cannot run, run each lead script that `INDEX-scripts.md` lists with a DIR usage directly, one `python3 -I <skill dir>/scripts/<name> .` command each, and list `review_checks.sh` under NOT RUN. Then build the entry-point table from `full-audit.md` section 1 (routes, actions, jobs, consumers, webhooks), then add the entries it omits for a security review: CLI and admin scripts, CI workflows and their triggers, agent tools and MCP servers, file-upload and import paths, third-party callbacks, and any host-platform integration that can deploy or read config. List every route from `python3 -I <skill dir>/scripts/route_auth_map.py --all <repo>` (Go, Spring, Django, Next, Express, Flask, FastAPI) in the route inventory, one line each. The list is heuristic: unresolved constants (`{?}`) and includes can leave routes out, so cross-check with a grep of the framework's route annotations or registrations.
1010
2. **Asset inventory.** One row each: data store and its sensitive columns, every secret and where it is stored and read (env var, CI secret, secret manager, file), signing keys, session store, third-party accounts the app can act on, and the identities (human, service, CI, agent) that can reach each. Mark the crown jewels: what an attacker most wants (money movement, other users' data, deploy rights).
1111
3. **Boundary pass.** Run `threat-modeling.md` over the inventory (data-flow table, per-boundary threats), then the matching domain files by archetype from `full-map.md`; do not re-derive their checklists here.
1212
4. **Chain, do not just list.** After domain findings exist, build attack paths: write each finding as `precondition -> capability gained`, then join findings where one's gain satisfies another's precondition. Typical joins to test: an SSRF or file-read that reaches a secret store or cloud metadata; a low-privilege account plus an object-level auth gap; XSS or an open redirect plus a token in a URL; a CI trigger that runs untrusted code plus a write-scoped token; an agent that reads untrusted content plus a tool that sends data out or executes. Report the path as one finding at the severity of its end state, citing each link with `file:line`, and name the cheapest link to cut. A chain of Low findings that ends in a crown jewel is High.
13-
5. **Close with the gaps below** (sections 2 to 6), then **read** `whole-project-report.md` and write the report in its template (per-standard table, NOT CHECKED, NOT RUN, static vs runtime, dependencies, under those exact headings), plus a `## Findings` section in the one-line-per-finding format `whole-project-report.md` defines (class label, one primary `path:line`), in `report-format.md` order with an attack-path section ahead of the flat list. **Before emitting**, pipe the draft to `python3 <skill dir>/scripts/report_contract_lint.py - --root .` and fix every violation.
13+
5. **Close with the gaps below** (sections 2 to 6), then **read** `whole-project-report.md` and write the report in its template (per-standard table, NOT CHECKED, NOT RUN, static vs runtime, dependencies, under those exact headings), plus a `## Findings` section in the one-line-per-finding format `whole-project-report.md` defines (class label, one primary `path:line`), in `report-format.md` order with an attack-path section ahead of the flat list. **Before emitting**, lint the draft by heredoc to stdin in ONE Bash command: `python3 -I <skill dir>/scripts/report_contract_lint.py - --root . <<'EOF'` then the draft, then `EOF`; keep each heredoc under 8,000 chars (the bash parser denies ~10,000+), so send a longer draft as `--part N` chunks split at line boundaries and sent in order and numbered from 1 (`... report_contract_lint.py --part 1 --root . <<'EOF'`, each prints the line count), then `... report_contract_lint.py --done --root .` to lint the joined draft. `--done` always deletes the draft, so after a failing `--done` fix the draft and resend it from `--part 1`. Never write the draft to a file in the repo (read-only audits deny Write and `cat >`); the single-command rule of step 1 applies; fix every printed line and re-run until exit 0. After exit 0 the final answer is that linted draft verbatim: never re-word, re-section, drop `[class]` labels, or give a summary-only answer.
1414

1515
## 2. Host-platform breach and secret-rotation runbook
1616

‎.claude/skills/deep-code-review/references/whole-project-report.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,9 @@ Needs a running system: deployed config, WAF, runtime authz, vault, infra outsid
3131

3232
A security audit (`security-review-mode.md`) has a `## Findings` section with one finding per top-level list item: `[<class>] <repo-relative path>:<line> <severity>: <text>`. The path is the vulnerable handler or line, never a gate, helper or config used as evidence; give exactly one `path:line`, and put evidence cites after `evidence:` in the text or in other sections. Never cite a bare file name. A chain finding cites its end-state line as the primary and the other links after `evidence:`. Indented continuation lines, prose, and subsections whose heading names an attack path or chain are not linted.
3333

34-
One finding per vulnerable site, never a group or a range: each state-changing cookie-session handler without a CSRF defence (`csrf`, anchored on the line that commits the change, not the signature), each session-cookie set call missing a flag (`cookie-flags`, including a cookie that sets HttpOnly but not Secure or SameSite), each unauthenticated handler (`missing-authn`). A GET that sets a cookie is not `csrf`. The per-standard Files cell and the primary `path:line` list vulnerable sites only: a gate, middleware, interceptor, filter, helper or token table you read as evidence goes after `evidence:`, and a route a verified global gate covers is not `missing-authn`. Each `csrf-cookie` row from `review_checks.sh --full` is a lead: verify it, then file it at its own line.
34+
One finding per vulnerable site, never a group or a range: each state-changing cookie-session handler without a CSRF defence (`csrf`, anchored on the line that commits the change, not the signature), each session-cookie set call missing a flag (`cookie-flags`, including a cookie that sets HttpOnly but not Secure or SameSite), each unauthenticated handler (`missing-authn`). `missing-authn`, `missing-authz` and `object-level-authz` anchor on the route's declaration line (the line `route_auth_map.py --all` prints), not the read or write inside the body; the commit-line anchor is `csrf` only. A GET that sets a cookie is not `csrf`. Each `pii-data` row of `review_checks.sh --full` (tool `pii-data`: a plaintext national id literal, an unencrypted national-id column, raw email or phone sent to an analytics host) is one `pii-data` finding at its line. The per-standard Files cell and the primary `path:line` list vulnerable sites only: a gate, middleware, interceptor, filter, helper or token table you read as evidence goes after `evidence:`, and a route a verified global gate covers is not `missing-authn`. Each `csrf-cookie` row from `review_checks.sh --full` is a lead: verify it, then file it at its own line. Each `missing-attempt-limit` or `missing-cost-limit` row of `review_checks.sh --full` is a lead: verify it, then file one `rate-limit` finding at its own line.
3535

36-
Fixed class vocabulary: `missing-authn`, `missing-authz`, `object-access`, `ssrf`, `tls-off`, `outbound-timeout`, `secret-in-url`, `secret-in-log`, `hardcoded-secret` (credentials or tokens in source), `csrf`, `cookie-flags`, `rate-limit`, `pii-in-log`, `pii-data` (personal data committed, stored in plaintext or sent to a third party), `prompt-injection` (including tool-call JSON built from input), `injection`, `xss`, `other`. Pick the closest class; use `other` only when none fits.
36+
Fixed class vocabulary: `missing-authn`, `missing-authz`, `object-level-authz`, `ssrf`, `tls-off`, `outbound-timeout`, `secret-in-url`, `secret-in-log`, `hardcoded-secret` (credentials or tokens in source), `csrf`, `cookie-flags`, `rate-limit`, `pii-in-log`, `pii-data` (personal data committed, stored in plaintext or sent to a third party), `prompt-injection` (including tool-call JSON built from input), `injection`, `xss`, `other`. Pick the closest class; use `other` only when none fits.
3737

3838
Example: `- [pii-data] src/api/users.py:42 high: serves SSNs and dates of birth unmasked. evidence: src/models.py:10`
3939

0 commit comments

Comments
 (0)