Skip to content

Report lint sink rules; G6 set2 result - #2230

Merged
remigiusz-antczak merged 1 commit into
mainfrom
lane/g6-land
Nov 4, 2026
Merged

remigiusz-antczak merged 1 commit into
mainfrom
lane/g6-land

Conversation

@remigiusz-antczak

Copy link
Copy Markdown
Collaborator

Adds two report_contract_lint.py rules aimed at the false positives found when a blind data/API set was scored and adjudicated. Most of those false positives cited the route or helper line instead of the sink, or used the wrong class.

  • finding-not-on-sink: a Findings row for an outbound class (ssrf, outbound-timeout, tls-off, secret-in-url), a log class (secret-in-log, pii-in-log) or cookie-flags is flagged when no line within two lines of the cite shows the matching sink. The sinks are an outbound call, URL or TLS option; a log or print call; or a cookie or session config.
  • class-mismatch: a pii-in-log row is flagged when it logs a credential-named value and no personal-data word (use secret-in-log). It is also flagged when the cited line is an outbound call with no log call nearby (use pii-data).

whole-project-report.md now tells the reviewer to cite the sink line. One existing fixture changed its class from SSRF to XSS so it keeps testing the rule it was written for.

This PR also lands the set2 blind fixtures (docs/bench/g6-data-api/set2/) and the scored, adjudicated result: FAIL, 3 of 8 parts. Recall passes in all 8 parts and every failure is precision. Set2 is burned. The raw and adjudicated score.json, the adjudication input and verdicts, and the scored event in ledger.jsonl are included.

Tests 89/89: lint 48 (41 existing plus 7 new sink and class tests), bench 41. Privacy, size-ratchet, changelog-fragment and SHA256SUMS gates pass locally. An independent review was done and its folds verified.

🤖 Generated with Claude Code

…scored result

report_contract_lint.py gains two rules aimed at the false positives found when blind data/API set2 was scored and adjudicated. Most of those false positives cited the route or helper line instead of the sink, or used the wrong class.

finding-not-on-sink flags a Findings row for an outbound class (ssrf, outbound-timeout, tls-off, secret-in-url), a log class (secret-in-log, pii-in-log) or cookie-flags when no line within two lines of the cite shows the matching sink. class-mismatch flags a pii-in-log row in two cases: when it logs a credential-named value and no personal-data word (use secret-in-log), and when the line is an outbound call with no log call nearby (use pii-data). whole-project-report.md now tells the reviewer to cite the sink line.

This also lands the set2 blind fixtures and the scored, adjudicated result: FAIL, 3 of 8 parts. Recall passes in all 8 parts and every failure is precision. Set2 is burned.

Tests: 89/89

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@remigiusz-antczak
remigiusz-antczak merged commit 2b5ef2b into main Nov 4, 2026
11 checks passed
@remigiusz-antczak
remigiusz-antczak deleted the lane/g6-land branch November 4, 2026 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant