Repository navigation
Bench: seed six web CWE classes and harden the Python taint guards - #2386
Merged
Merged
Conversation
… ceiling to 26 The CI real-hub job measured 11/11 recall and 26 false positives on the clean twins after the CWE-22 taint rule stopped firing on send_from_directory's file name. Both values are now frozen so a regression in any seeded class fails CI. Tests: 2/2 (py_compile; CI real-hub job on the merged rule measured recall 1.0, 26 false positives) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds CWE-79, 918, 611, 601, 295 and 347 keys; fixtures follow in the next commit. Tests: n/a (answer key only; scored in the next commit) No-Mechanism-Reason: answer key data, no skill prose changed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fictional XSS, SSRF, XXE, open-redirect, TLS-verify and JWT-verify fixtures. Local run without the full hub: 14/17 recall, 15 false positives; CWE-918, 611 and 601 missed locally because their rules also fire on the clean twins. Tests: 1/1 (REQUIRE_REAL_HUB=0 bench_tool_only.py run locally; FLOOR_RECALL unchanged, CI measures) No-Mechanism-Reason: bench fixtures only, no skill prose changed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…xml XXE rule
perun.py.ssrf skips sinks after `if url not in ALLOWED: abort(...)`.
perun.py.open-redirect skips sinks after the startswith("/")/("//") reassign guard
and after url_has_allowed_host_and_scheme guards. New perun.py.xxe-lxml (CWE-611)
flags lxml XMLParser(resolve_entities=True) and no_network=False.
Tests: 3/3 (test_semgrep_py_guards, test_semgrep_py_pathtrav, test_semgrep_defaults)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SSRF guard now requires the guarded value to be the URL argument itself and the container not to be a string literal. Redirect guard also requires a "/\" check. xxe-lxml also flags load_dtd=True and huge_tree=True. New negative fixtures and offline pins for each. Tests: 3/3 (test_semgrep_py_guards, test_semgrep_py_pathtrav, test_semgrep_defaults) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ng at 31 CI real-hub flagged ruff I001 and PIE810 on the new clean ssrf/redirect twins; fix them and set the ceiling to 31 for the 5 remaining registry/semgrep FPs on guarded code. Tests: 2/2 (ruff --select I001,PIE810 on both twins clean; bench_tool_only locally FP 13 <= 31, local recall 0.941 reflects missing semgrep, CI real-hub is authoritative) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ling at 32 The tuple rewrite of the redirect clean twin broke its guard, so the real-hub job missed CWE-601 and flagged a new false positive. Tests: 1/1 (py_compile scripts/bench_tool_only.py) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The tool-only bench now seeds CWE-79 (Express reflected XSS), CWE-918 (SSRF), CWE-611 (lxml XXE), CWE-601 (open redirect), CWE-295 (requests verify=False) and CWE-347 (PyJWT signature verification disabled), each with a clean twin, and the answer key lists all six.
Rule changes in the Python Semgrep pack:
New test scripts/test_semgrep_py_guards.py (11 ruleid cases, 6 ok cases) is wired through scripts/ci-tests.d/semgrep-py-guards.sh.
Tests: 3/3 (test_semgrep_py_guards, test_semgrep_py_pathtrav, test_semgrep_defaults; local bench recall 1.0 on every new class)
🤖 Generated with Claude Code