Skip to content

ansiescape: cover all control-sequence introducers, not just CSI/OSC#1980

Open
WatchTree-19 wants to merge 1 commit into
NVIDIA:mainfrom
WatchTree-19:ansiescape-introducer-coverage
Open

ansiescape: cover all control-sequence introducers, not just CSI/OSC#1980
WatchTree-19 wants to merge 1 commit into
NVIDIA:mainfrom
WatchTree-19:ansiescape-introducer-coverage

Conversation

@WatchTree-19

Copy link
Copy Markdown

Fixes #1977. Supersedes #1978 — that PR was closed after I force-pushed a malformed commit to its branch (an --amend inside a shallow clone produced a parentless commit that appeared to touch 811 files). The branch is now correct: one commit, three files, on top of main.

The gap

Both ansiescape detectors built their needles from a hardcoded suffixes = {"[", "]"}, so only CSI and OSC were recognised. ESC c (RIS) and the DCS/SOS/PM/APC string controls — plus their C1 single-byte forms 0x90/0x98/0x9e/0x9f — scored 0.0 in both Raw and Escaped. APC is not theoretical: it is live in the wild via the kitty graphics protocol.

As #1977 notes, this is a coverage boundary rather than a live false negative — all seven existing LIVE_PAYLOADS still score 1.0 today. The nine previously-uncovered sequences (RIS, DCS, SOS, PM, APC and the four C1 forms) go from 0.0 to 1.0.

Why it isn't just a wider suffix list

Widening the list naively introduces false positives, which is the substantive part of this change. StringDetector matches case-insensitively (case_sensitive defaults to False), so crossing the alphabetic introducers with the shell-style \e spelling produces the needles \ec, \ep and \ex — which match benign text like \echo, \epsilon and \export. I hit this via a failing test rather than by inspection.

So the introducers are split:

  • INTRODUCERS ([, ], ^, _) — punctuation, safe to cross with every command spelling.
  • ALPHA_INTRODUCERS (c, P, X) — pinned for the Escaped detector against the unambiguous command spellings only, derived from ESCAPED_COMMANDS so they track future additions automatically. RAW_COMMANDS are control bytes rather than text and carry no such risk, so Raw takes the full set.

RIS

RIS is covered for detection and included in ESCAPED_PAYLOADS as literal text, but deliberately kept out of LIVE_PAYLOAD_TOKENS. Unlike every other entry there — all inert, incomplete introducers — ESC c is a complete two-byte terminal reset, and the module note is explicit that these payloads land in both garak and remote logs. Happy to add it if you'd rather have the coverage.

ST (0x9c), charset designation and DECALN remain out of scope per the issue.

Verification

  • tests/detectors/test_detectors_ansiescape.py and tests/probes/test_probes_ansiescape.py — 11 passed.
  • New tests pin each newly covered introducer in both raw and escaped form, assert the two detectors do not cross-fire, and hold benign text (including the \e-prefixed words above) at 0.0.
  • Existing raw/escaped assertions are unchanged; I re-verified the LIVE_PAYLOADSRaw-only / ESCAPED_PAYLOADSEscaped-only separation still holds with 0 violations, including when simulated against the widened ESCAPED_COMMANDS from ansiescape: detect alternative escaped ESC encodings (2-digit octal, curly/8-digit unicode) #1929.
  • black clean, DCO signed off.

Note on overlap with #1929

This touches ansi.py and test_detectors_ansiescape.py, as does my open #1929. The two are orthogonal axes — #1929 widens the ESC encoding set (ESCAPED_COMMANDS), this widens the introducer set — and they compose, since needles are the cross-product. I checked that combination explicitly for false positives (clean, above). Whichever lands first, I'll rebase the other.

The ansiescape detectors built their needles from a hardcoded
suffixes = {[, ]}, so only CSI and OSC were recognised. ESC c (RIS)
and the DCS/SOS/PM/APC string controls -- plus their C1 single-byte forms
0x90/0x98/0x9e/0x9f -- scored 0.0 in both Raw and Escaped, leaving the
module narrower than the attack class its docstring describes. APC in
particular is live in the wild via the kitty graphics protocol.

Widening the suffix list naively introduces false positives, so the
introducers are split in two:

- INTRODUCERS ([, ], ^, _) are punctuation and safe to cross with every
  command spelling.
- ALPHA_INTRODUCERS (c, P, X) are not. StringDetector matches
  case-insensitively (case_sensitive defaults to False), so crossing them
  with the shell-style \e spelling yields the needles \ec, \ep and \ex,
  which match benign text such as \echo, \epsilon and \export. For the
  Escaped detector these are instead pinned against the unambiguous
  command spellings, derived from ESCAPED_COMMANDS so they track any
  future additions. RAW_COMMANDS are control bytes rather than text and
  carry no such risk, so Raw takes the full set.

RIS is covered for detection and appears in ESCAPED_PAYLOADS as literal
text, but is deliberately kept out of LIVE_PAYLOAD_TOKENS: unlike every
other entry there it is a complete two-byte terminal reset rather than an
inert introducer, and per the module note these payloads land in both
garak and remote logs.

ST (0x9c), charset designation and DECALN remain out of scope.

Tests pin each newly covered introducer in both raw and escaped form,
assert the two detectors do not cross-fire, and keep benign text --
including the \e-prefixed words above -- at 0.0. The pre-existing
LIVE/ESCAPED payload separation is unchanged.

Signed-off-by: WatchTree-19 <119982314+WatchTree-19@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ansiescape: detector and payloads cover only CSI and OSC introducers

1 participant