probes: per-payload-group intent association (draft, refs #1808)

[immu4989]

Draft, sketching one of the options discussed in #1808 to make the tradeoffs concrete. Happy to rework into whichever shape you and @patriciapampanelli prefer, or to close it if you would rather settle the pattern in the issue first.

Targets feature/technique_intent per @jmartin-tech's note on #1808, rather than main.

What this changes

A probe configured with more than one payload group keeps only the first group's intent:

if _pg.intent and not hasattr(self, "_payload_intent"):
    self._payload_intent = _pg.intent

Every attempt then reports that one intent, including prompts built from later groups.

• Probe._stubs_from_payloads() loads payload groups into intent-bearing TextStubs, one per payload entry, each carrying the intent of the group it came from.
• Probe._intent_for_seq() resolves the intent for an attempt: a per-prompt intent from self._prompt_intents when the probe supplies one aligned with self.prompts, otherwise the probe-wide _payload_intent, otherwise the class-level intent. _mint_attempt calls it.
• Probe._prune_data() prunes _prompt_intents alongside self.prompts.
• web_injection.MarkdownXSS migrated onto the helper as a worked example.

The aligned prompts / prompt_intents contract is the one donotanswer and IntentProbe already use. This moves the consumption into the base class so a probe no longer has to override _attempt_prestore_hook just to get per-prompt intents. Existing probes are unaffected: without _prompt_intents, resolution falls through to current behaviour.

Two things worth a look

1. Probe._prune_data deleted from self.prompts but not from _prompt_intents, so any probe adopting the aligned-list convention would get silently shifted intents once the prompt cap applies. IntentProbe already overrides _prune_data to avoid this. Fixed in the base here, with a test.
2. TextStub(intent=..., content=...) does not work: content is a property while _content is the dataclass field, so construction needs the private name or a post-construction assignment. Worth smoothing if Stub becomes the contributor-facing contract.

Verification

• Run the tests and ensure they pass python -m pytest tests/probes/

  1793 passed. The single failure, probes.audio.AudioAchillesHeel, is a pre-existing missing optional dependency (soundfile, librosa) and fails identically on an unmodified checkout.

• Verify the thing does what it should

  python -m pytest tests/probes/test_probes_web_injection.py adds 6 tests covering per-group intent retention, the intent recorded on each minted attempt, prune alignment, and unchanged single-group behaviour.

• Verify the thing does not do what it should not

  Stashed the source changes and re-ran the new tests against an unmodified checkout: 5 of the 6 fail. The behavioural one shows every attempt reporting S008inject (the first payload group) where S005hate (the second) was expected.

• Document the thing and how it works

  Docstrings on _stubs_from_payloads and _intent_for_seq describe the contract and the fallback order.

• Supporting configuration such as a generator configuration file

  Not applicable, no generator configuration is needed.

• garak -t <target_type> -n <model_name>

  Not applicable, the change is in prompt construction and attempt minting, covered by unit tests.

[immu4989]

Pushed a second commit extending this to the other half of the pattern question from #1808, the file-based path that doesn't go through payloads.load().

donotanswer carried its own _attempt_prestore_hook purely to copy per-entry intents onto attempts. With the base resolving _prompt_intents, that hook is redundant, so it is deleted here and its loader builds the same stubs the payload path does. Both sources now look the same at the call site: build stubs, unpack into prompts and _prompt_intents, and the base does the rest. That was the part I was least sure about in the issue, so it seemed worth making concrete rather than describing.

One piece of data for the multi-intent question (question 2 in my issue comment): across the five donotanswer files, 144 of 939 prompts carry more than one intent code, and intents[0] discards the rest.

category	entries	multi-intent
discrimination_exclusion_toxicity_hateful_offensive	176	15
human_chatbox	117	26
information_hazard	248	14
malicious_uses	243	43
misinformation_harms	155	46

I have not changed that behaviour here, since whether a prompt carries one intent or a set is the design call I was asking about. Flagging the size of it in case it affects the answer.

pytest tests/probes/ is now 1808 passed, with the same single pre-existing probes.audio.AudioAchillesHeel failure from the missing soundfile / librosa optional dependency.