diff --git a/DESIGN.md b/DESIGN.md index abaf35c..e31c348 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -2,7 +2,7 @@ A Go library for authoring coding-agent hooks once and running them everywhere: Claude Code, Cursor (IDE + CLI + cloud), OpenAI Codex, Gemini CLI, OpenCode, -Kimi Code, and GitHub Copilot CLI. +OpenClaw, Kimi Code, GitHub Copilot CLI, and Copilot Chat in VS Code. The core promise: **one clear interface, zero data-fidelity loss**. The library owns the per-provider glue, hacks, and workarounds so consumers don't have to. @@ -44,6 +44,20 @@ a crashed hook from denying every tool call. `prompt.submitted` gets an empty capability set because Copilot discards command-hook output for `userPromptSubmitted`.) +(**Copilot Chat in VS Code** is a ninth provider and a *second Copilot +dialect*, not a surface of the first: its wire shape is Claude Code's — +snake_case stdin, PascalCase event names, `hookSpecificOutput` response — so +both codecs delegate to the Claude ones and the distinct constant exists for +the capability row and the config renderer. It fires 8 of the CLI's 14 events, +parses `matcher` values and then ignores them, and injects no environment +marker into the hook child, so detection is flag-only. Two placements diverge +from Claude Code and are fixed up in `encodeVSCode`: `decision`/`reason` ride +*inside* `hookSpecificOutput` on `Stop`/`SubagentStop`, and `SubagentStart` +honors `additionalContext`, which Claude Code does not. The capability row was +read out of the extension source rather than the reference, which contradicts +itself on both the nesting and the timeout key — see quirk registry entries +#42–#49.) + Consequence: the unified contract should be **Claude-shaped semantics with typed extensions**, not a lowest-common-denominator invention. Three of five providers natively converge on it; Cursor half-converges; only OpenCode needs a @@ -610,6 +624,19 @@ Per-target rendering encodes the workaround knowledge: `command` is emitted, never `bash`/`powershell`: Copilot copies `command` into both when absent. No `failClosed` knob either — Copilot fixes the posture per event (`preToolUse` fail-closed, everything else fail-open). +- **VS Code Copilot Chat**: `agenthooks-vscode.json` with PascalCase event + keys, at `<~/.copilot>/hooks/` (user) or `.github/hooks/` (project); no + plugin scope. Both directories are globbed by the Copilot CLI *as well*, so + the distinct basename is what keeps this file from colliding with the CLI's + — and, being neither `settings.json` nor `hooks.json`, it is whole-file + owned rather than merged. Four omissions, each preventing a silent failure: + no `matcher` (VS Code parses matchers and ignores them, so `--filter` on the + argv is the only real enforcement), no `version` (absent from every VS Code + example; an unknown key is a schema-validation risk for zero benefit), no + `bash`/`powershell` (VS Code's split is `windows`/`linux`/`osx`, and the + rendered argv is valid in every shell), and **both** `timeout` and + `timeoutSec` at the same value, because the reference table names one and a + usage example on the same doc set names the other. Matchers: `ToolMatcher` compiles to the provider dialect where expressible (Claude regex/exact-list rules incl. the hyphen/comma version gates, Gemini @@ -690,7 +717,8 @@ upstream reference. Seeded from provider research and production observation: The table below is the initially seeded set; `quirks.go` is the authoritative registry and has grown past it (entries #21+, including the OpenClaw rows -#34–#37). +#34–#37, the Copilot CLI dialect rows #38–#41, and the VS Code Copilot Chat +rows #42–#49). | # | Quirk | Mitigation | |---|---|---| diff --git a/README.md b/README.md index 97cf0af..4894d3d 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@

agenthooks

-

Author coding-agent hooks once in Go; run them on Claude Code, Cursor, OpenAI Codex, Gemini CLI, OpenCode, Kimi Code, OpenClaw, and GitHub Copilot CLI.

+

Author coding-agent hooks once in Go; run them on Claude Code, Cursor, OpenAI Codex, Gemini CLI, OpenCode, Kimi Code, OpenClaw, GitHub Copilot CLI, and Copilot Chat in VS Code.

Go Doc @@ -210,9 +210,22 @@ config, and an absent one already means match-all. ambiguous or unrecoverable matches stay empty. Disable with `WithoutMCPResolution()` (everything) or `WithoutMCPListFallback()` (provider CLI probes). -- Copilot hooks are **CLI-only** (the IDE surfaces fire nothing) and the - dialect needs the most repair: most payloads omit their own event name, so - the codec reconstructs it from the payload shape — the shapes are disjoint, +- Copilot ships **two dialects behind one name**: the CLI's camelCase wire + (`copilot-cli`) and Copilot Chat in VS Code, which speaks the Claude-shaped + dialect (`vscode-copilot`). Both runtimes glob the same two hook + directories, so the `vscode-copilot` file serves both — its PascalCase + event keys are the CLI's Claude-compat mode, and the CLI's own `COPILOT_*` + env demotes the session to `copilot-cli` before decode, so each runtime gets + its own capability row and response schema (VS Code nested, the CLI flat). + Install `vscode-copilot` for both runtimes (8 events) or `copilot-cli` for 12 + mapped CLI events; the CLI's two other native events currently decode only as + `KindOther` and cannot be registered by generated configs. Installing both + targets double-fires in the CLI. Cross-runtime + registration is verified against Copilot CLI 1.0.81 for the five events a + headless turn can drive; `SubagentStart`, `SubagentStop` and `PreCompact` + are unmeasured there. +- The CLI dialect needs the most repair: most payloads omit their own event + name, so the codec reconstructs it from the shape — the shapes are disjoint, so this is exact. `toolArgs` is a JSON-encoded *string* on `pre`/`postToolUse` while `permissionRequest` ships a plain object in `toolInput`; both normalize to an object. No tool-call id ships at all, so diff --git a/agenthooks.go b/agenthooks.go index 493bdb1..bb14863 100644 --- a/agenthooks.go +++ b/agenthooks.go @@ -313,6 +313,10 @@ func (r *Runner) Run(ctx context.Context, args []string, stdin io.Reader, stdout } provider, conf := detectProvider(inv, payload) + // The generated vscode-copilot file is loaded by the Copilot CLI too; when + // the CLI is the runtime, its own env demotes the flag so the session gets + // the CLI's capability row and flat response schema. + provider = demoteVSCodeToCLI(provider) if provider == "" { r.logger.Error("agenthooks: cannot detect provider; emitting neutral no-op", "payload_bytes", len(payload)) _, _ = fmt.Fprint(stdout, "{}") diff --git a/agenthookstest/agenthookstest.go b/agenthookstest/agenthookstest.go index 3d5d41e..8d8734f 100644 --- a/agenthookstest/agenthookstest.go +++ b/agenthookstest/agenthookstest.go @@ -66,6 +66,10 @@ func FixtureDir(p agenthooks.Provider) string { return "openclaw" case agenthooks.ProviderKimi: return "kimi" + case agenthooks.ProviderCopilotCLI: + return "copilot" + case agenthooks.ProviderVSCodeCopilot: + return "vscode" } return string(p) } diff --git a/agenthookstest/fixtures/claude/subagent_start.json b/agenthookstest/fixtures/claude/subagent_start.json new file mode 100644 index 0000000..600d105 --- /dev/null +++ b/agenthookstest/fixtures/claude/subagent_start.json @@ -0,0 +1 @@ +{"session_id":"sess-claude-1","transcript_path":"/tmp/claude/transcript.jsonl","cwd":"/work/repo","hook_event_name":"SubagentStart","agent_id":"agent-9","agent_type":"Explore"} diff --git a/agenthookstest/fixtures/copilot/post_tool_use.json b/agenthookstest/fixtures/copilot/post_tool_use.json index e5baeff..48241f3 100644 --- a/agenthookstest/fixtures/copilot/post_tool_use.json +++ b/agenthookstest/fixtures/copilot/post_tool_use.json @@ -1 +1 @@ -{"sessionId":"sess-copilot-1","timestamp":1786820437772,"cwd":"/work/repo","toolName":"bash","toolArgs":"{\"command\":\"echo hello-from-gram\",\"description\":\"Echo hello-from-gram\"}","toolResult":{"resultType":"success","textResultForLlm":"hello-from-gram\n"}} +{"sessionId":"sess-copilot-1","timestamp":1786820437772,"cwd":"/work/repo","toolName":"bash","toolArgs":{"command":"echo hello-from-gram","description":"Echo hello-from-gram"},"toolResult":{"resultType":"success","textResultForLlm":"hello-from-gram\n"}} diff --git a/agenthookstest/fixtures/copilot/post_tool_use_failure.json b/agenthookstest/fixtures/copilot/post_tool_use_failure.json index 64ed9bb..2ea16b4 100644 --- a/agenthookstest/fixtures/copilot/post_tool_use_failure.json +++ b/agenthookstest/fixtures/copilot/post_tool_use_failure.json @@ -1 +1 @@ -{"sessionId":"sess-copilot-1","timestamp":1786820662904,"cwd":"/work/repo","toolName":"view","toolArgs":"{\"path\":\"/work/repo/no-such-file-here.txt\"}","error":"Path does not exist"} +{"sessionId":"sess-copilot-1","timestamp":1786820662904,"cwd":"/work/repo","toolName":"view","toolArgs":{"path":"/work/repo/no-such-file-here.txt"},"error":"Path does not exist"} diff --git a/agenthookstest/fixtures/copilot/pre_tool_use.json b/agenthookstest/fixtures/copilot/pre_tool_use.json index e0a42f5..cccd0e0 100644 --- a/agenthookstest/fixtures/copilot/pre_tool_use.json +++ b/agenthookstest/fixtures/copilot/pre_tool_use.json @@ -1 +1 @@ -{"sessionId":"sess-copilot-1","timestamp":1786820437717,"cwd":"/work/repo","toolName":"bash","toolArgs":"{\"command\":\"echo hello-from-gram\",\"description\":\"Echo hello-from-gram\"}"} +{"sessionId":"sess-copilot-1","timestamp":1786820437717,"cwd":"/work/repo","toolName":"bash","toolArgs":{"command":"echo hello-from-gram","description":"Echo hello-from-gram"}} diff --git a/agenthookstest/fixtures/vscode/pre_tool_use.json b/agenthookstest/fixtures/vscode/pre_tool_use.json new file mode 100644 index 0000000..c9e1d52 --- /dev/null +++ b/agenthookstest/fixtures/vscode/pre_tool_use.json @@ -0,0 +1 @@ +{"timestamp":"2026-08-29T17:29:22.255Z","hook_event_name":"PreToolUse","session_id":"a1b2c3d4-e5f6-4a7b-8c9d-000000000001","transcript_path":"/work/vscode/workspaceStorage/0123456789abcdef0123456789abcdef/GitHub.copilot-chat/transcripts/a1b2c3d4-e5f6-4a7b-8c9d-000000000001.jsonl","tool_name":"read_file","tool_input":{"filePath":"/work/repo/README.md","startLine":1,"endLine":80},"tool_use_id":"call_01ABCDEFGHIJKLMNOPQRST__vscode-1700000000000","cwd":"/work/repo"} diff --git a/agenthookstest/fixtures/vscode/session_start.json b/agenthookstest/fixtures/vscode/session_start.json new file mode 100644 index 0000000..05c7cab --- /dev/null +++ b/agenthookstest/fixtures/vscode/session_start.json @@ -0,0 +1 @@ +{"timestamp":"2026-08-29T17:29:20.031Z","hook_event_name":"SessionStart","session_id":"a1b2c3d4-e5f6-4a7b-8c9d-000000000001","transcript_path":"/work/vscode/workspaceStorage/0123456789abcdef0123456789abcdef/GitHub.copilot-chat/transcripts/a1b2c3d4-e5f6-4a7b-8c9d-000000000001.jsonl","source":"new","model":"auto","cwd":"/work/repo"} diff --git a/agenthookstest/fixtures/vscode/stop.json b/agenthookstest/fixtures/vscode/stop.json new file mode 100644 index 0000000..136a3ea --- /dev/null +++ b/agenthookstest/fixtures/vscode/stop.json @@ -0,0 +1 @@ +{"timestamp":"2026-08-29T17:29:39.042Z","hook_event_name":"Stop","session_id":"a1b2c3d4-e5f6-4a7b-8c9d-000000000001","transcript_path":"/work/vscode/workspaceStorage/0123456789abcdef0123456789abcdef/GitHub.copilot-chat/transcripts/a1b2c3d4-e5f6-4a7b-8c9d-000000000001.jsonl","stop_hook_active":false,"cwd":"/work/repo"} diff --git a/capability.go b/capability.go index 9bc31ca..21e1465 100644 --- a/capability.go +++ b/capability.go @@ -89,7 +89,7 @@ var capMatrix = map[Provider]map[EventKind]CapSet{ KindToolPre: caps(CapDeny, CapAsk, CapUpdateInput), KindPromptSubmitted: caps(CapDeny), }, - ProviderCopilot: { + ProviderCopilotCLI: { // preToolUse and permissionRequest are the only decision-capable // events: deny was observed enforced end to end, and it fires even // under --allow-all/--yolo. prompt.submitted is deliberately an empty @@ -102,6 +102,33 @@ var capMatrix = map[Provider]map[EventKind]CapSet{ KindSubagentStop: caps(CapContinueAgent), KindSessionStart: caps(CapAddContext), }, + ProviderVSCodeCopilot: { + // Verified against the extension source, not the docs, which contradict + // themselves on placement. PreToolUse is the only event whose decision + // rides hookSpecificOutput; PostToolUse and UserPromptSubmit block via + // TOP-LEVEL decision/reason; Stop/SubagentStop block via NESTED + // decision/reason (encodeVSCode moves them). SessionStart/SubagentStart + // are processed with ignoreErrors and drop stopReason silently, so no + // CapStopAgent there. No CapReplaceOutput anywhere: updatedToolOutput is + // a Claude extension VS Code does not read. No CapAsk outside ToolPre. + // PostToolUse CapAddContext is contract-level: VS Code 1.135 accepts it + // and appends it to the tool result, but its panel path starts that async + // work without awaiting it (quirk #49). Keep the capability so the same + // documented payload works once the upstream race is fixed. + // permission.request, session.end, tool.error and notification are absent: + // VS Code never fires them. + // + // SubagentStart and CompactPre are observe-only in the public runner, + // so their upstream output channels are not capabilities here. Stop and + // SubagentStop cannot advertise CapStopAgent because StopDecision has no + // operation that sets it. + KindToolPre: caps(CapDeny, CapAsk, CapAllow, CapUpdateInput, CapAddContext, CapSystemMessage, CapStopAgent), + KindToolPost: caps(CapAddContext, CapSystemMessage, CapStopAgent), + KindPromptSubmitted: caps(CapDeny, CapAddContext, CapSystemMessage, CapStopAgent), + KindSessionStart: caps(CapAddContext, CapSystemMessage), + KindStop: caps(CapContinueAgent, CapSystemMessage), + KindSubagentStop: caps(CapContinueAgent, CapSystemMessage), + }, ProviderKimi: { // Only UserPromptSubmit, PreToolUse and Stop are blockable; JSON // output understands deny|allow only — no ask, no updatedInput, no diff --git a/codec.go b/codec.go index 38c8ba6..bdce33c 100644 --- a/codec.go +++ b/codec.go @@ -58,8 +58,10 @@ func decodePayload(p Provider, v Variant, conf DetectionConfidence, now time.Tim return decodeOpenClawLine(v, conf, now, payload) case ProviderKimi: return decodeKimi(v, conf, now, payload) - case ProviderCopilot: + case ProviderCopilotCLI: return decodeCopilot(v, conf, now, payload) + case ProviderVSCodeCopilot: + return decodeVSCode(v, conf, now, payload) } return nil, fmt.Errorf("agenthooks: unknown provider %q", p) } @@ -103,8 +105,10 @@ func encodeDecision(typed any, d decisionCore) (wireResponse, error) { return wireResponse{Stdout: out}, nil case ProviderKimi: return encodeKimi(base, d) - case ProviderCopilot: + case ProviderCopilotCLI: return encodeCopilot(base, d) + case ProviderVSCodeCopilot: + return encodeVSCode(base, d) } return wireResponse{}, fmt.Errorf("agenthooks: unknown provider %q", base.Provider) } diff --git a/codec_copilot.go b/codec_copilot.go index 68636c0..c2b32a9 100644 --- a/codec_copilot.go +++ b/codec_copilot.go @@ -15,11 +15,12 @@ import ( // permissionRequest ships `hookName` and notification ships a PascalCase // `hook_event_name`. The native name is therefore reconstructed from the // payload shape (copilotEventName); the shapes are disjoint, so the -// reconstruction is exact for every documented event. The two events -// that cannot be driven from a test harness — preCompact and -// subagentStart — were read off the CLI's own bundled sources -// (app.js, `nativeHookProcessor.event("preCompact", …)` and -// `onSubagentStart`) rather than guessed. +// reconstruction is exact for every documented event. preCompact +// (`trigger`) and subagentStart (`agentName`) were read off the CLI's own +// bundled sources first (app.js, `nativeHookProcessor.event("preCompact", +// …)` and `onSubagentStart`); both are now driven live too — the `task` +// tool fires the subagent pair and `/compact` is accepted as a headless +// prompt (e2e TestCopilotSubagentEvents, TestCopilotPreCompact). // 2. `preToolUse` command hooks are fail-closed on ANY non-zero exit other // than a timeout: exit 2, a crash, or any other code denies the tool call // even when stdout says allow. So this codec NEVER signals through the @@ -52,6 +53,21 @@ var copilotPascalAliases = map[string]string{ "Notification": "notification", } +var copilotCompatKinds = map[string]EventKind{ + "SessionStart": KindSessionStart, + "SessionEnd": KindSessionEnd, + "UserPromptSubmit": KindPromptSubmitted, + "PreToolUse": KindToolPre, + "PostToolUse": KindToolPost, + "PostToolUseFailure": KindToolError, + "PermissionRequest": KindPermission, + "Stop": KindStop, + "SubagentStart": KindSubagentStart, + "SubagentStop": KindSubagentStop, + "PreCompact": KindCompactPre, + "Notification": KindNotification, +} + type copilotToolResult struct { ResultType string `json:"resultType"` TextResultForLM string `json:"textResultForLlm"` @@ -80,9 +96,11 @@ type copilotIn struct { CustomInstructions string `json:"customInstructions"` ToolName string `json:"toolName"` - // ToolArgs is a JSON-ENCODED STRING on pre/postToolUse; ToolInput is a - // plain object on permissionRequest. normalizeInput un-stringifies the - // former, so ToolCall.Input is an object either way. + // ToolArgs on pre/postToolUse is version-dependent: a JSON-ENCODED STRING + // through CLI 1.0.80, a plain object from 1.0.81. ToolInput is a plain + // object on permissionRequest. Raw either way, because normalizeInput + // un-stringifies when needed and ToolCall.Input is an object in all three + // cases. ToolArgs json.RawMessage `json:"toolArgs"` ToolInput json.RawMessage `json:"toolInput"` ToolResult *copilotToolResult `json:"toolResult"` @@ -98,6 +116,32 @@ type copilotIn struct { NotificationTyp string `json:"notification_type"` } +type copilotCompatIn struct { + SessionID string `json:"session_id"` + TranscriptPath string `json:"transcript_path"` + CWD string `json:"cwd"` + HookEventName string `json:"hook_event_name"` + PermissionMode string `json:"permission_mode"` + Model string `json:"model"` + PromptID string `json:"prompt_id"` + ToolName string `json:"tool_name"` + ToolInput json.RawMessage `json:"tool_input"` + ToolUseID string `json:"tool_use_id"` + ToolResponse json.RawMessage `json:"tool_response"` + ToolError string `json:"tool_error"` + Prompt string `json:"prompt"` + Message string `json:"message"` + LastAssistantMessage string `json:"last_assistant_message"` + DurationMS *float64 `json:"duration_ms"` + Source string `json:"source"` + Reason string `json:"reason"` + StopHookActive bool `json:"stop_hook_active"` + Trigger string `json:"trigger"` + CustomInstructions string `json:"custom_instructions"` + AgentID string `json:"agent_id"` + AgentType string `json:"agent_type"` +} + // copilotEventName resolves the native event name. Copilot omits it from most // payloads, so an explicit field wins and the shape decides otherwise. Field // order below is the discrimination order and must stay in it: sessionStart @@ -152,13 +196,27 @@ func decodeCopilot(v Variant, conf DetectionConfidence, now time.Time, payload [ if err := json.Unmarshal(payload, &in); err != nil { return nil, err } + // PascalCase compatibility fallthrough. A --provider=copilot-cli registration + // can receive this snake_case wire shape from two directions: the CLI running the + // PascalCase compat file this library installs for VS Code, and VS Code + // discovering a camelCase CLI file (both runtimes glob both hook + // directories). copilotEventName has no camelCase shape to reconstruct from + // there, so without this the event lands on KindOther with the tool fields + // lost. The discriminator is an explicit event name with no camelCase + // sessionId — every genuine Copilot payload keys the session on sessionId, + // including the one native event (notification) that also ships + // hook_event_name. The label stays ProviderCopilotCLI so encodeCopilot still + // answers in the CLI's flat schema. + if in.HookEventName != "" && in.SessionID == "" { + return decodeCopilotCompat(v, conf, now, payload) + } native := copilotEventName(&in) kind, ok := copilotKinds[native] if !ok { kind = KindOther } base := Event{ - Provider: ProviderCopilot, + Provider: ProviderCopilotCLI, Variant: v, NativeName: native, Kind: kind, @@ -181,10 +239,10 @@ func decodeCopilot(v Variant, conf DetectionConfidence, now time.Time, payload [ base.Agent = &AgentInfo{ID: in.AgentID, Type: typ} } - // Copilot carries Claude's shapes under renamed keys: project onto claudeIn - // and reuse the shared builder. Two normalizations happen here — the two - // argument shapes collapse to one (a JSON-encoded string in toolArgs on - // pre/postToolUse, a plain object in toolInput on permissionRequest), and + // Two normalizations happen here: the argument shapes collapse to one + // (toolArgs on pre/postToolUse, either a + // JSON-encoded string or a plain object depending on the CLI release, and + // a plain object in toolInput on permissionRequest), and // the toolResult block flattens to output + error text. Copilot ships no // tool-call id (so every id is synthesized) and no duration. args := in.ToolArgs @@ -203,21 +261,92 @@ func decodeCopilot(v Variant, conf DetectionConfidence, now time.Time, payload [ errText = in.ToolResult.TextResultForLM } } - shaped := claudeIn{ - ToolName: in.ToolName, - ToolInput: args, - ToolResponse: output, - ToolError: errText, - Prompt: in.Prompt, - Message: in.Message, - LastAssistantMessage: in.Response, - Source: in.Source, - Reason: in.Reason, - StopHookActive: in.StopHookActive, - Trigger: in.Trigger, - CustomInstructions: in.CustomInstructions, + switch kind { + case KindToolPre: + return &ToolPreEvent{Event: base, Tool: makeToolCall(base.Session, in.ToolName, "", args, args)}, nil + case KindPermission: + return &PermissionEvent{Event: base, Tool: makeToolCall(base.Session, in.ToolName, "", args, args)}, nil + case KindToolPost, KindToolError: + return &ToolPostEvent{ + Event: base, + Tool: makeToolCall(base.Session, in.ToolName, "", args, args), + Output: output, + Failed: kind == KindToolError, + Error: errText, + }, nil + case KindPromptSubmitted: + return &PromptEvent{Event: base, Prompt: in.Prompt}, nil + case KindStop, KindSubagentStop: + loopCount := 0 + if in.StopHookActive { + loopCount = 1 + } + return &StopEvent{Event: base, PreviouslyContinued: in.StopHookActive, LoopCount: loopCount, FinalMessage: in.Response}, nil + case KindSubagentStart: + return &SubagentStartEvent{Event: base}, nil + case KindSessionStart: + return &SessionStartEvent{Event: base, Source: in.Source}, nil + case KindSessionEnd: + return &SessionEndEvent{Event: base, Reason: in.Reason}, nil + case KindNotification: + return &NotificationEvent{Event: base, Message: in.Message}, nil + case KindCompactPre: + return &CompactEvent{Event: base, Trigger: in.Trigger, Instructions: in.CustomInstructions}, nil + default: + return &base, nil + } +} + +func decodeCopilotCompat(v Variant, conf DetectionConfidence, now time.Time, payload []byte) (any, error) { + var in copilotCompatIn + if err := json.Unmarshal(payload, &in); err != nil { + return nil, err + } + kind, ok := copilotCompatKinds[in.HookEventName] + if !ok { + kind = KindOther + } + base := Event{ + Provider: ProviderCopilotCLI, Variant: v, NativeName: in.HookEventName, + Kind: kind, Time: now, DetectionConfidence: conf, + Session: SessionInfo{ + ID: in.SessionID, TurnID: in.PromptID, CWD: in.CWD, + WorkspaceRoots: rootsFor(in.CWD), TranscriptPath: in.TranscriptPath, + Model: in.Model, PermissionMode: in.PermissionMode, + }, + Raw: json.RawMessage(payload), + } + if in.AgentID != "" || in.AgentType != "" { + base.Agent = &AgentInfo{ID: in.AgentID, Type: in.AgentType} + } + switch kind { + case KindToolPre: + return &ToolPreEvent{Event: base, Tool: makeToolCall(base.Session, in.ToolName, in.ToolUseID, in.ToolInput, in.ToolInput)}, nil + case KindPermission: + return &PermissionEvent{Event: base, Tool: makeToolCall(base.Session, in.ToolName, in.ToolUseID, in.ToolInput, in.ToolInput)}, nil + case KindToolPost, KindToolError: + return &ToolPostEvent{Event: base, Tool: makeToolCall(base.Session, in.ToolName, in.ToolUseID, in.ToolInput, in.ToolInput), Output: in.ToolResponse, Failed: kind == KindToolError, Error: in.ToolError, DurationMS: in.DurationMS}, nil + case KindPromptSubmitted: + return &PromptEvent{Event: base, Prompt: in.Prompt}, nil + case KindStop, KindSubagentStop: + loopCount := 0 + if in.StopHookActive { + loopCount = 1 + } + return &StopEvent{Event: base, PreviouslyContinued: in.StopHookActive, LoopCount: loopCount, FinalMessage: in.LastAssistantMessage}, nil + case KindSubagentStart: + return &SubagentStartEvent{Event: base}, nil + case KindSessionStart: + return &SessionStartEvent{Event: base, Source: in.Source}, nil + case KindSessionEnd: + return &SessionEndEvent{Event: base, Reason: in.Reason}, nil + case KindNotification: + return &NotificationEvent{Event: base, Message: in.Message}, nil + case KindCompactPre: + return &CompactEvent{Event: base, Trigger: in.Trigger, Instructions: in.CustomInstructions}, nil + default: + return &base, nil } - return buildClaudeShaped(base, &shaped), nil } // encodeCopilot writes the per-event output schema. It always exits 0: on diff --git a/codec_copilot_test.go b/codec_copilot_test.go index b8e11b6..a3b3a61 100644 --- a/codec_copilot_test.go +++ b/codec_copilot_test.go @@ -43,8 +43,10 @@ func TestCopilotEventNamesFromShape(t *testing.T) { } } -// toolArgs is a JSON-encoded string on pre/postToolUse and a plain object on -// permissionRequest; both must normalize to an object. +// toolArgs is a plain object on pre/postToolUse from CLI 1.0.81 and a +// JSON-encoded string through 1.0.80; permissionRequest's toolInput is an +// object throughout. All three must normalize to the same object, so a policy +// written against ToolCall.Input keeps working across the version boundary. func TestCopilotToolArgsNormalize(t *testing.T) { typed, err := decodeCopilot(VariantUnknown, DetectionConfig, testNow, fixture(t, "copilot/pre_tool_use.json")) if err != nil { @@ -67,6 +69,19 @@ func TestCopilotToolArgsNormalize(t *testing.T) { t.Errorf("tool = %+v; copilot ships no call id, so it must be synthesized", pre.Tool) } + legacy := []byte(`{"sessionId":"sess-copilot-1","timestamp":1786820437717,"cwd":"/work/repo","toolName":"bash","toolArgs":"{\"command\":\"echo hello-from-gram\"}"}`) + typed, err = decodeCopilot(VariantUnknown, DetectionConfig, testNow, legacy) + if err != nil { + t.Fatal(err) + } + pre, ok = typed.(*ToolPreEvent) + if !ok { + t.Fatalf("decoded %T, want *ToolPreEvent", typed) + } + if err := json.Unmarshal(pre.Tool.Input, &args); err != nil || args.Command != "echo hello-from-gram" { + t.Errorf("1.0.80 double-encoded toolArgs did not un-stringify: %s (%v)", pre.Tool.Input, err) + } + typed, err = decodeCopilot(VariantUnknown, DetectionConfig, testNow, fixture(t, "copilot/permission_request.json")) if err != nil { t.Fatal(err) @@ -137,7 +152,7 @@ func TestCopilotPerEventOutputSchemas(t *testing.T) { t.Errorf("permissionRequest body = %s", wire.Stdout) } - if Capabilities(ProviderCopilot, VariantUnknown, KindPromptSubmitted).Has(CapDeny) { + if Capabilities(ProviderCopilotCLI, VariantUnknown, KindPromptSubmitted).Has(CapDeny) { t.Error("prompt.submitted must not claim deny: copilot drops command-hook output for it") } @@ -320,7 +335,7 @@ func TestCopilotSessionStartAdditionalContext(t *testing.T) { // Every path must still exit 0: a non-zero exit propagates to preToolUse // semantics as an unconditional deny. func TestCopilotDegradesUnsupportedDecisions(t *testing.T) { - copilotArgs := []string{"agenthooks", "run", "--provider=copilot"} + copilotArgs := []string{"agenthooks", "run", "--provider=copilot-cli"} // permissionRequest declares deny+allow but no ask. FallbackDeny must // harden to a real behavior:deny, not fall through to the empty body. @@ -362,16 +377,121 @@ func TestCopilotDegradesUnsupportedDecisions(t *testing.T) { } func TestCopilotDetection(t *testing.T) { - inv, err := parseArgs([]string{"agenthooks", "run", "--provider=copilot"}) - if err != nil || inv.provider != ProviderCopilot { - t.Fatalf("--provider=copilot → %q (%v)", inv.provider, err) + inv, err := parseArgs([]string{"agenthooks", "run", "--provider=copilot-cli"}) + if err != nil || inv.provider != ProviderCopilotCLI { + t.Fatalf("--provider=copilot-cli → %q (%v)", inv.provider, err) } t.Setenv("COPILOT_CLI", "1") t.Setenv("CLAUDE_PLUGIN_ROOT", "/tmp/plugin") - if p, ok := detectFromEnv(); !ok || p != ProviderCopilot { + if p, ok := detectFromEnv(); !ok || p != ProviderCopilotCLI { t.Errorf("env detection = %q; copilot cross-sets CLAUDE_PLUGIN_ROOT and must win", p) } - if p, ok := detectFromShape(fixture(t, "copilot/pre_tool_use.json")); !ok || p != ProviderCopilot { + if p, ok := detectFromShape(fixture(t, "copilot/pre_tool_use.json")); !ok || p != ProviderCopilotCLI { t.Errorf("shape detection = %q", p) } } + +// A --provider=copilot-cli registration receives the Claude-shaped snake_case +// payload from two directions — the CLI running the PascalCase compat file +// this library installs for VS Code, and VS Code discovering a camelCase CLI +// file, because both runtimes glob both hook directories. copilotEventName has +// no camelCase shape to reconstruct from there, so before the fallthrough +// every one of these landed on KindOther with the tool fields empty: hooks +// that look installed and healthy while reporting nothing useful. +func TestCopilotClaudeShapedFallthrough(t *testing.T) { + for _, tc := range []struct { + fixture, native string + kind EventKind + }{ + {"claude/pre_tool_use.json", "PreToolUse", KindToolPre}, + {"claude/user_prompt_submit.json", "UserPromptSubmit", KindPromptSubmitted}, + {"claude/post_tool_use.json", "PostToolUse", KindToolPost}, + {"claude/session_start.json", "SessionStart", KindSessionStart}, + {"claude/stop.json", "Stop", KindStop}, + } { + typed, err := decodeCopilot(VariantUnknown, DetectionConfig, testNow, fixture(t, tc.fixture)) + if err != nil { + t.Fatalf("%s: %v", tc.fixture, err) + } + ev := eventOf(typed) + if ev.NativeName != tc.native || ev.Kind != tc.kind { + t.Errorf("%s decoded as native=%q kind=%q, want %q/%q", tc.fixture, ev.NativeName, ev.Kind, tc.native, tc.kind) + } + // The label must stay ProviderCopilotCLI: it selects the CLI's flat + // response schema downstream. + if ev.Provider != ProviderCopilotCLI { + t.Errorf("%s provider = %q, want %q", tc.fixture, ev.Provider, ProviderCopilotCLI) + } + if ev.Session.ID != "sess-claude-1" { + t.Errorf("%s session id = %q", tc.fixture, ev.Session.ID) + } + } + + // The bug this fixes, stated as the assertion: tool arguments survive. + typed, err := decodeCopilot(VariantUnknown, DetectionConfig, testNow, fixture(t, "claude/pre_tool_use.json")) + if err != nil { + t.Fatal(err) + } + pre, ok := typed.(*ToolPreEvent) + if !ok { + t.Fatalf("decoded %T, want *ToolPreEvent", typed) + } + if pre.Tool.Name != "Bash" || pre.Tool.Canonical != ToolShell { + t.Errorf("tool = %+v; the whole point of the fallthrough is that these are populated", pre.Tool) + } + + // The camelCase corpus must be untouched. copilot/notification.json is the + // trap: it is the one native Copilot event that ships hook_event_name, so + // only the sessionId half of the discriminator keeps it on this path. + for name, want := range map[string]string{ + "copilot/notification.json": "notification", + "copilot/pre_tool_use.json": "preToolUse", + "copilot/agent_stop.json": "agentStop", + "copilot/session_start.json": "sessionStart", + } { + typed, err := decodeCopilot(VariantUnknown, DetectionConfig, testNow, fixture(t, name)) + if err != nil { + t.Fatalf("%s: %v", name, err) + } + if ev := eventOf(typed); ev.NativeName != want { + t.Errorf("%s decoded as native=%q, want %q; the fallthrough stole a camelCase payload", name, ev.NativeName, want) + } + } +} + +// End to end for the shared file: ONE installed agenthooks-vscode.json, run by +// both runtimes. The Copilot CLI's own env demotes the --provider flag, and +// everything downstream follows from the provider constant — so the same +// PascalCase input produces the CLI's FLAT body here and VS Code's nested one +// without it. A deny answered in the wrong placement is accepted and ignored +// by either runtime, which is why this is asserted rather than reasoned about. +func TestCopilotPascalCaseSharedFile(t *testing.T) { + vscodeArgs := []string{"agenthooks", "run", "--provider=vscode-copilot"} + denier := func() *Runner { + r := quietRunner() + r.OnToolPre(func(ctx context.Context, e *ToolPreEvent) (ToolPreDecision, error) { + return Deny("blocked by policy"), nil + }) + return r + } + + t.Setenv("COPILOT_CLI", "1") + out, code := runWith(t, denier(), vscodeArgs, fixture(t, "claude/pre_tool_use.json")) + if out != `{"permissionDecision":"deny","permissionDecisionReason":"blocked by policy"}` || code != 0 { + t.Errorf("CLI session = %q (exit %d), want copilot's flat deny at exit 0", out, code) + } + + t.Setenv("COPILOT_CLI", "") + out, code = runWith(t, denier(), vscodeArgs, fixture(t, "claude/pre_tool_use.json")) + var body struct { + HSO struct { + PermissionDecision string `json:"permissionDecision"` + } `json:"hookSpecificOutput"` + } + if err := json.Unmarshal([]byte(out), &body); err != nil { + t.Fatalf("VS Code session stdout %q: %v", out, err) + } + if body.HSO.PermissionDecision != "deny" || code != 0 { + t.Errorf("VS Code session = %q (exit %d), want a nested deny at exit 0", out, code) + } +} diff --git a/codec_test.go b/codec_test.go index df6166b..6dc47f1 100644 --- a/codec_test.go +++ b/codec_test.go @@ -97,6 +97,52 @@ func TestDecodeClaudeSkillPostToolUseBackfillsModelOutput(t *testing.T) { } } +// The Skill backfill exists because Claude Code reports only the skill name and +// resolves the manifest from Claude's own on-disk layout. Other providers' +// real tool output must survive to the handler. +func TestDecodeClaudeShapedSkillBackfillsOnlyForClaudeCode(t *testing.T) { + isolateClaudeSkillRoots(t) + repo := filepath.Join(t.TempDir(), "repo") + cwd := filepath.Join(repo, "nested") + if err := os.MkdirAll(filepath.Join(repo, ".git"), 0o700); err != nil { + t.Fatal(err) + } + writeClaudeSkillManifest(t, filepath.Join(repo, ".claude", "skills", "review"), "manifest") + payload, err := json.Marshal(map[string]any{ + "session_id": "sess-skill", + "cwd": cwd, + "hook_event_name": "PostToolUse", + "tool_name": "Skill", + "tool_input": map[string]any{"skill": "review"}, + "tool_response": "provider output", + "tool_use_id": "toolu_skill", + }) + if err != nil { + t.Fatal(err) + } + + for _, tc := range []struct { + provider Provider + want string + }{ + {ProviderClaudeCode, `"manifest"`}, + {ProviderVSCodeCopilot, `"provider output"`}, + {ProviderCopilotCLI, `"provider output"`}, + } { + typed, err := decodePayload(tc.provider, VariantUnknown, DetectionConfig, testNow, payload) + if err != nil { + t.Fatalf("%s: %v", tc.provider, err) + } + event, ok := typed.(*ToolPostEvent) + if !ok { + t.Fatalf("%s decoded %T, want *ToolPostEvent", tc.provider, typed) + } + if string(event.Output) != tc.want { + t.Errorf("%s Output = %s, want %s", tc.provider, event.Output, tc.want) + } + } +} + func TestDecodeClaudeMCP(t *testing.T) { typed, err := decodeClaude(VariantUnknown, DetectionConfig, testNow, fixture(t, "claude/pre_tool_use_mcp.json")) if err != nil { diff --git a/codec_vscode.go b/codec_vscode.go new file mode 100644 index 0000000..ffaf263 --- /dev/null +++ b/codec_vscode.go @@ -0,0 +1,189 @@ +package agenthooks + +import ( + "encoding/json" + "time" +) + +// VS Code Copilot Chat dialect: snake_case JSON on stdin, PascalCase event +// names, and per-event response placement on stdout. Keep this codec separate +// from other providers: matching wire fields today do not imply shared future +// behavior. + +var vscodeKinds = map[string]EventKind{ + "SessionStart": KindSessionStart, + "SessionEnd": KindSessionEnd, + "UserPromptSubmit": KindPromptSubmitted, + "PreToolUse": KindToolPre, + "PostToolUse": KindToolPost, + "PostToolUseFailure": KindToolError, + "PermissionRequest": KindPermission, + "Stop": KindStop, + "SubagentStart": KindSubagentStart, + "SubagentStop": KindSubagentStop, + "PreCompact": KindCompactPre, +} + +type vscodeIn struct { + SessionID string `json:"session_id"` + TranscriptPath string `json:"transcript_path"` + CWD string `json:"cwd"` + HookEventName string `json:"hook_event_name"` + PermissionMode string `json:"permission_mode"` + Model string `json:"model"` + PromptID string `json:"prompt_id"` + ToolName string `json:"tool_name"` + ToolInput json.RawMessage `json:"tool_input"` + ToolUseID string `json:"tool_use_id"` + ToolResponse json.RawMessage `json:"tool_response"` + ToolError string `json:"tool_error"` + Prompt string `json:"prompt"` + LastAssistantMessage string `json:"last_assistant_message"` + DurationMS *float64 `json:"duration_ms"` + Source string `json:"source"` + Reason string `json:"reason"` + StopHookActive bool `json:"stop_hook_active"` + Trigger string `json:"trigger"` + CustomInstructions string `json:"custom_instructions"` + AgentID string `json:"agent_id"` + AgentType string `json:"agent_type"` +} + +func decodeVSCode(v Variant, conf DetectionConfidence, now time.Time, payload []byte) (any, error) { + var in vscodeIn + if err := json.Unmarshal(payload, &in); err != nil { + return nil, err + } + kind, ok := vscodeKinds[in.HookEventName] + if !ok { + kind = KindOther + } + base := Event{ + Provider: ProviderVSCodeCopilot, + Variant: v, + NativeName: in.HookEventName, + Kind: kind, + Time: now, + DetectionConfidence: conf, + Session: SessionInfo{ + ID: in.SessionID, + TurnID: in.PromptID, + CWD: in.CWD, + WorkspaceRoots: rootsFor(in.CWD), + TranscriptPath: in.TranscriptPath, + Model: in.Model, + PermissionMode: in.PermissionMode, + }, + Raw: json.RawMessage(payload), + } + if in.AgentID != "" || in.AgentType != "" { + base.Agent = &AgentInfo{ID: in.AgentID, Type: in.AgentType} + } + + switch kind { + case KindToolPre: + return &ToolPreEvent{Event: base, Tool: makeToolCall(base.Session, in.ToolName, in.ToolUseID, in.ToolInput, in.ToolInput)}, nil + case KindPermission: + return &PermissionEvent{Event: base, Tool: makeToolCall(base.Session, in.ToolName, in.ToolUseID, in.ToolInput, in.ToolInput)}, nil + case KindToolPost, KindToolError: + return &ToolPostEvent{ + Event: base, + Tool: makeToolCall(base.Session, in.ToolName, in.ToolUseID, in.ToolInput, in.ToolInput), + Output: in.ToolResponse, + Failed: kind == KindToolError, + Error: in.ToolError, + DurationMS: in.DurationMS, + }, nil + case KindPromptSubmitted: + return &PromptEvent{Event: base, Prompt: in.Prompt}, nil + case KindStop, KindSubagentStop: + loopCount := 0 + if in.StopHookActive { + loopCount = 1 + } + return &StopEvent{Event: base, PreviouslyContinued: in.StopHookActive, LoopCount: loopCount, FinalMessage: in.LastAssistantMessage}, nil + case KindSubagentStart: + return &SubagentStartEvent{Event: base}, nil + case KindSessionStart: + return &SessionStartEvent{Event: base, Source: in.Source}, nil + case KindSessionEnd: + return &SessionEndEvent{Event: base, Reason: in.Reason}, nil + case KindCompactPre: + return &CompactEvent{Event: base, Trigger: in.Trigger, Instructions: in.CustomInstructions}, nil + default: + return &base, nil + } +} + +// encodeVSCode follows VS Code's per-event field placement. Stop verdicts are +// nested; prompt and post-tool verdicts stay top-level. +func encodeVSCode(base *Event, d decisionCore) (wireResponse, error) { + out := map[string]any{} + hso := map[string]any{} + ctx := joinContext(d.context) + + switch base.Kind { + case KindToolPre, KindPermission: + switch d.kind { + case DecisionAllow: + hso["permissionDecision"] = "allow" + case DecisionDeny: + hso["permissionDecision"] = "deny" + case DecisionAsk: + hso["permissionDecision"] = "ask" + } + if d.reason != "" && d.kind != DecisionNoDecision { + hso["permissionDecisionReason"] = d.reason + } + if d.hasUpdatedInput { + hso["updatedInput"] = d.updatedInput + } + if ctx != "" { + hso["additionalContext"] = ctx + } + case KindPromptSubmitted: + if d.kind == DecisionBlockPrompt { + out["decision"] = "block" + out["reason"] = d.reason + } + if ctx != "" { + hso["additionalContext"] = ctx + } + case KindSessionStart, KindSubagentStart: + if ctx != "" { + hso["additionalContext"] = ctx + } + case KindStop, KindSubagentStop: + if d.kind == DecisionContinue { + hso["decision"] = "block" + hso["reason"] = d.instruction + } + case KindToolPost, KindToolError: + if d.kind == DecisionFlagOutput { + out["decision"] = "block" + out["reason"] = d.reason + } + if ctx != "" { + hso["additionalContext"] = ctx + } + } + + if d.systemMessage != "" { + out["systemMessage"] = d.systemMessage + } + if d.stopAgent { + out["continue"] = false + if d.stopReason != "" { + out["stopReason"] = d.stopReason + } + } + if len(hso) > 0 { + hso["hookEventName"] = base.NativeName + out["hookSpecificOutput"] = hso + } + b, err := json.Marshal(out) + if err != nil { + return wireResponse{}, err + } + return wireResponse{Stdout: b}, nil +} diff --git a/codec_vscode_test.go b/codec_vscode_test.go new file mode 100644 index 0000000..41aa8a3 --- /dev/null +++ b/codec_vscode_test.go @@ -0,0 +1,299 @@ +package agenthooks + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "testing" +) + +func vscodeArgs() []string { return []string{"agenthooks", "run", "--provider=vscode-copilot"} } + +// vscodeDecode runs the payload through the provider switch, not through +// decodeClaude directly: the relabel lives in decodePayload's case, and a +// direct call would pass while the wiring was missing. +func vscodeDecode(t *testing.T, name string) any { + t.Helper() + typed, err := decodePayload(ProviderVSCodeCopilot, VariantUnknown, DetectionConfig, testNow, fixture(t, name)) + if err != nil { + t.Fatalf("%s: %v", name, err) + } + return typed +} + +// TestVSCodeDecodesRecordedCorpus is the ground-truth pass: these payloads were +// recorded from a live VS Code Copilot Chat agent turn (see the Phase 4 capture +// runbook), not hand-authored from docs the research found self-contradictory. +// +// What they confirmed, beyond decoding: the field set matches claudeIn's tags +// exactly, transcript_path and source are both populated, and tool_use_id ships +// — so unlike the Copilot CLI, VS Code needs no ID synthesis. SessionStart's +// source is "new" where Claude Code says "startup"; Source is passed through as +// provider-specific vocabulary and nothing switches on it, so that is a +// recorded divergence rather than a bug. +func TestVSCodeDecodesRecordedCorpus(t *testing.T) { + paths, err := filepath.Glob(filepath.Join("agenthookstest", "fixtures", "vscode", "*.json")) + if err != nil || len(paths) == 0 { + t.Fatalf("vscode corpus: %v (%d files)", err, len(paths)) + } + assertVSCodeDecodes(t, paths) +} + +// TestVSCodeDecodesClaudeCorpus is a drift assertion, not a stand-in: the +// Claude corpus IS the VS Code wire shape — same snake_case fields, same +// PascalCase event names — and the recorded corpus above proved it. It stays so +// that a future divergence in either dialect fails here, and because it covers +// the four VS Code events the three recorded fixtures do not. +func TestVSCodeDecodesClaudeCorpus(t *testing.T) { + paths, err := filepath.Glob(filepath.Join("agenthookstest", "fixtures", "claude", "*.json")) + if err != nil || len(paths) == 0 { + t.Fatalf("claude corpus: %v (%d files)", err, len(paths)) + } + assertVSCodeDecodes(t, paths) +} + +func assertVSCodeDecodes(t *testing.T, paths []string) { + t.Helper() + for _, p := range paths { + name := filepath.Base(p) + t.Run(name, func(t *testing.T) { + payload, err := os.ReadFile(p) + if err != nil { + t.Fatal(err) + } + typed, err := decodePayload(ProviderVSCodeCopilot, VariantUnknown, DetectionConfig, testNow, payload) + if err != nil { + t.Fatal(err) + } + ev := eventOf(typed) + if ev.Provider != ProviderVSCodeCopilot { + t.Errorf("provider = %q; decodeClaude hardcodes claude-code, so the relabel is load-bearing", ev.Provider) + } + var wire struct { + Name string `json:"hook_event_name"` + } + if err := json.Unmarshal(payload, &wire); err != nil { + t.Fatal(err) + } + if ev.NativeName != wire.Name { + t.Errorf("native name = %q, want %q", ev.NativeName, wire.Name) + } + want, ok := vscodeKinds[wire.Name] + if !ok { + want = KindOther + } + if ev.Kind != want { + t.Errorf("kind = %q, want %q", ev.Kind, want) + } + if string(ev.Raw) != string(payload) { + t.Error("Raw must be byte-identical to the payload") + } + }) + } +} + +// Stop and SubagentStop are the ONE wire difference from Claude Code: +// toolCallingLoop reads the block verdict from inside hookSpecificOutput, so +// encodeClaude's top-level placement would be a continuation that silently +// never happens. The nested hookEventName has to match too — a mismatch there +// makes _toHookResult strip the whole hookSpecificOutput. +func TestVSCodeStopDecisionIsNested(t *testing.T) { + for _, tc := range []struct{ fixture, native string }{ + {"vscode/stop.json", "Stop"}, + {"claude/subagent_stop.json", "SubagentStop"}, + } { + typed := vscodeDecode(t, tc.fixture) + base := eventOf(typed) + wire, err := encodeVSCode(base, decisionCore{kind: DecisionContinue, instruction: "keep going"}) + if err != nil { + t.Fatal(err) + } + var out struct { + Decision string `json:"decision"` + Reason string `json:"reason"` + HSO struct { + Decision string `json:"decision"` + Reason string `json:"reason"` + HookEventName string `json:"hookEventName"` + } `json:"hookSpecificOutput"` + } + if err := json.Unmarshal(wire.Stdout, &out); err != nil { + t.Fatalf("%s stdout %q: %v", tc.native, wire.Stdout, err) + } + if out.HSO.Decision != "block" || out.HSO.Reason != "keep going" { + t.Errorf("%s: continuation must ride hookSpecificOutput; body = %s", tc.native, wire.Stdout) + } + if out.Decision != "" || out.Reason != "" { + t.Errorf("%s: top-level decision/reason left behind; body = %s", tc.native, wire.Stdout) + } + if out.HSO.HookEventName != tc.native { + t.Errorf("%s: nested hookEventName = %q; a mismatch strips the whole hookSpecificOutput", tc.native, out.HSO.HookEventName) + } + } +} + +// The mirror of the test above: PostToolUse and UserPromptSubmit block via +// TOP-LEVEL decision/reason, which is where encodeClaude already puts them. +// Moving those too — the symmetric-looking change — would break both. +func TestVSCodeBlockStaysTopLevel(t *testing.T) { + for _, tc := range []struct { + fixture string + core decisionCore + }{ + {"claude/post_tool_use.json", decisionCore{kind: DecisionFlagOutput, reason: "output flagged"}}, + {"claude/user_prompt_submit.json", decisionCore{kind: DecisionBlockPrompt, reason: "output flagged"}}, + } { + base := eventOf(vscodeDecode(t, tc.fixture)) + wire, err := encodeVSCode(base, tc.core) + if err != nil { + t.Fatal(err) + } + var out map[string]any + if err := json.Unmarshal(wire.Stdout, &out); err != nil { + t.Fatalf("%s stdout %q: %v", tc.fixture, wire.Stdout, err) + } + if out["decision"] != "block" || out["reason"] != "output flagged" { + t.Errorf("%s: block must stay top level; body = %s", tc.fixture, wire.Stdout) + } + } +} + +// PostToolUse context keeps the documented nested shape. VS Code 1.135 parses +// this correctly but can race its async append to the next model request +// (quirk #49); changing the wire shape cannot fix that upstream race. +func TestVSCodePostToolContextUsesDocumentedShape(t *testing.T) { + base := eventOf(vscodeDecode(t, "claude/post_tool_use.json")) + wire, err := encodeVSCode(base, decisionCore{kind: DecisionObserved, context: []string{"lint passed"}}) + if err != nil { + t.Fatal(err) + } + var out struct { + Decision string `json:"decision"` + HSO struct { + AdditionalContext string `json:"additionalContext"` + HookEventName string `json:"hookEventName"` + } `json:"hookSpecificOutput"` + } + if err := json.Unmarshal(wire.Stdout, &out); err != nil { + t.Fatal(err) + } + if out.Decision != "" || out.HSO.AdditionalContext != "lint passed" || out.HSO.HookEventName != "PostToolUse" { + t.Fatalf("PostToolUse context shape changed: %s", wire.Stdout) + } +} + +// A TOP-LEVEL hookEventName makes _toHookResult discard the ENTIRE result when +// it mismatches, so the library never emits one; the name lives inside +// hookSpecificOutput only, stamped from the VS Code event name. +func TestVSCodeHookEventNameNeverTopLevel(t *testing.T) { + for _, tc := range []struct { + fixture string + core decisionCore + }{ + {"vscode/pre_tool_use.json", decisionCore{kind: DecisionDeny, reason: "blocked by policy"}}, + {"vscode/stop.json", decisionCore{kind: DecisionContinue, instruction: "keep going"}}, + {"vscode/session_start.json", decisionCore{kind: DecisionContinueSession, context: []string{"repo is frozen"}}}, + } { + base := eventOf(vscodeDecode(t, tc.fixture)) + wire, err := encodeVSCode(base, tc.core) + if err != nil { + t.Fatal(err) + } + var out struct { + HookEventName any `json:"hookEventName"` + HSO struct { + HookEventName string `json:"hookEventName"` + } `json:"hookSpecificOutput"` + } + if err := json.Unmarshal(wire.Stdout, &out); err != nil { + t.Fatal(err) + } + if out.HookEventName != nil { + t.Errorf("%s: top-level hookEventName present; a mismatch discards the whole result: %s", tc.fixture, wire.Stdout) + } + if out.HSO.HookEventName != base.NativeName { + t.Errorf("%s: nested hookEventName = %q, want %q", tc.fixture, out.HSO.HookEventName, base.NativeName) + } + } +} + +// Degradation is enforced generically in applyPolicy against the capability +// row, so it only holds end to end through the runner. The row is narrower +// than Claude Code's in both directions, and each narrowing below is a place a +// handler would otherwise believe it had an effect it does not have. +func TestVSCodeDegradesUnsupportedDecisions(t *testing.T) { + // tool.post has no CapReplaceOutput: updatedToolOutput is a Claude + // extension VS Code does not read. Its documented CapAddContext remains even + // though VS Code 1.135 can race the async append (quirk #49). + post := quietRunner() + post.OnToolPost(func(ctx context.Context, e *ToolPostEvent) (ToolPostDecision, error) { + if e.Can(CapReplaceOutput) { + t.Error("vscode tool.post must not report CapReplaceOutput") + } + if !e.Can(CapAddContext) { + t.Error("vscode tool.post must retain its documented CapAddContext") + } + return ReplaceOutput("scrubbed"), nil + }) + if out, code := runWith(t, post, vscodeArgs(), fixture(t, "claude/post_tool_use.json")); out != "{}" || code != 0 { + t.Errorf("replace-output on tool.post = %q (exit %d), want {} at exit 0", out, code) + } + + // agent.stop takes a continuation and nothing else: no ask, no context. + // Neither is reachable through StopDecision, so the row is the assertion. + stop := quietRunner() + stop.OnStop(func(ctx context.Context, e *StopEvent) (StopDecision, error) { + if e.Can(CapAddContext) || e.Can(CapAsk) { + t.Error("vscode agent.stop must report neither CapAddContext nor CapAsk") + } + return Finish(), nil + }) + if out, code := runWith(t, stop, vscodeArgs(), fixture(t, "vscode/stop.json")); out != "{}" || code != 0 { + t.Errorf("finish on agent.stop = %q (exit %d), want {} at exit 0", out, code) + } + + // VS Code can encode subagent.start context, but the public handler is + // observe-only and therefore cannot produce it as a capability. + if got := Capabilities(ProviderVSCodeCopilot, VariantUnknown, KindSubagentStart); len(got) != 0 { + t.Errorf("vscode subagent.start must be observe-only; capabilities = %v", got) + } + sub := &Event{Provider: ProviderVSCodeCopilot, NativeName: "SubagentStart", Kind: KindSubagentStart} + wire, err := encodeVSCode(sub, decisionCore{context: []string{"repo is frozen for release"}}) + if err != nil { + t.Fatal(err) + } + var out struct { + HSO struct { + AdditionalContext string `json:"additionalContext"` + HookEventName string `json:"hookEventName"` + } `json:"hookSpecificOutput"` + } + if err := json.Unmarshal(wire.Stdout, &out); err != nil { + t.Fatal(err) + } + if out.HSO.AdditionalContext != "repo is frozen for release" || out.HSO.HookEventName != "SubagentStart" { + t.Errorf("subagent.start context dropped: %s", wire.Stdout) + } +} + +// Detection is flag-only by construction: VS Code injects no environment +// marker and sends no field Claude Code does not also send, so any shape or +// env branch would misroute real Claude Code sessions into this row. +func TestVSCodeDetection(t *testing.T) { + inv, err := parseArgs(vscodeArgs()) + if err != nil || inv.provider != ProviderVSCodeCopilot || inv.confidence != DetectionConfig { + t.Fatalf("--provider=vscode-copilot → %q/%q (%v)", inv.provider, inv.confidence, err) + } + if p, ok := detectFromShape(fixture(t, "claude/pre_tool_use.json")); !ok || p != ProviderClaudeCode { + t.Errorf("shape detection = %q; a VS Code branch here would steal real Claude Code sessions", p) + } + // The same assertion against a RECORDED VS Code payload, which is the half + // that could not be proven before the capture session: a real VS Code + // payload is genuinely indistinguishable from Claude Code by shape, so the + // --provider flag is not merely the chosen mechanism, it is the only one + // available. A config without the flag degrades to claude-code. + if p, ok := detectFromShape(fixture(t, "vscode/pre_tool_use.json")); !ok || p != ProviderClaudeCode { + t.Errorf("recorded VS Code payload shape-detects as %q, want claude-code: if it were distinguishable, flag-only detection would be a choice rather than a constraint", p) + } +} diff --git a/detect.go b/detect.go index 4f47402..3c7d398 100644 --- a/detect.go +++ b/detect.go @@ -33,7 +33,11 @@ var validProviders = map[Provider]bool{ ProviderOpenCode: true, ProviderOpenClaw: true, ProviderKimi: true, - ProviderCopilot: true, + ProviderCopilotCLI: true, + // VS Code ships no provider env marker and no field Claude Code doesn't + // also send, so this one is reachable by the generated --provider flag + // only: neither detectFromEnv nor detectFromShape can produce it. + ProviderVSCodeCopilot: true, } func parseArgs(args []string) (*invocation, error) { @@ -129,15 +133,46 @@ func detectFromEnv() (Provider, bool) { } // Copilot cross-sets CLAUDE_PLUGIN_ROOT/CLAUDE_PROJECT_DIR into hook // processes (observed on CLI 1.0.80), so it must be checked before Claude. - if os.Getenv("COPILOT_CLI") != "" || os.Getenv("COPILOT_PLUGIN_ROOT") != "" || os.Getenv("COPILOT_PLUGIN_DATA") != "" { - return ProviderCopilot, true + if copilotCLIEnv() { + return ProviderCopilotCLI, true } - if os.Getenv("CLAUDE_PROJECT_DIR") != "" || os.Getenv("CLAUDE_PLUGIN_ROOT") != "" { + if os.Getenv("CLAUDECODE") == "1" || os.Getenv("CLAUDE_PROJECT_DIR") != "" || os.Getenv("CLAUDE_PLUGIN_ROOT") != "" { return ProviderClaudeCode, true } return "", false } +// copilotCLIEnv reports whether the Copilot CLI spawned this process. These +// are the CLI's own variables, not ones agenthooks injects; VS Code Copilot +// Chat sets no marker at all, which is what makes their absence meaningful in +// demoteVSCodeToCLI. +func copilotCLIEnv() bool { + return os.Getenv("COPILOT_CLI") != "" || os.Getenv("COPILOT_PLUGIN_ROOT") != "" || os.Getenv("COPILOT_PLUGIN_DATA") != "" +} + +// demoteVSCodeToCLI resolves which of the two runtimes that read the SAME hook +// file is actually running us, and is the one place a --provider flag is +// overridden rather than obeyed. +// +// VS Code Copilot Chat and the Copilot CLI glob the same two directories +// (~/.copilot/hooks, .github/hooks), so every file install writes is loaded by +// both. PascalCase event keys make the INPUT identical — that is the CLI's +// documented Claude-compat mode — but the CLI reads decisions from a FLAT body +// while VS Code reads them from a nested hookSpecificOutput. The CLI is the +// only one of the pair that marks its hook processes, so its env is the +// discriminator and --provider=vscode-copilot is the default it overrides. +// +// Resolving the runtime here, before decode, means each session gets the +// already-correct capability row and encoder with no branch downstream: +// decodeCopilot's Claude-shaped fallthrough handles the snake_case payload and +// encodeCopilot answers flat. +func demoteVSCodeToCLI(p Provider) Provider { + if p == ProviderVSCodeCopilot && copilotCLIEnv() { + return ProviderCopilotCLI + } + return p +} + func detectFromShape(payload []byte) (Provider, bool) { var probe struct { HookEventName string `json:"hook_event_name"` @@ -166,7 +201,7 @@ func detectFromShape(payload []byte) (Provider, bool) { // its payloads carry no event-name field at all on most events, so this is // the discriminator (verified against Copilot CLI 1.0.80). case probe.SessionIDCamel != "": - return ProviderCopilot, true + return ProviderCopilotCLI, true case probe.ConversationID != "": return ProviderCursor, true case probe.HookEventName != "" && isCamel(probe.HookEventName): diff --git a/detect_test.go b/detect_test.go index 9c3f9cb..8d6a674 100644 --- a/detect_test.go +++ b/detect_test.go @@ -35,3 +35,45 @@ func TestParseArgsNoSentinel(t *testing.T) { t.Errorf("provider = %q, want %q", inv.provider, ProviderClaudeCode) } } + +func TestDetectClaudeCodeMarker(t *testing.T) { + for _, v := range []string{ + "CURSOR_VERSION", "CURSOR_TRACE_ID", "CURSOR_AGENT", "CODEX_HOME", "CODEX_SANDBOX", + "GEMINI_CWD", "GEMINI_CLI", "OPENCODE_SERVER", "OPENCODE", "COPILOT_CLI", + "COPILOT_PLUGIN_ROOT", "COPILOT_PLUGIN_DATA", "CLAUDE_PROJECT_DIR", "CLAUDE_PLUGIN_ROOT", + } { + t.Setenv(v, "") + } + t.Setenv("CLAUDECODE", "1") + if got, ok := detectFromEnv(); !ok || got != ProviderClaudeCode { + t.Errorf("CLAUDECODE=1 detection = %q, %v; want %q, true", got, ok, ProviderClaudeCode) + } +} + +// One installed file, two runtimes. --provider=vscode-copilot is the default +// the Copilot CLI overrides with its own env; VS Code sets no marker, so its +// absence is the only signal available and it has to mean "leave the flag +// alone". Getting this backwards sends a whole runtime the wrong response +// schema, silently — both runtimes accept and ignore the other's body. +func TestDemoteVSCodeToCLI(t *testing.T) { + // No COPILOT_* set: a VS Code session, flag obeyed. + for _, v := range []string{"COPILOT_CLI", "COPILOT_PLUGIN_ROOT", "COPILOT_PLUGIN_DATA"} { + t.Setenv(v, "") + } + if got := demoteVSCodeToCLI(ProviderVSCodeCopilot); got != ProviderVSCodeCopilot { + t.Errorf("no COPILOT_* → %q, want %q", got, ProviderVSCodeCopilot) + } + + t.Setenv("COPILOT_PLUGIN_ROOT", "/tmp/plugin") + if got := demoteVSCodeToCLI(ProviderVSCodeCopilot); got != ProviderCopilotCLI { + t.Errorf("COPILOT_PLUGIN_ROOT set → %q, want %q", got, ProviderCopilotCLI) + } + // Every other provider is untouched: Claude Code hooks run inside a + // Copilot CLI session under CLAUDE_* compat vars, and demoting those would + // hijack a provider that never shared the file. + for _, p := range []Provider{ProviderClaudeCode, ProviderCopilotCLI, ProviderCursor, ""} { + if got := demoteVSCodeToCLI(p); got != p { + t.Errorf("demote(%q) = %q, want unchanged", p, got) + } + } +} diff --git a/docs/research/vscode-copilot-posttooluse-context.md b/docs/research/vscode-copilot-posttooluse-context.md new file mode 100644 index 0000000..2bc4ac7 --- /dev/null +++ b/docs/research/vscode-copilot-posttooluse-context.md @@ -0,0 +1,49 @@ +# VS Code Copilot Chat `PostToolUse` context and feedback + +_Research date: 2026-08-30. Scope: command hooks in VS Code 1.135 / bundled Copilot Chat 0.63._ + +## Conclusion + +**The documented contract supports both model-visible `PostToolUse` forms, but VS Code 1.135's panel command-hook path does not reliably deliver either form to the immediate next model request.** The 1.135 source accepts and aggregates nested `hookSpecificOutput.additionalContext` and top-level `decision: "block"` / `reason`; however, it starts the asynchronous post-hook/context append without awaiting it. The next prompt can therefore be assembled before either message reaches the tool result. An open upstream fix describes the same symptom and changes that call to `await`. + +This reconciles the observation: “successful” in the hook log proves that the command ran and its JSON was parsed, not that the asynchronous context mutation completed before the next model request. A model's failure to repeat a marker is not by itself proof that it did not see it, but the 1.135 race makes absence from the actual next request payload expected. Verify delivery in the next LLM request input, not only in visible answer/reasoning text. + +## Documented response shape + +The current official [hooks reference, **PostToolUse output**](https://code.visualstudio.com/docs/agents/reference/hooks-reference#_posttooluse-output) documents this combined shape: + +```json +{ + "decision": "block", + "reason": "Post-processing validation failed", + "hookSpecificOutput": { + "hookEventName": "PostToolUse", + "additionalContext": "The edited file has lint errors that need to be fixed" + } +} +``` + +The same table says `reason` is shown to the model and `hookSpecificOutput.additionalContext` is injected into the conversation. `decision` has only the optional value `"block"`. The [official hooks guide](https://code.visualstudio.com/docs/copilot/customization/hooks#_hook-input-and-output) distinguishes common flow-control output from event-specific `hookSpecificOutput`. + +The release source's command contract agrees: `PostToolUse` input adds `tool_name`, `tool_input`, `tool_response`, and `tool_use_id`, while its nested output has only optional `hookEventName` and `additionalContext` ([1.135 source, lines 38–56](https://github.com/microsoft/vscode/blob/08d4889f9ec4a1685d257b9b95de036c8e1ce1e5/extensions/copilot/src/platform/chat/common/hookCommandTypes.ts#L38-L56)). `permissionDecision`, `permissionDecisionReason`, and `updatedInput` are **PreToolUse**, not documented PostToolUse feedback fields ([lines 14–34](https://github.com/microsoft/vscode/blob/08d4889f9ec4a1685d257b9b95de036c8e1ce1e5/extensions/copilot/src/platform/chat/common/hookCommandTypes.ts#L14-L34)). + +`decision: "block"` is post-execution feedback: the tool has already completed. In the implementation it becomes model-visible text saying the hook blocked the tool result; it does not undo the tool. + +## What the 1.135 command-hook runtime implements + +At the pinned `release/1.135` commit (Copilot extension package version 0.63.0): + +1. Successful command output remains structured JSON after common fields are removed; nested `hookEventName` is validated and matching `hookSpecificOutput` is preserved ([parser, lines 261–335](https://github.com/microsoft/vscode/blob/08d4889f9ec4a1685d257b9b95de036c8e1ce1e5/extensions/copilot/src/extension/chat/vscode-node/chatHookService.ts#L261-L335)). +2. `executePostToolUseHook` collects nested `additionalContext`, recognizes top-level `decision === "block"`, retains `reason`, and returns both in a collapsed result ([lines 491–586](https://github.com/microsoft/vscode/blob/08d4889f9ec4a1685d257b9b95de036c8e1ce1e5/extensions/copilot/src/extension/chat/vscode-node/chatHookService.ts#L491-L586)). Thus both observed payloads are valid and a success log is unsurprising. +3. The intended model handoff appends block feedback and additional context to the tool result as `LanguageModelTextPart` values inside `` tags ([lines 630–649](https://github.com/microsoft/vscode/blob/08d4889f9ec4a1685d257b9b95de036c8e1ce1e5/extensions/copilot/src/extension/prompts/node/panel/toolCalling.tsx#L630-L649)). +4. The defect is one level above: `appendHookContext(...)` is called without `await` ([line 341](https://github.com/microsoft/vscode/blob/08d4889f9ec4a1685d257b9b95de036c8e1ce1e5/extensions/copilot/src/extension/prompts/node/panel/toolCalling.tsx#L334-L345)), although that helper is async and awaits command execution ([lines 600–649](https://github.com/microsoft/vscode/blob/08d4889f9ec4a1685d257b9b95de036c8e1ce1e5/extensions/copilot/src/extension/prompts/node/panel/toolCalling.tsx#L600-L649)). Prompt rendering can finish first. Both nested context and block/reason use this same late append, so both can disappear from the immediate model turn. + +## Upstream corroboration + +- [microsoft/vscode#314118](https://github.com/microsoft/vscode/issues/314118) reports successful `PostToolUse` execution whose feedback arrives too late. +- Open PR [microsoft/vscode#331785](https://github.com/microsoft/vscode/pull/331785), **“fix: await PostToolUse context in panel tool calls,”** identifies the unawaited helper as root cause, says the panel may assemble the next model request before context is appended, and reports reproduction on VS Code 1.134 / Copilot Chat 0.62. Its production fix is the missing `await`. The same unawaited line remains in the pinned 1.135 / 0.63 source above. +- A separate agent-host path previously had the same externally visible failure: [issue #311138](https://github.com/microsoft/vscode/issues/311138) records context in the transcript but not the same-turn request; merged [PR #311984](https://github.com/microsoft/vscode/pull/311984) fixed that path by returning parsed PostToolUse command output instead of discarding it. This explains why documentation and some runtime paths/tests can indicate support while another live path still fails. + +## Practical answer + +Treat the shapes as **valid documented outputs but not reliable immediate model feedback on VS Code 1.135 / Copilot Chat 0.63's panel path**. Neither swapping nested `additionalContext` for top-level `decision/reason` nor combining them avoids the race. Reliability requires the upstream await fix (or a build containing an equivalent change); after that, inspect the next model request payload for the marker to distinguish transport from a model choosing not to mention it. diff --git a/e2e/copilot_test.go b/e2e/copilot_test.go index 26df325..c40cd42 100644 --- a/e2e/copilot_test.go +++ b/e2e/copilot_test.go @@ -1,6 +1,8 @@ package e2e import ( + "bytes" + "encoding/json" "strings" "testing" @@ -56,7 +58,7 @@ func TestCopilotEventFields(t *testing.T) { rec := newRecorder(t, "") home := copilotHome(t) proj := t.TempDir() - installHooks(t, rec, agenthooks.ProviderCopilot, install.ScopeUser, home) + installHooks(t, rec, agenthooks.ProviderCopilotCLI, install.ScopeUser, home) runCopilot(t, proj, home, shellMarkerPrompt("copilot-marker.txt")) return rec, proj }) @@ -90,7 +92,7 @@ func TestCopilotEventFields(t *testing.T) { requireNoBackfill(t, evs) for _, e := range ofKind(evs, agenthooks.KindSessionStart) { - ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilot, NativeName: e.Native, Raw: e.Raw} + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} in, ok := copilot.SessionStart(ev) if !ok { t.Fatalf("SessionStart view rejected native %q", e.Native) @@ -110,7 +112,7 @@ func TestCopilotEventFields(t *testing.T) { } for _, e := range ofKind(evs, agenthooks.KindSessionEnd) { - ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilot, NativeName: e.Native, Raw: e.Raw} + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} in, ok := copilot.SessionEnd(ev) if !ok { t.Fatalf("SessionEnd view rejected native %q", e.Native) @@ -127,7 +129,7 @@ func TestCopilotEventFields(t *testing.T) { if e.Backfilled { continue // no Raw to check: a backfill fabricates nothing } - ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilot, NativeName: e.Native, Raw: e.Raw} + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} in, ok := copilot.UserPromptSubmitted(ev) if !ok { t.Fatalf("UserPromptSubmitted view rejected native %q", e.Native) @@ -141,7 +143,7 @@ func TestCopilotEventFields(t *testing.T) { } for _, e := range ofKind(evs, agenthooks.KindToolPre) { - ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilot, NativeName: e.Native, Raw: e.Raw} + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} in, ok := copilot.PreToolUse(ev) if !ok { t.Fatalf("PreToolUse view rejected native %q", e.Native) @@ -149,11 +151,12 @@ func TestCopilotEventFields(t *testing.T) { if in.SessionID == "" || in.CWD == "" || in.ToolName == "" { t.Errorf("PreToolUse fields incomplete: %+v (raw: %s)", in, e.Raw) } - // ToolArgs is typed as a string on purpose: Copilot double-encodes - // the arguments here (a JSON object serialized into a JSON string), - // unlike permissionRequest's plain toolInput object. - if !strings.HasPrefix(in.ToolArgs, "{") { - t.Errorf("PreToolUse toolArgs is not a JSON-encoded object string — the double-encoding quirk changed: %q (raw: %s)", in.ToolArgs, e.Raw) + // ToolArgs is raw because the shape moved under a stable name: a + // JSON-encoded object string through CLI 1.0.80, a plain object from + // 1.0.81. Which one the installed CLI sends is logged rather than + // pinned; that both decode is what the view has to guarantee. + if !isObjectOrEncodedObject(in.ToolArgs) { + t.Errorf("PreToolUse toolArgs is neither an object nor a JSON-encoded object string — a third shape appeared: %s (raw: %s)", in.ToolArgs, e.Raw) } if len(in.Extra) > 0 { t.Errorf("PreToolUse has unknown fields %v (raw: %s)", keys(in.Extra), e.Raw) @@ -161,7 +164,7 @@ func TestCopilotEventFields(t *testing.T) { } for _, e := range ofKind(evs, agenthooks.KindToolPost) { - ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilot, NativeName: e.Native, Raw: e.Raw} + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} in, ok := copilot.PostToolUse(ev) if !ok { t.Fatalf("PostToolUse view rejected native %q", e.Native) @@ -177,7 +180,7 @@ func TestCopilotEventFields(t *testing.T) { } for _, e := range ofKind(evs, agenthooks.KindStop) { - ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilot, NativeName: e.Native, Raw: e.Raw} + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} in, ok := copilot.AgentStop(ev) if !ok { t.Fatalf("AgentStop view rejected native %q", e.Native) @@ -192,8 +195,8 @@ func TestCopilotEventFields(t *testing.T) { } } - // Normalization: toolArgs un-stringifies and the bash call classifies as - // canonical shell. + // Normalization: toolArgs lands as an object whichever shape it arrived + // in, and the bash call classifies as canonical shell. for _, e := range typedToolPres(evs) { if e.Canonical == string(agenthooks.ToolShell) { return @@ -202,6 +205,24 @@ func TestCopilotEventFields(t *testing.T) { t.Errorf("no shell tool.pre normalized from copilot; got:\n%s", summarize(evs)) } +// isObjectOrEncodedObject accepts both toolArgs shapes the Copilot CLI has +// shipped: a plain JSON object (1.0.81+) and an object serialized into a JSON +// string (through 1.0.80). Anything else is a third shape nothing normalizes. +func isObjectOrEncodedObject(raw json.RawMessage) bool { + trimmed := bytes.TrimSpace(raw) + if len(trimmed) == 0 { + return false + } + if trimmed[0] == '"' { + var inner string + if err := json.Unmarshal(trimmed, &inner); err != nil { + return false + } + trimmed = bytes.TrimSpace([]byte(inner)) + } + return len(trimmed) > 0 && trimmed[0] == '{' && json.Valid(trimmed) +} + // TestCopilotToolFailure requires a real failed file-view call to produce the // native postToolUseFailure event rather than treating failure coverage as an // optional side effect of denial. @@ -212,7 +233,7 @@ func TestCopilotToolFailure(t *testing.T) { rec := newRecorder(t, "") home := copilotHome(t) proj := t.TempDir() - installHooks(t, rec, agenthooks.ProviderCopilot, install.ScopeUser, home) + installHooks(t, rec, agenthooks.ProviderCopilotCLI, install.ScopeUser, home) runCopilot(t, proj, home, toolFailurePrompt()) return rec, proj }) @@ -226,7 +247,7 @@ func TestCopilotToolFailure(t *testing.T) { } matchedViewFailure := false for _, e := range ofKind(evs, agenthooks.KindToolError) { - ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilot, NativeName: e.Native, Raw: e.Raw} + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} in, ok := copilot.PostToolUseFailure(ev) if !ok { t.Fatalf("PostToolUseFailure view rejected native %q", e.Native) @@ -258,7 +279,7 @@ func TestCopilotModifiedArgs(t *testing.T) { rec := newRecorderWithConfig(t, recorderConfig{RewriteCommand: "touch " + rewritten}) home := copilotHome(t) proj := t.TempDir() - installHooks(t, rec, agenthooks.ProviderCopilot, install.ScopeUser, home) + installHooks(t, rec, agenthooks.ProviderCopilotCLI, install.ScopeUser, home) runCopilot(t, proj, home, oneShotShellMarkerPrompt(original)) return rec, proj }) @@ -294,7 +315,7 @@ func TestCopilotPluginScope(t *testing.T) { home := copilotHome(t) proj := t.TempDir() pluginDir := t.TempDir() - installHooks(t, rec, agenthooks.ProviderCopilot, install.ScopePlugin, pluginDir) + installHooks(t, rec, agenthooks.ProviderCopilotCLI, install.ScopePlugin, pluginDir) runCopilot(t, proj, home, oneShotShellMarkerPrompt(marker), "--plugin-dir", pluginDir) return rec, proj }) @@ -320,7 +341,7 @@ func TestCopilotDeny(t *testing.T) { rec := newRecorder(t, string(agenthooks.ToolShell)) home := copilotHome(t) proj := t.TempDir() - installHooks(t, rec, agenthooks.ProviderCopilot, install.ScopeUser, home) + installHooks(t, rec, agenthooks.ProviderCopilotCLI, install.ScopeUser, home) runCopilot(t, proj, home, shellMarkerPrompt("denied-marker.txt")) return rec, proj }) @@ -343,7 +364,7 @@ func TestCopilotDeny(t *testing.T) { // A blocked call may or may not produce a postToolUseFailure; when it // does, the failure view has to hold up too. for _, e := range ofKind(evs, agenthooks.KindToolError) { - ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilot, NativeName: e.Native, Raw: e.Raw} + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} in, ok := copilot.PostToolUseFailure(ev) if !ok { t.Fatalf("PostToolUseFailure view rejected native %q", e.Native) @@ -356,3 +377,287 @@ func TestCopilotDeny(t *testing.T) { } } } + +// TestCopilotPascalCaseCompat drives the file rendered for VS Code +// (agenthooks-vscode.json, PascalCase event keys) with the real Copilot CLI. +// VS Code and the CLI glob the same two hook directories, so this file is +// loaded by both whether we like it or not — the design bets that the CLI's +// documented Claude-compat mode makes that harmless. Three things only the +// real binary can witness: +// +// 1. WHICH of VS Code's eight PascalCase names the CLI actually registers. +// The CLI docs confirm the mode for PreToolUse and never enumerate the +// rest; UserPromptSubmit is the suspect one, since the CLI's native name +// is userPromptSubmitted. An unregistered name is silent — the hook +// simply never fires — so the recorded kinds are the answer. The t.Logf +// below is the output to read. +// 2. The COPILOT_* demotion fires in a real CLI hook child. Every event must +// be stamped copilot-cli, not vscode-copilot: the flag says vscode-copilot +// and only the CLI's own env overrides it. +// 3. The CLI honors a decision returned for a PascalCase registration, in +// the FLAT schema encodeCopilot emits. A body in VS Code's nested +// placement would be accepted and ignored, so a passing deny is the only +// proof the demotion picked the right encoder. +func TestCopilotPascalCaseCompat(t *testing.T) { + t.Parallel() + requireE2E(t, "copilot") + + rec, proj := runToolTurn(t, func() (recorder, string) { + rec := newRecorder(t, "") + home := copilotHome(t) + proj := t.TempDir() + installHooks(t, rec, agenthooks.ProviderVSCodeCopilot, install.ScopeUser, home) + runCopilot(t, proj, home, shellMarkerPrompt("pascal-marker.txt")) + return rec, proj + }) + evs := rec.events(t) + // sessionEnd and postToolUseFailure are deliberately absent: they have no + // VS Code counterpart, so kindToVSCode never renders them and the CLI + // cannot fire what was never registered. That is the documented cost of + // the PascalCase file versus the camelCase one. + requireKinds(t, evs, + agenthooks.KindSessionStart, + agenthooks.KindPromptSubmitted, + agenthooks.KindToolPre, + agenthooks.KindToolPost, + agenthooks.KindStop, + ) + registered := map[string]bool{} + for _, e := range evs { + if e.Native != "" { + registered[e.Native] = true + } + } + t.Logf("PascalCase names the Copilot CLI registered: %v", keys(registered)) + if !markerExists(proj, "pascal-marker.txt") { + t.Error("marker missing: shell command did not run under the PascalCase file") + } + for _, e := range evs { + if e.Provider != string(agenthooks.ProviderCopilotCLI) { + t.Errorf("event %s stamped provider %q, want %q — the COPILOT_* demotion did not fire, so this session got VS Code's capability row and nested encoder", e.Native, e.Provider, agenthooks.ProviderCopilotCLI) + } + } + + // Second turn, denying: a nested body would be silently ignored here. + denyRec, denyProj := runToolTurn(t, func() (recorder, string) { + rec := newRecorder(t, string(agenthooks.ToolShell)) + home := copilotHome(t) + proj := t.TempDir() + installHooks(t, rec, agenthooks.ProviderVSCodeCopilot, install.ScopeUser, home) + runCopilot(t, proj, home, shellMarkerPrompt("pascal-denied-marker.txt")) + return rec, proj + }) + denyEvs := denyRec.events(t) + requireKinds(t, denyEvs, agenthooks.KindToolPre) + deniedShell := false + for _, e := range typedToolPres(denyEvs) { + if e.Canonical == string(agenthooks.ToolShell) && e.Denied { + deniedShell = true + break + } + } + if !deniedShell { + t.Errorf("no denied shell tool.pre recorded; got:\n%s", summarize(denyEvs)) + } + if markerExists(denyProj, "pascal-denied-marker.txt") { + t.Error("marker exists: the flat deny was not honored for a PascalCase registration") + } +} + +// TestCopilotStopContinuation drives the one capability the Copilot row claims +// on agent.stop — CapContinueAgent — through a real turn. It was previously +// read off the shipped binary's executeStopHook and nothing else, so a wire +// change (or a wrong schema) would have been invisible: the CLI ignores an +// unrecognized stop body silently and simply finishes. +// +// The proof is the second marker. The recorder returns ContinueWith once, and +// only the CLI feeding that instruction back into the model can create it. +// The second stop's guard fields are asserted too: Copilot reports +// stop_hook_active, so PreviouslyContinued/LoopCount must be populated on the +// continued turn — that is what keeps Policy.ContinuationCap load-bearing +// rather than decorative here. +func TestCopilotStopContinuation(t *testing.T) { + t.Parallel() + requireE2E(t, "copilot") + const first = "stop-first-marker.txt" + const continued = "stop-continued-marker.txt" + rec, proj := runToolTurn(t, func() (recorder, string) { + rec := newRecorderWithConfig(t, recorderConfig{ContinueInstruction: oneShotShellMarkerPrompt(continued)}) + home := copilotHome(t) + proj := t.TempDir() + installHooks(t, rec, agenthooks.ProviderCopilotCLI, install.ScopeUser, home) + runCopilot(t, proj, home, oneShotShellMarkerPrompt(first)) + return rec, proj + }) + evs := rec.events(t) + requireKinds(t, evs, agenthooks.KindStop) + if !markerExists(proj, first) { + t.Error("first marker missing: the turn never ran its own shell command") + } + if !markerExists(proj, continued) { + t.Errorf("continuation marker missing: the CLI did not act on the ContinueWith instruction, so CapContinueAgent is not honored on agentStop; got:\n%s", summarize(evs)) + } + var stops []event + for _, e := range evs { + if e.Typed && e.Kind == string(agenthooks.KindStop) { + stops = append(stops, e) + } + } + if len(stops) < 2 { + t.Fatalf("want at least two agent.stop deliveries (the original turn and the continued one), got %d:\n%s", len(stops), summarize(evs)) + } + if !stops[0].Continued { + t.Errorf("first stop did not return a continuation: %+v", stops[0]) + } + last := stops[len(stops)-1] + if !last.PrevContinued || last.LoopCount < 1 { + t.Errorf("continued stop reports no native guard (PreviouslyContinued=%v LoopCount=%d) — Copilot stopped sending stop_hook_active, so the library-side continuation cap is now the only loop bound", last.PrevContinued, last.LoopCount) + } +} + +// subagentPrompt delegates to the CLI's `task` tool, which is what fires +// subagentStart/subagentStop. The nested agent is told to answer in one word: +// the delegation is the point, not the work. +func subagentPrompt() string { + return "Use the task tool exactly once to delegate this to a subagent: reply with the single word ok. " + + "Do not use any other tool." +} + +// TestCopilotSubagentEvents measures subagentStart and subagentStop, which the +// codec previously mapped from the CLI's bundled sources alone (subagentStart +// in particular has no explicit event name on the wire, so copilotEventName +// reconstructs it from `agentName` — an inference until this test). +// +// Both runtimes are driven, because one file's registrations do not predict +// the other's dialect. The camelCase CLI file gets camelCase payloads for +// both events; the PascalCase (VS Code) file gets Claude-shaped SubagentStop +// but a NATIVE camelCase subagentStart — the CLI's Claude-compat translation +// does not cover it. That mix is asserted below: it is exactly the kind of +// half-translation that would silently drop an event to KindOther. +func TestCopilotSubagentEvents(t *testing.T) { + t.Parallel() + requireE2E(t, "copilot") + for _, tc := range []struct { + name string + provider agenthooks.Provider + startName, opName string + }{ + {"camelCase", agenthooks.ProviderCopilotCLI, "subagentStart", "subagentStop"}, + {"pascalCase", agenthooks.ProviderVSCodeCopilot, "subagentStart", "SubagentStop"}, + } { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + rec, _ := runToolTurn(t, func() (recorder, string) { + rec := newRecorder(t, "") + home := copilotHome(t) + proj := t.TempDir() + installHooks(t, rec, tc.provider, install.ScopeUser, home) + runCopilot(t, proj, home, subagentPrompt()) + return rec, proj + }) + evs := rec.events(t) + requireKinds(t, evs, agenthooks.KindSubagentStart, agenthooks.KindSubagentStop) + for _, e := range ofKind(evs, agenthooks.KindSubagentStart) { + if e.Native != tc.startName { + t.Errorf("subagent.start native = %q, want %q — the CLI changed which dialect it sends for this registration", e.Native, tc.startName) + } + } + for _, e := range ofKind(evs, agenthooks.KindSubagentStop) { + if e.Native != tc.opName { + t.Errorf("subagent.stop native = %q, want %q — the CLI changed which dialect it sends for this registration", e.Native, tc.opName) + } + } + if tc.provider != agenthooks.ProviderCopilotCLI { + return // views below are the camelCase wire only + } + for _, e := range ofKind(evs, agenthooks.KindSubagentStart) { + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} + in, ok := copilot.SubagentStart(ev) + if !ok { + t.Fatalf("SubagentStart view rejected native %q", e.Native) + } + // AgentName is the sole discriminator copilotEventName has for + // this event; empty means the reconstruction was luck. + if in.SessionID == "" || in.AgentName == "" { + t.Errorf("SubagentStart fields incomplete: %+v (raw: %s)", in, e.Raw) + } + if len(in.Extra) > 0 { + t.Errorf("SubagentStart has unknown fields %v (raw: %s)", keys(in.Extra), e.Raw) + } + } + for _, e := range ofKind(evs, agenthooks.KindSubagentStop) { + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} + in, ok := copilot.SubagentStop(ev) + if !ok { + t.Fatalf("SubagentStop view rejected native %q", e.Native) + } + // AgentID/AgentType are what separate subagentStop from + // agentStop: both carry stopReason and nothing else does. + if in.AgentID == "" || in.AgentType == "" || in.StopReason == "" { + t.Errorf("SubagentStop fields incomplete: %+v (raw: %s)", in, e.Raw) + } + if len(in.Extra) > 0 { + t.Errorf("SubagentStop has unknown fields %v (raw: %s)", keys(in.Extra), e.Raw) + } + } + }) + } +} + +// TestCopilotPreCompact measures preCompact in both runtimes. `/compact` is +// accepted as a headless prompt and the hook fires BEFORE the CLI decides +// there is nothing to compact, so a fresh session drives the event for free — +// the turn ends in "Nothing to compact" and spends no model tokens. +// +// Unlike the subagent events, both runtimes translate this one: the PascalCase +// registration yields a Claude-shaped PreCompact payload. Note what this test +// does NOT establish — capMatrix[vscode-copilot][KindCompactPre] is a row about +// what VS Code honors on a decision, and the CLI running the PascalCase file +// gets the copilot row instead (demotion, quirk #46). That row stays inferred +// until a real VS Code session drives it. +func TestCopilotPreCompact(t *testing.T) { + t.Parallel() + requireE2E(t, "copilot") + for _, tc := range []struct { + name string + provider agenthooks.Provider + native string + }{ + {"camelCase", agenthooks.ProviderCopilotCLI, "preCompact"}, + {"pascalCase", agenthooks.ProviderVSCodeCopilot, "PreCompact"}, + } { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + rec := newRecorder(t, "") + home := copilotHome(t) + installHooks(t, rec, tc.provider, install.ScopeUser, home) + runCopilot(t, t.TempDir(), home, "/compact") + evs := rec.events(t) + requireKinds(t, evs, agenthooks.KindCompactPre) + for _, e := range ofKind(evs, agenthooks.KindCompactPre) { + if e.Native != tc.native { + t.Errorf("compact.pre native = %q, want %q", e.Native, tc.native) + } + if e.Provider != string(agenthooks.ProviderCopilotCLI) { + t.Errorf("compact.pre stamped provider %q, want %q", e.Provider, agenthooks.ProviderCopilotCLI) + } + if tc.provider != agenthooks.ProviderCopilotCLI { + continue // camelCase view only + } + ev := &agenthooks.Event{Provider: agenthooks.ProviderCopilotCLI, NativeName: e.Native, Raw: e.Raw} + in, ok := copilot.PreCompact(ev) + if !ok { + t.Fatalf("PreCompact view rejected native %q", e.Native) + } + // Trigger is the only field copilotEventName can discriminate + // preCompact on, and an explicit /compact must report "manual". + if in.Trigger != "manual" { + t.Errorf("PreCompact trigger = %q, want \"manual\" (raw: %s)", in.Trigger, e.Raw) + } + if len(in.Extra) > 0 { + t.Errorf("PreCompact has unknown fields %v (raw: %s)", keys(in.Extra), e.Raw) + } + } + }) + } +} diff --git a/e2e/harness_test.go b/e2e/harness_test.go index dcaba54..99ce086 100644 --- a/e2e/harness_test.go +++ b/e2e/harness_test.go @@ -71,8 +71,9 @@ type recorder struct { } type recorderConfig struct { - Deny string `json:"deny,omitempty"` - RewriteCommand string `json:"rewrite_command,omitempty"` + Deny string `json:"deny,omitempty"` + RewriteCommand string `json:"rewrite_command,omitempty"` + ContinueInstruction string `json:"continue_instruction,omitempty"` } // newRecorder links the recorder binary into a per-test directory and writes @@ -121,6 +122,11 @@ func manifest(bin string) install.Manifest { {Kind: agenthooks.KindToolPost, Blocking: true, Timeout: 30 * time.Second}, {Kind: agenthooks.KindToolError, Blocking: true, Timeout: 30 * time.Second}, {Kind: agenthooks.KindStop, Blocking: true, Timeout: 30 * time.Second}, + // Registered everywhere, driven only where a headless turn can + // reach them; renderers skip the kinds a provider has no event for. + {Kind: agenthooks.KindSubagentStart, Blocking: true, Timeout: 30 * time.Second}, + {Kind: agenthooks.KindSubagentStop, Blocking: true, Timeout: 30 * time.Second}, + {Kind: agenthooks.KindCompactPre, Blocking: true, Timeout: 30 * time.Second}, }, Identity: install.Identity{Name: "agenthooks-e2e", Version: "0.0.1", Description: "agenthooks e2e recorder"}, Fail: agenthooks.FailOpen, @@ -198,7 +204,11 @@ type event struct { Prompt string `json:"prompt"` Denied bool `json:"denied"` Rewritten bool `json:"rewritten"` - Raw json.RawMessage `json:"raw"` + + Continued bool `json:"continued"` + PrevContinued bool `json:"prev_continued"` + LoopCount int `json:"loop_count"` + Raw json.RawMessage `json:"raw"` } // events reads the recorder's sink; missing file means no events (yet). diff --git a/e2e/testdata/recorder/main.go b/e2e/testdata/recorder/main.go index 4df2945..a35fdf0 100644 --- a/e2e/testdata/recorder/main.go +++ b/e2e/testdata/recorder/main.go @@ -10,6 +10,7 @@ package main import ( + "bytes" "context" "encoding/json" "os" @@ -24,6 +25,12 @@ type config struct { Deny string `json:"deny,omitempty"` // RewriteCommand replaces shell tool arguments and explicitly allows the call. RewriteCommand string `json:"rewrite_command,omitempty"` + // ContinueInstruction is returned from the first agent.stop as + // ContinueWith, so a test can assert the agent really kept working. It + // fires at most once per sink: providers that never report their own + // continuation guard (stop_hook_active) would otherwise loop forever, + // and the library cap only trips on a reported LoopCount. + ContinueInstruction string `json:"continue_instruction,omitempty"` } // record is one JSONL line. Kind "tool.pre" lines are emitted twice: once by @@ -45,7 +52,13 @@ type record struct { Prompt string `json:"prompt,omitempty"` Denied bool `json:"denied,omitempty"` Rewritten bool `json:"rewritten,omitempty"` - Raw json.RawMessage `json:"raw,omitempty"` + Continued bool `json:"continued,omitempty"` + // PrevContinued/LoopCount are the provider's own continuation guard as + // the library unified it; both are recorded rather than asserted, since + // whether a provider reports one at all is what the test measures. + PrevContinued bool `json:"prev_continued,omitempty"` + LoopCount int `json:"loop_count,omitempty"` + Raw json.RawMessage `json:"raw,omitempty"` } func main() { @@ -104,6 +117,25 @@ func main() { } return agenthooks.NoDecision(), nil }) + r.OnStop(func(_ context.Context, e *agenthooks.StopEvent) (agenthooks.StopDecision, error) { + cont := cfg.ContinueInstruction != "" && !e.PreviouslyContinued && !alreadyContinued(cfg.Out) + appendRecord(cfg.Out, record{ + Typed: true, + TimeMS: e.Time.UnixMilli(), + Provider: string(e.Provider), + Variant: string(e.Variant), + Native: e.NativeName, + Kind: string(e.Kind), + Session: e.Session.ID, + Continued: cont, + PrevContinued: e.PreviouslyContinued, + LoopCount: e.LoopCount, + }) + if cont { + return agenthooks.ContinueWith(cfg.ContinueInstruction), nil + } + return agenthooks.Finish(), nil + }) agenthooks.Main(r) } @@ -122,6 +154,14 @@ func loadConfig() config { return cfg } +// alreadyContinued reports whether an earlier hook process for this sink +// already returned a continuation. Each hook firing is its own process, so +// the sink is the only shared state available to bound the loop. +func alreadyContinued(path string) bool { + data, err := os.ReadFile(path) + return err == nil && bytes.Contains(data, []byte(`"continued":true`)) +} + func appendRecord(path string, rec record) { if path == "" { return diff --git a/event.go b/event.go index e19a8bc..46f8aa0 100644 --- a/event.go +++ b/event.go @@ -21,7 +21,10 @@ const ( ProviderOpenCode Provider = "opencode" ProviderOpenClaw Provider = "openclaw" // OpenClaw Gateway (typed plugin hooks via shim) ProviderKimi Provider = "kimi-code" // Kimi Code CLI ("kimi" accepted as a flag alias) - ProviderCopilot Provider = "copilot" // GitHub Copilot CLI (hooks are CLI-only) + // Copilot ships two dialects behind one product name, so a consumer + // switching on "is this Copilot?" must consider both constants. + ProviderCopilotCLI Provider = "copilot-cli" // GitHub Copilot CLI (camelCase dialect) + ProviderVSCodeCopilot Provider = "vscode-copilot" // Copilot Chat in VS Code (Claude-shaped dialect) ) // Variant refines Provider where runtime behavior genuinely differs. @@ -207,27 +210,66 @@ var canonicalNames = map[string]CanonicalTool{ "bash": ToolShell, "shell": ToolShell, "run_shell_command": ToolShell, "run_terminal_cmd": ToolShell, "exec": ToolShell, "local_shell": ToolShell, "terminal": ToolShell, + // VS Code Copilot Chat, enumerated from the extension's wire-name table + // (not its package.json, which registers different names). Everything here + // executes an arbitrary command or arbitrary code, so a deny-shell policy + // that missed any of them would have a hole. + // + // Deliberately NOT shell, having no execution of their own: the read-only + // get_terminal_output / get_task_output / terminal_selection / + // terminal_last_command, the kill_terminal control, and run_vscode_command + // — which can reach a terminal command indirectly, but classing the whole + // command palette as shell would make an allow-shell policy far too broad. + "run_in_terminal": ToolShell, "send_to_terminal": ToolShell, + "run_task": ToolShell, "create_and_run_task": ToolShell, + "runtests": ToolShell, "run_notebook_cell": ToolShell, + "run_playwright_code": ToolShell, "read": ToolFileRead, "read_file": ToolFileRead, "readfile": ToolFileRead, "view_file": ToolFileRead, "read_many_files": ToolFileRead, "notebookread": ToolFileRead, + // The Copilot CLI names its file tools with bare verbs no other provider + // uses: `view` reads a path and `create` writes one ({path, file_text}). + // Neither matched anything above, so a deny-file-read or deny-file-write + // policy silently covered nothing on that CLI. + "view": ToolFileRead, "write": ToolFileWrite, "write_file": ToolFileWrite, "writefile": ToolFileWrite, "create_file": ToolFileWrite, "save_file": ToolFileWrite, + "create": ToolFileWrite, "edit": ToolFileEdit, "multiedit": ToolFileEdit, "apply_patch": ToolFileEdit, "replace": ToolFileEdit, "edit_file": ToolFileEdit, "notebookedit": ToolFileEdit, "str_replace": ToolFileEdit, "search_replace": ToolFileEdit, "patch": ToolFileEdit, - "strreplacefile": ToolFileEdit, + "strreplacefile": ToolFileEdit, + "replace_string_in_file": ToolFileEdit, "multi_replace_string_in_file": ToolFileEdit, + "insert_edit_into_file": ToolFileEdit, "edit_notebook_file": ToolFileEdit, + "edit_files": ToolFileEdit, "grep": ToolSearch, "glob": ToolSearch, "search": ToolSearch, "codebase_search": ToolSearch, "search_file_content": ToolSearch, "find_files": ToolSearch, "list_directory": ToolSearch, "ls": ToolSearch, "glob_file_search": ToolSearch, "grep_search": ToolSearch, "file_search": ToolSearch, + "list_dir": ToolSearch, "semantic_search": ToolSearch, + "search_workspace_symbols": ToolSearch, "test_search": ToolSearch, + "github_repo": ToolSearch, "github_text_search": ToolSearch, + "read_project_structure": ToolSearch, "webfetch": ToolFetch, "web_fetch": ToolFetch, "websearch": ToolFetch, "web_search": ToolFetch, "fetch": ToolFetch, "google_web_search": ToolFetch, + "fetch_webpage": ToolFetch, + // VS Code's browser tools, split by what they actually do: these four pull + // external content into the session, which is what a deny-fetch policy is + // for. The rest of the suite — click_element, type_in_page, hover_element, + // drag_element, handle_dialog — only manipulates an already-open page and + // stays ToolOther. (run_playwright_code executes code, so it is shell.) + "open_browser_page": ToolFetch, "navigate_page": ToolFetch, + "read_page": ToolFetch, "screenshot_page": ToolFetch, "task": ToolTask, "agent": ToolTask, "subagent": ToolTask, + // VS Code spells its four subagent tools distinctly; none matched the + // generic names above, so a ToolTask matcher saw no VS Code subagent. + "runsubagent": ToolTask, "search_subagent": ToolTask, + "explore_subagent": ToolTask, "execution_subagent": ToolTask, } // CanonicalToolFor classifies a native tool name. MCP names (any dialect) @@ -243,8 +285,8 @@ func CanonicalToolFor(name string) CanonicalTool { } // ParseMCPName decodes the three MCP tool-name dialects: -// mcp__server__tool (Claude/Codex), mcp_server_tool (Gemini, best-effort -// since "_" is ambiguous), and MCP:tool (Cursor, server unknown). +// mcp__server__tool (Claude/Codex), mcp_server_tool (Gemini/VS Code, +// best-effort since "_" is ambiguous), and MCP:tool (Cursor, server unknown). // It returns nil when the name is not MCP-shaped. func ParseMCPName(name string) *MCPCall { switch { diff --git a/event_test.go b/event_test.go index 93d23ae..fa221f6 100644 --- a/event_test.go +++ b/event_test.go @@ -47,6 +47,47 @@ func TestCanonicalToolFor(t *testing.T) { "mcp__gh__issues": ToolMCP, "MCP:issues": ToolMCP, "SomethingCustom": ToolOther, + + // Copilot CLI's file tools, observed live on 1.0.81: bare verbs that + // matched nothing, so every read and write on that CLI was ToolOther. + "view": ToolFileRead, + "create": ToolFileWrite, + + // VS Code Copilot Chat. run_in_terminal and replace_string_in_file were + // observed in a live capture session; the rest come from the + // extension's wire-name table. Before these landed they all classified + // as ToolOther, so a deny-shell policy silently did not cover any of + // VS Code's execution tools and a ToolTask matcher saw no subagent. + "run_in_terminal": ToolShell, + "send_to_terminal": ToolShell, + "create_and_run_task": ToolShell, + "run_task": ToolShell, + "runTests": ToolShell, + "run_notebook_cell": ToolShell, + "run_playwright_code": ToolShell, + "replace_string_in_file": ToolFileEdit, + "edit_notebook_file": ToolFileEdit, + "semantic_search": ToolSearch, + "github_text_search": ToolSearch, + "fetch_webpage": ToolFetch, + "runSubagent": ToolTask, + "execution_subagent": ToolTask, + + // The browser suite splits: these pull external content in... + "open_browser_page": ToolFetch, + "navigate_page": ToolFetch, + "read_page": ToolFetch, + "screenshot_page": ToolFetch, + // ...while these only manipulate an already-open page. + "click_element": ToolOther, + "type_in_page": ToolOther, + + // Deliberately ToolOther — no execution of their own. Pinned so a + // future "classify everything terminal-ish as shell" sweep has to + // argue with a test rather than silently widen an allow-shell policy. + "get_terminal_output": ToolOther, + "kill_terminal": ToolOther, + "run_vscode_command": ToolOther, } for name, want := range cases { if got := CanonicalToolFor(name); got != want { diff --git a/install/install.go b/install/install.go index 3201d4f..7121ca4 100644 --- a/install/install.go +++ b/install/install.go @@ -95,8 +95,10 @@ func Render(m Manifest, t Target) (fs.FS, error) { return renderOpenClaw(m, t) case agenthooks.ProviderKimi: return renderKimi(m, t) - case agenthooks.ProviderCopilot: + case agenthooks.ProviderCopilotCLI: return renderCopilot(m, t) + case agenthooks.ProviderVSCodeCopilot: + return renderVSCode(m, t) } return nil, fmt.Errorf("install: unknown provider %q", t.Provider) } diff --git a/install/install_test.go b/install/install_test.go index 5a8b62f..4fad926 100644 --- a/install/install_test.go +++ b/install/install_test.go @@ -9,6 +9,7 @@ import ( "io/fs" "os" "path/filepath" + "runtime" "strings" "testing" "time" @@ -411,7 +412,7 @@ func TestRenderOpenCodeShim(t *testing.T) { // - bash/powershell keys (Copilot fills both from command; splitting them // here would render the argv twice with no test on the second copy). func TestRenderCopilotPlugin(t *testing.T) { - fsys, err := Render(testManifest(), Target{Provider: agenthooks.ProviderCopilot, Scope: ScopePlugin}) + fsys, err := Render(testManifest(), Target{Provider: agenthooks.ProviderCopilotCLI, Scope: ScopePlugin}) if err != nil { t.Fatal(err) } @@ -458,7 +459,7 @@ func TestRenderCopilotPlugin(t *testing.T) { if pre[0].Type != "command" || pre[0].TimeoutSec != 30 { t.Errorf("preToolUse entry wrong: %+v", pre[0]) } - if !strings.Contains(pre[0].Command, "agenthooks run --provider=copilot") { + if !strings.Contains(pre[0].Command, "agenthooks run --provider=copilot-cli") { t.Errorf("command wrong: %q", pre[0].Command) } if len(pre[0].Bash) > 0 || len(pre[0].PowerShell) > 0 { @@ -474,13 +475,13 @@ func TestRenderCopilotPlugin(t *testing.T) { func TestRenderCopilotScopes(t *testing.T) { // Project scope goes to .github/hooks/, user scope to hooks/hooks.json // under Target.Dir (~/.copilot). Plugin scope has nowhere to put the name. - proj, err := Render(testManifest(), Target{Provider: agenthooks.ProviderCopilot, Scope: ScopeProject}) + proj, err := Render(testManifest(), Target{Provider: agenthooks.ProviderCopilotCLI, Scope: ScopeProject}) if err != nil { t.Fatal(err) } readRendered(t, proj, ".github/hooks/agenthooks.json") - user, err := Render(testManifest(), Target{Provider: agenthooks.ProviderCopilot, Scope: ScopeUser}) + user, err := Render(testManifest(), Target{Provider: agenthooks.ProviderCopilotCLI, Scope: ScopeUser}) if err != nil { t.Fatal(err) } @@ -488,7 +489,131 @@ func TestRenderCopilotScopes(t *testing.T) { m := testManifest() m.Identity.Name = "" - if _, err := Render(m, Target{Provider: agenthooks.ProviderCopilot, Scope: ScopePlugin}); err == nil { + if _, err := Render(m, Target{Provider: agenthooks.ProviderCopilotCLI, Scope: ScopePlugin}); err == nil { t.Error("plugin scope with no Identity.Name must fail, not emit a nameless package") } } + +// TestRenderVSCodeScopes pins the two paths and the basename. Both directories +// are globbed by VS Code AND by the Copilot CLI, so the basename is the only +// thing keeping this file from colliding with render_copilot.go's — and +// agenthooks-vscode.json is neither settings.json nor hooks.json, so the file +// stays whole-file owned instead of being merged into. +func TestRenderVSCodeScopes(t *testing.T) { + proj, err := Render(testManifest(), Target{Provider: agenthooks.ProviderVSCodeCopilot, Scope: ScopeProject}) + if err != nil { + t.Fatal(err) + } + readRendered(t, proj, ".github/hooks/agenthooks-vscode.json") + + user, err := Render(testManifest(), Target{Provider: agenthooks.ProviderVSCodeCopilot, Scope: ScopeUser}) + if err != nil { + t.Fatal(err) + } + raw := readRendered(t, user, "hooks/agenthooks-vscode.json") // Target.Dir is ~/.copilot + if isMergeableJSON("hooks/agenthooks-vscode.json") { + t.Error("agenthooks-vscode.json must not be merge-eligible; a merge would fold it into the CLI's config") + } + + if _, err := Render(testManifest(), Target{Provider: agenthooks.ProviderVSCodeCopilot, Scope: ScopePlugin}); err == nil { + t.Error("plugin scope must fail: VS Code loads plugin hooks through ~/.copilot, which user scope already covers") + } + + // PascalCase event keys, one per declared kind. A camelCase key here would + // still resolve in VS Code (it accepts the CLI's names) but would pick up + // the CLI's event vocabulary, where Stop is spelled agentStop. + var cfg struct { + Hooks map[string][]struct { + Type string `json:"type"` + Command string `json:"command"` + Timeout int `json:"timeout"` + TimeoutSec int `json:"timeoutSec"` + } `json:"hooks"` + } + if err := json.Unmarshal(raw, &cfg); err != nil { + t.Fatal(err) + } + for _, event := range []string{"PreToolUse", "Stop", "PostToolUse"} { + if len(cfg.Hooks[event]) != 1 { + t.Fatalf("%s not registered: %v", event, cfg.Hooks) + } + } + if len(cfg.Hooks) != 3 { + t.Errorf("hooks = %v, want one entry per declared kind", cfg.Hooks) + } + pre := cfg.Hooks["PreToolUse"][0] + if pre.Type != "command" { + t.Errorf("type = %q, want command", pre.Type) + } + if !strings.Contains(pre.Command, "agenthooks run --provider=vscode-copilot") { + t.Errorf("command wrong: %q", pre.Command) + } + // VS Code parses matcher values and ignores them, so --filter is the only + // enforcement that is actually true. + if !strings.Contains(pre.Command, "--filter=") { + t.Errorf("no --filter in %q; VS Code ignores matchers, so a scoped hook would fire on every tool", pre.Command) + } +} + +// TestRenderVSCodeOmissions pins the keys whose wrong presence or absence fails +// silently rather than loudly: a matcher that reads as enforcement VS Code does +// not perform, a version key no VS Code example carries (an unknown key is a +// schema-validation risk), bash/powershell keys VS Code does not understand at +// all (its platform override is windows), and the two unreconciled timeout +// spellings — the reference table says timeout, a usage example says +// timeoutSec, and reading the missing one silently means the 30s default. +func TestRenderVSCodeOmissions(t *testing.T) { + fsys, err := Render(testManifest(), Target{Provider: agenthooks.ProviderVSCodeCopilot, Scope: ScopeProject}) + if err != nil { + t.Fatal(err) + } + raw := readRendered(t, fsys, ".github/hooks/agenthooks-vscode.json") + for _, key := range []string{`"matcher"`, `"version"`, `"bash"`, `"powershell"`, `"osx"`} { + if bytes.Contains(raw, []byte(key)) { + t.Errorf("%s key present:\n%s", key, raw) + } + } + var cfg struct { + Hooks map[string][]struct { + Timeout *int `json:"timeout"` + TimeoutSec *int `json:"timeoutSec"` + } `json:"hooks"` + } + if err := json.Unmarshal(raw, &cfg); err != nil { + t.Fatal(err) + } + pre := cfg.Hooks["PreToolUse"][0] + if pre.Timeout == nil || pre.TimeoutSec == nil || *pre.Timeout != 30 || *pre.TimeoutSec != 30 { + t.Errorf("both timeout spellings must carry the same value, got timeout=%v timeoutSec=%v", pre.Timeout, pre.TimeoutSec) + } + stop := cfg.Hooks["Stop"][0] + if stop.Timeout == nil || *stop.Timeout != 60 { + t.Errorf("Stop timeout = %v, want the 60s default", stop.Timeout) + } +} + +// TestHookCommandQuotesSpacedBinary pins the quoting of a consumer binary whose +// path contains spaces. The dialect follows the HOST OS, not the provider +// (shellQuote, install.go:355-374): configs are rendered on the machine that +// runs them, cmd.exe has no single-quote syntax and POSIX shells have no +// cmd-style escaping. +// +// The Windows form is cmd.exe's, and deliberately so: PowerShell parses a +// statement that begins with a quote as an expression, so it needs a leading +// call operator (& "C:\Program Files\...") that cmd.exe in turn rejects — no +// single string is valid in both. Both shells are reachable on Windows (the +// Copilot CLI copies command into its powershell key, render_copilot.go:34-36), +// so if a spaced path is ever observed failing under PowerShell the fix is a +// per-shell rendering for those providers, not a change to this quoting. +func TestHookCommandQuotesSpacedBinary(t *testing.T) { + m := testManifest() + m.Command = []string{"/opt/My Hooks/myhooks"} + want := `'/opt/My Hooks/myhooks' agenthooks run --provider=vscode-copilot` + if runtime.GOOS == "windows" { + want = `"/opt/My Hooks/myhooks" agenthooks run --provider=vscode-copilot` + } + got := hookCommand(m, agenthooks.ProviderVSCodeCopilot, m.Hooks[0]) + if !strings.HasPrefix(got, want) { + t.Errorf("spaced binary path unquoted or wrong dialect:\n got %s\nwant prefix %s", got, want) + } +} diff --git a/install/render_copilot.go b/install/render_copilot.go index 688bc93..12622f1 100644 --- a/install/render_copilot.go +++ b/install/render_copilot.go @@ -49,7 +49,7 @@ func renderCopilot(m Manifest, t Target) (fs.FS, error) { } hooks[event] = append(hooks[event], copilotHookEntry{ Type: "command", - Command: hookCommand(m, agenthooks.ProviderCopilot, spec), + Command: hookCommand(m, agenthooks.ProviderCopilotCLI, spec), TimeoutSec: timeoutSeconds(spec), }) } diff --git a/install/render_vscode.go b/install/render_vscode.go new file mode 100644 index 0000000..d2f4f7d --- /dev/null +++ b/install/render_vscode.go @@ -0,0 +1,110 @@ +package install + +import ( + "errors" + "io/fs" + "path" + "strings" + + "github.com/speakeasy-api/agenthooks" +) + +// kindToVSCode maps unified kinds to the eight PascalCase events Copilot Chat +// fires in VS Code (HOOKS_BY_TARGET[Target.VSCode] upstream). The six Copilot +// CLI events with no VS Code counterpart — sessionEnd, userPromptTransformed, +// postToolUseFailure, errorOccurred, permissionRequest, notification — are +// absent, and are reachable only through the camelCase CLI file that +// render_copilot.go writes. +var kindToVSCode = map[agenthooks.EventKind]string{ + agenthooks.KindSessionStart: "SessionStart", + agenthooks.KindPromptSubmitted: "UserPromptSubmit", + agenthooks.KindToolPre: "PreToolUse", + agenthooks.KindToolPost: "PostToolUse", + agenthooks.KindStop: "Stop", + agenthooks.KindSubagentStart: "SubagentStart", + agenthooks.KindSubagentStop: "SubagentStop", + agenthooks.KindCompactPre: "PreCompact", +} + +// vscodeHookEntry is one command entry, with three deliberate omissions and +// two deliberate duplications: +// +// - No matcher: VS Code parses matcher values and then ignores them, so a +// key here would read as enforcement that does not exist. --filter on the +// rendered argv is the only true enforcement (hookCommand emits it for +// every non-empty matcher, because CompileMatcher has no VS Code dialect). +// - No version: no VS Code example carries one, and an unknown key is a +// schema-validation risk for zero benefit. That is also what keeps this +// file distinguishable from render_copilot.go's CLI document. +// - No bash/powershell: VS Code's platform split is windows/linux/osx. The +// windows override uses PowerShell's call operator so a quoted executable +// path is invoked instead of being parsed as a string expression. +// - Both timeout spellings, same value: the VS Code reference table says +// `timeout` while a usage example on the same doc set says `timeoutSec`, +// and nothing reconciles them. Reading the wrong one silently falls back +// to the 30s default; two JSON keys cost nothing. +type vscodeHookEntry struct { + Type string `json:"type"` + Command string `json:"command"` + Windows string `json:"windows"` + Timeout int `json:"timeout,omitempty"` + TimeoutSec int `json:"timeoutSec,omitempty"` +} + +func renderVSCode(m Manifest, t Target) (fs.FS, error) { + if t.Scope == ScopePlugin { + return nil, errors.New("install: vscode-copilot has no plugin scope; install at user or project scope") + } + hooks := map[string][]vscodeHookEntry{} + for _, spec := range m.Hooks { + event, ok := kindToVSCode[spec.Kind] + if !ok { + continue + } + secs := timeoutSeconds(spec) + hooks[event] = append(hooks[event], vscodeHookEntry{ + Type: "command", + Command: hookCommand(m, agenthooks.ProviderVSCodeCopilot, spec), + Windows: vscodePowerShellCommand(m, spec), + Timeout: secs, + TimeoutSec: secs, + }) + } + content, err := jsonFile(map[string]any{"hooks": hooks}) + if err != nil { + return nil, err + } + // Both directories are globbed by VS Code AND by the Copilot CLI, so the + // basename is what keeps this file distinct from the CLI's. It is neither + // settings.json nor hooks.json, so isMergeableJSON says no and the file is + // whole-file owned — the same posture render_copilot.go's project file has. + files := map[string][]byte{} + if t.Scope == ScopeProject { + files[path.Join(".github", "hooks", "agenthooks-vscode.json")] = content + } else { + files[path.Join("hooks", "agenthooks-vscode.json")] = content // Target.Dir is ~/.copilot + } + return memFS(files), nil +} + +func vscodePowerShellCommand(m Manifest, spec HookSpec) string { + parts := make([]string, 0, len(m.Command)+5) + for _, arg := range m.Command { + parts = append(parts, powerShellQuote(arg)) + } + parts = append(parts, powerShellQuote("agenthooks"), powerShellQuote("run"), + powerShellQuote("--provider="+string(agenthooks.ProviderVSCodeCopilot))) + if spec.Timeout > 0 { + parts = append(parts, powerShellQuote("--timeout="+spec.Timeout.String())) + } + if !spec.Tools.IsEmpty() { + if _, ok := agenthooks.CompileMatcher(agenthooks.ProviderVSCodeCopilot, spec.Tools); !ok { + parts = append(parts, powerShellQuote("--filter="+spec.Tools.Encode())) + } + } + return "& " + strings.Join(parts, " ") +} + +func powerShellQuote(arg string) string { + return "'" + strings.ReplaceAll(arg, "'", "''") + "'" +} diff --git a/install/render_vscode_test.go b/install/render_vscode_test.go new file mode 100644 index 0000000..eb99a2e --- /dev/null +++ b/install/render_vscode_test.go @@ -0,0 +1,31 @@ +package install + +import ( + "encoding/json" + "testing" + + "github.com/speakeasy-api/agenthooks" +) + +func TestRenderVSCodePowerShellCommand(t *testing.T) { + m := Manifest{ + Command: []string{`C:\Program Files\Agent Hooks\hook.exe`, `arg's`}, + Hooks: []HookSpec{{Kind: agenthooks.KindStop}}, + } + fsys, err := renderVSCode(m, Target{Scope: ScopeProject}) + if err != nil { + t.Fatal(err) + } + var cfg struct { + Hooks map[string][]struct { + Windows string `json:"windows"` + } `json:"hooks"` + } + if err := json.Unmarshal(readRendered(t, fsys, ".github/hooks/agenthooks-vscode.json"), &cfg); err != nil { + t.Fatal(err) + } + const want = `& 'C:\Program Files\Agent Hooks\hook.exe' 'arg''s' 'agenthooks' 'run' '--provider=vscode-copilot'` + if got := cfg.Hooks["Stop"][0].Windows; got != want { + t.Errorf("windows command = %q, want %q", got, want) + } +} diff --git a/matcher_test.go b/matcher_test.go index ec4643a..42db2d8 100644 --- a/matcher_test.go +++ b/matcher_test.go @@ -73,6 +73,17 @@ func TestCapabilityDivergences(t *testing.T) { if Capabilities(ProviderOpenCode, "", KindStop).Has(CapContinueAgent) { t.Error("opencode session.idle cannot continue the agent") } + for _, kind := range []EventKind{KindSubagentStart, KindCompactPre} { + if got := Capabilities(ProviderVSCodeCopilot, "", kind); len(got) != 0 { + t.Errorf("VS Code %s is observe-only; capabilities = %v", kind, got) + } + } + for _, kind := range []EventKind{KindStop, KindSubagentStop} { + got := Capabilities(ProviderVSCodeCopilot, "", kind) + if !got.Has(CapContinueAgent) || !got.Has(CapSystemMessage) || got.Has(CapStopAgent) { + t.Errorf("VS Code %s stop capabilities = %v", kind, got) + } + } e := &Event{Provider: ProviderCursor, Kind: KindToolPre} if !e.Can(CapDeny) { t.Error("Event.Can should reflect the matrix") @@ -84,8 +95,8 @@ func TestCapabilityDivergences(t *testing.T) { func TestQuirkRegistry(t *testing.T) { qs := Quirks() - if len(qs) != 37 { - t.Fatalf("expected the 37 seeded quirks, got %d", len(qs)) + if len(qs) != 49 { + t.Fatalf("expected the 49 seeded quirks, got %d", len(qs)) } seen := map[int]bool{} for _, q := range qs { @@ -97,8 +108,12 @@ func TestQuirkRegistry(t *testing.T) { } seen[q.ID] = true } - if got := len(QuirksFor(ProviderCursor)); got == 0 { - t.Error("cursor quirks missing") + // The two Copilot dialects are registered separately on purpose: the CLI + // rows and the VS Code rows are what make the split legible. + for _, p := range []Provider{ProviderCursor, ProviderCopilotCLI, ProviderVSCodeCopilot} { + if len(QuirksFor(p)) == 0 { + t.Errorf("%s quirks missing", p) + } } } diff --git a/mcpresolve.go b/mcpresolve.go index d8e02f1..32bf776 100644 --- a/mcpresolve.go +++ b/mcpresolve.go @@ -5,6 +5,7 @@ import ( "encoding/json" "os" "path/filepath" + "regexp" "sort" "strings" @@ -74,8 +75,8 @@ func (r *Runner) resolveMCPWithOpenCodeInventory(ctx context.Context, typed any, // Copilot branches here rather than in resolveMCPProvider's switch: that // path only enriches a tc.MCP the codec already built, and Copilot's // unmarked - names leave it nil, so it would return early. - if base.Provider == ProviderCopilot { - r.resolveCopilotMCP(tc, loadMCPConfigEntries(ProviderCopilot, base.Session.CWD)) + if base.Provider == ProviderCopilotCLI { + r.resolveCopilotMCP(tc, loadMCPConfigEntries(ProviderCopilotCLI, base.Session.CWD)) return } r.resolveMCPProvider(ctx, typed) @@ -120,6 +121,12 @@ func (r *Runner) resolveMCPProvider(ctx context.Context, typed any) { // is ambiguous whenever the server name contains "_" (quirk #15), so // match configured names longest-first. matched, server, tool = matchSanitizedPrefix(entries, tc.Name, "mcp_", "_", verbatimMCPName) + case ProviderVSCodeCopilot: + entries := loadMCPConfigEntries(base.Provider, base.Session.CWD) + // VS Code uses mcp__ and truncates its sanitized + // server prefix. Colliding runtime-assigned numeric suffixes remain + // unresolved because their assignment depends on registration order. + matched, server, tool = matchSanitizedPrefix(entries, tc.Name, "mcp_", "_", vscodeSanitizeMCPName) case ProviderCursor: entries := loadMCPConfigEntries(base.Provider, base.Session.CWD) if tc.MCP.Server != "" { @@ -340,6 +347,19 @@ func verbatimMCPName(name string) string { return strings.ReplaceAll(name, " ", "_") } +var vscodeUnsafeMCPName = regexp.MustCompile(`[^a-z0-9_.-]+`) + +// vscodeSanitizeMCPName mirrors VS Code's McpPrefixGenerator. The full MCP +// prefix is capped at 18 characters: "mcp_", at most 13 server characters, +// then "_" before the tool name. +func vscodeSanitizeMCPName(name string) string { + s := vscodeUnsafeMCPName.ReplaceAllString(strings.ToLower(name), "_") + if len(s) > 13 { + s = s[:13] + } + return s +} + // loadMCPConfigEntries reads the provider's MCP server config files. Missing // or malformed files contribute nothing. More specific scopes come first and // win name collisions (project over user). @@ -400,7 +420,7 @@ func loadMCPConfigEntries(p Provider, cwd string) []mcpConfigEntry { if home != "" { groups = append(groups, readMCPServersJSON(filepath.Join(home, ".kimi", "mcp.json"))) } - case ProviderCopilot: + case ProviderCopilotCLI: // Workspace scope first: Copilot auto-loads .mcp.json and // .github/mcp.json from the repo and those win name collisions against the // user-scope file. COPILOT_HOME overrides the user directory @@ -420,6 +440,12 @@ func loadMCPConfigEntries(p Provider, cwd string) []mcpConfigEntry { if dir != "" { groups = append(groups, readMCPServersJSON(filepath.Join(dir, "mcp-config.json"))) } + case ProviderVSCodeCopilot: + if cwd != "" { + groups = append(groups, + readVSCodeMCPJSON(filepath.Join(cwd, ".vscode", "mcp.json")), + readMCPServersJSON(filepath.Join(cwd, ".mcp.json"))) + } case ProviderOpenCode: // opencode.json(c) at project root; global config under // $XDG_CONFIG_HOME/opencode (default ~/.config/opencode). The .jsonc @@ -472,6 +498,20 @@ func parseMCPServersJSON(data []byte) []mcpConfigEntry { return mcpEntriesFromJSON(doc.MCPServers) } +func readVSCodeMCPJSON(path string) []mcpConfigEntry { + data, err := os.ReadFile(path) + if err != nil { + return nil + } + var doc struct { + Servers map[string]mcpServerJSON `json:"servers"` + } + if err := json.Unmarshal(stripJSONCComments(data), &doc); err != nil { + return nil + } + return mcpEntriesFromJSON(doc.Servers) +} + func firstMCPEntries(groups ...[]mcpConfigEntry) []mcpConfigEntry { seen := map[string]bool{} var out []mcpConfigEntry @@ -587,9 +627,9 @@ func openCodeMCPEntries(servers map[string]opencodeMCPJSON) []mcpConfigEntry { return out } -// stripJSONCComments blanks // and /* */ comments (string-aware) so the -// stdlib JSON decoder accepts .jsonc config files. Byte positions are -// preserved by replacing comment bytes with spaces. +// stripJSONCComments blanks // and /* */ comments plus trailing commas +// (string-aware) so the stdlib JSON decoder accepts .jsonc config files. Byte +// positions are preserved by replacing removed syntax with spaces. func stripJSONCComments(data []byte) []byte { out := make([]byte, len(data)) copy(out, data) @@ -630,6 +670,32 @@ func stripJSONCComments(data []byte) []byte { inBlock = true } } + + // JSONC permits a comma before a closing object or array delimiter. Comments + // are spaces now, so a second string-aware pass can inspect the next + // non-whitespace byte without needing to understand comment boundaries. + inStr = false + for i := 0; i < len(out); i++ { + switch out[i] { + case '\\': + if inStr { + i++ + } + case '"': + inStr = !inStr + case ',': + if inStr { + continue + } + j := i + 1 + for j < len(out) && (out[j] == ' ' || out[j] == '\t' || out[j] == '\r' || out[j] == '\n') { + j++ + } + if j < len(out) && (out[j] == '}' || out[j] == ']') { + out[i] = ' ' + } + } + } return out } diff --git a/mcpresolve_test.go b/mcpresolve_test.go index a8749d8..0b9fdad 100644 --- a/mcpresolve_test.go +++ b/mcpresolve_test.go @@ -925,7 +925,7 @@ func TestResolveMCPCopilotDetection(t *testing.T) { }`) r := mcpTestRunner(t) - ev := mcpToolPre(ProviderCopilot, "", "github-create_issue") + ev := mcpToolPre(ProviderCopilotCLI, "", "github-create_issue") if ev.Tool.MCP != nil || ev.Tool.Canonical == ToolMCP { t.Fatalf("precondition: codec must not classify copilot MCP names: %+v", ev.Tool) } @@ -938,7 +938,7 @@ func TestResolveMCPCopilotDetection(t *testing.T) { t.Errorf("copilot identity/transport wrong: %+v", ev.Tool.MCP) } - ev = mcpToolPre(ProviderCopilot, "", "tracker-list_issues") + ev = mcpToolPre(ProviderCopilotCLI, "", "tracker-list_issues") r.resolveMCP(ev) if ev.Tool.MCP == nil || ev.Tool.MCP.URL != "https://tracker.example.com/mcp" || ev.Tool.MCP.Tool != "list_issues" { t.Errorf("copilot remote server wrong: %+v", ev.Tool.MCP) @@ -946,13 +946,13 @@ func TestResolveMCPCopilotDetection(t *testing.T) { // Hyphenated native tool with no matching server: stays native. This is // the boundary that keeps the "-" separator usable at all. - ev = mcpToolPre(ProviderCopilot, "", "read-file") + ev = mcpToolPre(ProviderCopilotCLI, "", "read-file") r.resolveMCP(ev) if ev.Tool.MCP != nil { t.Errorf("unconfigured hyphenated native tool must not detect: %+v", ev.Tool.MCP) } - ev = mcpToolPre(ProviderCopilot, "", "shell") + ev = mcpToolPre(ProviderCopilotCLI, "", "shell") r.resolveMCP(ev) if ev.Tool.MCP != nil || ev.Tool.Canonical != ToolShell { t.Errorf("native copilot tool must stay native: %+v", ev.Tool) @@ -969,7 +969,7 @@ func TestResolveMCPCopilotHyphenCollision(t *testing.T) { writeConfig(t, filepath.Join(home, ".copilot", "mcp-config.json"), `{ "mcpServers": {"read": {"url": "https://read.example.com/mcp"}} }`) - ev := mcpToolPre(ProviderCopilot, "", "read-file") + ev := mcpToolPre(ProviderCopilotCLI, "", "read-file") mcpTestRunner(t).resolveMCP(ev) if ev.Tool.MCP == nil || ev.Tool.MCP.Server != "read" || ev.Tool.MCP.Tool != "file" { t.Errorf("expected the documented hyphen collision, got: %+v", ev.Tool.MCP) @@ -986,7 +986,7 @@ func TestResolveMCPCopilotLongestPrefixWins(t *testing.T) { "a-b": {"url": "https://long.example.com"} } }`) - ev := mcpToolPre(ProviderCopilot, "", "a-b-do") + ev := mcpToolPre(ProviderCopilotCLI, "", "a-b-do") mcpTestRunner(t).resolveMCP(ev) if ev.Tool.MCP == nil || ev.Tool.MCP.Server != "a-b" || ev.Tool.MCP.Tool != "do" || ev.Tool.MCP.URL != "https://long.example.com" { @@ -1020,28 +1020,28 @@ func TestResolveMCPCopilotProjectScope(t *testing.T) { r := mcpTestRunner(t) // Same name in both scopes: the workspace transport wins. - ev := mcpToolPre(ProviderCopilot, cwd, "github-create_issue") + ev := mcpToolPre(ProviderCopilotCLI, cwd, "github-create_issue") r.resolveMCP(ev) if ev.Tool.MCP == nil || ev.Tool.MCP.URL != "https://workspace.example.com/mcp" { t.Errorf("workspace scope must outrank user scope: %+v", ev.Tool.MCP) } // .mcp.json is read too. - ev = mcpToolPre(ProviderCopilot, cwd, "tracker-list_issues") + ev = mcpToolPre(ProviderCopilotCLI, cwd, "tracker-list_issues") r.resolveMCP(ev) if ev.Tool.MCP == nil || ev.Tool.MCP.URL != "https://tracker.example.com/mcp" { t.Errorf(".mcp.json server not resolved: %+v", ev.Tool.MCP) } // At workspace scope, .mcp.json takes precedence over .github/mcp.json. - ev = mcpToolPre(ProviderCopilot, cwd, "shared-search") + ev = mcpToolPre(ProviderCopilotCLI, cwd, "shared-search") r.resolveMCP(ev) if ev.Tool.MCP == nil || ev.Tool.MCP.URL != "https://root-file.example.com/mcp" { t.Errorf(".mcp.json must outrank .github/mcp.json: %+v", ev.Tool.MCP) } // User scope still resolves for names the workspace does not define. - ev = mcpToolPre(ProviderCopilot, cwd, "notes-search") + ev = mcpToolPre(ProviderCopilotCLI, cwd, "notes-search") r.resolveMCP(ev) if ev.Tool.MCP == nil || ev.Tool.MCP.URL != "https://notes.example.com/mcp" { t.Errorf("user scope must still resolve: %+v", ev.Tool.MCP) @@ -1058,13 +1058,46 @@ func TestResolveMCPCopilotHomeOverride(t *testing.T) { writeConfig(t, filepath.Join(dir, "mcp-config.json"), `{ "mcpServers": {"notes": {"url": "https://notes.example.com/mcp"}} }`) - ev := mcpToolPre(ProviderCopilot, "", "notes-search") + ev := mcpToolPre(ProviderCopilotCLI, "", "notes-search") mcpTestRunner(t).resolveMCP(ev) if ev.Tool.MCP == nil || ev.Tool.MCP.URL != "https://notes.example.com/mcp" { t.Errorf("COPILOT_HOME override not honoured: %+v", ev.Tool.MCP) } } +func TestResolveMCPVSCode(t *testing.T) { + isolateHome(t) + cwd := t.TempDir() + writeConfig(t, filepath.Join(cwd, ".vscode", "mcp.json"), `{ + // VS Code uses a servers block and permits JSONC comments and trailing commas. + "servers": { + "github": { + "type": "http", + "url": "https://example.test/vscode", + }, + }, + }`) + writeConfig(t, filepath.Join(cwd, ".mcp.json"), `{ + "mcpServers": {"My Long Server Name!!!": {"url": "https://example.test/root"}} + }`) + r := mcpTestRunner(t) + for _, tc := range []struct { + name, server, tool, url string + }{ + {"mcp_github_list_issues", "github", "list_issues", "https://example.test/vscode"}, + {"mcp_my_long_serve_search", "my_long_serve", "search", "https://example.test/root"}, + } { + ev := mcpToolPre(ProviderVSCodeCopilot, cwd, tc.name) + r.resolveMCP(ev) + if ev.Tool.MCP == nil { + t.Fatalf("%s: MCP call was not recognized", tc.name) + } + if got := ev.Tool.MCP; got.Server != tc.server || got.Tool != tc.tool || got.URL != tc.url || !got.FromConfig { + t.Errorf("%s: VS Code MCP resolution = %+v", tc.name, got) + } + } +} + func TestResolveMCPOpenCodeJSONCGlobal(t *testing.T) { home := isolateHome(t) // Real-world shape: the global config is a .jsonc with comments under diff --git a/provider/copilot/copilot.go b/provider/copilot/copilot.go index 7d84f30..a37e41f 100644 --- a/provider/copilot/copilot.go +++ b/provider/copilot/copilot.go @@ -3,10 +3,11 @@ // // Two wire quirks the unified layer normalizes: Copilot omits the event name // from most payloads (only permissionRequest carries hookName, only -// notification carries hook_event_name), and tool arguments arrive as a -// JSON-ENCODED STRING in toolArgs on pre/postToolUse while permissionRequest -// ships a plain object in toolInput. These views hand you the wire truth; use -// the unified Event.NativeName and ToolCall for the normalized form. +// notification carries hook_event_name), and toolArgs has changed shape across +// CLI releases — a JSON-ENCODED STRING through 1.0.80, a plain object from +// 1.0.81 — while permissionRequest has always shipped a plain object in +// toolInput. These views hand you the wire truth; use the unified +// Event.NativeName and ToolCall for the normalized form. package copilot import ( @@ -46,12 +47,18 @@ type UserPromptSubmittedInput struct { Extra map[string]json.RawMessage `json:"-"` } -// PreToolUseInput is the native preToolUse payload. ToolArgs is a JSON-encoded -// string, and no tool-call id is supplied. +// PreToolUseInput is the native preToolUse payload. No tool-call id is +// supplied. +// +// ToolArgs is raw because the CLI changed its shape without changing its name: +// through 1.0.80 it was a JSON-encoded string, from 1.0.81 it is a plain +// object. Typing it as either one makes the view reject every tool event on +// the other release — silently, since callers key on ok. Handle both, or read +// the normalized object off the unified ToolCall.Input. type PreToolUseInput struct { Base ToolName string `json:"toolName"` - ToolArgs string `json:"toolArgs"` + ToolArgs json.RawMessage `json:"toolArgs"` Extra map[string]json.RawMessage `json:"-"` } @@ -65,7 +72,7 @@ type ToolResult struct { type PostToolUseInput struct { Base ToolName string `json:"toolName"` - ToolArgs string `json:"toolArgs"` + ToolArgs json.RawMessage `json:"toolArgs"` ToolResult ToolResult `json:"toolResult"` Extra map[string]json.RawMessage `json:"-"` } @@ -77,7 +84,7 @@ type PostToolUseInput struct { type PostToolUseFailureInput struct { Base ToolName string `json:"toolName"` - ToolArgs string `json:"toolArgs"` + ToolArgs json.RawMessage `json:"toolArgs"` Error string `json:"error"` ToolResult ToolResult `json:"toolResult"` Extra map[string]json.RawMessage `json:"-"` @@ -146,7 +153,7 @@ type NotificationInput struct { } func view[T any](e *agenthooks.Event, native string) (*T, bool) { - if e == nil || e.Provider != agenthooks.ProviderCopilot || e.NativeName != native { + if e == nil || e.Provider != agenthooks.ProviderCopilotCLI || e.NativeName != native { return nil, false } var v T diff --git a/provider/copilot/copilot_test.go b/provider/copilot/copilot_test.go index d10f9d1..dc2d1f5 100644 --- a/provider/copilot/copilot_test.go +++ b/provider/copilot/copilot_test.go @@ -11,7 +11,7 @@ import ( func event(t *testing.T, fixture, native string) *agenthooks.Event { t.Helper() return &agenthooks.Event{ - Provider: agenthooks.ProviderCopilot, + Provider: agenthooks.ProviderCopilotCLI, NativeName: native, Raw: json.RawMessage(agenthookstest.Fixture(t, "copilot/"+fixture)), } @@ -82,24 +82,44 @@ func TestViewsDecodeRecordedPayloads(t *testing.T) { } // The one wire quirk the views deliberately expose rather than smooth over: -// toolArgs is a JSON-encoded STRING on pre/postToolUse while permissionRequest -// ships a plain object in toolInput. Typing either one wrong makes the view -// fail to decode at all. +// toolArgs is the one field whose shape moved under the same name: a +// JSON-encoded STRING through CLI 1.0.80, a plain object from 1.0.81. +// permissionRequest's toolInput was an object throughout. A view typed to one +// toolArgs shape returns ok=false for the other, and callers keyed on ok then +// do nothing at all — so both shapes are asserted here, against the recorded +// corpus (current) and an inline legacy payload (1.0.80). func TestToolArgumentShapes(t *testing.T) { - pre, ok := PreToolUse(event(t, "pre_tool_use.json", "preToolUse")) - if !ok { - t.Fatal("PreToolUse did not decode; toolArgs must be a string, not an object") - } var args struct { Command string `json:"command"` } - if err := json.Unmarshal([]byte(pre.ToolArgs), &args); err != nil { - t.Fatalf("toolArgs is not a JSON-encoded string: %q", pre.ToolArgs) + pre, ok := PreToolUse(event(t, "pre_tool_use.json", "preToolUse")) + if !ok { + t.Fatal("PreToolUse did not decode the recorded 1.0.81 payload; toolArgs must stay raw") + } + if err := json.Unmarshal(pre.ToolArgs, &args); err != nil { + t.Fatalf("toolArgs is not a plain object: %s", pre.ToolArgs) } if args.Command != "echo hello-from-gram" { t.Errorf("command = %q", args.Command) } + legacy := &agenthooks.Event{ + Provider: agenthooks.ProviderCopilotCLI, + NativeName: "preToolUse", + Raw: json.RawMessage(`{"sessionId":"sess-copilot-1","timestamp":1786820437717,"cwd":"/work/repo","toolName":"bash","toolArgs":"{\"command\":\"echo hello-from-gram\"}"}`), + } + old, ok := PreToolUse(legacy) + if !ok { + t.Fatal("PreToolUse rejected the 1.0.80 double-encoded payload; an older CLI must still decode") + } + var inner string + if err := json.Unmarshal(old.ToolArgs, &inner); err != nil { + t.Fatalf("legacy toolArgs did not survive as a JSON string: %s", old.ToolArgs) + } + if err := json.Unmarshal([]byte(inner), &args); err != nil || args.Command != "echo hello-from-gram" { + t.Errorf("legacy toolArgs inner object = %q (%v)", inner, err) + } + perm, ok := PermissionRequest(event(t, "permission_request.json", "permissionRequest")) if !ok { t.Fatal("PermissionRequest did not decode; toolInput must be raw, not a string") @@ -130,7 +150,7 @@ func TestViewsRejectMismatchedEvent(t *testing.T) { func TestPreToolUseViewWithExtraCapture(t *testing.T) { raw := `{"sessionId":"sess-copilot-1","timestamp":1786820437717,"cwd":"/work/repo","toolName":"bash","toolArgs":"{}","brand_new_field":"surprise"}` e := &agenthooks.Event{ - Provider: agenthooks.ProviderCopilot, + Provider: agenthooks.ProviderCopilotCLI, NativeName: "preToolUse", Raw: json.RawMessage(raw), } diff --git a/quirks.go b/quirks.go index b77ad8b..303d754 100644 --- a/quirks.go +++ b/quirks.go @@ -177,4 +177,59 @@ var quirkRegistry = []Quirk{ Behavior: "after_tool_call still fires for a call the plugin just blocked, carrying the block text as the tool result — indistinguishable from success by shape", Mitigation: "the serve loop tracks denied toolCallIds per connection and decodes the blocked sibling as tool.error (Failed=true, Error \"blocked: \")", Reference: "verified against openclaw 2026.6.34"}, + // Copilot CLI: the four dialect behaviors DESIGN.md §1 has described since + // the provider landed. They sit next to the VS Code rows below because the + // CLI-vs-IDE split is only legible with both halves present. + {ID: 38, Provider: ProviderCopilotCLI, Versions: "observed 1.0.80", Event: KindOther, + Behavior: "most payloads carry no event name at all: only permissionRequest ships hookName, and only notification ships hook_event_name (PascalCase, aliased back)", + Mitigation: "copilotEventName reconstructs the name from which fields are populated, in a fixed discrimination order; the shapes are disjoint, so the reconstruction is exact", + Reference: "verified against copilot CLI 1.0.80"}, + {ID: 39, Provider: ProviderCopilotCLI, Versions: "all", Event: KindToolPre, + Behavior: "argument shape is split: toolArgs is a JSON-encoded STRING on pre/postToolUse while permissionRequest ships a plain object in toolInput", + Mitigation: "decodeCopilot collapses both onto one args value and makeToolCall un-stringifies, so ToolCall.Input is an object either way; Raw keeps the wire form", + Reference: "copilot CLI hooks reference"}, + {ID: 40, Provider: ProviderCopilotCLI, Versions: "all", Event: KindToolPre, + Behavior: "no tool-call id ships on any event, so pre and post cannot be correlated from the payload", + Mitigation: "SynthesizeToolID(sessionID, turnID, toolName, input) mints a stable id and Tool.Synthesized marks it as ours", + Reference: "copilot CLI hooks reference"}, + {ID: 41, Provider: ProviderCopilotCLI, Versions: "all", Event: KindToolPre, Capability: CapDeny, + Behavior: "preToolUse denies the call on ANY non-zero exit regardless of stdout, so a crashed hook or an expired credential is a total tool-call outage", + Mitigation: "encodeCopilot never signals through the exit code: it always exits 0 and puts the verdict on stdout", + Reference: "copilot CLI hooks reference"}, + // VS Code Copilot Chat. Every row below was read from the extension source + // in microsoft/vscode (vscode-copilot-chat was archived 2026-05-20); the + // published reference contradicts itself on Stop nesting and on the timeout + // key, so it is not the authority for any of them. + {ID: 42, Provider: ProviderVSCodeCopilot, Versions: "VS Code 1.109+ / Copilot Chat 0.38+, Preview", Event: KindOther, + Behavior: "matcher values are parsed and then ignored, so every registered hook runs on every tool invocation", + Mitigation: "renderVSCode emits no matcher key and hookCommand appends --filter for every non-empty matcher; in-process filtering is the only real enforcement", + Reference: "vscode agent hooks FAQ"}, + {ID: 43, Provider: ProviderVSCodeCopilot, Versions: "VS Code 1.109+ / Copilot Chat 0.38+, Preview", Event: KindOther, + Behavior: "the hook child gets {...process.env, ...hook.env} with no hook-specific COPILOT_*/CLAUDE_*/VSCODE_* marker, and no payload field relevant to provider detection that Claude Code does not also send (recorded VS Code payloads additionally carry timestamp)", + Mitigation: "detection is flag-only: generated config carries --provider=vscode-copilot and neither detectFromEnv nor detectFromShape gains a branch; a hand-written config without the flag degrades to claude-code", + Reference: "microsoft/vscode hookExecutor.ts"}, + {ID: 44, Provider: ProviderVSCodeCopilot, Versions: "VS Code 1.109+ / Copilot Chat 0.38+, Preview", Event: KindOther, + Behavior: "8 events, not the CLI's 14: permissionRequest, notification, sessionEnd, postToolUseFailure, userPromptTransformed and errorOccurred never fire", + Mitigation: "those kinds are absent from capMatrix[vscode-copilot] rather than present-and-empty, and kindToVSCode renders only the eight; the camelCase CLI file is the only way to reach the other six", + Reference: "microsoft/vscode hookTypes.ts (HOOKS_BY_TARGET)"}, + {ID: 45, Provider: ProviderVSCodeCopilot, Versions: "VS Code 1.109+ / Copilot Chat 0.38+, Preview", Event: KindStop, + Behavior: "a block verdict is read from hookSpecificOutput on Stop/SubagentStop but from the TOP LEVEL on PostToolUse/UserPromptSubmit; separately, a top-level hookEventName mismatch discards the entire result and a nested one strips hookSpecificOutput", + Mitigation: "encodeVSCode moves decision/reason into hookSpecificOutput for Stop/SubagentStop only, stamps the nested hookEventName from NativeName, and never writes a top-level one", + Reference: "microsoft/vscode toolCallingLoop.ts, chatHookService.ts"}, + {ID: 46, Provider: ProviderVSCodeCopilot, Versions: "VS Code 1.109+ / Copilot Chat 0.38+, Preview", Event: KindOther, + Behavior: "VS Code and the Copilot CLI glob the SAME two hook directories (~/.copilot/hooks, .github/hooks), so every installed file is loaded by both; PascalCase keys make the input identical — the CLI's Claude-compat mode — but the CLI reads decisions flat while VS Code reads them nested", + Mitigation: "demoteVSCodeToCLI rewrites vscode-copilot to copilot-cli when the CLI's own COPILOT_* env is set, so the runtime is resolved before decode and each session gets its own capability row and encoder; decodeCopilot falls through to the Claude codec for the snake_case payload", + Reference: "verified against copilot CLI 1.0.81: PreToolUse, PostToolUse, UserPromptSubmit, SessionStart and Stop all register from a PascalCase file; SubagentStart/SubagentStop/PreCompact are unmeasured because copilot -p cannot drive them"}, + {ID: 47, Provider: ProviderVSCodeCopilot, Versions: "VS Code 1.109+ / Copilot Chat 0.38+, Preview", Event: KindOther, + Behavior: "hooks stop firing mid-session: userPromptSubmitted/agentStop fire at session init and no further event arrives", + Mitigation: "none in-library (open upstream); a session that goes silent after the first turn is this, not a misconfiguration", + Reference: "microsoft/vscode#300193"}, + {ID: 48, Provider: ProviderVSCodeCopilot, Versions: "VS Code 1.135 / Copilot Chat 0.63", Event: KindToolPre, Capability: CapDeny, + Behavior: "tool names on the wire are NOT the names the extension registers: package.json declares copilot_readFile/copilot_replaceString, while a hook receives read_file/replace_string_in_file from a separate wire-name table, and the terminal tools are not in package.json at all. Auditing tool names from the obvious source maps the wrong strings entirely", + Mitigation: "canonicalNames carries the wire names, enumerated from the extension's own table and pinned by TestCanonicalToolFor. Until it did, every VS Code tool classified as ToolOther, so a deny-shell policy silently did not cover run_in_terminal and a ToolTask matcher saw no subagent", + Reference: "observed live 2026-08-29: a capture session ran 'echo hello' straight through a deny that should have blocked it"}, + {ID: 49, Provider: ProviderVSCodeCopilot, Versions: "VS Code 1.135 / Copilot Chat 0.63", Event: KindToolPost, Capability: CapAddContext, + Behavior: "PostToolUse context and block feedback are documented and parsed, but the panel path calls the async appendHookContext helper without await, so the next model request can be assembled before either reaches the tool result", + Mitigation: "retain the documented response shape and capability so it works once the upstream await fix lands; on affected versions treat immediate model feedback as best-effort and verify the outgoing model request rather than response text", + Reference: "VS Code 1.135 toolCalling.tsx:334-345,600-649; microsoft/vscode#314118 and fix PR #331785; reproduced live 2026-08-30 with both response placements"}, } diff --git a/roundtrip_test.go b/roundtrip_test.go index fa13d42..72db4ff 100644 --- a/roundtrip_test.go +++ b/roundtrip_test.go @@ -23,7 +23,8 @@ func TestRoundTripNoOpAndRawFidelity(t *testing.T) { agenthooks.ProviderGemini, agenthooks.ProviderOpenCode, agenthooks.ProviderKimi, - agenthooks.ProviderCopilot, + agenthooks.ProviderCopilotCLI, + agenthooks.ProviderVSCodeCopilot, } quiet := agenthooks.WithLogger(slog.New(slog.NewTextHandler(io.Discard, nil))) for _, p := range providers { diff --git a/runner_test.go b/runner_test.go index 08b06aa..ae67af6 100644 --- a/runner_test.go +++ b/runner_test.go @@ -537,7 +537,7 @@ func TestArgvPayloadMode(t *testing.T) { func TestUndetectableProviderNoOps(t *testing.T) { for _, v := range []string{ "CURSOR_VERSION", "CURSOR_TRACE_ID", "CURSOR_AGENT", "CODEX_HOME", "CODEX_SANDBOX", - "GEMINI_CWD", "GEMINI_CLI", "OPENCODE_SERVER", "OPENCODE", "CLAUDE_PROJECT_DIR", "CLAUDE_PLUGIN_ROOT", + "GEMINI_CWD", "GEMINI_CLI", "OPENCODE_SERVER", "OPENCODE", "CLAUDECODE", "CLAUDE_PROJECT_DIR", "CLAUDE_PLUGIN_ROOT", } { t.Setenv(v, "") }