feat(claude-code): use stream-json protocol for persistent sessions

[codefromthecrypt]

Summary

Switch the Claude Code provider from one-shot -p invocations to the stream-json protocol (--input-format stream-json --output-format stream-json). This is the same protocol used by the official Claude Agent SDKs (Go, TypeScript, Elixir) to communicate with the CLI as a persistent subprocess.

The old approach spawned a new claude -p <messages_json> process per turn, passing the entire conversation as a CLI argument. This hit OS argument size limits (E2BIG) on long conversations and lost all session context between turns. The new approach keeps a single CLI process alive, sends messages as NDJSON on stdin, and reads responses line-by-line from stdout -- giving goose persistent multi-turn sessions, image support, and proper error handling for free.

Type of Change

• Refactor / Code quality
• Bug fix

AI Assistance

• This PR was created or reviewed with AI assistance

Testing

GOOSE_PROVIDER=claude-code GOOSE_MODEL=sonnet target/release/goose run --text "say just hello"
# starting session | provider: claude-code model: sonnet
#     session id: 20260206_26
#     working directory: /Users/codefromthecrypt/oss/goose-goosed
# Hello! 👋

Related Issues

Relates to #6972

[Copilot]

Pull request overview

This PR refactors the Claude Code provider to use a persistent subprocess with the stream-json protocol instead of spawning a new process per turn with the -p flag. This change addresses OS argument size limits (E2BIG errors) on long conversations and enables persistent multi-turn sessions with proper state management.

Changes:

• Switched from one-shot -p <messages_json> invocations to persistent --input-format stream-json --output-format stream-json protocol
• Added CliProcess struct to manage persistent stdin/stdout/stderr of the Claude CLI subprocess
• Converted message formatting from Anthropic-style structured tool blocks to text-based representations (tools are rendered as [tool_use: ...] and [tool_result: ...])
• Implemented incremental message sending (only new messages are sent per turn)
• Added comprehensive unit tests for message conversion, NDJSON protocol, response parsing, and error handling

[alexhancock]

is this part in theory useful for other cli providers?

[rabi]

The new approach keeps a single CLI process alive, sends messages as NDJSON on stdin, and reads responses line-by-line from stdout -- giving goose persistent multi-turn sessions, image support, and proper error handling for free.

I think this breaks the existing functionality of using Ctrl+C to send a new message, as Ctrl+C (SIGINT) kills the persistent child process and, since it's stored in a OnceCell, it can never be respawned — all subsequent writes fail with a broken pipe. I've proposed #7070 fix and refactor it a bit, so that I can rebase my streaming PR.