feat: llm prompt caching with cache-usage visibility

[teriyakichild]

Summary

Adds opt-in provider prompt caching and end-to-end cache-usage visibility. Depends on the fork changes merged in mezmo/rig#12 (pinned here).

• prompt_caching flag on [agent.llm] for Anthropic (cache_control breakpoints on the system prompt + last message, covering tools via render order) and Bedrock (cachePoint blocks after system, tools, and last message). Off by default: writes bill at a premium, and Bedrock rejects cachePoints on models without caching support. OpenAI caching is automatic — this PR only surfaces its cached_tokens. Wired at all three construction sites (single-agent, coordinator, workers); workers inherit or override per [orchestration.worker.<name>.llm].
• aura.usage gains optional cache_read_input_tokens / cache_creation_input_tokens — sub-counts of prompt_tokens, omitted entirely when no provider reports cache usage, so existing consumers see byte-identical payloads. Orchestration accumulates per-worker turns into the same totals via the widened StreamItem::TurnUsage.
• CLI status line shows in N (M cached) / out K; the split lands in the replayable event log (old logs still parse).
• OpenInference: gen_ai.usage.cache_* span attributes translate to llm.token_count.prompt_details.cache_read/cache_write, which Phoenix (v13+) renders in the token breakdown and prices.
• rig-bedrock moves from crates.io =0.3.10 to the fork's vendored copy at the same version, retiring the 0.3.11-semver pin hazard.

Breaking change (aura-events API)

The wire format is backward compatible (new fields are optional and omitted when absent), but the aura-events Rust API breaks for external consumers:

• AuraStreamEvent::usage() takes a new cache_usage: Option<(u64, u64)> parameter.
• The AuraStreamEvent::Usage variant gains two fields, which breaks exhaustive struct patterns even if the constructor change were papered over — so no purely additive option exists.
• aura-cli's StreamHandler::on_usage gains the same parameter (all in-tree impls updated).

Accepted as a 0.2.x break rather than adding a parallel usage_with_cache constructor (which would only cover half the break) or rewriting history for a BREAKING CHANGE footer. External matchers add ../None; this note is the changelog record of the break.

Testing

• cargo test --workspace green (2149); clippy + fmt clean; new unit tests for config parsing, usage accumulation, event serde (including the untagged-enum ordering guard), status-line rendering, and the OpenInference translation.
• Wire-verified with capture servers: Anthropic requests carry cache_control only when the flag is set; Bedrock requests carry cachePoint {"type": "default"}.
• Live-verified against real Bedrock (us.anthropic.claude-sonnet-5): cold run reports the 8.8k-token prefix as cache creation, warm run reads all of it back; aura.usage, the status line, and Phoenix's token breakdown all agree with the provider totals.

Refs: #630

[greptile-apps]

Greptile Summary

Adds opt-in Anthropic and Bedrock prompt caching while propagating provider cache-token usage through runtime events, orchestration, telemetry, persistence, and CLI rendering.

• Wires prompt_caching through single-agent, coordinator, and worker construction.
• Extends usage events and OpenInference attributes with cache-read and cache-creation token counts.
• Restores persisted token totals during CLI replay while preserving in-flight counters.
• Pins Rig core and Bedrock dependencies to the prompt-caching fork revision.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
crates/aura-cli/src/ui/event_replay.rs	Replays persisted display events, preserves live counters during streaming, and replaces reconstructed token totals with authoritative ledger totals while idle.
crates/aura-web-server/src/streaming/types.rs	Extends per-turn streaming state with cache-usage data.
crates/aura-web-server/src/streaming/handlers.rs	Captures provider cache usage and emits it with cumulative usage responses.
crates/aura/src/builder.rs	Applies the opt-in prompt-caching configuration to supported provider builders.
crates/aura/src/orchestration/orchestrator.rs	Accumulates cache-token usage across orchestration turns.
crates/aura/src/openinference_exporter.rs	Translates cache usage into OpenInference prompt-detail token attributes.
crates/aura-events/src/lib.rs	Widens the shared usage event contract with backward-compatible optional cache-token counts.
Cargo.toml	Pins Rig core and Bedrock to a common fork revision containing prompt-caching support.

Sequence Diagram

sequenceDiagram
  participant C as Agent config
  participant R as Aura runtime
  participant P as LLM provider
  participant E as aura.usage stream
  participant U as CLI and telemetry
  C->>R: prompt_caching flag
  R->>P: request with provider cache breakpoints
  P-->>R: prompt, completion, and cache usage
  R->>E: cumulative usage event
  E->>U: status line, ledger, and span attributes

Loading

_{Reviews (5): Last reviewed commit: "fix(cli): leave live token counters alon..." | Re-trigger Greptile}

[Shearerbeard]

Greptile Summary

This PR adds opt-in Anthropic and Bedrock prompt caching and propagates provider cache usage through runtime events, orchestration, telemetry, persistence, replay, and the CLI status line.

• Pins the Rig fork revision and uses its Bedrock implementation.
• Extends usage contracts with optional cache-read and cache-creation token counts.
• Restores conversation token totals from the authoritative usage ledger during replay.
• Exposes cache metrics in OpenInference-compatible telemetry.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
crates/aura/src/streaming_request_hook.rs Captures provider cache usage and accumulates it alongside billed token usage.
crates/aura-cli/src/ui/event_replay.rs Restores replayed token counters from the persistent usage ledger.
crates/aura-cli/src/repl/conversations.rs Persists and totals cache-read usage for resumed conversations.
crates/aura/src/orchestration/orchestrator.rs Propagates and accumulates per-turn cache usage across orchestration.
crates/aura-events/src/lib.rs Adds backward-compatible optional cache token fields to usage events.
crates/aura/src/builder.rs Applies opt-in prompt caching when constructing Anthropic and Bedrock models.

Sequence Diagram

sequenceDiagram
    participant P as LLM Provider
    participant R as Aura Runtime
    participant E as aura.usage Event
    participant L as Usage Ledger
    participant U as CLI Status
    P-->>R: token and cache usage
    R->>E: optional cache read/write counts
    E->>L: persist per-turn usage
    E->>U: update live counters
    L->>U: restore authoritative totals on replay

Loading

Reviews (3): Last reviewed commit: "fix(cli): reseed ledger totals on every ..." | Re-trigger Greptile

@teriyakichild - Okay trying something out here - I didn't actually find anything wrong with this by hand but all the layers of token counting gets pretty complex. I DID however find a few things that came up in a multi provider adversarial review that might be worth looking into that seems decently convincing:

1. Blocking. crates/aura-web-server/src/streaming/handlers.rs:516 - the final aura.usage event takes its cache split from the streaming hook (callbacks.usage_state.get_cache_usage()), but that hook only sees turns with assistant text. The prompt/completion totals next to it come from rig's turn-aggregated Final usage, which includes tool-only turns (the doc comment on resolve_billed_usage around line 446 admits the hook under-reports). Totals and cache split come from different populations. The "subset of prompt_tokens" invariant can break on Bedrock tool loops. The fork already exposes what's needed: FinalResponse::cache_usage() in rig (prompt_request/streaming.rs, "summed across every turn"). Fix: carry it through FinalResponseInfo in crates/aura/src/provider_agent.rs:553 (currently drops it) and prefer the aggregate in send_final_events, same pattern as resolve_billed_usage. Add a regression test with a tool-only turn.
2. Blocking. examples/reference.toml doesn't mention prompt_caching at all, and it's the canonical config reference. Needs the flag documented: default off, cache writes bill at a premium. Bedrock rejects cachePoint on unsupported models. Also note the worker inherit/override semantics. OpenAI needs no flag (automatic). Worth saying there too.
3. Upstream (mezmo/rig, not this diff): the non-streaming Converse path (rig-bedrock types/assistant_content.rs) copies inputTokens raw and drops cacheReadInputTokens/cacheWriteInputTokens even though converse_output.rs parses them. AWS defines inputTokens as excluding cached tokens, so prompt usage is understated there. The streaming path does the fold correctly. mezmo/rig#12 never touched this file. File it on the fork and move the pin when fixed.
4. Call to make: StreamHandler::on_usage (crates/aura-cli/src/api/stream.rs:78) and AuraStreamEvent::usage (crates/aura-events/src/lib.rs:650) changed signature. Every in-tree impl is updated and 0.2.x makes it legal, but aura-events is the published shared crate, so either add an additive variant or note the break explicitly in the changelog.
5. Minor. Stale doc comment in crates/aura-cli/src/repl/commands.rs:476 - still describes a two-element usage tuple, now three values.
6. Minor. Cargo.toml:41 pin comment covers the core pin but not that rig-derive silently downgrades 0.1.14 -> 0.1.10 and the fork's rig-bedrock reuses the "0.3.10" version string. One sentence in the comment fixes it.
Fix 1 and 2 before merge; 3 goes to the fork; 4 is your call; 5 and 6 in passing.

[teriyakichild]

Good catches from the adversarial pass — dispositions:

1. Fixed (78112ad^..): confirmed real — the doc comment on resolve_billed_usage even documents the hook's text-turn-only view. Rig's turn-aggregated cache split now rides FinalResponseInfo into TurnState, and resolve_billed_usage resolves totals and cache split from the same population (aggregated Final when present; hook counters — which see every turn via TurnUsage — on the orchestration fallback). Regression test covers the tool-only-turn mismatch.
2. Fixed (78112ad): examples/reference.toml documents prompt_caching for anthropic (billing tradeoff, worker inheritance) and bedrock (cachePoint placement, supported models), plus a note that OpenAI is automatic.
3. Filed and fixed on the fork: fix(bedrock): fold cache tokens into non-streaming converse usage rig#13 folds cache read/write into the non-streaming Converse usage with a regression test. aura is streaming-only so no behavior change here; pin moves on its next bump.
4. Open — deciding on the aura-events compat note vs. additive constructor. One input to that call: the Usage enum variant gained fields, so any external exhaustive struct-pattern matcher breaks regardless of what we do with the constructor — the constructor is the smaller half of the break.
5. Fixed: resume_conversation doc now describes the three-element tuple.
6. Fixed: pin comment covers the reused 0.3.10 version string and the rig-derive resolution.

[teriyakichild]

@Shearerbeard closing out finding 4: going with an accepted, documented 0.2.x break rather than an additive constructor or a history rewrite. The Usage variant's new fields break exhaustive matchers no matter what we do to the usage() signature, so a parallel usage_with_cache would only paper over half of it. The PR description now has a Breaking change (aura-events API) section spelling out exactly what breaks (constructor arity, variant fields, StreamHandler::on_usage) and that the wire format stays backward compatible. Also: Greptile's round-4 finding (mid-stream replay clobbering live counters) is fixed in c3d2112. That's all 6 findings dispositioned — 1, 2, 5, 6 fixed here, 3 open as mezmo/rig#13, 4 accepted + documented.