Request: Aggregate traces by session_id in Jaeger

[seeger12138]

Feature Request: Aggregate traces by session_id in Jaeger

Summary

Plano already defines a plano.session_id span attribute (crates/brightstaff/src/tracing/constants.rs:142) and sets it on plano(llm) spans when the client sends an X-Model-Affinity header. In practice, agentic clients like Claude Code do not send X-Model-Affinity — they propagate session identity either via a x-claude-code-session-id header (as other open-source gateways have started converging on) or embedded in the Anthropic request body's metadata.user_id JSON. As a result, plano.session_id is effectively never populated for Claude Code traffic, and Jaeger has no first-class way to filter or aggregate traces by session.

This issue requests two things:

1. Extract session_id from additional sources (the x-claude-code-session-id header and the Anthropic body's metadata.user_id.session_id), not just X-Model-Affinity.
2. Propagate plano.session_id to every span in a Plano trace (inbound, routing, llm, outbound), so Jaeger's tag-based search and any future session-grouped views work across the full request lifecycle.

Current Behavior

Source 1: X-Model-Affinity header is the only source Plano reads

crates/brightstaff/src/handlers/routing_service.rs:76-79:

let session_id: Option<String> = request_headers
    .get(MODEL_AFFINITY_HEADER)   // X-Model-Affinity
    .and_then(|h| h.to_str().ok())
    .map(|s| s.to_string());

Same in crates/brightstaff/src/handlers/llm/mod.rs:97-100. Claude Code does not send this header — it's a Plano-internal header intended for the second turn of an agentic loop after Plano returns it in a response.

Source 2: plano.session_id only lands on plano(llm) spans

crates/brightstaff/src/handlers/llm/mod.rs:120-128:

if let Some(ref sid) = session_id {
    get_active_span(|span| {
        span.set_attribute(opentelemetry::KeyValue::new(
            tracing_plano::SESSION_ID,
            sid.clone(),
        ));
    });
}

The routing_decision span (routing_service.rs:89-95) only carries request_id — session_id is read but not attached to the span. The plano (inbound) and plano(outbound) (Envoy egress) spans also do not carry plano.session_id.

Source 3: /routing/v1/* debug endpoint never sets plano.session_id

crates/brightstaff/src/main.rs:504-519 routes /routing/v1/messages to routing_decision directly, bypassing llm_chat. So the only place that currently sets plano.session_id is never executed for debug-mode traces.

Result

For a Claude Code session that issues N turns, each turn produces a trace with:

• A plano inbound span — no plano.session_id
• A plano(routing) routing_decision span — no plano.session_id
• A plano(llm) LLM call span — no plano.session_id (because Claude Code doesn't send X-Model-Affinity)
• A plano(outbound) Envoy egress span — no plano.session_id

To answer "what happened in session X?", a user must manually correlate N traces by request_id and message timestamps. There is no Jaeger tag search that returns all traces for a session, because no span carries the session identifier.

Proposed Solution

Three independent changes. Any subset is useful; together they close the gap.

1. Extract session_id from additional sources

Add a resolution chain (first non-empty wins), in this order:

Source	Header / path	Notes
X-Model-Affinity	header	existing behavior, kept for backward compat
x-claude-code-session-id	header	emerging convention across open-source LLM gateways
metadata.user_id JSON	Anthropic body	parse JSON, extract session_id field (Claude Code format: {"device_id":"...","account_uuid":"...","session_id":"<uuid>"})

The Anthropic body extraction should happen during request parsing in crates/hermesllm/src/apis/anthropic.rs (where MessagesRequest is deserialized, lines 104-123) and be threaded into parse_and_validate_request so both llm_chat and routing_decision paths can use it.

2. Set plano.session_id on every span in a Plano trace

Specifically:

• routing_decision span in crates/brightstaff/src/handlers/routing_service.rs:89-95 — add session_id = %sid to the info_span! macro (or set_attribute after creation when session_id is Some)
• Inbound plano span (whatever creates the root request span) — same
• plano(llm) span — already done at handlers/llm/mod.rs:120-128, keep as-is
• plano(outbound) — propagate via OTel context (the OTLP exporter already inherits parent span attributes through traceparent); no explicit action needed if the upstream spans are children of the Plano trace

3. Auto-extract via header_prefixes config

Document that users can add x-claude-code-session-id to tracing.span_attributes.header_prefixes in their config as a stopgap:

tracing:
  span_attributes:
    header_prefixes:
      - x-request-id
      - x-claude-code-session-id   # surfaces as span attribute "x-claude-code-session-id"

This is a partial fix — it captures the header as a span attribute but doesn't normalize it to plano.session_id and doesn't help with the Anthropic body case. The code changes in (1) and (2) are still needed for a unified view.

Why This Matters

Agentic loops (Claude Code, Cursor, Continue.dev, Aider, etc.) are the primary use case for model affinity and session pinning. A single "session" can span many turns, each producing a separate trace. Without session-scoped observability:

• Debugging "why did session X route to model Y on turn 5?" requires manually correlating 5+ traces by request_id and timestamps.
• Cost attribution per session is impossible without joining traces on session_id.
• Signals (misalignment, loops, stagnation) are computed per-request; aggregating them per-session would give a much clearer picture of agent quality over a full task.

With session_id on every span, Jaeger's existing tag search (plano.session_id=<uuid>) immediately becomes a session-scoped trace finder, and downstream tools (PostHog, Grafana, planoai obs TUI from #891) can build session-grouped views on top.

Evidence

A real Claude Code request body (Anthropic /v1/messages):

{
  "model": "claude-opus-4-7",
  "metadata": {
    "user_id": "{\"device_id\":\"xx\",\"account_uuid\":\"\",\"session_id\":\"xx-22c3-4543-xx-xx\"}"
  },
  "thinking": { "type": "adaptive" },
  "messages": [...]
}

The session_id is JSON-encoded inside metadata.user_id. No X-Model-Affinity header is sent. Plano currently extracts neither.

Prior Art

• Custom trace attributes from request headers #707 / add Custom Trace Attributes to extend observability #708 — Custom trace attributes from request headers: the foundation header_prefixes builds on. This feature request extends that pattern with structured extraction (JSON parsing) and propagation to all spans.
• Model affinity for consistent model selection in agentic loops #827 — Model affinity for consistent model selection: introduced the X-Model-Affinity header and session cache. The session_id concept exists; this issue is about making it observable for clients that don't use Plano's header.
• planoai obs: live LLM observability TUI #891 — planoai obs live LLM observability TUI: natural consumer of a session-grouped view once plano.session_id is reliably populated.
• Other open-source LLM gateways have begun standardizing on x-claude-code-session-id as a header for exactly this use case; supporting it would improve interop.

Acceptance Criteria

• Sending a Claude Code request through POST /v1/messages produces a trace where every Plano span (plano, plano(routing), plano(llm), plano(outbound)) carries plano.session_id matching the session_id from metadata.user_id.
• Sending a request with x-claude-code-session-id: <uuid> header produces the same result without body parsing.
• Sending a request with X-Model-Affinity: <id> continues to work as before (backward compat).
• /routing/v1/messages debug endpoint also sets plano.session_id on its routing_decision span.
• Jaeger tag search plano.session_id=<uuid> returns all traces for that session.
• Documentation in docs/source/guides/observability/tracing.rst updated to describe the session_id sources and Jaeger workflow.

Related Files

• crates/brightstaff/src/handlers/routing_service.rs:76-79, 89-95 — session_id extraction + routing_decision span
• crates/brightstaff/src/handlers/llm/mod.rs:97-100, 120-128 — session_id extraction + LLM span attribute
• crates/brightstaff/src/tracing/constants.rs:142 — plano.session_id constant
• crates/brightstaff/src/main.rs:504-519 — /routing/v1/* dispatch (bypasses llm_chat)
• crates/hermesllm/src/apis/anthropic.rs:104-123 — MessagesRequest deserialization (where body parsing would hook in)
• crates/brightstaff/src/tracing/init.rs — header_prefixes config wiring