Skip to content

fix(proxy): add provider-aware Anthropic thinking policy#4413

Draft
DeliciousBuding wants to merge 2 commits into
farion1231:mainfrom
DeliciousBuding:codex/provider-aware-thinking-placeholder
Draft

fix(proxy): add provider-aware Anthropic thinking policy#4413
DeliciousBuding wants to merge 2 commits into
farion1231:mainfrom
DeliciousBuding:codex/provider-aware-thinking-placeholder

Conversation

@DeliciousBuding

@DeliciousBuding DeliciousBuding commented Jun 18, 2026

Copy link
Copy Markdown

Summary

This is a smaller provider-aware alternative to #4210.

Instead of replacing proactive placeholder injection with a reactive retry path, this PR keeps the current default behavior for compatibility and adds an explicit provider metadata policy for Anthropic-format tool-use thinking history:

  • auto / unset: keep historical behavior for reasoning-vendor hints.
  • preserve_only: preserve real thinking blocks and safe structural cleanup only; do not synthesize {"type":"thinking","thinking":"tool call"}.
  • placeholder_always: explicitly opt a provider into placeholder synthesis, even without a vendor hint.

The intent is to stop treating synthetic thinking placeholders as a hardcoded behavior for every reasoning-vendor Anthropic path. They become a provider capability / compatibility choice.

Context collected

Why this shape

A pure reactive retry path is useful as a compatibility fallback, but it has drawbacks:

  • it depends on upstream error text staying stable;
  • it adds a failed round trip for vendors that still require backpass;
  • some relays wrap or normalize errors before cc-switch can match them;
  • it still treats placeholder synthesis as an error-path repair rather than an explicit provider capability.

This PR does not claim that every DeepSeek-compatible Anthropic endpoint has relaxed thinking backpass requirements. It only makes the existing synthetic placeholder behavior configurable and safe to disable for providers where it is harmful.

Private A/B validation

I also ran a sanitized private A/B against a DeepSeek-compatible Anthropic relay. No private endpoint names, machine names, or credentials are included here.

  • Auth/model probe: bearer auth worked for the test model; x-api-key was rejected by that relay.
  • Anthropic /v1/messages, small tool-use history:
    • no synthetic placeholder: HTTP 200, response contained thinking + text blocks.
    • with synthetic "tool call" placeholder: HTTP 200, response contained thinking + text blocks.
  • Anthropic /v1/messages, non-streaming long-history fixture (~63 KB request, 12 tool turns, ~21.9k input tokens):
    • no synthetic placeholder: HTTP 200, thinking + text, stop_reason=end_turn.
    • with synthetic placeholder: HTTP 200, but one run emitted only thinking before max_tokens.
  • Anthropic /v1/messages, streaming stress fixture (~304 KB request, 18 tool turns, ~107k input tokens):
    • no synthetic placeholder: HTTP 200, thinking + text, stop_reason=end_turn.
    • with synthetic placeholder: HTTP 200, thinking + text, stop_reason=end_turn.

Interpretation: this is not proof that disabling placeholders fixes every long-context freeze. It does confirm that this tested DeepSeek-compatible Anthropic path accepts tool-use history without synthetic placeholders, and that synthetic placeholders can still affect the output block mix / budget under longer histories. That is why this PR exposes a provider-scoped policy instead of changing the global default.

Changes

File Change
src-tauri/src/provider.rs Add AnthropicToolThinkingPolicy and ProviderMeta.anthropicToolThinkingPolicy.
src-tauri/src/proxy/providers/claude.rs Parameterize placeholder synthesis while keeping signature stripping and redacted_thinking conversion; soften outdated comments that implied all DeepSeek-compatible endpoints still require placeholder replay.
src/types.ts Add frontend TypeScript metadata type for the new policy.

Behavior

  • Existing providers are unchanged because unset policy maps to historical auto behavior.
  • preserve_only prevents missing / empty thinking blocks from being filled with synthetic "tool call", but still strips invalid signature fields from real thinking blocks.
  • placeholder_always allows explicit opt-in for a provider that requires synthetic placeholders even when it does not match the built-in vendor hints.

Verification

Passed:

cargo test --manifest-path src-tauri/Cargo.toml --lib proxy::providers::claude -- --nocapture
# 66 passed

cargo fmt --manifest-path src-tauri/Cargo.toml --check
cargo check --manifest-path src-tauri/Cargo.toml --lib
pnpm typecheck
pnpm format:check

Also ran:

cargo test --manifest-path src-tauri/Cargo.toml --lib

Result: 1654 passed, 8 failed, 2 ignored. The failures are outside this PR's touched area (codex_history_migration, commands::misc::anchored_upgrade_windows, database::dao::usage_rollup) and appear unrelated to Anthropic provider normalization.

Draft notes

This is intentionally draft because the maintainer may prefer one of two follow-ups:

  1. expose anthropicToolThinkingPolicy in the provider UI, or
  2. seed preserve_only / placeholder_always on specific presets after more provider verification.

I kept this PR backend-focused to separate the protocol behavior from UI/preset policy decisions.

Copy link
Copy Markdown
Owner

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: c97604e669

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants