Skip to content

[BUG]: Billing/quota provider errors are misclassified as generic agent errors and surfaced to the user as a raw nested provider payload #593

Description

@justintime4tea

Summary

When the LLM provider rejects a request because the account is out of credit, Anthropic returns HTTP 400 with "type": "invalid_request_error" and the message Your credit balance is too low to access the Anthropic API. This is a hard, account-level, non-retryable failure — no amount of replanning or retrying can clear it.

AURA does not recognize it. Orchestrator::categorize_failure_error has branches for overload (429/503), auth (401/403/"api key"), and not-found (404), but nothing for billing, credit, or quota exhaustion:

// crates/aura/src/orchestration/orchestrator.rs:3506-3527
} else if lower.contains("rate limit") || lower.contains("429") || ... {
    FailureCategory::ProviderOverloaded
} else if lower.contains("authentication")
    || lower.contains("unauthorized")
    || lower.contains("403")
    || lower.contains("401")
    || lower.contains("api key")
{
    FailureCategory::ProviderAuthError
} else if lower.contains("404") || ... {
    FailureCategory::ProviderNotFound
} else {
    FailureCategory::AgentError   // <-- credit-balance errors land here
}

Three consequences follow from the misclassification:

1. The orchestrator keeps spending on a dead account. Because the failure is AgentError and not a Provider* category, should_short_circuit_provider_errors (orchestrator.rs:3534-3549) returns false, so the orchestrator proceeds to the post-execute coordinator call. That call hits the identical 400. is_transient_planning_error (orchestrator.rs:4588) is also false, so planning bails out — after having issued another billable request against an account that had already told us it has no balance.

2. The user-facing message is three layers of internal jargon wrapped around a raw JSON payload. Each layer prefixes the one below it — orchestrator.rs:1570 produces Planning failed: {err}, and orchestrator.rs:4261's catch-all arm wraps that again:

_ => format!("Post-execute coordinator call failed: {}", err_str),

The note becomes the first line of build_raw_task_results, i.e. the first thing the user reads. Nothing in it tells them the actionable fact: add credit to your Anthropic account.

3. It bypasses the debug_provider_errors redaction. The web server already has build_provider_error_message (crates/aura-web-server/src/streaming/handlers.rs:1327), which hides raw provider detail unless --debug-provider-errors is set (default off, handlers.rs:1808). That guard only covers stream-level errors. The orchestrator folds the raw provider string into the response content, so it streams as ordinary assistant text and never passes through the redaction — the provider's raw JSON and its request_id reach the user on a default-configured server.

Expected: billing/credit/quota exhaustion should be its own terminal FailureCategory, should short-circuit replanning immediately (no further provider calls), and should surface as a plain-language message naming the cause and the fix.

Reproduction

Pre-requisites

  • AURA 0.2.4 (05169982), orchestration enabled, Anthropic provider
  • An Anthropic API key on an account with zero credit balance

Steps

  1. Start aura-web-server against an orchestration config (no --debug-provider-errors).
  2. Issue any query that routes to orchestrated.
  3. Let the worker wave run; the first provider call after the balance hits zero returns 400.
  4. [BUG] The worker failure is logged as (agent_error), the orchestrator issues a further post-execute coordinator call which fails identically, and the response body opens with the nested error string below — including the raw provider JSON, despite debug_provider_errors = false.

Relevant log output

2026-08-24T23:24:27.532368Z  WARN aura::orchestration::orchestrator: Worker 'github_analyst' failed
  task 0 after 73121ms (agent_error): Worker failed task 0 after 2 attempts: CompletionError:
  ProviderError: SSE Error: Invalid status code 400 Bad Request with message: {"type":"error",
  "error":{"type":"invalid_request_error","message":"Your credit balance is too low to access the
  Anthropic API. Please go to Plans & Billing to upgrade or purchase credits."},
  "request_id":"req_011CeNSF3my2nQECgwDCGSSN"}

2026-08-24T23:24:27.535405Z  WARN aura::orchestration::orchestrator: Execution had failures:
  1 failed (1 agent_error), 1 blocked

2026-08-24T23:24:27.552302Z  INFO aura::orchestration::orchestrator: Planning attempt 1/3
  (per_call_timeout=360s, conversation_len=2)          <-- billable call on a known-dead account

2026-08-24T23:24:27.991754Z  WARN aura::orchestration::orchestrator: Planning attempt 1 failed
  after 0.4s: CompletionError: ProviderError: SSE Error: Invalid status code 400 ...
  "request_id":"req_011CeNSF5BZ7CQWKKVgGqHv7"

2026-08-24T23:24:27.992264Z  WARN aura::orchestration::orchestrator: Post-execute coordinator call
  failed: Planning failed: CompletionError: ProviderError: SSE Error: Invalid status code 400 ...

Note agent_error in the first line — that single misclassification is what allows the 0.4s billable planning attempt four lines later.

What the user saw (verbatim, as the opening of the assistant response):

Post-execute coordinator call failed: Planning failed: CompletionError: ProviderError: SSE Error:
Invalid status code 400 Bad Request with message:
  {"type":"error","error":{"type":"invalid_request_error","message":"Your credit balance is too low
  to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits."},
  "request_id":"req_011CeNSF5BZ7CQWKKVgGqHv7"}

Additional Context

The provider's own message is perfectly clear. Everything AURA adds around it — Post-execute coordinator call failed, Planning failed, CompletionError, ProviderError, SSE Error, Invalid status code 400 Bad Request with message — is internal implementation detail that pushes the one useful sentence to the middle of a JSON blob.

This is not Anthropic-specific. The equivalent conditions on other providers are also unclassified today:

Provider Status Error body
Anthropic 400 invalid_request_error / credit balance is too low
OpenAI 429 insufficient_quota / You exceeded your current quota
OpenRouter 402 Insufficient credits

OpenAI's insufficient_quota is worse than unclassified — it arrives as a 429, so the existing lower.contains("429") branch classifies it as ProviderOverloaded, which is_transient_planning_error treats as retryable. AURA will burn all three planning attempts retrying a quota failure that can never succeed.

Suggested fix

  1. Add FailureCategory::ProviderQuotaExhausted to crates/aura/src/orchestration/types.rs:489.

  2. Match it in categorize_failure_error before the 429 branch, so OpenAI's 429-with-insufficient_quota is not swallowed by ProviderOverloaded. Suggested matches: credit balance, insufficient_quota, insufficient credits, billing, exceeded your current quota, payment required, 402.

  3. Include the new variant in should_short_circuit_provider_errors (orchestrator.rs:3534) so the orchestrator stops rather than issuing another billable call, and exclude it from is_transient_planning_error (orchestrator.rs:4588).

  4. Give it a dedicated arm in the orchestrator.rs:4249-4261 match, alongside the existing AgentTimeout / DepthExhausted arms, that emits a plain-language note instead of the nested error — e.g.

    Stopped: the model provider rejected the request because the account has no remaining credit or quota. Add credit to the provider account and retry.

  5. Route the raw provider detail through the existing debug_provider_errors gate rather than embedding it in response content unconditionally, so a default-configured server does not leak provider payloads and request_ids.

Test convention already exists at orchestrator.rs:5420-5435 (test_categorize_failure_provider_errors); the new variant should get cases there, including an explicit assertion that a 429 carrying insufficient_quota classifies as quota-exhausted rather than overloaded.

Searched Issues

  • No similar issues found

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions