Skip to content

fix(axir): retry cut-off actor replies in the five generated languages - #912

Merged
dosco merged 3 commits into
mainfrom
claude/axir-cut-off-reply-retry
Nov 18, 2026
Merged

dosco merged 3 commits into
mainfrom
claude/axir-cut-off-reply-retry

Conversation

@dosco

@dosco dosco commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Ports the cut-off reply retry from TypeScript (#906) to the five generated languages (Python, Java, Go, Rust, C++) through AxIR, and closes the backlog entry axir-2026-11-17-retry-an-agent-actor-turn-after-a-cut-off-model-reply.

A distiller or executor reply that is cut off at the output limit no longer ends the run. The same turn asks again, up to three more times: the first retry repeats the request unchanged, later ones add the TypeScript note after the action log. Any other failure, and a reply still cut off after every retry, is raised as before.

What changed

  • Agent loop (ir/axcore/agent.axir): two new ops at the end of the file, @agent_is_cut_off_reply_error and @agent_actor_stage_forward. The distiller and executor actor loops call the model through the second one. forward and streamingForward share that loop, so both are covered. The limit (3) and the note text are the TypeScript ones.
  • Empty cut-off replies (ir/axcore/gen.axir): a cut-off reply with no text used to be parsed as an answer with a missing field. The validation retry that followed then failed its own request check, so the run ended with Generate failed: Assistant content is required when no tool calls are provided. TypeScript does not parse an empty answer and reports Max tokens reached before completion. The ports now do the same (@parse_answer_outputs, a new op at the end of the file), so the retry recognizes the most common real case: a reply that spent its whole allowance on reasoning.
  • Fixtures: nine new ones, all recorded from real TypeScript runs by the existing extractors (agent-streaming-goldens.ts, axgen-streaming-goldens.ts). The agent extractor's scripted model learned two reply forms the port runners already had: {results: [...]} with its own finish reason, and {error: {type: 'plain', message}}.
  • Generated output: the five packages, one line in each generated agent skill doc, and the website skill mirrors.
Fixture What it pins
agent-forward-cut-off-reply-retried one cut-off executor reply is retried and the run completes
agent-forward-cut-off-reply-note three cut-off replies, then success; all four executor requests in full: the first retry is identical to the first request, the second and third carry the note
agent-forward-cut-off-reply-exhausted a reply that stays cut off fails after four executor calls with Max tokens reached before completion
agent-forward-cut-off-reply-other-error an unrelated failure is not retried
agent-forward-cut-off-reply-distiller the distiller turn recovers the same way; its three requests in full
agent-forward-cut-off-reply-with-content a cut-off reply whose text still parses is retried, and its code does not run
agent-forward-cut-off-reply-function-arguments a failure reading Invalid function arguments is retried
agent-streaming-forward-cut-off-reply-retried the streaming forward shares the loop
errors-max-tokens-empty-content (axgen) an empty cut-off reply fails as cut off, after one request

How the ports report a cut-off reply

Checked by running the packages, not only by reading the IR:

  • Cut off with text that parses: already raised Generate failed: Max tokens reached before completion. Unchanged.
  • Cut off with no text: raised the unrelated message above. Fixed here.
  • Truncated tool-call payload: TypeScript ends the generation with Invalid function arguments (one call). The ports never raise that: the generation answers the bad call with a tool error and asks the model again (Python and Go: three requests, run completes). That difference predates this PR and is not changed. The retry still recognizes the message, as TypeScript does, and a fixture pins it with a scripted failure.
  • TypeScript walks the error's cause chain. The ports test the error message, which is equivalent here: a port generation failure repeats the failure it wraps in its own message (Generate failed: ...).

Behaviour changes for port users

  • A cut-off actor reply now costs up to three extra model calls in that turn before the error surfaces, as in TypeScript. There is no option to turn it off.
  • An empty cut-off reply from any generator now fails with Generate failed: Max tokens reached before completion instead of Generate failed: Assistant content is required when no tool calls are provided. It still fails after one request.

Not included

  • The port-only actorMode: 'completion' stages are unchanged. TypeScript has no counterpart, so there is no reference behaviour to port.
  • An empty reply that was not cut off (finish reason stop) is unchanged. Checked on the Python package: it still ends with Assistant content is required when no tool calls are provided, where TypeScript returns an empty output after one call. That difference predates this PR.

Verification

Run locally on 0b7e5b844, which is this branch on top of main at 5706a70d2 (#910). The second commit only closes the backlog entry.

  • npm run axir:verify:dev: exit 0. Python, Java, C++, Go and Rust each report ok conformance: shard 1/1: 2209 fixtures (2,200 before, plus the nine new ones).
  • npm run axir:check-packages: AxIR committed packages are up to date.
  • npm run axir:conformance:check: AxIR conformance fixtures and provider catalog are in sync. Re-recording after feat(agent): raise the default maxRuntimeChars from 3,000 to 12,000 #910 changed none of the new fixtures.
  • npm run test --workspace=@ax-llm/ax: 277 files, 4,243 passed, 1 skipped; templates, index, type-check, type-tests, lint and format pass.
  • npm run test:spelling: passes.
  • npm run test:axir:go-fast: 72 fast AxIR Go tests ... 0 failing. npm run test:axir:tools: 137 passed.
  • npm run axir:gate:response-perturb: 522 response mutations across 203 fixtures: every fixture has a model-dependent assertion on every target.
  • npm run skills:check, axir:gate:prompt-sync, axir:gate:anti-facade, axir:gate:ledger, axir:backlog:validate, axir:backlog:check: pass.
  • Real-engine agent suite (ir/conformance/axagent-real, 14 fixtures, built as CI's antidote job builds it): passes on Go, Java, C++ and Rust.

Do the new fixtures have teeth?

  • On the unmodified Python package, eight of the nine fail. The ninth pins that an unrelated error is not retried, which was already true.
  • All five runners reject eight deliberately wrong expectations: a wrong request count, one executor call too few or too many before giving up, the note missing from the second retry, the note present on the first retry, the note missing from the distiller's retry, and an unrelated error expected to be retried.

Not verified locally:

  • The Python real-engine agent suite: no Python on the machine has the quickjs module. CI runs it.
  • The full npm run test:axir aggregate (the slow Go test suite) was not run; the fast subset and the five-language verification were.

🤖 Generated with Claude Code

dosco and others added 2 commits November 17, 2026 12:16
Port the TypeScript cut-off reply retry (#906) to Python, Java, Go, Rust
and C++ through AxIR. A distiller or executor reply that is cut off at the
output limit no longer ends the run: the same turn asks again, up to three
more times, first unchanged and then with the TypeScript note after the
action log. Any other failure, and a reply still cut off after every retry,
is raised as before.

A cut-off reply with no text used to fail in the ports as a missing field,
which then surfaced as "Assistant content is required when no tool calls
are provided". It now fails as TypeScript reports it, "Max tokens reached
before completion", so the retry recognizes it.

Nine conformance fixtures recorded from TypeScript cover both changes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The five generated languages now retry a cut-off actor reply as TypeScript
does, so the entry added with #906 is done.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@dosco
dosco merged commit 03b1d87 into main Nov 18, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant