A hard dollar cap across a whole AgentChat conversation, via the model client

[domondi1]

One team.run(...) can turn into many model calls: every turn of a RoundRobinGroupChat or Swarm, each agent's tool-calling loop, reflection, a termination check that keeps going longer than expected. Token usage is visible per call, but nothing stops the whole conversation at a dollar amount, and a loop between two agents can spend a lot before a human notices.

OpenAIChatCompletionClient passes default_headers straight to the underlying AsyncOpenAI, so you can put a hard per-conversation budget in front of the model without changing your agents: point the client at an OpenAI-compatible gateway that enforces the budget, and tag every call with the conversation's id and budget. Share one client across the agents and they all draw from the same budget.

Run the gateway next to your app (it uses your provider key):

pip install inferrail
export OPENAI_API_KEY=sk-...
inferrail serve --quickstart --app-mode   # http://127.0.0.1:8000

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination
from autogen_ext.models.openai import OpenAIChatCompletionClient

def client_for_run(run_id: str, budget_usd: str) -> OpenAIChatCompletionClient:
    return OpenAIChatCompletionClient(
        model="gpt-4o-mini",
        base_url="http://127.0.0.1:8000/v1",
        api_key="unused",
        max_tokens=400,          # the reservation is based on it
        max_retries=0,           # a 402 is a decision, not a transient error
        default_headers={
            "X-Inferrail-Attribute-Work-Id": run_id,
            "X-Inferrail-Budget-Usd": budget_usd,
        },
    )

client = client_for_run("conv-4812", "0.50")
researcher = AssistantAgent("researcher", model_client=client)
writer = AssistantAgent("writer", model_client=client)
team = RoundRobinGroupChat([researcher, writer], termination_condition=MaxMessageTermination(6))
await team.run(task="...")   # every agent call in this run shares $0.50

Afterwards:

inferrail work conv-4812

shows how many calls the conversation made, the total tokens, and what it cost.

Each call reserves its worst-case cost (input + max_tokens) before it's sent, so when agents make calls concurrently (parallel tool calls, a Swarm) they can't overspend together. Once the conversation's budget can't cover the next call, it's refused with HTTP 402 before it reaches the provider. In AgentChat that comes back as an APIStatusError (status 402) and ends the run (wrapped in a GroupChatError, so catch it where you call team.run). The two headers aren't forwarded to the provider.

What I tested: autogen-agentchat / autogen-ext 0.7.5, inferrail 0.4.13, against a local stub upstream standing in for OpenAI. A two-agent RoundRobinGroupChat with a generous budget ran to its message limit and all calls were attributed to the one id. With a budget below the conversation's cost, the first calls succeeded and the next was refused before reaching the upstream, ending the run. I haven't run it against the real OpenAI API from AutoGen yet, so reports are welcome.

Caveats: chat completions only (keep embeddings/other clients separate); set max_tokens; one gateway (the budget is a SQLite file on its host); and the model needs a price in Inferrail (gpt-4o-mini, gpt-4.1-mini, gpt-4.1 are built in, inferrail models lists others, and you can add your own). To budget per agent instead of per conversation, give each agent a client with its own work-id.

Setup details: guide. I maintain Inferrail (open source, Apache-2.0), so take the suggestion with that in mind. Curious how people here currently bound the cost of a long multi-agent run.