Skip to content

Commit 68e29cc

Browse files
authored
feat(providers): add shared authentication and resilience (#16)
## Summary Adds shared authentication and resilience configuration across AgentKit model providers. Introduces separate OpenAI Chat Completions and Responses adapters, including configurable support for public OpenAI Responses and private ChatGPT deployments. ## Motivation Provider adapters previously stored raw credential strings and implemented inconsistent authentication, retry, timeout, and stream-recovery behavior. This change gives providers reusable HTTP-layer authentication and resilience primitives while keeping each provider’s authentication scheme and wire protocol explicit. ## Impact Existing constructors continue to accept string credentials, and omitted resilience preserves the current single-attempt behavior. The following public API changes require migration: - Provider config fields such as `api_key` and `auth_token` become `authentication: Authentication`; Ollama and vLLM use `Option<Authentication>` because authentication remains optional. - Provider config structs gain `resilience: Option<ResilienceConfig>`. Exhaustive struct literals must add `resilience: None` or move to constructors and builders. - `SessionConfig` gains `consumer_capabilities`; direct struct literals must add the field or use `SessionConfig::new(...)`. - `ModelTurnEvent` and `AgentEvent` gain `ResponseAttemptSuperseded`; exhaustive matches must handle the new variant. - Anthropic still distinguishes `x-api-key` from bearer authentication. A bare string converted directly to `Authentication` always means bearer authentication. ## Technical details ### Shared authentication API `agentkit-http` now exports `Authentication`, `AuthenticationAttempt`, and the asynchronous `AuthenticationProvider` contract. A provider receives `None` for initial authentication and the rejected attempt after a `401`, allowing one controlled refresh while keeping provider-private state opaque. Static bearer authentication remains concise: ```rust use agentkit_http::Authentication; let from_string: Authentication = "sk-example".into(); let explicit = Authentication::bearer("sk-example"); ``` Refreshable authentication can retain rejected-attempt state and attach a stable, non-secret binding: ```rust use agentkit_http::{ header, Authentication, AuthenticationAttempt, AuthenticationProvider, HeaderMap, HeaderValue, HttpError, }; use async_trait::async_trait; #[derive(Clone, Copy)] struct RotatingTokens; #[async_trait] impl AuthenticationProvider for RotatingTokens { async fn authenticate( &self, previous: Option<&AuthenticationAttempt>, ) -> Result<AuthenticationAttempt, HttpError> { let generation = previous .and_then(|attempt| attempt.state::<u64>()) .copied() .map_or(0, |generation| generation + 1); let value = match generation { 0 => HeaderValue::from_static("Bearer initial-token"), _ => HeaderValue::from_static("Bearer refreshed-token"), }; let mut headers = HeaderMap::new(); headers.insert(header::AUTHORIZATION, value); Ok(AuthenticationAttempt::new(headers, generation) .with_binding(format!("credential-generation-{generation}"))) } } let authentication = Authentication::new(RotatingTokens); ``` The rejected-attempt refresh is independent of resilience retries. OpenAI Responses additionally requires the refreshed attempt to retain the original non-secret binding before replaying continuation state. ### Optional resilience All provider configurations can opt into a shared retry and timeout policy: ```rust use std::time::Duration; use agentkit_http::ResilienceConfig; use agentkit_provider_openai::{ OpenAIChatCompletionsAdapter, OpenAIConfig, }; let resilience = ResilienceConfig { max_retries: 3, retry_budget: Duration::from_secs(60), attempt_timeout: Some(Duration::from_secs(30)), stream_idle_timeout: Some(Duration::from_secs(30)), initial_backoff: Duration::from_millis(200), max_backoff: Duration::from_secs(10), }; let adapter = OpenAIChatCompletionsAdapter::new( OpenAIConfig::new("sk-openai", "gpt-4o") .with_resilience(resilience), )?; ``` `resilience: None` preserves the existing no-timeout, single-attempt behavior. `Some(ResilienceConfig::default())` enables bounded retries and timeouts; it is not equivalent to omission. Authentication schemes remain provider-specific: | Providers | Authentication behavior | |---|---| | OpenAI, OpenRouter, Groq, Mistral, Baseten, Cerebras | Bearer by default | | Anthropic | `x-api-key` via `new`, bearer via `with_auth_token` | | Ollama, vLLM | Optional; unauthenticated by default | For Anthropic, use the constructor matching the intended scheme: ```rust use agentkit_provider_anthropic::AnthropicConfig; let api_key = AnthropicConfig::new( "sk-ant", "claude-sonnet-4-6", 4096, )?; // x-api-key let bearer = AnthropicConfig::with_auth_token( "oauth-token", "claude-sonnet-4-6", 4096, )?; // Authorization: Bearer ``` Protected Ollama and vLLM deployments can now opt in without changing local defaults: ```rust use agentkit_provider_ollama::OllamaConfig; use agentkit_provider_vllm::VllmConfig; let local = OllamaConfig::new("llama3.2"); let protected = VllmConfig::new("Qwen/Qwen3-8B") .with_api_key("server-token"); ``` ### Separate OpenAI schema adapters Chat Completions and Responses remain separate request codecs and stream state machines. The explicit Chat Completions name is `OpenAIChatCompletionsAdapter`; the historical `OpenAIAdapter` alias remains available. ```rust use agentkit_provider_openai::{ OpenAIChatCompletionsAdapter, OpenAIConfig, OpenAIResponsesAdapter, OpenAIResponsesConfig, }; let chat = OpenAIChatCompletionsAdapter::new( OpenAIConfig::new("sk-openai", "gpt-4o") .with_max_completion_tokens(4096), )?; let responses = OpenAIResponsesAdapter::new( OpenAIResponsesConfig::new("sk-openai", "gpt-5") .with_reasoning_effort("medium") .with_max_output_tokens(4096), )?; ``` `OpenAIResponsesConfig::new(authentication, model)` follows the existing `OpenAIConfig` argument order. Profile-specific constructors are model-first: ```rust let public = OpenAIResponsesConfig::public( "gpt-5", "sk-openai", ); let private = OpenAIResponsesConfig::chatgpt_private( "gpt-5-codex", Authentication::new(RotatingTokens), ) .with_originator("my-client") .with_user_agent("my-client/1.0"); ``` The Responses adapter supports configurable endpoints, request policy, limits, attribution, encrypted reasoning continuation, idempotency, response-size bounds, and private ChatGPT request/stream behavior without embedding any consumer-specific metadata or persistence formats. ### Typed response-attempt supersession OpenAI Responses retries only before visible output by default. A consumer that can discard an already-rendered attempt may opt into post-output recovery with a typed capability: ```rust use agentkit_loop::{ModelTurnEvent, SessionConfig}; let session = SessionConfig::new("session-1") .with_response_attempt_supersession(); fn retain_latest_attempt( events: impl IntoIterator<Item = ModelTurnEvent>, ) -> Vec<ModelTurnEvent> { let mut current_attempt = Vec::new(); for event in events { match event { ModelTurnEvent::ResponseAttemptSuperseded => { current_attempt.clear(); } other => current_attempt.push(other), } } current_attempt } ``` Opting in asserts that the consumer can discard all deltas, tool calls, usage, and reconstruction state from the failed attempt. AgentKit emits `ResponseAttemptSuperseded` after the failed attempt and before any replacement output. Without the capability, a failure after visible output is returned instead of replayed.
1 parent db9713c commit 68e29cc

47 files changed

Lines changed: 9910 additions & 542 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

Cargo.lock

Lines changed: 9 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

README.md

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,9 +122,25 @@ Shell:
122122

123123
The filesystem crate also supports session-scoped read-before-write enforcement through `FileSystemToolResources` and `FileSystemToolPolicy`.
124124

125+
## Provider authentication and resilience
126+
127+
Provider configs use `agentkit_http::Authentication` as their first-class
128+
credential type. For Baseten, Cerebras, Groq, Mistral, OpenAI, OpenRouter, and
129+
authenticated Ollama/vLLM endpoints, passing a bare string to an authentication
130+
argument is shorthand for bearer authentication. Custom refresh-capable
131+
credentials can be installed with `.with_authentication_provider(...)`.
132+
Anthropic is the exception: `AnthropicConfig::new(...)` sends `x-api-key`, while
133+
`AnthropicConfig::with_auth_token(...)` explicitly selects a bearer auth token.
134+
Ollama and vLLM authentication is optional.
135+
136+
Provider resilience is also opt-in. Configs store
137+
`Option<agentkit_http::ResilienceConfig>` and default to `None`; call
138+
`.with_resilience(...)` to enable retries and timeouts. Leaving it as `None`
139+
preserves the existing single-attempt behavior.
140+
125141
## Quick start
126142

127-
1. Set your OpenRouter API key and model — either through environment variables or directly in code via `OpenRouterConfig::new(api_key, model)`.
143+
1. Set your OpenRouter API key and model — either through environment variables or directly in code via `OpenRouterConfig::new(authentication, model)`.
128144
2. Run one of the examples.
129145

130146
Example commands:

book/src/ch01-model-adapter.md

Lines changed: 28 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -254,10 +254,17 @@ pub enum ModelTurnEvent {
254254
Delta(Delta),
255255
ToolCall(ToolCallPart),
256256
Usage(Usage),
257+
ResponseAttemptSuperseded,
257258
Finished(ModelTurnResult),
258259
}
259260
```
260261

262+
`ResponseAttemptSuperseded` is capability-gated. An adapter emits it after events
263+
from a failed visible attempt and before any replacement-attempt events. The loop
264+
forwards it as `AgentEvent::ResponseAttemptSuperseded`. A consumer that enables
265+
`SessionConfig::with_response_attempt_supersession()` must discard all deltas,
266+
tool calls, usage updates, and reconstruction state from the preceding attempt.
267+
261268
## Building an adapter from scratch
262269

263270
To see what the traits require, consider a hypothetical model provider that does not use the OpenAI format. Suppose "AcmeAI" has a proprietary REST API:
@@ -287,12 +294,14 @@ No `messages` array. No `choices` wrapper. No `tool_calls`. A completely differe
287294
```rust
288295
pub struct AcmeAdapter {
289296
client: Client,
290-
api_key: String,
297+
authentication: Authentication,
298+
resilience: Option<ResilienceConfig>,
291299
}
292300

293301
pub struct AcmeSession {
294302
client: Client,
295-
api_key: String,
303+
authentication: Authentication,
304+
resilience: Option<ResilienceConfig>,
296305
}
297306

298307
#[async_trait]
@@ -302,7 +311,8 @@ impl ModelAdapter for AcmeAdapter {
302311
async fn start_session(&self, _config: SessionConfig) -> Result<AcmeSession, LoopError> {
303312
Ok(AcmeSession {
304313
client: self.client.clone(),
305-
api_key: self.api_key.clone(),
314+
authentication: self.authentication.clone(),
315+
resilience: self.resilience.clone(),
306316
})
307317
}
308318
}
@@ -348,10 +358,15 @@ impl ModelSession for AcmeSession {
348358
"config": { "temperature": 0.5, "max_tokens": 256 },
349359
});
350360

351-
let resp: AcmeResponse = self.client
361+
let authentication = self.authentication.authenticate(None).await
362+
.map_err(|e| LoopError::Provider(e.to_string()))?;
363+
let mut request = self.client
352364
.post("https://api.acme.ai/v1/generate")
353-
.bearer_auth(&self.api_key)
354-
.json(&body)
365+
.json(&body);
366+
for (name, value) in authentication.headers() {
367+
request = request.header(name, value);
368+
}
369+
let resp: AcmeResponse = request
355370
.send().await
356371
.map_err(|e| LoopError::Provider(e.to_string()))?
357372
.json().await
@@ -619,9 +634,9 @@ This is the complete provider. All of the transcript conversion, tool call seria
619634

620635
Not all OpenAI-compatible providers are identical. The three hooks exist for providers that need to customise the standard request/response flow.
621636

622-
OpenRouter uses all three:
637+
OpenRouter uses all three, while exposing its first-class credential through `authentication()`:
623638

624-
1. **`preprocess_request`** — adds bearer auth, `X-Title`, and `HTTP-Referer` headers
639+
1. **`preprocess_request`** — adds `X-Title` and `HTTP-Referer` headers
625640
2. **`preprocess_response`** — the API sometimes returns HTTP 200 with an error payload instead of a proper error status; the hook parses these and converts them to errors before the adapter attempts normal deserialization
626641
3. **`postprocess_response`** — extracts the `cost` field from the usage object (OpenRouter-specific, not part of the standard format) and adds `openrouter.model` and `openrouter.refusal` to the item metadata
627642

@@ -633,11 +648,15 @@ impl CompletionsProvider for OpenRouterProvider {
633648
fn endpoint_url(&self) -> &str { &self.base_url }
634649
fn config(&self) -> &OpenRouterRequestConfig { &self.request_config }
635650

651+
fn authentication(&self) -> Option<Authentication> {
652+
Some(self.authentication.clone())
653+
}
654+
636655
fn preprocess_request(
637656
&self,
638657
builder: agentkit_http::HttpRequestBuilder,
639658
) -> agentkit_http::HttpRequestBuilder {
640-
let mut builder = builder.bearer_auth(&self.api_key);
659+
let mut builder = builder;
641660
if let Some(app_name) = &self.app_name {
642661
builder = builder.header("X-Title", app_name);
643662
}

book/src/ch03-transcript-model.md

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -49,18 +49,23 @@ pub enum ItemKind {
4949

5050
The variants are ordered: `System < Developer < User < Assistant < Tool < Context`. This ordering is used by compaction strategies that need to sort or prioritise items by role.
5151

52-
Role mapping to provider wire formats:
53-
54-
| agentkit `ItemKind` | OpenAI role | What it carries |
55-
| ------------------- | ------------- | ---------------------------------- |
56-
| `System` | `"system"` | Hardcoded application instructions |
57-
| `Developer` | `"system"` | Developer-level instructions |
58-
| `User` | `"user"` | End-user messages |
59-
| `Assistant` | `"assistant"` | Model-generated text + tool calls |
60-
| `Tool` | `"tool"` | Tool execution results |
61-
| `Context` | `"system"` | Project context (AGENTS.md, etc.) |
62-
63-
System, Developer, and Context all map to `"system"` in the OpenAI wire format, but they carry different semantic intent. The distinction matters for compaction: system items are never trimmed, context items may be refreshed, and developer items sit between the two. Collapsing them into a single kind would lose information that compaction strategies need.
52+
Role mapping depends on the OpenAI API and profile. Chat Completions uses:
53+
54+
| agentkit `ItemKind` | Chat Completions role | What it carries |
55+
| ------------------- | --------------------- | ---------------------------------- |
56+
| `System` | `"system"` | Hardcoded application instructions |
57+
| `Developer` | `"developer"` | Developer-level instructions |
58+
| `User` | `"user"` | End-user messages |
59+
| `Assistant` | `"assistant"` | Model-generated text + tool calls |
60+
| `Tool` | `"tool"` | Tool execution results |
61+
| `Context` | `"system"` | Project context (AGENTS.md, etc.) |
62+
63+
For the Responses API, the public profile keeps System and Context items as
64+
`system`; the private profile sends both as `developer`. Context content is
65+
encoded unchanged, without consumer-specific prose. These kinds retain distinct
66+
semantic intent even when a provider maps them to the same role. The distinction
67+
matters for compaction: system items are never trimmed, context items may be
68+
refreshed, and developer items sit between the two.
6469

6570
### Why item-based, not message-based
6671

book/src/ch05-model-adapter.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,7 @@ pub enum ModelTurnEvent {
105105
Delta(Delta),
106106
ToolCall(ToolCallPart),
107107
Usage(Usage),
108+
ResponseAttemptSuperseded,
108109
Finished(ModelTurnResult),
109110
}
110111
```
@@ -129,11 +130,21 @@ Turn event timeline:
129130
130131
Usage(Usage) ← token counts
131132
133+
ResponseAttemptSuperseded ← optional; invalidates every event above
134+
Delta(...) ← replacement attempt starts after the marker
135+
132136
Finished(ModelTurnResult) ← always last
133137
```
134138

135139
`Finished` always comes last. `Usage` typically comes just before `Finished` but some providers interleave it with deltas. `ToolCall` events represent fully assembled tool calls — the adapter has already accumulated the streaming chunks internally.
136140

141+
`ResponseAttemptSuperseded` is emitted only when the consumer enabled
142+
`SessionConfig::with_response_attempt_supersession()`. It appears after the failed
143+
visible attempt and before replacement output. The loop resets its attempt-local
144+
tool-call and usage state and forwards `AgentEvent::ResponseAttemptSuperseded`;
145+
consumers must discard every delta, tool call, usage update, and reconstruction
146+
state from the preceding attempt.
147+
137148
### ModelTurnResult
138149

139150
```rust

book/src/ch15-caching.md

Lines changed: 13 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ pub struct SessionConfig {
1313
pub session_id: SessionId,
1414
pub metadata: MetadataMap,
1515
pub cache: Option<PromptCacheRequest>,
16+
pub consumer_capabilities: SessionConsumerCapabilities,
1617
}
1718

1819
pub struct TurnRequest {
@@ -83,16 +84,12 @@ The simplest place to configure caching is the session:
8384

8485
```rust
8586
let mut driver = agent
86-
.start(SessionConfig {
87-
session_id: SessionId::new("coding-agent"),
88-
metadata: MetadataMap::new(),
89-
cache: Some(PromptCacheRequest {
90-
mode: PromptCacheMode::BestEffort,
91-
strategy: PromptCacheStrategy::Automatic,
92-
retention: Some(PromptCacheRetention::Short),
93-
key: None,
94-
}),
95-
})
87+
.start(SessionConfig::new("coding-agent").with_cache(PromptCacheRequest {
88+
mode: PromptCacheMode::BestEffort,
89+
strategy: PromptCacheStrategy::Automatic,
90+
retention: Some(PromptCacheRetention::Short),
91+
key: None,
92+
}))
9693
.await?;
9794
```
9895

@@ -225,16 +222,12 @@ This makes caching visible to reporters and host-side cost accounting without ex
225222
For most hosts, start here:
226223

227224
```rust
228-
SessionConfig {
229-
session_id: SessionId::new("demo"),
230-
metadata: MetadataMap::new(),
231-
cache: Some(PromptCacheRequest {
232-
mode: PromptCacheMode::BestEffort,
233-
strategy: PromptCacheStrategy::Automatic,
234-
retention: Some(PromptCacheRetention::Short),
235-
key: None,
236-
}),
237-
}
225+
SessionConfig::new("demo").with_cache(PromptCacheRequest {
226+
mode: PromptCacheMode::BestEffort,
227+
strategy: PromptCacheStrategy::Automatic,
228+
retention: Some(PromptCacheRetention::Short),
229+
key: None,
230+
})
238231
```
239232

240233
Then reach for explicit breakpoints only when you need to control exact cache boundaries.

0 commit comments

Comments
 (0)