Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,25 @@ Shell:

The filesystem crate also supports session-scoped read-before-write enforcement through `FileSystemToolResources` and `FileSystemToolPolicy`.

## Provider authentication and resilience

Provider configs use `agentkit_http::Authentication` as their first-class
credential type. For Baseten, Cerebras, Groq, Mistral, OpenAI, OpenRouter, and
authenticated Ollama/vLLM endpoints, passing a bare string to an authentication
argument is shorthand for bearer authentication. Custom refresh-capable
credentials can be installed with `.with_authentication_provider(...)`.
Anthropic is the exception: `AnthropicConfig::new(...)` sends `x-api-key`, while
`AnthropicConfig::with_auth_token(...)` explicitly selects a bearer auth token.
Ollama and vLLM authentication is optional.

Provider resilience is also opt-in. Configs store
`Option<agentkit_http::ResilienceConfig>` and default to `None`; call
`.with_resilience(...)` to enable retries and timeouts. Leaving it as `None`
preserves the existing single-attempt behavior.

## Quick start

1. Set your OpenRouter API key and model — either through environment variables or directly in code via `OpenRouterConfig::new(api_key, model)`.
1. Set your OpenRouter API key and model — either through environment variables or directly in code via `OpenRouterConfig::new(authentication, model)`.
2. Run one of the examples.

Example commands:
Expand Down
37 changes: 28 additions & 9 deletions book/src/ch01-model-adapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,10 +254,17 @@ pub enum ModelTurnEvent {
Delta(Delta),
ToolCall(ToolCallPart),
Usage(Usage),
ResponseAttemptSuperseded,
Finished(ModelTurnResult),
}
```

`ResponseAttemptSuperseded` is capability-gated. An adapter emits it after events
from a failed visible attempt and before any replacement-attempt events. The loop
forwards it as `AgentEvent::ResponseAttemptSuperseded`. A consumer that enables
`SessionConfig::with_response_attempt_supersession()` must discard all deltas,
tool calls, usage updates, and reconstruction state from the preceding attempt.

## Building an adapter from scratch

To see what the traits require, consider a hypothetical model provider that does not use the OpenAI format. Suppose "AcmeAI" has a proprietary REST API:
Expand Down Expand Up @@ -287,12 +294,14 @@ No `messages` array. No `choices` wrapper. No `tool_calls`. A completely differe
```rust
pub struct AcmeAdapter {
client: Client,
api_key: String,
authentication: Authentication,
resilience: Option<ResilienceConfig>,
}

pub struct AcmeSession {
client: Client,
api_key: String,
authentication: Authentication,
resilience: Option<ResilienceConfig>,
}

#[async_trait]
Expand All @@ -302,7 +311,8 @@ impl ModelAdapter for AcmeAdapter {
async fn start_session(&self, _config: SessionConfig) -> Result<AcmeSession, LoopError> {
Ok(AcmeSession {
client: self.client.clone(),
api_key: self.api_key.clone(),
authentication: self.authentication.clone(),
resilience: self.resilience.clone(),
})
}
}
Expand Down Expand Up @@ -348,10 +358,15 @@ impl ModelSession for AcmeSession {
"config": { "temperature": 0.5, "max_tokens": 256 },
});

let resp: AcmeResponse = self.client
let authentication = self.authentication.authenticate(None).await
.map_err(|e| LoopError::Provider(e.to_string()))?;
let mut request = self.client
.post("https://api.acme.ai/v1/generate")
.bearer_auth(&self.api_key)
.json(&body)
.json(&body);
for (name, value) in authentication.headers() {
request = request.header(name, value);
}
let resp: AcmeResponse = request
.send().await
.map_err(|e| LoopError::Provider(e.to_string()))?
.json().await
Expand Down Expand Up @@ -619,9 +634,9 @@ This is the complete provider. All of the transcript conversion, tool call seria

Not all OpenAI-compatible providers are identical. The three hooks exist for providers that need to customise the standard request/response flow.

OpenRouter uses all three:
OpenRouter uses all three, while exposing its first-class credential through `authentication()`:

1. **`preprocess_request`** — adds bearer auth, `X-Title`, and `HTTP-Referer` headers
1. **`preprocess_request`** — adds `X-Title` and `HTTP-Referer` headers
2. **`preprocess_response`** — the API sometimes returns HTTP 200 with an error payload instead of a proper error status; the hook parses these and converts them to errors before the adapter attempts normal deserialization
3. **`postprocess_response`** — extracts the `cost` field from the usage object (OpenRouter-specific, not part of the standard format) and adds `openrouter.model` and `openrouter.refusal` to the item metadata

Expand All @@ -633,11 +648,15 @@ impl CompletionsProvider for OpenRouterProvider {
fn endpoint_url(&self) -> &str { &self.base_url }
fn config(&self) -> &OpenRouterRequestConfig { &self.request_config }

fn authentication(&self) -> Option<Authentication> {
Some(self.authentication.clone())
}

fn preprocess_request(
&self,
builder: agentkit_http::HttpRequestBuilder,
) -> agentkit_http::HttpRequestBuilder {
let mut builder = builder.bearer_auth(&self.api_key);
let mut builder = builder;
if let Some(app_name) = &self.app_name {
builder = builder.header("X-Title", app_name);
}
Expand Down
29 changes: 17 additions & 12 deletions book/src/ch03-transcript-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,18 +49,23 @@ pub enum ItemKind {

The variants are ordered: `System < Developer < User < Assistant < Tool < Context`. This ordering is used by compaction strategies that need to sort or prioritise items by role.

Role mapping to provider wire formats:

| agentkit `ItemKind` | OpenAI role | What it carries |
| ------------------- | ------------- | ---------------------------------- |
| `System` | `"system"` | Hardcoded application instructions |
| `Developer` | `"system"` | Developer-level instructions |
| `User` | `"user"` | End-user messages |
| `Assistant` | `"assistant"` | Model-generated text + tool calls |
| `Tool` | `"tool"` | Tool execution results |
| `Context` | `"system"` | Project context (AGENTS.md, etc.) |

System, Developer, and Context all map to `"system"` in the OpenAI wire format, but they carry different semantic intent. The distinction matters for compaction: system items are never trimmed, context items may be refreshed, and developer items sit between the two. Collapsing them into a single kind would lose information that compaction strategies need.
Role mapping depends on the OpenAI API and profile. Chat Completions uses:

| agentkit `ItemKind` | Chat Completions role | What it carries |
| ------------------- | --------------------- | ---------------------------------- |
| `System` | `"system"` | Hardcoded application instructions |
| `Developer` | `"developer"` | Developer-level instructions |
| `User` | `"user"` | End-user messages |
| `Assistant` | `"assistant"` | Model-generated text + tool calls |
| `Tool` | `"tool"` | Tool execution results |
| `Context` | `"system"` | Project context (AGENTS.md, etc.) |

For the Responses API, the public profile keeps System and Context items as
`system`; the private profile sends both as `developer`. Context content is
encoded unchanged, without consumer-specific prose. These kinds retain distinct
semantic intent even when a provider maps them to the same role. The distinction
matters for compaction: system items are never trimmed, context items may be
refreshed, and developer items sit between the two.

### Why item-based, not message-based

Expand Down
11 changes: 11 additions & 0 deletions book/src/ch05-model-adapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ pub enum ModelTurnEvent {
Delta(Delta),
ToolCall(ToolCallPart),
Usage(Usage),
ResponseAttemptSuperseded,
Finished(ModelTurnResult),
}
```
Expand All @@ -129,11 +130,21 @@ Turn event timeline:

Usage(Usage) ← token counts

ResponseAttemptSuperseded ← optional; invalidates every event above
Delta(...) ← replacement attempt starts after the marker

Finished(ModelTurnResult) ← always last
```

`Finished` always comes last. `Usage` typically comes just before `Finished` but some providers interleave it with deltas. `ToolCall` events represent fully assembled tool calls — the adapter has already accumulated the streaming chunks internally.

`ResponseAttemptSuperseded` is emitted only when the consumer enabled
`SessionConfig::with_response_attempt_supersession()`. It appears after the failed
visible attempt and before replacement output. The loop resets its attempt-local
tool-call and usage state and forwards `AgentEvent::ResponseAttemptSuperseded`;
consumers must discard every delta, tool call, usage update, and reconstruction
state from the preceding attempt.

### ModelTurnResult

```rust
Expand Down
33 changes: 13 additions & 20 deletions book/src/ch15-caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ pub struct SessionConfig {
pub session_id: SessionId,
pub metadata: MetadataMap,
pub cache: Option<PromptCacheRequest>,
pub consumer_capabilities: SessionConsumerCapabilities,
}

pub struct TurnRequest {
Expand Down Expand Up @@ -83,16 +84,12 @@ The simplest place to configure caching is the session:

```rust
let mut driver = agent
.start(SessionConfig {
session_id: SessionId::new("coding-agent"),
metadata: MetadataMap::new(),
cache: Some(PromptCacheRequest {
mode: PromptCacheMode::BestEffort,
strategy: PromptCacheStrategy::Automatic,
retention: Some(PromptCacheRetention::Short),
key: None,
}),
})
.start(SessionConfig::new("coding-agent").with_cache(PromptCacheRequest {
mode: PromptCacheMode::BestEffort,
strategy: PromptCacheStrategy::Automatic,
retention: Some(PromptCacheRetention::Short),
key: None,
}))
.await?;
```

Expand Down Expand Up @@ -225,16 +222,12 @@ This makes caching visible to reporters and host-side cost accounting without ex
For most hosts, start here:

```rust
SessionConfig {
session_id: SessionId::new("demo"),
metadata: MetadataMap::new(),
cache: Some(PromptCacheRequest {
mode: PromptCacheMode::BestEffort,
strategy: PromptCacheStrategy::Automatic,
retention: Some(PromptCacheRetention::Short),
key: None,
}),
}
SessionConfig::new("demo").with_cache(PromptCacheRequest {
mode: PromptCacheMode::BestEffort,
strategy: PromptCacheStrategy::Automatic,
retention: Some(PromptCacheRetention::Short),
key: None,
})
```

Then reach for explicit breakpoints only when you need to control exact cache boundaries.
Expand Down
Loading
Loading