Skip to content

Commit b1c7767

Browse files
Add documentation for resumable streaming feature
This documentation covers the new automatic resumable streaming feature in AIChatAgent and useAgentChat, which allows AI chat responses to automatically resume when clients reconnect after disconnection. Key features documented: - Automatic stream persistence to SQLite - Client reconnection handling - Server-side and client-side implementation examples - Configuration options for disabling resume Based on PR cloudflare/agents#673 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent 8e164c7 commit b1c7767

File tree

1 file changed

+102
-29
lines changed

1 file changed

+102
-29
lines changed

src/content/docs/agents/guides/resumable-streaming.mdx

Lines changed: 102 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,12 @@ sidebar:
55
order: 6
66
---
77

8-
import { PackageManagers, TypeScriptExample, WranglerConfig } from "~/components";
8+
import { TypeScriptExample, WranglerConfig, PackageManagers } from "~/components";
99

1010
The `AIChatAgent` class provides automatic resumable streaming out of the box. When a client disconnects and reconnects during an active stream, the response automatically resumes from where it left off.
1111

12+
This is particularly useful for long-running AI responses from reasoning models (such as OpenAI's o3, DeepSeek R1, or Anthropic's Claude) where users might disconnect due to network issues, page refreshes, or other interruptions.
13+
1214
## How it works
1315

1416
When you use `AIChatAgent` with `useAgentChat`:
@@ -17,37 +19,81 @@ When you use `AIChatAgent` with `useAgentChat`:
1719
2. **On disconnect**: The stream continues server-side, buffering chunks
1820
3. **On reconnect**: Client receives all buffered chunks and continues streaming
1921

22+
No additional configuration is required.
23+
2024
## Example
2125

22-
### Server
26+
### Server-side implementation
2327

24-
<TypeScriptExample>
28+
<TypeScriptExample filename="src/index.ts">
2529

2630
```ts
2731
import { AIChatAgent } from "agents/ai-chat-agent";
28-
import { streamText } from "ai";
32+
import { streamText, convertToModelMessages, createUIMessageStream, createUIMessageStreamResponse } from "ai";
2933
import { openai } from "@ai-sdk/openai";
3034

35+
type Env = {
36+
OPENAI_API_KEY: string;
37+
ChatAgent: AgentNamespace<ChatAgent>;
38+
};
39+
3140
export class ChatAgent extends AIChatAgent<Env> {
3241
async onChatMessage() {
33-
const result = streamText({
34-
model: openai("gpt-4o"),
35-
messages: this.messages
42+
const stream = createUIMessageStream({
43+
execute: async ({ writer }) => {
44+
const result = streamText({
45+
model: openai("gpt-4o"),
46+
messages: convertToModelMessages(this.messages)
47+
});
48+
49+
writer.merge(result.toUIMessageStream());
50+
}
3651
});
3752

3853
// Automatic resumable streaming - no extra code needed
39-
return result.toUIMessageStreamResponse();
54+
return createUIMessageStreamResponse({ stream });
4055
}
4156
}
4257
```
4358

4459
</TypeScriptExample>
4560

46-
### Client
61+
Configure your Wrangler configuration to include the Durable Object binding:
62+
63+
<WranglerConfig>
64+
65+
```jsonc
66+
{
67+
"compatibility_date": "2025-03-14",
68+
"compatibility_flags": [
69+
"nodejs_compat",
70+
"nodejs_compat_populate_process_env"
71+
],
72+
"durable_objects": {
73+
"bindings": [
74+
{
75+
"class_name": "ChatAgent",
76+
"name": "ChatAgent"
77+
}
78+
]
79+
},
80+
"main": "src/index.ts",
81+
"migrations": [
82+
{
83+
"new_sqlite_classes": ["ChatAgent"],
84+
"tag": "v1"
85+
}
86+
]
87+
}
88+
```
89+
90+
</WranglerConfig>
4791

48-
<TypeScriptExample>
92+
### Client-side implementation
4993

50-
```ts
94+
<TypeScriptExample filename="src/client.tsx">
95+
96+
```tsx
5197
import { useAgent } from "agents/react";
5298
import { useAgentChat } from "agents/ai-react";
5399

@@ -57,13 +103,30 @@ function Chat() {
57103
name: "my-chat"
58104
});
59105

60-
const { messages, input, handleInputChange, handleSubmit, status } =
61-
useAgentChat({
62-
agent
63-
// resume: true is the default - streams automatically resume on reconnect
64-
});
106+
const { messages, sendMessage, status } = useAgentChat({
107+
agent
108+
// resume: true is the default - streams automatically resume on reconnect
109+
});
65110

66-
// ... render your chat UI
111+
const handleSubmit = async (userMessage: string) => {
112+
await sendMessage({
113+
role: "user",
114+
parts: [{ type: "text", text: userMessage }]
115+
});
116+
};
117+
118+
return (
119+
<div>
120+
{messages.map((message) => (
121+
<div key={message.id}>
122+
{message.parts.map((part) =>
123+
part.type === "text" ? part.text : null
124+
)}
125+
</div>
126+
))}
127+
{status === "streaming" && <div>Streaming...</div>}
128+
</div>
129+
);
67130
}
68131
```
69132

@@ -73,26 +136,30 @@ function Chat() {
73136

74137
### Server-side (`AIChatAgent`)
75138

76-
- Creates SQLite tables for stream chunks and metadata on startup
77-
- Each stream gets a unique ID and tracks chunk indices
78-
- Chunks are buffered and flushed to SQLite every 100ms for performance
79-
- On client connect, checks for active streams and sends `CF_AGENT_STREAM_RESUMING`
80-
- Old completed streams are cleaned up after 24 hours
139+
The `AIChatAgent` class handles resumable streaming by:
140+
141+
- Creating SQLite tables for stream chunks and metadata on startup
142+
- Assigning each stream a unique ID and tracking chunk indices
143+
- Buffering chunks and flushing to SQLite every 100ms for performance
144+
- Detecting active streams on client connect and sending `CF_AGENT_STREAM_RESUMING` notification
145+
- Cleaning up old completed streams after 24 hours
81146

82147
### Client-side (`useAgentChat`)
83148

84-
- Listens for `CF_AGENT_STREAM_RESUMING` notification
85-
- Sends `CF_AGENT_STREAM_RESUME_ACK` when ready
86-
- Receives all buffered chunks and reconstructs the message
87-
- Continues receiving live chunks as they arrive
149+
The `useAgentChat` hook enables resumable streaming by:
150+
151+
- Listening for `CF_AGENT_STREAM_RESUMING` notification
152+
- Sending `CF_AGENT_STREAM_RESUME_ACK` when ready
153+
- Receiving all buffered chunks and reconstructing the message
154+
- Continuing to receive live chunks as they arrive
88155

89156
## Disabling resume
90157

91158
If you do not want automatic resume (for example, for short responses), disable it:
92159

93-
<TypeScriptExample>
160+
<TypeScriptExample filename="src/client.tsx">
94161

95-
```ts
162+
```tsx
96163
const { messages } = useAgentChat({
97164
agent,
98165
resume: false // Disable automatic stream resumption
@@ -103,4 +170,10 @@ const { messages } = useAgentChat({
103170

104171
## Try it
105172

106-
Refer to the [resumable-stream-chat example](https://github.com/cloudflare/agents/tree/main/examples/resumable-stream-chat) for a complete working example. Start a long response, refresh the page mid-stream, and watch it resume automatically.
173+
Refer to the [resumable-stream-chat example](https://github.com/cloudflare/agents/tree/main/examples/resumable-stream-chat) for a complete working implementation. Start a long response, refresh the page mid-stream, and watch it resume automatically.
174+
175+
## Related resources
176+
177+
- [Using AI Models](/agents/api-reference/using-ai-models/)
178+
- [WebSockets](/agents/api-reference/websockets/)
179+
- [Store and Sync State](/agents/api-reference/store-and-sync-state/)

0 commit comments

Comments
 (0)