Skip to content
70 changes: 56 additions & 14 deletions docs/mkdocs/en/session/summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,11 +231,38 @@ boundary rules.

One important branch-summary behavior: after `WithCacheSafeForking(true)` is
enabled, a non-empty branch trigger may fork the current parent request for the
branch summary, but it will not also run the cascaded full-session summary in
that same summary pass. The framework skips that full-session target instead of
falling back to a standalone full-session prompt or reusing the branch-scoped
fork request. Trigger a full-session summary separately when you need an
all-branch summary.
branch summary, but that same summary pass does not make a second standalone
full-session LLM call. This applies to the common single-`filterKey` session as
well as sessions that contain multiple filter keys. The framework skips that
extra LLM target instead of falling back to a standalone full-session prompt or
reusing the branch-scoped fork request. When every event loaded on the session
has the same `filterKey`, a materialized branch summary is copied to
`SummaryFilterKeyAllContents` in the same pass. This is a loaded-window
optimization: a storage event limit can omit older events from other branches,
so do not infer historical branch/full equivalence from the copy. On a
multi-`filterKey` session, the full-session key is left untouched in that pass;
trigger a full-session summary separately when you need an all-branch summary.

More generally, a branch-triggered full-session cascade depends on the branch
Comment thread
liuzengh marked this conversation as resolved.
target producing a summary in that pass. If the branch gate declines to update
its summary, the framework stops the cascade instead of independently advancing
the full-session summary. A failed dependent target returns an error but does
not create a separate durable recovery protocol. A later ordinary call must
pass the branch gate again and can return `nil` without completing the earlier
full target when that gate does not fire. To recover immediately, directly
force `SummaryFilterKeyAllContents`, or retry the branch cascade with
`force=true` from a context that does not carry a cache-safe parent fork.
Forcing a branch cascade with a cache-safe parent still intentionally skips its
dependent full-session LLM target.

Asynchronous workers log dependent-target errors after processing. A successful
enqueue only confirms that the job was accepted; it does not synchronously
return errors produced later by the worker.

`WithSummaryJobTimeout(...)` is the deadline for the entire summary job. A
multi-`filterKey` cascade runs the branch and full-session targets sequentially,
and both targets share that deadline. Size the timeout for their combined model
Comment thread
liuzengh marked this conversation as resolved.
and persistence latency.

Prompt rules:

Expand Down Expand Up @@ -555,10 +582,11 @@ sent, the mode is `custom_response` and the prompt estimate remains zero.
Advanced integrations can attach a report before entering a higher-level
summary flow with `summary.ContextWithReport(ctx, report)` and retrieve it with
`summary.ReportFromContext(ctx)`. The framework reuses that report for a single
summary path; when a cascade generates multiple summaries in parallel, each
worker receives a cloned report so branch-specific writes do not race. Those
forked reports are emitted through their per-call hooks and are not merged back
into the root report.
summary path. Distinct branch and full-session targets in a multi-`filterKey`
cascade each receive a cloned report so target-specific writes remain isolated.
Those forked reports are emitted through their per-call hooks and are not merged
back into the root report. The single-`filterKey` copy-persistence optimization
does not create this pair of target reports.

For private deployments, endpoint IDs, fine-tuned models, newly released
models, or multi-tenant custom model configuration, prefer the instance or
Expand Down Expand Up @@ -1579,11 +1607,25 @@ Behavior notes:
- `WithCascadeFullSessionSummary(...)` controls whether a non-empty branch
trigger also refreshes the full-session summary.
- With `WithCacheSafeForking(true)`, a branch-triggered summary pass only runs
the branch summary target when a parent fork request is available. The
full-session cascade target is skipped in that pass; it does not fall back to
the standalone full-session prompt and does not reuse the branch-scoped fork
request. Request a full-session summary separately when you need one for all
branches.
the branch summary LLM target when a parent fork request is available. It
does not fall back to a standalone full-session prompt and does not reuse the
branch-scoped fork request for a second LLM call. When every event loaded on
the session has the same `filterKey`, a materialized branch summary is copied
to `SummaryFilterKeyAllContents` in that pass. This loaded-window optimization
does not prove that older, unloaded history contains no other branches. On a
multi-`filterKey` session, the full-session cascade target is skipped; request
a full-session summary separately when you need one for all branches.
- A full-session cascade is conditional on the branch target producing a
summary in the same pass. If the branch is not updated, the full-session
target is not run independently. Failed dependent targets are not retried
from inferred or framework-persisted recovery state; a later pass must
materialize the branch again. For immediate recovery, force the full-session
key directly, or force the branch cascade without a cache-safe parent fork.
- Async enqueue success does not report later worker failures; dependent-target
errors are logged by the worker.
- `WithSummaryJobTimeout(...)` applies to the complete summary job. Branch and
full-session targets run sequentially and share the same deadline, so allow
for their combined model and persistence latency.
- To keep only full-session summaries from branch-triggered automatic summary,
pass an explicit empty allowlist and leave cascade enabled:

Expand Down
52 changes: 43 additions & 9 deletions docs/mkdocs/zh/session/summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,9 +216,31 @@ standalone 重试;这次重试也可以按相同的边界规则选择更小的

这里有一个重要的 branch 摘要行为:开启 `WithCacheSafeForking(true)` 后,非空
branch 触发摘要时,可以用当前父请求 fork 来生成 branch 摘要;但同一轮 summary
pass 不会再跑级联出来的全量会话摘要。框架会直接跳过这个全量摘要目标,而不是
回退到独立的全量摘要 prompt,也不会复用这个 branch 视角的 fork request。如果
需要覆盖所有 branch 的全量摘要,需要单独触发一次全量会话摘要。
pass 不会再发第二次独立的全量会话 LLM 调用。这既适用于最常见的单
`filterKey` 会话,也适用于包含多个 filterKey 的会话。框架会跳过这次额外的
LLM 目标,而不是回退到独立的全量摘要 prompt,也不会复用这个 branch 视角的
fork request。当 session 当前加载的所有事件都属于同一个 `filterKey` 时,如果
本轮实际产出了 branch 摘要,该摘要会复制到 `SummaryFilterKeyAllContents`。这是
基于当前加载窗口的优化:存储层事件数量限制可能省略其他 branch 的更早事件,
因此不能仅凭这次复制推断 branch/full 的完整历史等价。多 filterKey 会话里,
全量摘要 key 在这一轮保持不动;如果需要覆盖所有 branch 的全量摘要,请单独
触发一次全量会话摘要。

更一般地,branch 触发的全量摘要级联依赖 branch 目标在本轮实际产出摘要。如果
branch gate 决定不更新摘要,框架会停止级联,不会独立推进全量会话摘要。后续
目标失败时会返回错误,但不会创建独立的持久化恢复协议;后续普通调用必须重新
通过 branch gate。如果 gate 没有触发,该调用可能返回 `nil`,但此前失败的全量
目标仍未补齐。需要立即恢复时,应直接对 `SummaryFilterKeyAllContents` 强制生成
摘要,或者在不携带 cache-safe parent fork 的 context 中用 `force=true` 重试
branch cascade。携带 cache-safe parent 时,即使强制 branch cascade,后续全量
LLM 目标仍会按设计跳过。

异步 worker 会在执行结束后记录后续目标的错误;enqueue 成功只表示任务已接收,
不会把 worker 稍后产生的错误同步返回给已经结束的 enqueue 调用。

`WithSummaryJobTimeout(...)` 是整条 summary job 的 deadline。多 filterKey 级联会
串行执行 branch 与全量摘要目标,两个目标共享同一个 deadline;配置时需要覆盖
两段模型调用和持久化的总延迟。

Prompt 规则:

Expand Down Expand Up @@ -512,9 +534,10 @@ summarizer := summary.NewSummarizer(

高级集成如果要在高层 summary 流程前放入同一个 report,可以使用
`summary.ContextWithReport(ctx, report)`,需要从 context 取出时使用
`summary.ReportFromContext(ctx)`。单一路径会复用这个 report;cascade 并行生成多个
summary 时,框架会给每个 worker 克隆一份 report,避免不同分支同时写同一个对象。
这些 fork 出来的 report 会通过各自调用的 hook 发出,不会再合并回 root report。
`summary.ReportFromContext(ctx)`。单一路径会复用这个 report;多 filterKey cascade
中的独立 branch 和全量目标会各自获得一份克隆的 report,以隔离各自的写入。这些
fork 出来的 report 会通过各自调用的 hook 发出,不会再合并回 root report。单
filterKey 的 copy-persistence 优化不会产生这一对 target report。

对于私有部署、endpoint ID、微调模型、新模型或多租户自定义模型配置,优先使用模型实例或单次运行 option,
避免不同用户覆盖同一个进程级注册表:
Expand Down Expand Up @@ -1505,9 +1528,20 @@ sessionService := inmemory.NewSessionService(
- `WithCascadeFullSessionSummary(...)` 控制非空分支触发摘要时,是否同时刷新
全量会话摘要。
- 开启 `WithCacheSafeForking(true)` 后,如果当前有父请求可 fork,branch 触发的
summary pass 只会生成 branch 摘要;级联出来的全量会话摘要目标会被跳过,不会
回退到独立的全量摘要 prompt,也不会复用这个 branch 视角的 fork request。如果
确实需要覆盖所有 branch 的全量摘要,请单独触发一次全量会话摘要。
summary pass 只会跑 branch 摘要的 LLM 目标,不会回退到独立的全量摘要
prompt,也不会把这个 branch 视角的 fork request 再拿去发第二次 LLM 调用。
当 session 当前加载的所有事件都属于同一个 `filterKey` 时,如果本轮实际产出
了 branch 摘要,该摘要会复制到 `SummaryFilterKeyAllContents`。这是基于当前
加载窗口的优化,不能证明更早但未加载的历史中没有其他 branch。多 filterKey
会话里,级联出来的全量摘要目标会被跳过;如果确实需要覆盖所有 branch 的全量
摘要,请单独触发一次全量会话摘要。
- 全量摘要级联以本轮 branch 目标实际产出摘要为前提;如果 branch 没有更新,
不会独立运行全量摘要目标。失败的后续目标不会根据推断或框架持久化的恢复状态
自动重试;后续调用必须重新产出 branch 摘要。需要立即恢复时,应直接强制生成
全量 key,或在不携带 cache-safe parent fork 的 context 中强制 branch cascade。
- 异步 enqueue 成功不会返回 worker 稍后发生的错误;后续目标错误由 worker 记录。
- `WithSummaryJobTimeout(...)` 作用于整条 summary job。branch 与全量摘要目标会
串行执行并共享同一个 deadline,因此配置时需要覆盖两段模型调用和持久化的总延迟。
- 如果只想保留 branch 触发出来的全量摘要,不写任何 branch 摘要,可以显式传入
空 allowlist,并保持默认 cascade 开启:

Expand Down
69 changes: 68 additions & 1 deletion session/internal/summary/async_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -525,6 +525,61 @@ func TestAsyncSummaryWorker_EnqueueJob(t *testing.T) {
require.ErrorIs(t, err, context.DeadlineExceeded)
})

t.Run("cascade targets share the summary job deadline", func(t *testing.T) {
const filterKey = "branch1"
var calls []string
var deadlines []time.Time
config := AsyncSummaryConfig{
Summarizer: &mockSummarizer{shouldSummarize: true, summaryText: "test"},
SummaryJobTimeout: 20 * time.Millisecond,
SummaryDispatchPolicy: NewSummaryDispatchPolicy(nil, true),
CreateSummaryFunc: func(
ctx context.Context,
sess *session.Session,
target string,
_ bool,
) error {
calls = append(calls, target)
deadline, ok := ctx.Deadline()
require.True(t, ok)
deadlines = append(deadlines, deadline)
if target == filterKey {
sess.SummariesMu.Lock()
sess.Summaries[target] = &session.Summary{
Summary: "branch summary",
UpdatedAt: time.Now(),
}
sess.SummariesMu.Unlock()
recordSummaryMaterialized(ctx, target)
return nil
}
<-ctx.Done()
return ctx.Err()
},
}
worker := NewAsyncSummaryWorker(config)
sess := &session.Session{
ID: "deadline-session",
AppName: "test-app",
UserID: "test-user",
Events: []event.Event{
{FilterKey: filterKey, Version: event.CurrentVersion},
{FilterKey: "branch2", Version: event.CurrentVersion},
},
Summaries: make(map[string]*session.Summary),
}

err := worker.runJob(&summaryJob{
ctx: context.Background(),
filterKey: filterKey,
session: sess,
})
require.ErrorIs(t, err, context.DeadlineExceeded)
require.Equal(t, []string{filterKey, ""}, calls)
require.Len(t, deadlines, 2)
require.True(t, deadlines[0].Equal(deadlines[1]))
})

t.Run("enqueue with filter key", func(t *testing.T) {
summarizer := &mockSummarizer{shouldSummarize: true, summaryText: "test"}
filterKeyCh := make(chan string, 10)
Expand All @@ -534,8 +589,20 @@ func TestAsyncSummaryWorker_EnqueueJob(t *testing.T) {
SummaryQueueSize: 10,
SummaryJobTimeout: time.Second,
SummaryDispatchPolicy: NewSummaryDispatchPolicy(nil, true),
CreateSummaryFunc: func(_ context.Context, _ *session.Session, fk string, _ bool) error {
CreateSummaryFunc: func(ctx context.Context, sess *session.Session, fk string, _ bool) error {
filterKeyCh <- fk
if fk != session.SummaryFilterKeyAllContents {
sess.SummariesMu.Lock()
if sess.Summaries == nil {
sess.Summaries = make(map[string]*session.Summary)
}
sess.Summaries[fk] = &session.Summary{
Summary: "branch summary",
UpdatedAt: time.Now(),
}
sess.SummariesMu.Unlock()
recordSummaryMaterialized(ctx, fk)
}
return nil
},
}
Expand Down
Loading
Loading