Skip to content
53 changes: 39 additions & 14 deletions docs/mkdocs/en/session/summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,11 +231,27 @@ boundary rules.

One important branch-summary behavior: after `WithCacheSafeForking(true)` is
enabled, a non-empty branch trigger may fork the current parent request for the
branch summary, but it will not also run the cascaded full-session summary in
that same summary pass. The framework skips that full-session target instead of
falling back to a standalone full-session prompt or reusing the branch-scoped
fork request. Trigger a full-session summary separately when you need an
all-branch summary.
branch summary, but that same summary pass does not make a second standalone
full-session LLM call. This applies to the common single-`filterKey` session as
well as sessions that contain multiple filter keys. The framework skips that
extra LLM target instead of falling back to a standalone full-session prompt or
reusing the branch-scoped fork request. When every event loaded on the session
has the same `filterKey`, a materialized branch summary is copied to
`SummaryFilterKeyAllContents` in the same pass. This is a loaded-window
optimization: a storage event limit can omit older events from other branches,
so do not infer historical branch/full equivalence from the copy. On a
multi-`filterKey` session, the full-session key is left untouched in that pass;
trigger a full-session summary separately when you need an all-branch summary.

More generally, a branch-triggered full-session cascade depends on the branch
Comment thread
liuzengh marked this conversation as resolved.
target producing a summary in that pass. If the branch gate declines to update
its summary, the framework stops the cascade instead of independently advancing
the full-session summary.

`WithSummaryJobTimeout(...)` is the deadline for the entire summary job. A
multi-`filterKey` cascade runs the branch and full-session targets sequentially,
and both targets share that deadline. Size the timeout for their combined model
Comment thread
liuzengh marked this conversation as resolved.
and persistence latency.

Prompt rules:

Expand Down Expand Up @@ -555,10 +571,10 @@ sent, the mode is `custom_response` and the prompt estimate remains zero.
Advanced integrations can attach a report before entering a higher-level
summary flow with `summary.ContextWithReport(ctx, report)` and retrieve it with
`summary.ReportFromContext(ctx)`. The framework reuses that report for a single
summary path; when a cascade generates multiple summaries in parallel, each
worker receives a cloned report so branch-specific writes do not race. Those
forked reports are emitted through their per-call hooks and are not merged back
into the root report.
summary path; when a cascade targets both a branch and the full session, each
target receives a cloned report so target-specific writes remain isolated.
Comment thread
liuzengh marked this conversation as resolved.
Outdated
Those forked reports are emitted through their per-call hooks and are not
merged back into the root report.

For private deployments, endpoint IDs, fine-tuned models, newly released
models, or multi-tenant custom model configuration, prefer the instance or
Expand Down Expand Up @@ -1579,11 +1595,20 @@ Behavior notes:
- `WithCascadeFullSessionSummary(...)` controls whether a non-empty branch
trigger also refreshes the full-session summary.
- With `WithCacheSafeForking(true)`, a branch-triggered summary pass only runs
the branch summary target when a parent fork request is available. The
full-session cascade target is skipped in that pass; it does not fall back to
the standalone full-session prompt and does not reuse the branch-scoped fork
request. Request a full-session summary separately when you need one for all
branches.
the branch summary LLM target when a parent fork request is available. It
does not fall back to a standalone full-session prompt and does not reuse the
branch-scoped fork request for a second LLM call. When every event loaded on
the session has the same `filterKey`, a materialized branch summary is copied
to `SummaryFilterKeyAllContents` in that pass. This loaded-window optimization
does not prove that older, unloaded history contains no other branches. On a
multi-`filterKey` session, the full-session cascade target is skipped; request
a full-session summary separately when you need one for all branches.
- A full-session cascade is conditional on the branch target producing a
summary in the same pass. If the branch is not updated, the full-session
target is not run independently.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
- `WithSummaryJobTimeout(...)` applies to the complete summary job. Branch and
full-session targets run sequentially and share the same deadline, so allow
for their combined model and persistence latency.
- To keep only full-session summaries from branch-triggered automatic summary,
pass an explicit empty allowlist and leave cascade enabled:

Expand Down
39 changes: 30 additions & 9 deletions docs/mkdocs/zh/session/summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,9 +216,22 @@ standalone 重试;这次重试也可以按相同的边界规则选择更小的

这里有一个重要的 branch 摘要行为:开启 `WithCacheSafeForking(true)` 后,非空
branch 触发摘要时,可以用当前父请求 fork 来生成 branch 摘要;但同一轮 summary
pass 不会再跑级联出来的全量会话摘要。框架会直接跳过这个全量摘要目标,而不是
回退到独立的全量摘要 prompt,也不会复用这个 branch 视角的 fork request。如果
需要覆盖所有 branch 的全量摘要,需要单独触发一次全量会话摘要。
pass 不会再发第二次独立的全量会话 LLM 调用。这既适用于最常见的单
`filterKey` 会话,也适用于包含多个 filterKey 的会话。框架会跳过这次额外的
LLM 目标,而不是回退到独立的全量摘要 prompt,也不会复用这个 branch 视角的
fork request。当 session 当前加载的所有事件都属于同一个 `filterKey` 时,如果
本轮实际产出了 branch 摘要,该摘要会复制到 `SummaryFilterKeyAllContents`。这是
基于当前加载窗口的优化:存储层事件数量限制可能省略其他 branch 的更早事件,
因此不能仅凭这次复制推断 branch/full 的完整历史等价。多 filterKey 会话里,
全量摘要 key 在这一轮保持不动;如果需要覆盖所有 branch 的全量摘要,请单独
触发一次全量会话摘要。

更一般地,branch 触发的全量摘要级联依赖 branch 目标在本轮实际产出摘要。如果
branch gate 决定不更新摘要,框架会停止级联,不会独立推进全量会话摘要。
Comment thread
liuzengh marked this conversation as resolved.
Outdated

`WithSummaryJobTimeout(...)` 是整条 summary job 的 deadline。多 filterKey 级联会
串行执行 branch 与全量摘要目标,两个目标共享同一个 deadline;配置时需要覆盖
两段模型调用和持久化的总延迟。

Prompt 规则:

Expand Down Expand Up @@ -512,9 +525,9 @@ summarizer := summary.NewSummarizer(

高级集成如果要在高层 summary 流程前放入同一个 report,可以使用
`summary.ContextWithReport(ctx, report)`,需要从 context 取出时使用
`summary.ReportFromContext(ctx)`。单一路径会复用这个 report;cascade 并行生成多个
summary 时,框架会给每个 worker 克隆一份 report,避免不同分支同时写同一个对象。
这些 fork 出来的 report 会通过各自调用的 hook 发出,不会再合并回 root report。
`summary.ReportFromContext(ctx)`。单一路径会复用这个 report;cascade 同时包含 branch
和全量目标时,框架会给每个目标克隆一份 report,隔离各自的写入。这些 fork 出来的
report 会通过各自调用的 hook 发出,不会再合并回 root report。

对于私有部署、endpoint ID、微调模型、新模型或多租户自定义模型配置,优先使用模型实例或单次运行 option,
避免不同用户覆盖同一个进程级注册表:
Expand Down Expand Up @@ -1505,9 +1518,17 @@ sessionService := inmemory.NewSessionService(
- `WithCascadeFullSessionSummary(...)` 控制非空分支触发摘要时,是否同时刷新
全量会话摘要。
- 开启 `WithCacheSafeForking(true)` 后,如果当前有父请求可 fork,branch 触发的
summary pass 只会生成 branch 摘要;级联出来的全量会话摘要目标会被跳过,不会
回退到独立的全量摘要 prompt,也不会复用这个 branch 视角的 fork request。如果
确实需要覆盖所有 branch 的全量摘要,请单独触发一次全量会话摘要。
summary pass 只会跑 branch 摘要的 LLM 目标,不会回退到独立的全量摘要
prompt,也不会把这个 branch 视角的 fork request 再拿去发第二次 LLM 调用。
当 session 当前加载的所有事件都属于同一个 `filterKey` 时,如果本轮实际产出
了 branch 摘要,该摘要会复制到 `SummaryFilterKeyAllContents`。这是基于当前
加载窗口的优化,不能证明更早但未加载的历史中没有其他 branch。多 filterKey
会话里,级联出来的全量摘要目标会被跳过;如果确实需要覆盖所有 branch 的全量
摘要,请单独触发一次全量会话摘要。
- 全量摘要级联以本轮 branch 目标实际产出摘要为前提;如果 branch 没有更新,
不会独立运行全量摘要目标。
- `WithSummaryJobTimeout(...)` 作用于整条 summary job。branch 与全量摘要目标会
串行执行并共享同一个 deadline,因此配置时需要覆盖两段模型调用和持久化的总延迟。
- 如果只想保留 branch 触发出来的全量摘要,不写任何 branch 摘要,可以显式传入
空 allowlist,并保持默认 cascade 开启:

Expand Down
42 changes: 42 additions & 0 deletions session/inmemory/summary_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,48 @@ func TestMemoryService_CreateSessionSummary_FilterAllowlistAndCascade(t *testing
require.False(t, ok)
}

func TestMemoryService_EnqueueSummaryJob_CascadePreservesObserverContext(
t *testing.T,
) {
const filterKey = "branch"
s := NewSessionService(
WithSummarizer(&fakeSummarizer{allow: true, out: "summary"}),
WithAsyncSummaryNum(1),
)
defer s.Close()

ctx := context.Background()
key := session.Key{
AppName: "app",
UserID: "user",
SessionID: "observer-context",
}
sess, err := s.CreateSession(ctx, key, session.StateMap{})
require.NoError(t, err)
for i, target := range []string{filterKey, "other"} {
evt := event.New(fmt.Sprintf("inv-%d", i), "author")
evt.Timestamp = time.Now().Add(time.Duration(i) * time.Second)
evt.FilterKey = target
evt.Response = &model.Response{Choices: []model.Choice{{
Message: model.Message{Role: model.RoleUser, Content: target},
}}}
require.NoError(t, s.AppendEvent(ctx, sess, evt))
}

sess, err = s.GetSession(ctx, key)
require.NoError(t, err)
require.NoError(t, s.EnqueueSummaryJob(ctx, sess, filterKey, false))

require.Eventually(t, func() bool {
got, getErr := s.GetSession(ctx, key)
if getErr != nil || got == nil {
return false
}
return got.Summaries[filterKey] != nil &&
got.Summaries[session.SummaryFilterKeyAllContents] != nil
}, 2*time.Second, 10*time.Millisecond)
}

func TestMemoryService_EnqueueSummaryJob_NoSummarizer_NoOp(t *testing.T) {
// Create service with async summary enabled but no summarizer
s := NewSessionService(
Expand Down
69 changes: 68 additions & 1 deletion session/internal/summary/async_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -525,6 +525,61 @@ func TestAsyncSummaryWorker_EnqueueJob(t *testing.T) {
require.ErrorIs(t, err, context.DeadlineExceeded)
})

t.Run("cascade targets share the summary job deadline", func(t *testing.T) {
const filterKey = "branch1"
var calls []string
var deadlines []time.Time
config := AsyncSummaryConfig{
Summarizer: &mockSummarizer{shouldSummarize: true, summaryText: "test"},
SummaryJobTimeout: 20 * time.Millisecond,
SummaryDispatchPolicy: NewSummaryDispatchPolicy(nil, true),
CreateSummaryFunc: func(
ctx context.Context,
sess *session.Session,
target string,
_ bool,
) error {
calls = append(calls, target)
deadline, ok := ctx.Deadline()
require.True(t, ok)
deadlines = append(deadlines, deadline)
if target == filterKey {
sess.SummariesMu.Lock()
sess.Summaries[target] = &session.Summary{
Summary: "branch summary",
UpdatedAt: time.Now(),
}
sess.SummariesMu.Unlock()
recordSummaryMaterialized(ctx, sess, target)
return nil
}
<-ctx.Done()
return ctx.Err()
},
}
worker := NewAsyncSummaryWorker(config)
sess := &session.Session{
ID: "deadline-session",
AppName: "test-app",
UserID: "test-user",
Events: []event.Event{
{FilterKey: filterKey, Version: event.CurrentVersion},
{FilterKey: "branch2", Version: event.CurrentVersion},
},
Summaries: make(map[string]*session.Summary),
}

err := worker.runJob(&summaryJob{
ctx: context.Background(),
filterKey: filterKey,
session: sess,
})
require.ErrorIs(t, err, context.DeadlineExceeded)
require.Equal(t, []string{filterKey, ""}, calls)
require.Len(t, deadlines, 2)
require.True(t, deadlines[0].Equal(deadlines[1]))
})

t.Run("enqueue with filter key", func(t *testing.T) {
summarizer := &mockSummarizer{shouldSummarize: true, summaryText: "test"}
filterKeyCh := make(chan string, 10)
Expand All @@ -534,8 +589,20 @@ func TestAsyncSummaryWorker_EnqueueJob(t *testing.T) {
SummaryQueueSize: 10,
SummaryJobTimeout: time.Second,
SummaryDispatchPolicy: NewSummaryDispatchPolicy(nil, true),
CreateSummaryFunc: func(_ context.Context, _ *session.Session, fk string, _ bool) error {
CreateSummaryFunc: func(ctx context.Context, sess *session.Session, fk string, _ bool) error {
filterKeyCh <- fk
if fk != session.SummaryFilterKeyAllContents {
sess.SummariesMu.Lock()
if sess.Summaries == nil {
sess.Summaries = make(map[string]*session.Summary)
}
sess.Summaries[fk] = &session.Summary{
Summary: "branch summary",
UpdatedAt: time.Now(),
}
sess.SummariesMu.Unlock()
recordSummaryMaterialized(ctx, sess, fk)
}
return nil
},
}
Expand Down
Loading
Loading