Reader: search the full conversation and load more history by default

[lvwerra]

Reader: search the full conversation and load more history by default

Goal and agreed scope

The operator approved whole-conversation Reader search and added:

“in any case i want the loading of history a bit more extensive, at the moment it feels like to often i only see 1-2 messages by default”

Deliver these together in one focused PR: a useful amount of recent conversation loaded automatically, and an explicit way to search older messages without manually paging through the transcript. Keep Reader responsive and independent of the terminal. This is an issue specification, not a claim that either change has been implemented.

The planning default is about 20 recent prompt/reply exchanges, progressively loaded. This is an implementation starting target, not a count specified by the operator or permission to read unlimited bytes to reach it. Validate and tune against representative fixtures. An exchange means an actual user prompt and its associated work/reply, not two raw trace records. The history should be available in scrollback; do not shrink text or change disclosure defaults to fit 20 exchanges on screen.

Additional operator requirement: the first rendered conversation should cover at least one Reader viewport when enough history is available, so already-visible text does not move downward while older turns are loaded or measured. Viewport coverage and the approximately 20-exchange history target are separate criteria: a tall answer can fill a page without providing enough history, and many tiny exchanges might still not fill a tall viewport.

Search is limited to the selected conversation's available transcript. Keep existing loaded-history search. No fleet-wide search, recursive search of separate child conversations, new settings panel, background indexing of every session, or broad Reader rewrite.

Current evidence and integration points

Planning checked main at ef08e843d1c67fcc4e1c463415819ea351734d6d on 2026-09-09. Recheck current main and related work before implementation.

• ReaderStore currently requests an initial 128 KiB tail with INITIAL_WINDOW_TURNS = 2; ordinary pages request 384 KiB. Retaining an already-loaded store refreshes forward rather than automatically filling older context.
• Server byte windows and index windows interpret that minimum differently. Byte windows grow until the parsed-record floor, source start or 8 MiB per-window ceiling; windowIndex takes exactly the last min message positions on a tail request. With min=2, an indexed source initially returns at most two transport messages. Raw records can also be tool/result/lifecycle fragments rather than readable exchanges. These are source findings, not a fresh reproduction of the operator's particular session.
• ConversationView searches only loaded exchanges through readerSearch. It already says “Search loaded conversation” and discloses that earlier history has not been searched. Preserve that truthful behavior; missing scope disclosure is not the reported defect.
• Reuse readerModel.ts reconciliation, exchanges.ts grouping, useVirtualRows.ts and readingPosition.ts. The source adapters are in traceWindows.ts, api.ts, index.js trace routes, traces.js and trace-revision.js. Current full parsers have message/text display caps; searching their clipped output alone cannot establish a complete no-match result.
• Reader architecture and review follow-up document the contracts merged in #119. Inactive retained stores currently have an eight-entry / approximately 32 MiB budget; database/full-summary parsing remains a known cost. Do not assume merely enlarging a global transport constant solves the user-facing history problem.

Requirements

1. Fuller initial history without a slower usable Reader

• On an ordinary cold visit, automatically make roughly 20 recent exchanges available, or all available exchanges when the conversation is shorter. Count reconciled, visible conversation exchanges, not tool records, lifecycle events, raw API turns or DOM nodes. An active last exchange need not be complete; never wait for the agent to finish before showing history.
• Render the usable composer promptly and prepare a stable first transcript viewport as specified below; once that viewport is ready, fill the remaining recent-history target in bounded background steps. Do not wait for all 20 exchanges or a whole-transcript summary before presenting a usable first page. Initial fill must not depend on a wheel event, clicking Earlier or opening Search. A summary failure/hang must not prevent a usable recent history.
• Choose and document byte, retained-data, request-count and work/time limits for automatic fill. On tool-heavy or huge-message traces, reaching a limit may yield fewer exchanges; disclose remaining history and keep Earlier usable. Do not loop forever trying to reach a count when the cursor cannot advance or an oversized record blocks progress. Preserve existing per-record protections or justify any scoped adjustment with tests.
• Prefer complete prompt/work/reply boundaries within the budget. Prepending must reconcile partial messages/results without duplicates, missing answers, fabricated completion or reordered tools. Do not discard relevant work records simply to make the exchange count look larger.
• Apply the selected-session policy consistently across supported byte-oriented and indexed/database traces. Inventory shared hook consumers: do not blindly change child readers' explicit top-of-context behavior or multiply automatic loading across collapsed Overview previews. Only active visible readers do automatic work; shared consumers must not duplicate it.
• Retain already-loaded history on warm return. If the desired recent context is still missing, fill only the missing part; do not throw away a warm store or repeat the entire preload on every mount, poll, visibility event or mode switch.
• Preserve newest user intent during prepends: Latest stays at the current bottom through late measurements, while a deliberate older reading position stays on the same message/offset. Coordinate with saved-position restoration so automatic fill neither consumes its retry budget incorrectly nor moves the user back to a stale anchor. Preserve draft, attachments, selections and disclosure state.
• User navigation/search-hit loading and normal live updates must not be starved by speculative history fill. Cancel or pause unnecessary work on source changes, hidden/inactive readers or disposal, respecting other active subscribers. Failure must leave existing text and composer available and release busy state.

First-viewport coverage and visual stability

• Measure the actual conversation scroller's usable height, excluding the composer, headers and other fixed controls, at the current width, zoom and normal disclosure state. On an ordinary cold load with sufficient readable history, make at least that much real conversation content available for the first presented transcript viewport. Continue beyond 20 exchanges if unusually short messages require it, within the documented safety bounds. A count of transport records, estimated virtual heights, blank spacers or a skeleton does not prove that a page of conversation is available. Keep virtualization; do not mount the entire loaded history just to measure it.
• Prioritize obtaining and measuring that first page, then progressively fill the larger history target. A brief stable loading surface is acceptable during bounded first-page preparation; do not flash a top-aligned one-message transcript and then push it down with subsequent prepends. Do not add an arbitrary sleep, wait for the entire preload, hide already-readable warm content, or block the composer to achieve apparent stability. An actually short/empty conversation must render its available content/empty state promptly. A read failure, no-progress cursor or safety limit must end preparation with a stable, honestly partial view rather than an indefinite spinner or a false claim of full coverage.
• Establish the initial semantic anchor/follow intent and scroll geometry before exposing the first transcript paint. Subsequent older-history prepends and virtual-row measurements must preserve the on-screen position of already-visible text, including the transition from content shorter than the viewport to scrollable content. Apply necessary scroll compensation before paint, not a visible jump followed by correction on a later frame; do not smooth-scroll preload corrections. Reserve known media dimensions and handle font/image/layout completion without resetting the anchor. Preserve a remembered manual position when one exists, and let new user scrolling override pending automatic placement.
• Re-evaluate coverage if the viewport changes during initial loading, with coalesced and bounded work rather than a resize-fetch loop. Preserve stable anchors after initialization as well. Distinguish older-history layout corrections from genuinely new live output: normal Latest-follow behavior for a new reply remains valid, but delayed rendering of already-existing older turns must not move the current reply down the screen.

2. Explicit whole-conversation search

• Keep the fast loaded-history search and add a clearly named whole-conversation scope/action. The user must deliberately request whole-history work; no full scan on each normal keypress or Reader mount. Define when an edited query is submitted, cancellation, and how Enter/Shift+Enter navigate results without accidentally starting duplicate scans.
• Use literal, case-insensitive text matching initially, consistent with the existing Reader's searchable text. Search readable prompts, assistant content and tool text/names already exposed by Reader search; exclude encoded image data and unrelated transport metadata. Do not substitute searching raw JSON syntax for searching the displayed conversation. Document the exact content scope and any format-specific limitations.
• Find matches outside the currently loaded window, including earlier than summary/parser display limits where the source still contains readable text. Do not silently skip text after a display truncation and then report a complete search. If some portion genuinely cannot be searched safely, report incomplete coverage and its reason.
• Support the local session transcript formats already supported by Reader, including JSONL and indexed/SQLite sources. Reuse the existing selected-source resolver, pins, source identity and conversation boundaries; never accidentally search another conversation in the same database or a guardian/fork trace. Existing unsupported sources keep an explicit unsupported state. Shared file/bundle/child views must remain compatible; adding new search entry points to those separate surfaces is not required here.
• Return bounded, paginated results with useful text snippets and stable message/context locators. State whether counts are matching messages/exchanges or occurrences; do not present a capped page count as a full total. Empty results, still searching, canceled, failed, unsupported and partially searched are distinct outcomes. Only a completed search of its stated scope may say no matches were found in that scope.
• Define a finite scan boundary for a live conversation so continuing output cannot make a search run forever. Make freshness/coverage explicit if new output arrives. Bind requests/results to query, session/source generation and the relevant source revision or snapshot; old completions must not overwrite a newer query. Append-only growth need not invalidate valid old hits, but replacements or incompatible mutable-row changes must not silently retarget them.
• Validate query and continuation inputs and resolve sources on the server. Scope any search cache to the exact source/query/version, bound its memory and lifetime, and invalidate it correctly. Do not put transcripts, queries or snippets into diagnostic/audit logs, URLs, analytics or new persistent browser storage. No external search service or new database infrastructure is needed by default.

3. Navigate to old results without breaking live reading

• Selecting a hit loads a bounded window around that message and brings the match into view with surrounding conversation. The user must not need to download every intervening page from the live tail first. Do not splice separated history windows together as if the missing interval were continuous.
• Keep result identity stable across pagination and deduplicate cross-window fragments. Preserve existing message/tool reconciliation and sanitized Markdown/highlighting. If a match is in text beyond the ordinary display cap, show a bounded matching excerpt/context or clearly explain the display limit; do not navigate to a row where the match cannot be found without explanation.
• Result loading must not corrupt live-tail cursors, mix source generations, mark a partial context window as the full conversation, or overwrite the retained pre-search view. Use an explicit historical window/navigation state or another small design with equivalent guarantees rather than overloading the contiguous live-history store.
• Clearing/leaving search restores the original semantic reading position or current Latest-follow intent; an explicit Latest action wins over old search restoration. New direct scrolling cancels stale programmatic targets. Handle variable-height rows, images, resize, narrow viewports and virtualization without moving the terminal host or page ancestors.
• Keep search/results keyboard accessible and reuse existing Reader styling/controls. Preserve the composer and attachments throughout searches and failures. Loading an old hit is read-only: it must not start, attach, claim, resize or type into a PTY.
• Coordinate #128: searching or viewing an old reply must not mark a newer unseen reply as read. Automatic history preload is not evidence of reading either.

4. Resource and lifecycle design

• Keep first paint, recent-history fill, whole-history search and match-context loading as distinct operations with clear ownership and priority. Reuse the established store/request/generation contracts; do not start a second uncontrolled poll loop or make every source adapter implement inconsistent cancellation rules.
• Bound work on both client and server. A browser AbortSignal alone is not enough if a synchronous server parse keeps blocking the manager. Use incremental/asynchronous scanning, query-level paging or narrowly scoped worker offload where needed for the supported source. Optimize only the parsing/search paths necessary for this feature; a general parser rewrite is not required.
• Bound concurrent scans/results/retained contexts across several open panes. Superseded work must stop consuming resources, not merely have its result ignored forever. Repeated queries may reuse a valid bounded cache; opening every session must not create a persistent index or trigger a full scan.
• Reuse current error/empty/reconnect handling and maintain one scroll owner and the virtualized DOM bound. Keep inactive-reader eviction effective with the increased preload; do not silently remove cache budgets or retain unlimited search histories.

Required tests and evidence

Test real source/store/view integration as well as helpers. Existing browser fixtures sometimes return all synthetic turns regardless of bytes/min; add realistic fixtures that honor window requests so a passing test cannot hide the current two-message default.

Area	Required cases and assertions
Default context	Cold-open JSONL and indexed/SQLite conversations with 0, 1, 2, 5, 20 and many exchanges. Ordinary fixtures with sufficient history reach the documented recent-context target automatically, without scrolling/search or waiting on a summary. Shorter conversations load all available content. Assert logical exchanges, not raw record count.
First viewport	Cold-open with very short turns, one tall answer, collapsed tool-heavy exchanges and varied-height content at desktop, tall/narrow mobile and zoomed sizes. When sufficient readable history exists, the first presented transcript covers the actual usable Reader viewport; test a case needing more than 20 tiny exchanges and another where one answer fills a page but the broader history target must still load. Short/empty sources, blocked records, slow reads and limits settle honestly without indefinite waiting. Blank padding or estimated spacer height must not satisfy the assertion.
No initial text drift	Hold the source fixed and delay older pages, virtual-row measurement, fonts and images separately. Track the screen coordinates of a stable visible text anchor from the first transcript paint through successive prepends, including an underfilled-to-scrollable fallback transition. Assert no history-induced displacement beyond a small documented tolerance (target 2 px) on intermediate visible frames, not only the final position. Cover a user scroll or resize during preparation, remembered manual anchors and a warm return; stale corrections must not override newer intent. Test genuine live appends separately so expected Latest-follow movement is not confused with the regression.
Tool-heavy and large records	Many tool/lifecycle records per prompt, split prompt/answer fragments, multiline messages, large tool output, empty trailing records and a single oversized record. Useful context increases within documented budgets; no duplicate/missing/reordered content, infinite paging, false beginning or fake completion.
Progressive responsiveness	Delay initial response, summary, later preload pages and one entire reader independently. First readable content and composer remain usable; one pane cannot block another. Cancel/fail preload and verify the busy slot releases, retained text survives and Earlier/search/live refresh still work.
Warm history and scroll	A/B/A navigation, reload with a saved older anchor, actual Reader/Terminal round trips, foreground return, new replies during fill and late layout changes. No redundant cold refill, lost draft or jump away from manual position; Latest stays at the geometric bottom.
Search correctness	Unique match only near the start of a transcript larger than the initial/history budgets; matches crossing scan chunks or message fragments; quoted/Unicode/literal punctuation; prompt, assistant and tool text; duplicate content/timestamps; matches beyond ordinary display caps. Compare results with an independent fixture oracle, with consistent counts and no duplicated page-boundary hits.
Coverage and pagination	No matches, many matches exceeding a result page, bounded-scan continuation, unsupported format, unreadable/truncated/oversized source, canceled scan and server failure. Never label partial coverage as a complete empty result; page through deterministic results without omissions.
Source isolation and races	Switch session or query mid-scan; resolve canceled requests late; replace/shrink a trace; append output during a search; mutate a SQLite streaming row and WAL. Results/cursors stay tied to the right source and query, and old hits are verified rather than silently opening different content.
Result navigation	Jump directly to a far-old hit without loading the intervening transcript into the browser; verify surrounding prompt/answer context, sanitized highlight and bounded DOM/data. Clear search, choose Latest, scroll during pending navigation and revisit a hit. Correct anchors and live-tail continuity survive.
Shared surfaces and Unread	Multiple active subscribers, hidden/released stores, children with their existing 2 MiB/top-of-context policy, Overview previews, file/trace consumers, attachments and input-required notices. No duplicate speculative readers, new PTY/socket/input/resize, terminal bleed or read acknowledgements from background preload/old search results.
Resource limits	Large synthetic byte and database histories, repeated queries, many open panes, slow I/O, rapid cancellation and cache eviction. Assert bounded request counts/bytes/results/DOM and cleanup; measure browser responsiveness and server event-loop impact, not just response size.

• Extend/add normally discovered suites alongside web/test/readerModel.test.mjs, readerRedesign.test.mjs, traceWindows.test.mjs and exchanges.test.mjs; server coverage should include reader-protocol.test.mjs, trace-window.test.mjs, trace-tail.test.mjs and relevant source adapters. Add actual endpoint/browser wiring coverage, not only unused helpers.
• Record before/after under the same synthetic fixtures: time to usable composer and first stable viewport, actual first-viewport coverage and maximum history-induced anchor displacement, time and bytes to the recent-context target, number of available exchanges, time to first/complete search result, hit navigation latency, mounted rows and server responsiveness. Include cold/warm, tool-heavy, large JSONL and SQLite cases. Set reproducible regression budgets and explain any tuning away from the approximately 20-exchange starting target. Do not trade away first-page responsiveness by waiting for all background history merely to eliminate visible drift.
• Run focused new suites, normal web/server suites, npm run test:render and npm run build in web, plus relevant full-app Reader checks. Report exact commands/results. If a baseline suite fails (including the previously documented date-sensitive cron test), verify/report it separately and run otherwise-skipped relevant suites; do not hide unrelated failures or fix unrelated code in this PR.
• Use disposable sources, databases, state roots and isolated test servers with fake transports. Never send paid prompts, search private operator transcripts for fixtures, write installed skills, restart production or deploy. Performance diagnostics should contain counts/timings only, not user conversation contents.

Coordination and handoff

• Coordinate #127 for mode-switch scroll preservation, #124 for app return/reconnect refresh, #122 for attachments and #120 for file-preview navigation. Do not duplicate their controllers or broaden their scopes. #80 is older overlapping reader work; preserve the merged Rebuild reader with retained transcripts, recovery and bounded rendering #119 protocol rather than restoring an older path.
• Update Reader architecture/validation docs with the new initial-history policy, search content/coverage contract, live-source behavior and actual resource bounds. Include a short demonstration of cold-opening a long conversation and finding/navigating an unloaded old message.
• Open one focused PR referencing this issue with design rationale, tests, measurements and known limitations. No merge or deployment without separate authorization. Follow SECURITY.md for any security-sensitive discovery; keep private data and exploit narratives out of public issue/PR content.