The aiida-agents REPL used to stream the agent's reply token by token; now it blocks and dumps the whole answer at once after the spinner. On a local Ollama model that's a few seconds of spinner followed by a wall of text, which reads worse than watching it type out.
It regressed in #8 (commit 7b4d333), the human-gated write path, where ask switched from agent.run_stream(...) + stream_text to a blocking await agent.run(question). That was deliberate rather than careless: #8 added output_type=(str, DeferredToolRequests) and _handle_deferred, which need the full AgentRunResult to inspect result.output, and run_stream is geared to streaming text, so blocking was the quick way to get the result object back.
I think we can have both: stream while the model is producing a plain-text answer, and fall back to the blocking path only for the DeferredToolRequests (approval) case, which can't stream anyway. Worth checking first how pydantic-ai's streaming composes with a union output_type that includes deferred tool requests before committing to an approach.
This is independent of the prompt_toolkit REPL rewrite in #7: streaming is output-side (consuming run_stream instead of blocking on agent.run), the input library is input-side, so switching to prompt_toolkit neither fixes nor removes this. The ask change stands either way.
For reference, the streaming version #8 removed:
old ask
async with agent.run_stream(question) as result:
print("Agent: ", end="", flush=True)
printed_len = 0
async for chunk in result.stream_text(debounce_by=None):
print(chunk[printed_len:], end="", flush=True)
printed_len = len(chunk)
print()
Noticed while reviewing #19, which only added history threading to ask, not the streaming change. Ties into the CLI work in #7.
The
aiida-agentsREPL used to stream the agent's reply token by token; now it blocks and dumps the whole answer at once after the spinner. On a local Ollama model that's a few seconds of spinner followed by a wall of text, which reads worse than watching it type out.It regressed in #8 (commit
7b4d333), the human-gated write path, whereaskswitched fromagent.run_stream(...)+stream_textto a blockingawait agent.run(question). That was deliberate rather than careless: #8 addedoutput_type=(str, DeferredToolRequests)and_handle_deferred, which need the fullAgentRunResultto inspectresult.output, andrun_streamis geared to streaming text, so blocking was the quick way to get the result object back.I think we can have both: stream while the model is producing a plain-text answer, and fall back to the blocking path only for the
DeferredToolRequests(approval) case, which can't stream anyway. Worth checking first how pydantic-ai's streaming composes with a unionoutput_typethat includes deferred tool requests before committing to an approach.This is independent of the
prompt_toolkitREPL rewrite in #7: streaming is output-side (consumingrun_streaminstead of blocking onagent.run), the input library is input-side, so switching toprompt_toolkitneither fixes nor removes this. Theaskchange stands either way.For reference, the streaming version #8 removed:
old
askNoticed while reviewing #19, which only added history threading to
ask, not the streaming change. Ties into the CLI work in #7.