Fix pier verify silently dropping trajectory data#3
Conversation
There was a problem hiding this comment.
Pull request overview
Fixes pier verify dropping trajectory data when an agent isn’t explicitly selected by inferring the agent from the pier session (when unambiguous) and delegating container-mode log extraction to Harbor. Updates CLI behavior/messaging and documentation to make trajectory capture expectations explicit, and aligns the test suite with the new flow.
Changes:
- Infer agent for
pier verify/pier capturefrom the session when exactly one is registered; error on multiple; provide clearer messaging when none. - Remove pier-side agent/session-dir detection heuristics; introduce
extract_agent_context()and route container-mode extraction through Harbor directly. - Update README and tests for new
--session-dirsemantics and required flags.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents new trajectory capture behavior, --session-dir semantics, and pier’s agent-agnostic architecture stance. |
| pier/cli.py | Implements agent inference, container-path --session-dir copying via docker cp, and revised capture/verify assembly logic. |
| pier/harbor_bridge.py | Removes agent-detection helpers and adds extract_agent_context() to delegate extraction to Harbor. |
| pier/tests/test_pier_cli.py | Adds/updates tests for agent inference, multi-agent errors, container --session-dir copy, and -a requirements. |
| pier/tests/test_harbor_bridge.py | Updates default agent name expectation and removes tests for deleted detection helpers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
3970350 to
a6e24f7
Compare
There was a problem hiding this comment.
Pull request overview
This PR fixes pier verify silently dropping trajectory data when the agent can’t be determined, and moves agent/session detection responsibilities toward Harbor to keep pier agent-agnostic.
Changes:
- Infer the agent for
pier verify/pier capturefrom the saved session (when unambiguous), emit clearer messaging, and error on multi-agent ambiguity. - In container mode, delegate trajectory extraction to Harbor via
extract_agent_context, and reinterpret--session-diras a container path copied out viadocker cp. - Remove pier-side agent auto-detection heuristics and update docs/tests accordingly.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| README.md | Updates CLI docs/messaging around --no-mount, agent inference, and --session-dir semantics; adds architecture note. |
| pier/cli.py | Implements agent inference, new container session copy helper, revised verify/capture flow, and improved user-facing guidance. |
| pier/harbor_bridge.py | Removes agent/session-dir heuristics and introduces extract_agent_context to delegate parsing to Harbor. |
| pier/tests/test_pier_cli.py | Adds/updates coverage for agent inference, ambiguity errors, container --session-dir copy behavior, and -a requirement. |
| pier/tests/test_harbor_bridge.py | Updates trial-result default agent name expectation; removes tests for deleted heuristics. |
Comments suppressed due to low confidence (1)
pier/harbor_bridge.py:909
extract_agent_context()callsAgentName(agent_name)/AgentFactory.create_agent_from_name(...)outside thetryblock. If the agent name is invalid (user typo) or Harbor rejects the logs dir, this will raise and can crashpier verify/pier captureinstead of returningNoneas the docstring promises. Wrap agent creation and name parsing in the sametry/except(logging a warning and returningNone) so extraction failures remain non-fatal and consistent.
from harbor.agents.factory import AgentFactory
from harbor.models.agent.context import AgentContext
from harbor.models.agent.name import AgentName
agent = AgentFactory.create_agent_from_name(
AgentName(agent_name), logs_dir=logs_dir
)
context = AgentContext()
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
a6e24f7 to
2bcddd8
Compare
Bug:
|
Problem
When
pier verifycouldn't determine which agent was used — becausenone was registered, or
--agentwasn't passed — it silently savedthe reward without the trajectory (conversation, tool use, cost data).
Users would see the score and assume everything was captured.
Changes
Fix trajectory loss:
pier verifyauto-infers the agent from thesession (set by
pier start --agent) and prints which one it's using.No agent → reward still saved, but with clear guidance on how to
capture the trajectory. Multiple agents → error asking to pick one.
Remove agent-specific session detection from pier: Container mode
now delegates to Harbor's
populate_context_post_rundirectly(
extract_agent_context), removingfind_container_agent_session_dirand its per-agent detection code.
extract_agent_logscallsextract_agent_contextafter bridging.Container
--session-dir: Now refers to a path inside thecontainer (copied via
docker cp). Supports agents not installedthrough pier, e.g. baked into the image.
--session-dirrequires-a: When--session-diris explicitlypassed,
-a/--agentis required (session inference is skipped to avoidmismatching the agent with unrelated session files). Agent
auto-detection from file patterns is also removed.
Messaging:
pier startwithout--agenthints about agentinstallation and trajectory capture.
--agentin host mode errorswith explanation. Extraction failure in container mode warns explicitly.
--no-mountdoc fix: Clarified that Harbor's internal mounts(agent logs, verifier output) still write to the host under
.pier/.Architecture note in README: pier stays agent-agnostic — agent
details belong in Harbor.
Breaking changes
pier capture/verify --session-dirwithout-anow errors--session-diris now interpreted as a containerpath (previously a host path)