Save trajectory after each turn

[li-boxuan]

Currently mini-swe-agent only saves trajectory after finishing the task. This is not ideal in environments where mini-swe-agent could get killed. In fact, in terminal-bench experiments, we kill agents after they timeout; without saving trajectories after each turn, no trajectory would be persisted.

[li-boxuan]

@john-b-yang would you be able to review this? It would be nice to have harbor/terminal-bench not depend on my own fork :)

[john-b-yang]

Will check it out ~2 weeks from now! We have some of our things we're trying to get out.

[klieret]

Hi @li-boxuan sorry for the late response, John & I were busy working on https://codeclash.ai/

Hmm, so so far, saving the trajectory simply wasn't part of the responsibilities of this class.

Why don't you just do

try:
    agent.run()
finally:
    save_traj(agent)

[klieret]

(This is already what we do for the CLI of mini)

[li-boxuan]

Hi @li-boxuan sorry for the late response, John & I were busy working on https://codeclash.ai/

Hmm, so so far, saving the trajectory simply wasn't part of the responsibilities of this class.

Why don't you just do

try:
    agent.run()
finally:
    save_traj(agent)

That didn't work - at least in terminal-bench, where we kill the agent process if it times out

[klieret]

Hi @li-boxuan I see, I guess if you SIGKILL the process, there's no way any at-exit behavior can work. We never needed that, but I'm sure you have a reason for that.

I really want to keep the default class as lean as possible, so I am hesitating, but I also want to support tbench. Would the following be acceptable:

1. I create a TbenchMiniAgent subclass so you have the python bindings
2. If you need CLI support, I could add an --agent-class switch similar to how we support --model-class and you would specify --agent-class minisweagent.agents.extra.tbench.TbenchMiniAgent (somewhat clunky, we could add a shortcut it if necessary)

Having your own class would also make it easy to add any future adjustments you might have, I would happily support anything there as long as it doesn't interfere with the rest

Let me know if you use python bindings or need the CLI

[li-boxuan]

@klieret Sounds great!

We need CLI support (this is how we use mini-swe-agent: https://github.com/laude-institute/harbor/blob/133719fee1fa0a08357b789a30894f008d646e98/src/harbor/agents/installed/mini_swe_agent.py#L398)

HarborMiniAgent might be a better name, since we have moved all harness code from original https://github.com/laude-institute/terminal-bench repository to https://github.com/laude-institute/harbor

[li-boxuan]

Forgot to say - feel free to close this PR in favor of yours

[klieret]

Got it, thanks! Let me make this happen today or over the weekend :) I'll let you know here

[li-boxuan]

Gentle ping on this; we are using my fork in https://github.com/laude-institute/harbor/blob/main/src/harbor/agents/installed/install-mini-swe-agent.sh.j2 which means we will miss any great new improvement from mini-swe-agent team. On the other hand we have this workaround so not urgent :)