Currently Ultron seems to rely on SFT-based LoRA training for evolution.
Have you considered introducing a GRPO stage after SFT?
A possible pipeline could be:
Trajectory Collection
→ SFT LoRA
→ GRPO Optimization
Reward signals could come from:
- Task success
- Tool execution success
- Planning efficiency
- Skill reuse quality
I think reinforcement learning may be a natural next step for Agent Evolution because it optimizes trajectories instead of only imitating them.
In addition, compared with pure SFT, reinforcement learning may help preserve previously acquired capabilities while optimizing new behaviors, potentially reducing catastrophic forgetting during continuous evolution.
Curious whether this has already been discussed in the roadmap.
Currently Ultron seems to rely on SFT-based LoRA training for evolution.
Have you considered introducing a GRPO stage after SFT?
A possible pipeline could be:
Trajectory Collection
→ SFT LoRA
→ GRPO Optimization
Reward signals could come from:
I think reinforcement learning may be a natural next step for Agent Evolution because it optimizes trajectories instead of only imitating them.
In addition, compared with pure SFT, reinforcement learning may help preserve previously acquired capabilities while optimizing new behaviors, potentially reducing catastrophic forgetting during continuous evolution.
Curious whether this has already been discussed in the roadmap.