Skip to content

Conversation

@lorenss-m
Copy link
Contributor

@lorenss-m lorenss-m commented Dec 17, 2025

Note

Introduces remote BYOK support, tighter error propagation, stricter scenario validation, and broader HTTP client instrumentation.

  • Remote BYOK support: Adds --byok CLI flag, byok config, and use_byok in SingleTaskRequest; forwarded via submit_rollouts while stripping api_key/model_client from remote agent_params.
  • Agent error propagation: MCPAgent.run() now sets ctx.error on failures and step errors; new tests cover exception and step-level propagation.
  • Scenario robustness: Environment.run_scenario_setup() now raises detailed errors for missing/invalid scenarios and malformed/empty content; MCPAgent.run() gives clearer messages when ctx.prompt is missing.
  • Gateway auth rules: OpenAIChatAgent forbids custom api_key with HUD Gateway; CLI injects HUD API key for OpenAI-compatible when targeting gateway.
  • HTTP instrumentation: Extends auto-instrumentation to aiohttp alongside httpx to inject trace/auth headers.
  • Eval context stability: EvalContext.__aenter__ cleans up connections and contextvars on setup failure.
  • Minor: log auto-respond errors; lazy-import questionary/typer; display byok in eval summary.

Written by Cursor Bugbot for commit 4ddaa33. This will update automatically on new commits. Configure here.

@lorenss-m lorenss-m marked this pull request as ready for review December 28, 2025 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants