Skip to content

Use Logfire managed variables for prompt instructions#22

Open
dmontagu wants to merge 6 commits intomainfrom
managed-variable-instructions
Open

Use Logfire managed variables for prompt instructions#22
dmontagu wants to merge 6 commits intomainfrom
managed-variable-instructions

Conversation

@dmontagu
Copy link
Copy Markdown
Contributor

@dmontagu dmontagu commented Apr 8, 2026

Summary

  • move the agent instruction string into a Logfire managed variable with the bad demo prompt as the code default
  • resolve instructions at runtime via @relations_agent.instructions so the same code path works for local GEPA runs and a deployed server
  • switch eval, golden generation, and GEPA candidate testing to update the local Logfire variable provider instead of overriding agent instructions directly

Validation

  • uv run ruff check task.py main.py cases.py adapter.py README.md
  • uv run basedpyright task.py main.py cases.py adapter.py
  • managed-variable GEPA check with openai:gpt-4.1-mini task model and gateway/anthropic:claude-opus-4-6 proposer on the existing golden dataset:
    • baseline held-out test score on 25 cases: 69.04%
    • GEPA validation score on 8-case val slice: 57.31% -> 82.31%
    • optimized held-out test score on the same 25 cases: 81.90%

Notes

  • the CLI now uses LocalVariablesOptions so GEPA can mutate the same variable-backed instruction path locally
  • a deployed server can switch to remote Logfire managed variables without changing the agent instruction code path

Base automatically changed from 2026-04-ai-engineer to main April 8, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants