-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Implement adapter retry for Pydantic Validation Error #8050
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
dspy/adapters/base.py
Outdated
|
||
logger.debug("A ValueError occurred while parsing the LM output. Retrying with a new signature.", exc_info=True) | ||
if retry_count == 0: | ||
signature = create_signature_for_retry(signature) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note to self: How does modifying the signature interact with demonstrations and with optimization?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work! Can we include some demo code + logging/screenshot on retry resolves the pydantic parsing error? Although I am almost sure that this is helpful for small models, I don't have a concrete idea on how robust this retry works.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Left two minor comments
This PR introduces RetryAdapter, which
max_retries
times if fallback adapter failed.Resolve #7693
Alternative design of adapter retry:
Implement the decorator method around the adapter call
This design can separate the call function from the retry logic. However, faces difficulty in accessing the previous LLM call output.
Implement Retry as a dspy module
This is the previous design before LiteLLM. However, Pydantic validation is a responsibility of adapter abstraction and having the retry logic in this layer should be cohesive.
Have the logic in BaseAdapter.call
This approach is not suitable since several different adapters are used in the retry logic instead of retrying one adapter for multiple times.