Skip to content

Implement adapter retry for Pydantic Validation Error #8050

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

TomeHirata
Copy link
Collaborator

@TomeHirata TomeHirata commented Apr 8, 2025

This PR introduces RetryAdapter, which

  1. Call the main_adapter.
  2. Call the fallback_adapter if the main_adapter fails, c
  3. Retry the main_adapter including previous response for max_retries times if fallback adapter failed.

Resolve #7693

Alternative design of adapter retry:

  1. Implement the decorator method around the adapter call
    This design can separate the call function from the retry logic. However, faces difficulty in accessing the previous LLM call output.

  2. Implement Retry as a dspy module
    This is the previous design before LiteLLM. However, Pydantic validation is a responsibility of adapter abstraction and having the retry logic in this layer should be cohesive.

  3. Have the logic in BaseAdapter.call
    This approach is not suitable since several different adapters are used in the retry logic instead of retrying one adapter for multiple times.


logger.debug("A ValueError occurred while parsing the LM output. Retrying with a new signature.", exc_info=True)
if retry_count == 0:
signature = create_signature_for_retry(signature)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: How does modifying the signature interact with demonstrations and with optimization?

Copy link
Collaborator

@chenmoneygithub chenmoneygithub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! Can we include some demo code + logging/screenshot on retry resolves the pydantic parsing error? Although I am almost sure that this is helpful for small models, I don't have a concrete idea on how robust this retry works.

Copy link
Collaborator

@chenmoneygithub chenmoneygithub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Left two minor comments

@okhat okhat self-assigned this Apr 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Retries on Pydantic Validation Errors
3 participants