-
Notifications
You must be signed in to change notification settings - Fork 36
fix: Correctly format LLM model names for litellm providers #322
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,20 +1,49 @@ | ||
| import litellm | ||
|
|
||
| from openhands.sdk.llm import UNVERIFIED_MODELS_EXCLUDING_BEDROCK, VERIFIED_MODELS | ||
|
|
||
|
|
||
| # Get set of valid litellm provider names for filtering | ||
| # See: https://docs.litellm.ai/docs/providers | ||
| _VALID_LITELLM_PROVIDERS: set[str] = { | ||
| str(getattr(p, "value", p)) for p in litellm.provider_list | ||
| } | ||
|
|
||
|
|
||
| def get_provider_options() -> list[tuple[str, str]]: | ||
| """Get list of available LLM providers.""" | ||
| providers = list(VERIFIED_MODELS.keys()) + list( | ||
| UNVERIFIED_MODELS_EXCLUDING_BEDROCK.keys() | ||
| ) | ||
| return [(provider, provider) for provider in providers] | ||
| """Get list of available LLM providers. | ||
| Includes: | ||
| - All VERIFIED_MODELS providers (openhands, openai, anthropic, mistral) | ||
| even if not in litellm.provider_list (e.g. 'openhands' is custom) | ||
| - UNVERIFIED providers that are known to litellm (filters out invalid | ||
| "providers" like 'meta-llama', 'Qwen' which are vendor names) | ||
| Sorted alphabetically. | ||
| """ | ||
| # Verified providers always included (includes custom like 'openhands') | ||
| verified_providers = set(VERIFIED_MODELS.keys()) | ||
|
|
||
| # Unverified providers are filtered to only valid litellm providers | ||
| unverified_providers = set(UNVERIFIED_MODELS_EXCLUDING_BEDROCK.keys()) | ||
| valid_unverified = unverified_providers & _VALID_LITELLM_PROVIDERS | ||
|
|
||
| # Combine and sort | ||
| all_valid_providers = sorted(verified_providers | valid_unverified) | ||
|
|
||
| return [(provider, provider) for provider in all_valid_providers] | ||
|
|
||
|
|
||
| def get_model_options(provider: str) -> list[tuple[str, str]]: | ||
| """Get list of available models for a provider.""" | ||
| """Get list of available models for a provider, sorted alphabetically.""" | ||
| models = VERIFIED_MODELS.get( | ||
| provider, [] | ||
| ) + UNVERIFIED_MODELS_EXCLUDING_BEDROCK.get(provider, []) | ||
| return [(model, model) for model in models] | ||
|
|
||
| # Remove duplicates and sort | ||
| unique_models = sorted(set(models)) | ||
|
|
||
| return [(model, model) for model in unique_models] | ||
|
|
||
|
|
||
| provider_options = get_provider_options() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -79,10 +79,11 @@ def get_full_model_name(self): | |
| return str(self.custom_model) | ||
|
|
||
| model_str = str(self.model) | ||
| full_model = ( | ||
| f"{self.provider}/{model_str}" if "/" not in model_str else model_str | ||
| ) | ||
| return full_model | ||
|
|
||
| # Always add provider prefix - litellm requires it for routing. | ||
| # Even if model contains '/' (e.g. "openai/gpt-4.1" from openrouter) | ||
| # See: https://docs.litellm.ai/docs/providers | ||
| return f"{self.provider}/{model_str}" | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This makes perfect sense, yes; there are multiple providers like that, which proxy to others, basically |
||
|
|
||
|
|
||
| class SettingsSaveResult(BaseModel): | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@OpenHands investigate where do unverified models come from, in the sdk, and make an issue on the sdk repo to clean those up / verify as best we can, e.g. maybe we should refactor this code to be there.
(clone yourself agent-sdk from the same org)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm on it! enyst can track my progress at all-hands.dev
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Final summary (work completed)
What I investigated
/workspace/project/agent-sdk.openhands-sdk/openhands/sdk/llm/utils/unverified_models.pyget_supported_llm_models()aggregateslitellm.model_list+litellm.model_cost.keys()(plus optional Bedrock via boto3).get_unverified_models()then naively splits model identifiers on/(or.) and treats the first segment as a provider.1024-x-1024,low,medium,us,eu, etc.): LiteLLM includes parameterized/region-prefixed entries in its registries, which the SDK currently misclassifies as providers.Issue created on the SDK repo
Checklist against your request
Git / code changes
View full conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SDK PR for the root cause: