You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #1540, I reported that the /openai integration endpoint silently drops provider-specific extra parameters (e.g., AWS Bedrock guardrailConfig). This was fixed in #1579 by adding a RequestWithSettableExtraParams interface and SetExtraParams() methods to OpenAI request types, gated behind the x-bf-passthrough-extra-params: true header.
Gap Analysis
After the merge, I looked at the other integration endpoints and found that the fix only covers /openai types. The router-level extraction logic in createHandler() (router.go) is already generic -- it uses interface assertion:
ifrws, ok:=req.(RequestWithSettableExtraParams); ok {
rws.SetExtraParams(wrapper.ExtraParams)
}
But the assertion silently no-ops for types that don't implement the interface. Here's the current state:
Integration
Request Types
Has SetExtraParams?
extra_params works?
/openai
OpenAIChatRequest, etc. (8 types)
Yes
Fixed
/anthropic
AnthropicMessageRequest, AnthropicTextRequest, etc.
No
Broken
/bedrock
BedrockConverseRequest, BedrockTextCompletionRequest, etc.
No
Broken
/genai
GeminiGenerationRequest, GeminiBatchEmbeddingRequest, etc.
No
Broken
/cohere
CohereChatRequest, CohereEmbeddingRequest, etc.
No
Broken
/langchain
Reuses all above routes
Only OpenAI routes
Partial
/litellm
Reuses all above routes
Only OpenAI routes
Partial
/pydanticai
Reuses all above routes
Only OpenAI routes
Partial
All provider types already have ExtraParams map[string]interface{} fields (with json:"-" tag) and GetExtraParams() getters -- the only missing piece is the SetExtraParams() setter.
Add SetExtraParams() to every Anthropic, Bedrock, GenAI, and Cohere request type. Each is a 3-5 line method.
Subtlety: OpenAI types write to both r.ExtraParams AND r.Parameters.ExtraParams due to nested structs. Other providers have different struct layouts, so each setter needs to be aware of its type's structure.
Pros: Consistent with merged PR, low risk, easy to review
Define a reusable embedded struct or helper that provides SetExtraParams/GetExtraParams for any type. Since all provider types already share the same field pattern, this could reduce boilerplate.
Pros: DRY, one-time structural change, future-proof for new providers
Cons: Bigger refactor, touches more files, riskier to review
Which approach (A or B) aligns better with the project's direction?
Are there specific provider types where extra_params support is higher priority? (e.g., Bedrock for guardrailConfig, Anthropic for thinking/caching params)
I'm happy to implement whichever approach is preferred. Just want to align before writing code across multiple packages.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Context
In #1540, I reported that the
/openaiintegration endpoint silently drops provider-specific extra parameters (e.g., AWS BedrockguardrailConfig). This was fixed in #1579 by adding aRequestWithSettableExtraParamsinterface andSetExtraParams()methods to OpenAI request types, gated behind thex-bf-passthrough-extra-params: trueheader.Gap Analysis
After the merge, I looked at the other integration endpoints and found that the fix only covers
/openaitypes. The router-level extraction logic increateHandler()(router.go) is already generic -- it uses interface assertion:But the assertion silently no-ops for types that don't implement the interface. Here's the current state:
SetExtraParams?extra_paramsworks?/openaiOpenAIChatRequest, etc. (8 types)/anthropicAnthropicMessageRequest,AnthropicTextRequest, etc./bedrockBedrockConverseRequest,BedrockTextCompletionRequest, etc./genaiGeminiGenerationRequest,GeminiBatchEmbeddingRequest, etc./cohereCohereChatRequest,CohereEmbeddingRequest, etc./langchain/litellm/pydanticaiAll provider types already have
ExtraParams map[string]interface{}fields (withjson:"-"tag) andGetExtraParams()getters -- the only missing piece is theSetExtraParams()setter.Proposed Approaches
Approach A: Extend the same pattern from #1579
Add
SetExtraParams()to every Anthropic, Bedrock, GenAI, and Cohere request type. Each is a 3-5 line method.Subtlety: OpenAI types write to both
r.ExtraParamsANDr.Parameters.ExtraParamsdue to nested structs. Other providers have different struct layouts, so each setter needs to be aware of its type's structure.Approach B: Shared ExtraParams base/embed
Define a reusable embedded struct or helper that provides
SetExtraParams/GetExtraParamsfor any type. Since all provider types already share the same field pattern, this could reduce boilerplate.Questions for Maintainers
extra_paramssupport is higher priority? (e.g., Bedrock forguardrailConfig, Anthropic for thinking/caching params)I'm happy to implement whichever approach is preferred. Just want to align before writing code across multiple packages.
cc @akshaydeo @Pratham-Mishra04
All reactions