feat: add reasoning support for chat completions #978
Conversation
|
Caution Review failedThe pull request is closed. 📝 WalkthroughSummary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings. WalkthroughStructured reasoning (Effort + MaxTokens) and reasoning details were introduced across schemas and providers; many converters now return errors and validate reasoning limits; streaming and non‑streaming flows propagate reasoning content, details, and signatures; call-sites, transports, utilities, and UI were updated accordingly. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Client
participant BifrostHTTP as Bifrost-HTTP
participant Schemas
participant Converter as ToProviderRequest
participant Provider
participant Model
participant Stream as Streaming
Client->>BifrostHTTP: POST /chat or /responses (with nested Reasoning)
BifrostHTTP->>Schemas: Unmarshal (custom JSON -> Reasoning)
Schemas->>Converter: ToAnthropic/ToCohere/ToBedrockRequest(bifrostReq)
alt conversion error
Converter-->>BifrostHTTP: error
BifrostHTTP-->>Client: HTTP error
else conversion ok
Converter->>Provider: provider-specific request (Model set)
Provider->>Provider: validate Reasoning.MaxTokens >= MinimumReasoningMaxTokens
Provider->>Model: call external model (stream or non-stream)
Model->>Stream: emit deltas (include Reasoning & ReasoningDetails)
Stream->>Client: stream chunks with reasoning deltas and signatures
Note right of Stream: Non-stream responses also include Reasoning & Details
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Areas to focus on:
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (30)
Comment |
This stack of pull requests is managed by Graphite. Learn more about stacking. |
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
d24ab88 to
e1c5d21
Compare
Pratham-Mishra04
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
78eb82c to
3540e84
Compare
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
e1c5d21 to
b5f6f04
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
plugins/jsonparser/utils.go (1)
303-323: Preservereasoning_detailsin deep copy of stream deltas
deepCopyChatStreamResponseChoiceDeltanow copiesReasoning, but it still omits the newReasoningDetailsslice. When the JSON parser plugin is enabled, this means anyreasoning_detailson the original delta will be dropped in the copied response.You likely want to keep these details intact, similar to
RefusalandToolCalls:func (p *JsonParserPlugin) deepCopyChatStreamResponseChoiceDelta(original *schemas.ChatStreamResponseChoiceDelta) *schemas.ChatStreamResponseChoiceDelta { if original == nil { return nil } result := &schemas.ChatStreamResponseChoiceDelta{ - Role: original.Role, - Reasoning: original.Reasoning, // Shallow copy - Refusal: original.Refusal, // Shallow copy - ToolCalls: original.ToolCalls, // Shallow copy - we don't modify tool calls + Role: original.Role, + Reasoning: original.Reasoning, // Shallow copy + ReasoningDetails: original.ReasoningDetails, // Shallow copy - not modified by plugin + Refusal: original.Refusal, // Shallow copy + ToolCalls: original.ToolCalls, // Shallow copy - we don't modify tool calls } // Deep copy Content pointer if it exists (this is what we modify) if original.Content != nil { contentCopy := *original.Content result.Content = &contentCopy } return result }core/providers/anthropic/responses.go (1)
296-403: Fix Type field and Nil Effort handling in Anthropic reasoning logicThe
ToAnthropicResponsesRequestreasoning block has two critical bugs:
AnthropicThinking.Type incorrectly set to Effort string. Line 332 sets
Type: string(*bifrostReq.Params.Reasoning.Effort), but the field is documented to accept only"enabled"or"disabled". Effort values like"minimal","low","medium","high","auto"are invalid and will create malformed Anthropic requests.Nil Effort semantics inconsistent with chat path and Bedrock. The current condition (line 326)
Effort != nil && Effort != "none"means nil Effort → disabled. However, the chat.go path and Bedrock both treat nil Effort as enabled (only"none"triggers disabled). This causes the same input to enable reasoning on one path and disable it on another.Apply the suggested fix:
- Change condition to
Effort == nil || Effort != "none"- Change Type to
"enabled"(not the Effort string)
🧹 Nitpick comments (11)
core/schemas/responses.go (1)
231-236: Responses reasoning schema extension is consistentAdding
MaxTokens *int \json:"max_tokens,omitempty"`alongsideEffort` cleanly models reasoning configuration for Responses; this struct remains backward compatible and matches how chat reasoning is modeled elsewhere. Any provider-specific constraints (e.g., Anthropics’s minimum) can be enforced in the converters rather than here.core/schemas/chatcompletions.go (2)
154-185: ChatParameters reasoning migration and legacy support look correctThe
ChatParametersUnmarshalJSON alias pattern cleanly:
- Accepts the new nested
reasoningobject,- Maps legacy
reasoning_effortintoChatReasoning.Effort, and- Rejects payloads that set both, which prevents ambiguous configuration.
ChatReasoningitself mirrors the Responses reasoning shape and keeps the Anthropics-specificmax_tokensas an optional field, which is appropriate here.Also applies to: 186-220, 222-226
468-478: Reasoning details synthesis across assistant messages and streaming deltas is well‑designedThe new JSON handling for:
ChatMessage.UnmarshalJSON(splitting base fields vs embedded assistant/tool structs),ChatAssistantMessage.UnmarshalJSON, andChatStreamResponseChoiceDelta.UnmarshalJSONcorrectly keeps the core message fields intact while auto‑populating
ReasoningDetailsfrom the flatreasoningstring when no details are present. This gives downstream code a single, structured way to consume reasoning without breaking existing payloads that only sendreasoning. The mutual‑exclusion checks and alias usage avoid recursion issues.Also applies to: 480-521, 625-667, 714-731, 755-763, 765-794
core/providers/openai/openai.go (1)
981-988: LGTM! Token details propagation matches text completion pattern.The propagation logic is correct. Minor note: the field order here (PromptTokensDetails, then CompletionTokensDetails) differs from the text completion streaming path (lines 515-520) which does CompletionTokensDetails first. Consider aligning the order for consistency, though this has no functional impact.
framework/streaming/chat.go (1)
55-105: LGTM! ReasoningDetails merging logic is correct.The merge-by-Index approach correctly handles:
- Text, Summary, Data: accumulated via string concatenation
- Signature, Type: overwritten (appropriate for final/immutable values)
- New details: appended when Index not found
The nested loop is O(n×m) but acceptable given typical small detail counts.
For improved readability, consider extracting the detail update logic into a helper function:
func mergeReasoningDetail(existing, incoming *schemas.ChatReasoningDetails) { if incoming.Text != nil { if existing.Text == nil { existing.Text = incoming.Text } else { *existing.Text += *incoming.Text } } // ... similar for other fields }core/providers/openai/utils.go (1)
5-24: Converters look correct; note that ReasoningDetails is intentionally omitted.The OpenAI↔Bifrost message converters correctly map name, role, content, tool fields, and assistant metadata (refusal, reasoning, annotations, tool calls). They don’t map
ChatAssistantMessage.ReasoningDetails, which is fine as long as we never expect those details to survive a round‑trip through OpenAI models; if that becomes a requirement, we’ll need to extendOpenAIChatAssistantMessageor carry details elsewhere.Also applies to: 26-45
core/providers/bedrock/chat.go (1)
58-60: Reasoning extraction for Bedrock chat looks correct; only minor polish possible.
- In
ToBifrostChatResponse, reasoning text/signature fromReasoningContentblocks are mapped intoChatReasoningDetails(type: reasoning.text) and concatenated into a singleassistantMessage.Reasoningstring. The single‑text‑block unwrap (promoting it toContentStr) is a nice compatibility touch.- In
ToBifrostChatCompletionStream,ReasoningContentdeltas emit chunks withReasoningplus a one‑elementReasoningDetailsarray, and signature‑only deltas emitReasoningDetailswith a populatedSignaturefield.Two small nits you might consider (not blockers):
reasoningTextaccumulates with a trailing newline; trimming the final\nbefore assigning toassistantMessage.Reasoningwould avoid a cosmetic artifact.- For signature‑only deltas you reuse
ChatReasoningDetailsTypeText; if the intention is to distinguish encrypted/signature‑only entries (e.g.,"reasoning.encrypted"), adjusting theTypewould make that clearer to downstream consumers.Functionally this is solid and matches the new reasoning schema.
Also applies to: 103-112, 117-121, 135-141, 293-341
core/providers/cohere/chat.go (2)
7-7: Cross-provider coupling: Cohere uses Anthropic's minimum token constant.The Cohere provider imports and uses
anthropic.MinimumReasoningMaxTokensfor validation. This creates a coupling where Cohere's validation depends on Anthropic's constraints.Consider extracting this constant to a shared location (e.g.,
schemaspackage or a dedicatedreasoningconstants file) to decouple providers:-import ( - "github.com/maximhq/bifrost/core/providers/anthropic" -) +import ( + "github.com/maximhq/bifrost/core/schemas" +)Then use
schemas.MinimumReasoningMaxTokensor similar. This makes the validation logic self-contained and allows each provider to potentially define its own minimum in the future.Also applies to: 112-113
598-598: Minor: Trailing newline appended to reasoningText.Line 598 appends
"\n"after each thinking block's text, which will leave a trailing newline in the finalreasoningText. This is consistent with the Anthropic implementation (line 355) but may be undesirable.Consider trimming the trailing newline when setting the final reasoning text, or use
strings.Joinwith newline separator:// Alternative approach using strings.Builder or Join reasoningText = strings.TrimSuffix(reasoningText, "\n")transports/bifrost-http/handlers/inference.go (1)
63-89: Custom ChatRequest unmarshalling and reasoning handling look correct
- Adding
"reasoning"tochatParamsKnownFieldsand wiring it throughChatParameterskeeps reasoning out ofExtraParamsand aligned with the new structuredReasoningobject.- The
ChatRequest.UnmarshalJSONimplementation cleanly separates concerns: it independently unmarshalsBifrostParams,Messages, and then delegates toChatParameters’s own unmarshaller, which avoids the embedded-struct interference you mention in the comment.- Switching
extractExtraParamsover tosonic.Unmarshalkeeps JSON parsing consistent with the rest of the file, while still usingjson.RawMessageas a lightweight buffer type.- Initializing
req := ChatRequest{ChatParameters: &schemas.ChatParameters{}}before unmarshalling is a good defensive step so the custom unmarshaller can safely populate parameters.If this path ever shows up as a hot spot, you could reduce the triple-unmarshal in
ChatRequest.UnmarshalJSONby unmarshalling once into a small helper struct that embeds all three views, but that’s purely an optimization, not a correctness issue.Also applies to: 169-199, 268-288, 472-509
core/providers/anthropic/anthropic.go (1)
303-311: Anthropic conversion error handling is solid; consider aligning cached-token aggregation
- Using
ToAnthropicChatRequest/ToAnthropicResponsesRequestinside theCheckContextAndGetRequestBodyclosures with expliciterrchecks is the right way to surface conversion failures early and avoid sending bad payloads.- Enabling
Stream = Ptr(true)only after a successful conversion in both chat and responses streaming paths is also correct.For the new cached-token usage fields:
- In
HandleAnthropicChatCompletionStreaming, you currently overwritePromptTokensDetails/CompletionTokensDetailseach timeusageToProcesshasCacheReadInputTokensorCacheCreationInputTokens.- In
HandleAnthropicResponsesStream, you do the same forInputTokensDetails/OutputTokensDetails.If Anthropic ever emits multiple usage records per stream, you might want to mirror the “take the max seen so far” pattern you already use for token counts (and that Bedrock uses for cached tokens) instead of blind overwrite, e.g.:
if usageToProcess.CacheReadInputTokens > 0 { if usage.PromptTokensDetails == nil { usage.PromptTokensDetails = &schemas.ChatPromptTokensDetails{} } if usageToProcess.CacheReadInputTokens > usage.PromptTokensDetails.CachedTokens { usage.PromptTokensDetails.CachedTokens = usageToProcess.CacheReadInputTokens } }Same idea for the other cached-token fields. Not strictly required now, but it keeps aggregation semantics consistent across providers.
Also applies to: 354-365, 551-560, 647-655, 696-707, 893-902
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (29)
core/providers/anthropic/anthropic.go(6 hunks)core/providers/anthropic/chat.go(10 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(4 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(3 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(1 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/semanticcache/utils.go(1 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (20)
core/providers/anthropic/errors.go (2)
ui/lib/types/logs.ts (1)
BifrostError(233-239)core/providers/anthropic/types.go (2)
AnthropicMessageError(376-379)AnthropicMessageErrorStruct(382-385)
core/providers/openai/openai.go (3)
plugins/mocker/main.go (1)
Usage(138-142)core/providers/perplexity/types.go (1)
Usage(69-78)ui/lib/types/logs.ts (1)
CompletionTokensDetails(197-202)
core/providers/vertex/vertex.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-282)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
ui/app/workspace/logs/views/logChatMessageView.tsx (2)
ui/lib/utils/validation.ts (2)
isJson(539-546)cleanJson(548-558)ui/app/workspace/logs/views/codeEditor.tsx (1)
CodeEditor(82-245)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/chatcompletions.go (2)
ChatMessage(469-478)ChatParameters(155-184)ui/lib/types/logs.ts (1)
ChatMessage(115-124)
core/schemas/mux.go (3)
core/schemas/responses.go (3)
ResponsesParametersReasoning(231-236)BifrostResponsesStreamResponse(1422-1460)ResponsesStreamResponseTypeReasoningSummaryTextDelta(1388-1388)core/schemas/chatcompletions.go (1)
ChatReasoning(223-226)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/openai/utils.go (3)
core/providers/openai/types.go (2)
OpenAIMessage(56-65)OpenAIChatAssistantMessage(67-72)core/schemas/chatcompletions.go (3)
ChatMessage(469-478)ChatToolMessage(621-623)ChatAssistantMessage(626-632)ui/lib/types/logs.ts (1)
ChatMessage(115-124)
core/providers/azure/azure.go (4)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-282)core/utils.go (1)
Ptr(51-53)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/anthropic/responses.go (3)
core/schemas/responses.go (1)
ResponsesParametersReasoning(231-236)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/types.go (2)
MinimumReasoningMaxTokens(14-14)AnthropicThinking(65-68)
core/providers/openai/chat.go (3)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/utils.go (1)
Ptr(51-53)
core/providers/anthropic/anthropic.go (7)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-282)core/utils.go (1)
Ptr(51-53)core/schemas/utils.go (1)
Ptr(16-18)core/schemas/chatcompletions.go (1)
ChatPromptTokensDetails(821-827)ui/lib/types/logs.ts (3)
CompletionTokensDetails(197-202)ResponsesResponseInputTokens(542-544)ResponsesResponseOutputTokens(546-548)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)core/schemas/responses.go (2)
ResponsesResponseInputTokens(260-266)ResponsesResponseOutputTokens(268-278)
core/providers/bedrock/bedrock.go (2)
core/providers/utils/utils.go (1)
CreateBifrostChatCompletionChunkResponse(687-716)core/schemas/bifrost.go (1)
ChatCompletionStreamRequest(90-90)
core/providers/bedrock/responses.go (4)
core/schemas/utils.go (4)
SafeExtractFromMap(519-525)SafeExtractString(280-297)SafeExtractInt(300-342)Ptr(16-18)core/schemas/responses.go (1)
ResponsesParametersReasoning(231-236)core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/cohere/cohere.go (1)
core/providers/cohere/chat.go (1)
ToCohereChatCompletionRequest(12-199)
core/providers/openai/types.go (2)
core/schemas/models.go (1)
Model(109-129)core/schemas/chatcompletions.go (9)
ChatMessageRole(457-457)ChatMessageContent(524-527)ChatToolMessage(621-623)ChatAssistantMessageAnnotation(670-673)ChatAssistantMessageToolCall(686-692)Alias(189-189)Alias(643-643)Alias(770-770)Alias(860-860)
core/schemas/chatcompletions.go (4)
core/providers/openai/types.go (1)
Alias(78-78)core/schemas/bifrost.go (2)
Alias(383-383)Alias(400-400)ui/lib/types/logs.ts (3)
ChatMessage(115-124)ChatMessageContent(113-113)ReasoningDetails(126-133)core/providers/gemini/types.go (2)
Role(13-13)Content(922-930)
core/providers/bedrock/utils.go (3)
core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)OrderedMap(268-268)core/providers/bedrock/types.go (1)
BedrockConverseRequest(44-57)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
framework/streaming/chat.go (4)
core/schemas/chatcompletions.go (1)
ChatAssistantMessage(626-632)core/utils.go (1)
Ptr(51-53)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)core/providers/gemini/types.go (1)
Type(778-778)
core/providers/bedrock/chat.go (2)
core/schemas/chatcompletions.go (4)
ChatReasoningDetails(723-730)ChatReasoningDetailsTypeText(719-719)ChatContentBlockTypeText(582-582)ChatAssistantMessage(626-632)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)
core/providers/anthropic/chat.go (2)
core/providers/anthropic/types.go (7)
AnthropicMessageRequest(40-59)AnthropicThinking(65-68)MinimumReasoningMaxTokens(14-14)AnthropicContentBlock(147-159)AnthropicContentBlockTypeThinking(143-143)AnthropicContentBlockTypeText(135-135)AnthropicContentBlockTypeToolUse(137-137)core/providers/anthropic/utils.go (1)
ConvertAnthropicFinishReasonToBifrost(27-32)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (34)
ui/lib/types/logs.ts (2)
122-124: LGTM! Type definitions properly structured.The addition of
reasoningandreasoning_detailsfields toChatMessageis well-designed and aligns with the PR's goal of adding unified reasoning support. The optional fields ensure backward compatibility.
126-133: LGTM! Well-designed interface for structured reasoning data.The
ReasoningDetailsinterface is properly structured with appropriate discriminated union types and optional fields that support various reasoning formats (summary, encrypted, text) as mentioned in the PR objectives.ui/app/workspace/logs/views/logChatMessageView.tsx (1)
73-94: Consistent renaming from thought to reasoning.The update is well-executed with consistent changes throughout the rendering logic. The JSON/plain-text rendering approach is maintained appropriately.
Note: The
reasoning_details?: ReasoningDetails[]field in the ChatMessage type is currently not rendered in this view, only themessage.reasoningstring is displayed. This mirrors how therefusalfield is handled (string only, despite optional detailed fields in the type). Verify whether this is intentional (e.g., for future use) or ifreasoning_detailsshould be rendered to display signatures and verification data as suggested by the PR objectives.core/providers/anthropic/types.go (1)
11-15: Reasoning min-tokens constant looks appropriateDefining
MinimumReasoningMaxTokens = 1024at the Anthropic provider level is a good way to centralize the API’s minimum reasoning budget; no issues from this file alone.core/providers/perplexity/chat.go (1)
29-35: Reasoning effort mapping correctly updated to nested paramsThe switch to
bifrostReq.Params.Reasoning.Effortwhile preserving the"minimal" → "low"mapping keeps Perplexity behaviour consistent with the legacyreasoning_effortfield and aligns with the newChatReasoningshape. No further changes needed here.core/providers/bedrock/types.go (1)
83-108: Bedrock reasoning content wiring is structurally soundThe new
ReasoningContentfields andBedrockReasoningContent{,Text}types integrate cleanly into both full content blocks and streaming deltas, and mirror the nested structure used elsewhere in the Bedrock path. From this file alone the types look correct and ready to be consumed by the Bedrock chat/response converters.Also applies to: 152-160, 436-441
core/providers/cohere/types.go (1)
519-524: Streaming thinking deltas are wired consistentlyExtending
CohereStreamContentwithThinking *stringreuses the existing thinking shape fromCohereContentBlockand cleanly surfaces thinking deltas in the stream; no issues found.core/schemas/chatcompletions.go (1)
829-836: ImageTokens field addition is schema-only; provider mapper integration pendingThe new
ImageTokens *intfield inChatCompletionTokensDetailsadds a forward-compatible extension to token accounting, similar toAudioTokensandReasoningTokens. However, verification shows no provider mappers currently populate this field—none of the provider implementations reference or assign values toimage_tokens. If provider support is intended, the mapper implementations need to be updated to extract and populate this field from provider responses.core/providers/cohere/cohere.go (2)
279-286: LGTM! Proper error propagation from request converter.The change correctly propagates errors from
ToCohereChatCompletionRequestthrough the request body builder, aligning with the error-first pattern established across providers in this PR.
331-342: LGTM! Streaming path now properly handles conversion errors.Good defensive pattern: the error is captured and checked before enabling streaming mode. This ensures conversion failures surface to the caller rather than being silently ignored.
core/providers/openai/openai.go (1)
515-521: LGTM! Token details propagation for streaming.Correctly propagates
CompletionTokensDetailsandPromptTokensDetails(includingreasoning_tokens) into the aggregated usage for text completion streams.core/providers/openai/chat.go (3)
14-14: LGTM! Uses centralized message conversion utility.The change to
ConvertOpenAIMessagesToBifrostMessagesproperly handles the new message structure includingChatAssistantMessagefields likeReasoningandReasoningDetails.
28-28: LGTM! Bidirectional conversion with new utility.Uses
ConvertBifrostMessagesToOpenAIMessagesfor the reverse transformation, maintaining consistency with the input conversion path.
62-65: LGTM! Nil-safe nested field access for reasoning effort.The triple nil-check (
Reasoning != nil && Reasoning.Effort != nil && *Effort == "minimal") correctly handles the transition from flatReasoningEffortto nestedReasoning.Effort. The "minimal" to "low" mapping maintains compatibility with non-OpenAI providers.core/providers/bedrock/utils.go (3)
9-11: LGTM! Import for reasoning validation constant.The import of
anthropic.MinimumReasoningMaxTokensenables consistent validation of reasoning token budgets across providers.
14-17: LGTM! Error return enables proper error propagation.The function signature change to return
errorallows callers to handle conversion failures gracefully. The nil check for params is appropriate.
133-133: LGTM! Explicit success return.The
return nilon successful completion completes the error-returning function contract.framework/streaming/chat.go (2)
38-43: LGTM! Consistent pointer initialization for Refusal.Using
bifrost.Ptr(*chunk.Delta.Refusal)aligns with the pattern used elsewhere in the codebase for initializing optional string pointers.
44-54: LGTM! Reasoning content accumulation.The reasoning accumulation logic correctly mirrors the existing Refusal handling pattern, properly initializing
ChatAssistantMessageif needed and concatenating reasoning content across chunks.core/providers/bedrock/responses.go (1)
656-660: New Bedrock→Responses output mapping is straightforward and consistent.The added conversion of
response.Output.MessageviaconvertBedrockMessageToResponsesMessagescleanly populatesbifrostResp.Outputand aligns with the existing Bedrock↔Responses content‑block helpers. No obvious edge‑case or correctness issues here.core/providers/anthropic/responses.go (1)
190-201: Thinking→Reasoning mapping on inbound Anthropic requests looks sound.
AnthropicMessageRequest.Thinkingis correctly surfaced asResponsesParametersReasoning:"enabled"becomesEffort: "auto"withMaxTokensfromBudgetTokens, and any other type producesEffort: "none". This gives Bifrost a normalized view without over‑encoding Anthropic‑specific details.core/providers/bedrock/chat.go (1)
37-40: Good: convertChatParameters errors are now surfaced to callers.Wrapping
convertChatParameterswith error propagation prevents silently constructing invalid Bedrock requests (e.g., bad reasoning.max_tokens) and aligns this path with the new validation logic in the utilities.core/providers/azure/azure.go (1)
361-373: Azure–Anthropic integration correctly handles new error‑returning converters.In all Anthropic branches (chat, chat stream, responses, responses stream):
- You now call
anthropic.ToAnthropicChatRequest/ToAnthropicResponsesRequest, check the returned error, and abort early if non‑nil instead of assuming success.- On success, you override
reqBody.Modelwith the Azure deployment and, for streaming paths, setreqBody.Stream = Ptr(true)before serialization.This keeps the Azure provider aligned with the new Anthropic converter signatures and ensures invalid reasoning configs or other request issues surface as proper errors instead of malformed upstream calls.
Also applies to: 461-471, 537-547, 636-646
core/providers/openai/types.go (2)
56-65: Embedded pointer structs design is sound, but the mutual exclusion is only documented.The comment on lines 61-62 correctly warns that only one embedded pointer can be non-nil at a time. However, this invariant is enforced only by convention. Consider adding a validation method or documenting this more prominently in the consuming code to prevent accidental misuse.
74-101: Custom MarshalJSON correctly shadows and transforms the reasoning field.The implementation properly:
- Uses the type alias pattern to avoid infinite recursion
- Shadows the
Reasoningfield to omit it from JSON output- Extracts
Reasoning.Effortintoreasoning_effortfor OpenAI compatibilitycore/providers/anthropic/chat.go (5)
421-435: Cached token mapping appears semantically inverted.The code maps
CacheReadInputTokenstoPromptTokensDetails.CachedTokens(line 422) andCacheCreationInputTokenstoCompletionTokensDetails.CachedTokens(lines 426, 434).Anthropic's
cache_creation_input_tokensrepresents tokens written to cache (a cost incurred during input processing), whilecache_read_input_tokensrepresents tokens read from cache. Both are input-side metrics. Mapping cache creation tokens toCompletionTokensDetailsseems semantically incorrect since completion tokens typically refer to output tokens.Please verify the intended semantics of these token mappings. If cache creation tokens should be tracked separately from completion details, consider adding a dedicated field or documenting this mapping decision.
647-654: Reasoning details in stream deltas include both summary and details.The streaming response correctly populates both
Reasoning(summary text) andReasoningDetails(structured details) for thinking content. This provides flexibility for consumers.
664-687: Signature streaming correctly creates separate reasoning detail entries.The signature delta handling properly creates a
ReasoningDetailsentry with just the signature, allowing consumers to associate signatures with their corresponding thinking blocks.
744-751: Reasoning-to-thinking conversion in stream response is correct.The conversion from Bifrost's
delta.Reasoningto Anthropic'sThinkingdelta type maintains consistency with the forward conversion path.
118-122: Unreachable code: condition on line 118 can never be true.The condition
bifrostReq.Params.Reasoning != nilon line 118 is inside anelseblock that only executes whenbifrostReq.Params == nil(from line 25). SinceParamsis nil, accessingParams.Reasoningwould cause a nil pointer dereference if the condition were ever evaluated.However, looking more closely, line 118 is at the same indentation level as line 122's closing brace, suggesting this is meant to be outside the
if bifrostReq.Params != nilblock but the logic is incorrect.- } else if bifrostReq.Params.Reasoning != nil && bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort == "none" { - anthropicReq.Thinking = &AnthropicThinking{ - Type: "disabled", - } }This dead code should be removed. If
bifrostReq.Paramsis nil, thenbifrostReq.Params.Reasoningcannot be accessed.Likely an incorrect or invalid review comment.
core/providers/cohere/chat.go (2)
431-446: Thinking content streaming correctly populates both Reasoning and ReasoningDetails.The streaming handler for thinking content mirrors the Anthropic implementation, providing consistent behavior across providers.
556-672: ToBifrostChatMessage implementation is thorough and handles edge cases well.The method correctly:
- Handles string content, block content, and thinking blocks
- Safely handles nil Function in tool calls (lines 619-622)
- Collapses single text blocks to ContentStr (lines 604-607)
- Accumulates reasoning details with proper indexing
core/providers/bedrock/bedrock.go (1)
22-22: Consistent per-stream UUID looks goodGenerating a single UUID per Bedrock chat stream and reusing it across all chunks and the final summary response is a clean way to provide a stable ID when the provider doesn’t. No functional issues spotted.
Also applies to: 783-785, 871-872, 894-895
core/providers/vertex/vertex.go (1)
291-345: Error propagation from Anthropic converters is correct and saferUsing
ToAnthropicChatRequest/ToAnthropicResponsesRequestwith expliciterrchecks inside theCheckContextAndGetRequestBodyclosures is the right pattern here and avoids streaming or sending requests when conversion fails. The updated blocks inChatCompletion,ChatCompletionStream,Responses, andResponsesStreamall look consistent.Also applies to: 532-539, 696-707, 860-868
Pratham-Mishra04
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
3540e84 to
e8afa58
Compare
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
b5f6f04 to
facc630
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (5)
core/providers/bedrock/utils.go (1)
26-47: Reasoning config may still be overwritten by ExtraParams handling.The reasoning config is correctly added to
AdditionalModelRequestFieldsat lines 29-47, but lines 72-73 can overwrite the entire map whenadditionalModelRequestFieldPathsexists in ExtraParams:bedrockReq.AdditionalModelRequestFields = orderedFields // Overwrites reasoning_config!This would silently discard the
reasoning_configset earlier.Consider merging instead of replacing:
if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = orderedFields + } else { + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } + } } }Also applies to: 70-75
core/schemas/mux.go (1)
772-787: PreserveReasoning.MaxTokensin Chat ↔ Responses conversionsThe mux conversions still only copy
Reasoning.Effortand gate on it, soReasoning.MaxTokensis silently dropped and MaxTokens‑only configs are ignored:
ToResponsesRequestonly setsbrr.Params.Reasoningwhenbcr.Params.Reasoning.Effort != nil, and only copies Effort.ToChatRequestonly copies Effort back, never MaxTokens.Given
MaxTokensis what actually drives the Anthropic/Bedrock reasoning budget, this is configuration loss across Chat↔Responses flows.Consider something along these lines:
- // Handle Reasoning from reasoning_effort - if bcr.Params.Reasoning != nil && bcr.Params.Reasoning.Effort != nil { - brr.Params.Reasoning = &ResponsesParametersReasoning{ - Effort: bcr.Params.Reasoning.Effort, - } - } + // Handle structured Reasoning (Effort and MaxTokens) + if bcr.Params.Reasoning != nil && + (bcr.Params.Reasoning.Effort != nil || bcr.Params.Reasoning.MaxTokens != nil) { + brr.Params.Reasoning = &ResponsesParametersReasoning{ + Effort: bcr.Params.Reasoning.Effort, + MaxTokens: bcr.Params.Reasoning.MaxTokens, + } + } - // Handle Reasoning from Reasoning - if brr.Params.Reasoning != nil { - bcr.Params.Reasoning = &ChatReasoning{ - Effort: brr.Params.Reasoning.Effort, - } - } + // Handle structured Reasoning from Responses -> Chat + if brr.Params.Reasoning != nil { + bcr.Params.Reasoning = &ChatReasoning{ + Effort: brr.Params.Reasoning.Effort, + MaxTokens: brr.Params.Reasoning.MaxTokens, + } + }This keeps reasoning budgets intact in both directions, even when Effort is omitted but MaxTokens is set.
Also applies to: 809-856
core/providers/anthropic/errors.go (1)
9-31: Guard against nilbifrostErr.ErrorinToAnthropicChatCompletionError
ToAnthropicChatCompletionErrorstill assumesbifrostErr.Erroris non‑nil and dereferencesbifrostErr.Error.Message, which can panic if upstream code ever omits the nested error struct.You can keep this helper robust with a small nil‑check:
func ToAnthropicChatCompletionError(bifrostErr *schemas.BifrostError) *AnthropicMessageError { if bifrostErr == nil { return nil } // Provide blank strings for nil pointer fields errorType := "" if bifrostErr.Type != nil { errorType = *bifrostErr.Type } - - // Handle nested error fields with nil checks - errorStruct := AnthropicMessageErrorStruct{ - Type: errorType, - Message: bifrostErr.Error.Message, - } + // Safely extract nested error message + message := "" + if bifrostErr.Error != nil { + message = bifrostErr.Error.Message + } + + errorStruct := AnthropicMessageErrorStruct{ + Type: errorType, + Message: message, + }This avoids panics regardless of which code path constructs the
BifrostError.core/providers/bedrock/responses.go (1)
347-367: Unifyreasoning_configmap type and avoid overwriting it with extra field pathsTwo issues remain around
reasoning_configthat can lead to lost reasoning settings:
Type mismatch for disabled reasoning
InToBedrockResponsesRequestyou write:bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ "type": "disabled", }but
ToBifrostResponsesRequestlater does:reasoningConfig, ok := schemas.SafeExtractFromMap(request.AdditionalModelRequestFields, "reasoning_config") if reasoningConfigMap, ok := reasoningConfig.(map[string]interface{}); ok { // ... }If a
BedrockConverseRequestever carries the map produced by your own writer, this type assertion will fail andParams.Reasoningwill not be populated. Use a uniformmap[string]anyfor both branches:
- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{
"type": "disabled",- }
- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{
"type": "disabled",- }
2. **Overwriting `AdditionalModelRequestFields` with `additionalModelRequestFieldPaths`** After setting `reasoning_config` from `Params.Reasoning`, this block: ```go if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { bedrockReq.AdditionalModelRequestFields = orderedFields } }replaces the entire
AdditionalModelRequestFieldsmap, discarding the earlierreasoning_config. If both are meant to coexist, mergeorderedFieldsinto the existing map instead of overwriting it.For example:
if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { if bedrockReq.AdditionalModelRequestFields == nil { bedrockReq.AdditionalModelRequestFields = orderedFields } else { for k, v := range orderedFields { bedrockReq.AdditionalModelRequestFields[k] = v } } }This keeps reasoning budgets configured via
Params.Reasoningintact even when callers also useadditionalModelRequestFieldPaths.Also applies to: 452-482
core/providers/cohere/chat.go (1)
126-137: ExtraParams["thinking"] can silently override Params.Reasoning.This issue was flagged in a previous review. When both
Params.ReasoningandExtraParams["thinking"]are provided, ExtraParams wins silently, which may not be the intended behavior.
🧹 Nitpick comments (7)
transports/bifrost-http/handlers/inference.go (1)
64-89: CustomChatRequestunmarshalling fixes embedded params, but double‑checkExtraParamsbehaviorThe custom
ChatRequest.UnmarshalJSONplus initializingChatParametersinchatCompletionis a good way to keepBifrostParams(model,fallbacks,stream, …) separate fromChatParametersdespite its custom unmarshaller, and adding"reasoning"tochatParamsKnownFieldskeeps it out ofExtraParams.One thing to verify: since you now:
- let
schemas.ChatParameters.UnmarshalJSONsee the full payload, and- later always compute
extraParamsviaextractExtraParamsand assignreq.ChatParameters.ExtraParams = extraParams,make sure
ChatParameters.UnmarshalJSONitself is not also trying to buildExtraParams, or its work will be silently overwritten here. If it is, consider consolidatingExtraParamspopulation in one place (either the type’s unmarshaller or this handler) for predictable behavior.Also applies to: 169-199, 476-513
core/schemas/mux.go (1)
1359-1369: Reasoning streaming delta mapping is aligned with newdelta.ReasoningfieldThe new block that emits
ResponsesStreamResponseTypeReasoningSummaryTextDeltafromdelta.Reasoning(withOutputIndex: Ptr(0)andDelta: delta.Reasoning) is consistent with the newChatStreamResponseChoiceDelta.Reasoningfield and the Responses streaming schema. For single‑output streams this looks fine; if you later support multi‑output streaming, you’ll likely want to plumb the actual choice index through instead of hard‑coding 0.core/providers/bedrock/chat.go (1)
38-40: Bedrock reasoning mapping (non‑stream & stream) is coherent with the new schema
ToBedrockChatCompletionRequestnow surfaces errors fromconvertChatParameters, which is important given the new reasoning validation there.ToBifrostChatResponsecorrectly liftsReasoningContentintoChatReasoningDetailsand a consolidatedassistantMessage.Reasoningstring.- Streaming
ReasoningContentdeltas are converted intoChatStreamResponseChoiceDelta.ReasoningplusReasoningDetails, which matches howmux.ToBifrostResponsesStreamResponsenow consumes reasoning deltas.Only very minor nits you might consider later:
reasoningTextis built with a trailing newline; trim if that ever shows up awkwardly in clients.- If a Bedrock reasoning delta ever contains both text and a signature in the same payload, you currently only surface one or the other; merging both into a single
ChatReasoningDetailsentry would preserve more information.Overall, the reasoning plumbing here looks solid.
Also applies to: 58-60, 103-112, 117-120, 135-141, 293-341
core/providers/anthropic/anthropic.go (1)
551-560: Aggregate cached‑token usage with max semantics instead of overwritingIn both streaming handlers you now populate cached‑token details:
- Chat streaming:
usage.PromptTokensDetails/usage.CompletionTokensDetailsfromCacheReadInputTokensandCacheCreationInputTokens.- Responses streaming:
usage.InputTokensDetails/usage.OutputTokensDetailsfrom the same fields.Right now this is “last writer wins”:
if usageToProcess.CacheReadInputTokens > 0 { usage.PromptTokensDetails = &schemas.ChatPromptTokensDetails{ CachedTokens: usageToProcess.CacheReadInputTokens } }If Anthropic ever emits multiple events with incremental or non‑monotonic cache counts, you may want to mirror your token counters and Bedrock’s behavior by taking the max instead of overwriting:
if usageToProcess.CacheReadInputTokens > 0 { if usage.PromptTokensDetails == nil || usageToProcess.CacheReadInputTokens > usage.PromptTokensDetails.CachedTokens { usage.PromptTokensDetails = &schemas.ChatPromptTokensDetails{ CachedTokens: usageToProcess.CacheReadInputTokens, } } }(and similarly for the other cached‑token fields). This keeps the final usage snapshot consistent regardless of emission order.
Also applies to: 893-902
core/providers/anthropic/responses.go (1)
325-344: Simplify nested else clause for cleaner code flow.The
elseat line 330 is redundant since the precedingifreturns. This can be flattened.if bifrostReq.Params.Reasoning != nil { if bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort != "none" { if bifrostReq.Params.Reasoning.MaxTokens != nil { if *bifrostReq.Params.Reasoning.MaxTokens < MinimumReasoningMaxTokens { return nil, fmt.Errorf("reasoning.max_tokens must be greater than or equal to %d", MinimumReasoningMaxTokens) - } else { - anthropicReq.Thinking = &AnthropicThinking{ - Type: "enabled", - BudgetTokens: bifrostReq.Params.Reasoning.MaxTokens, - } } + anthropicReq.Thinking = &AnthropicThinking{ + Type: "enabled", + BudgetTokens: bifrostReq.Params.Reasoning.MaxTokens, + } } else { return nil, fmt.Errorf("reasoning.max_tokens is required for reasoning") }core/providers/cohere/chat.go (2)
556-672: Well-implemented ToBifrostChatMessage with comprehensive content handling.The method properly handles:
- String vs block content
- Image content blocks
- Thinking/reasoning blocks with details accumulation
- Tool calls with nil-safety
- Assistant message assembly with reasoning
Minor note: Line 598 appends a newline after each thinking block text, which may leave a trailing newline in
reasoningText.} else if block.Type == CohereContentBlockTypeThinking && block.Thinking != nil { reasoningDetails = append(reasoningDetails, schemas.ChatReasoningDetails{ Index: len(reasoningDetails), Type: schemas.BifrostReasoningDetailsTypeText, Text: block.Thinking, }) - reasoningText += *block.Thinking + "\n" + if reasoningText != "" { + reasoningText += "\n" + } + reasoningText += *block.Thinking }
104-121: Move reasoning minimum token validation constant to shared location.The constant
MinimumReasoningMaxTokens = 1024is correctly validated here and matches Cohere's official minimum (≥1000 tokens for reasoning), but it's currently defined in the anthropic package. This creates an unnecessary cross-provider dependency; both Cohere and Bedrock import from the anthropic package for this validation. Consider defining this constant incore/schemaswhere other shared reasoning types (ChatReasoning, ResponsesParametersReasoning) are already centralized, making the constraint universally accessible without provider-specific imports.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (29)
core/providers/anthropic/anthropic.go(6 hunks)core/providers/anthropic/chat.go(10 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(4 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(3 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/semanticcache/utils.go(1 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (11)
- core/providers/vertex/vertex.go
- framework/streaming/chat.go
- plugins/semanticcache/utils.go
- ui/lib/types/logs.ts
- core/providers/openai/openai.go
- core/providers/cohere/types.go
- core/providers/openai/utils.go
- core/providers/cohere/cohere.go
- core/providers/openai/types.go
- core/providers/perplexity/chat.go
- core/schemas/responses.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
core/providers/anthropic/types.gotransports/bifrost-http/handlers/inference.goplugins/jsonparser/utils.gocore/providers/azure/azure.gocore/providers/bedrock/utils.gocore/providers/bedrock/responses.gocore/providers/openai/chat.gocore/schemas/mux.goui/app/workspace/logs/views/logChatMessageView.tsxcore/providers/anthropic/errors.gocore/providers/bedrock/types.gocore/providers/anthropic/responses.gocore/providers/cohere/chat.gocore/providers/bedrock/bedrock.gocore/providers/anthropic/anthropic.gocore/providers/bedrock/chat.gocore/providers/anthropic/chat.gocore/schemas/chatcompletions.go
🧬 Code graph analysis (12)
core/providers/azure/azure.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-282)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/bedrock/utils.go (3)
core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)OrderedMap(268-268)core/providers/bedrock/types.go (1)
BedrockConverseRequest(44-57)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/openai/chat.go (3)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/utils.go (1)
Ptr(51-53)
core/schemas/mux.go (3)
core/schemas/responses.go (3)
ResponsesParametersReasoning(232-237)BifrostResponsesStreamResponse(1424-1462)ResponsesStreamResponseTypeReasoningSummaryTextDelta(1390-1390)core/schemas/chatcompletions.go (1)
ChatReasoning(223-226)core/schemas/utils.go (1)
Ptr(16-18)
ui/app/workspace/logs/views/logChatMessageView.tsx (2)
ui/lib/utils/validation.ts (2)
isJson(539-546)cleanJson(548-558)ui/app/workspace/logs/views/codeEditor.tsx (1)
CodeEditor(82-245)
core/providers/anthropic/errors.go (1)
core/providers/anthropic/types.go (2)
AnthropicMessageError(376-379)AnthropicMessageErrorStruct(382-385)
core/providers/anthropic/responses.go (2)
core/schemas/responses.go (1)
ResponsesParametersReasoning(232-237)core/providers/anthropic/types.go (2)
MinimumReasoningMaxTokens(14-14)AnthropicThinking(65-68)
core/providers/cohere/chat.go (2)
core/schemas/chatcompletions.go (10)
BifrostChatRequest(12-19)ChatMessage(469-478)ChatReasoning(223-226)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)ChatReasoningDetails(723-730)ChatAssistantMessageToolCall(686-692)ChatMessageContent(524-527)ChatAssistantMessage(626-632)ChatMessageRole(457-457)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/bedrock/bedrock.go (2)
core/providers/utils/utils.go (1)
CreateBifrostChatCompletionChunkResponse(687-716)core/schemas/bifrost.go (1)
ChatCompletionStreamRequest(90-90)
core/providers/anthropic/anthropic.go (7)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-282)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)core/schemas/chatcompletions.go (1)
ChatPromptTokensDetails(821-827)ui/lib/types/logs.ts (3)
CompletionTokensDetails(197-202)ResponsesResponseInputTokens(542-544)ResponsesResponseOutputTokens(546-548)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)core/schemas/responses.go (2)
ResponsesResponseInputTokens(261-267)ResponsesResponseOutputTokens(269-279)
core/providers/bedrock/chat.go (1)
core/schemas/chatcompletions.go (6)
ChatReasoningDetails(723-730)BifrostReasoningDetailsTypeText(719-719)ChatContentBlockTypeText(582-582)ChatAssistantMessage(626-632)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)
core/schemas/chatcompletions.go (1)
ui/lib/types/logs.ts (3)
ChatMessage(115-124)ChatMessageContent(113-113)ReasoningDetails(126-133)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (27)
core/providers/anthropic/types.go (1)
14-14: LGTM! Well-placed constant for reasoning token validation.The
MinimumReasoningMaxTokensconstant is appropriately defined here and exported for use by other providers (e.g., Bedrock) that need to validate reasoning configurations against Anthropic's requirements.core/providers/anthropic/chat.go (2)
647-687: LGTM! Proper handling of reasoning and signature streaming deltas.The streaming conversion correctly:
- Emits
Reasoningcontent withReasoningDetailsfor thinking deltas- Emits signature-only
ReasoningDetailsfor signature deltasThis aligns with Anthropic's streaming format where thinking content and signatures arrive in separate delta events.
744-751: LGTM! Consistent use of Reasoning field in streaming.The streaming response conversion correctly uses the new
Reasoningfield (renamed fromThought) for thinking content deltas.ui/app/workspace/logs/views/logChatMessageView.tsx (1)
73-94: LGTM! UI correctly updated to display reasoning content.The UI changes properly:
- Rename the field from
thoughttoreasoningto match the backend schema changes- Update the display label to "Reasoning"
- Preserve the JSON detection and formatting behavior
plugins/jsonparser/utils.go (1)
309-314: LGTM! Deep copy correctly updated for Reasoning field rename.The field rename from
ThoughttoReasoningis correctly applied in the deep copy function, maintaining the shallow copy behavior appropriate for string pointers.core/providers/bedrock/bedrock.go (1)
21-22: Per‑stream UUID for Bedrock chat streaming looks correctUsing a goroutine‑local
uuid.New().String()as the stream ID and wiring it through each chunk and the finalCreateBifrostChatCompletionChunkResponsekeeps Bedrock in line with other providers that emit stableresponse.IDs during streaming. No issues spotted.Also applies to: 771-783, 859-885
core/providers/openai/chat.go (1)
11-18: Message and reasoning conversions for OpenAI look consistentUsing
ConvertOpenAIMessagesToBifrostMessages/ConvertBifrostMessagesToOpenAIMessagesensures assistant‑side fields (reasoning, annotations, tool calls) survive round‑trips, and updatingfilterOpenAISpecificParametersto operate onChatParameters.Reasoning.Effortwhile normalizing"minimal"→"low"keeps the previous behavior with the new structured reasoning object. No issues spotted.Also applies to: 21-33, 61-75
core/providers/anthropic/anthropic.go (1)
303-311: Error‑aware Anthropic request construction is wired correctlyUsing
ToAnthropicChatRequest/ToAnthropicResponsesRequestdirectly inside theCheckContextAndGetRequestBodyclosures (and settingStream = Ptr(true)only after a successful conversion in the streaming cases) ensures model/parameter validation errors surface asBifrostErrors instead of slipping through. This applies consistently to both non‑streaming and streaming Chat/Responses paths and matches the broader error‑handling refactor in this stack.Also applies to: 353-365, 647-655, 695-707
core/providers/bedrock/responses.go (1)
656-660: Single conversion of Bedrock output message to Responses output is fineThe new early conversion of
response.Output.MessageviaconvertBedrockMessageToResponsesMessagesintobifrostResp.Outputis straightforward and avoids duplicated work later in the function. No issues here.core/providers/anthropic/responses.go (2)
190-201: LGTM! Clean conversion from Anthropic Thinking to Bifrost Reasoning.The mapping correctly handles both enabled and disabled thinking states, preserving the budget tokens when present.
296-299: Good defensive nil check added.The early return with a descriptive error message prevents nil pointer dereferences downstream.
core/providers/azure/azure.go (2)
353-360: Proper error propagation from converter function.The updated code correctly handles the error return from
ToAnthropicChatRequestand sets the deployment model after validation.
529-533: LGTM! Consistent error handling pattern.Error propagation for
ToAnthropicResponsesRequestfollows the same pattern as the chat completion methods.core/providers/bedrock/types.go (3)
103-104: LGTM! ReasoningContent field properly added.The optional field follows the existing pattern for content blocks and enables reasoning support in Bedrock requests/responses.
152-159: Well-structured reasoning types.The nested structure with
ReasoningTextcontainingTextand optionalSignaturealigns with the Anthropic thinking feature and supports verification via signatures.
438-440: Clean delta structure for streaming reasoning content.The updated
BedrockContentBlockDeltaproperly separates text, reasoning, and tool use deltas, enabling proper streaming of thinking content.core/providers/cohere/chat.go (3)
214-221: LGTM! Clean message conversion using new helper.The refactored code using
ToBifrostChatMessage()is cleaner and more maintainable.
242-256: LGTM! Bidirectional reasoning conversion is consistent.The conversion from Cohere Thinking to Bifrost Reasoning properly mirrors the forward conversion logic, handling both enabled and disabled states.
405-446: LGTM! Stream delta handling properly distinguishes text and thinking content.The nested conditionals correctly route text deltas and thinking deltas to their respective response fields, including populating
ReasoningDetails.core/schemas/chatcompletions.go (8)
165-165: Clean replacement of deprecated field with structured type.Replacing
ReasoningEffort *stringwithReasoning *ChatReasoningprovides a cleaner API while maintaining backward compatibility through the custom unmarshaller.
186-220: Robust backward-compatible unmarshalling with clear error handling.The custom
UnmarshalJSON:
- Correctly prevents both
reasoning_effortandreasoningfrom being set simultaneously- Converts legacy
reasoning_effortto the newReasoningstruct transparentlyThis ensures smooth migration for API consumers.
222-226: LGTM! ChatReasoning struct is well-designed.The struct with
EffortandMaxTokensfields provides the necessary flexibility for reasoning configuration across providers.
480-521: Necessary custom unmarshaller for ChatMessage with embedded types.The explicit handling of embedded
ChatToolMessageandChatAssistantMessageis required because embedded structs with their ownUnmarshalJSONmethods don't compose automatically. The check at lines 514-517 properly gates assistant message assignment.
634-667: Good UX: Synthesizing ReasoningDetails from Reasoning string.When a simple
reasoningstring is provided withoutreasoning_details, the unmarshaller automatically creates a structuredChatReasoningDetailsentry. This improves API ergonomics while maintaining full detail support.
714-730: Well-defined reasoning details types with comprehensive fields.The
ChatReasoningDetailsstruct withIndex,Type,Summary,Text,Signature, andDatafields covers all reasoning output variations across providers (summary, encrypted, text). The type constants align with the UI types inui/lib/types/logs.ts.
768-794: Consistent synthesizing behavior for streaming deltas.The
UnmarshalJSONforChatStreamResponseChoiceDeltafollows the same pattern asChatAssistantMessage, ensuring consistent behavior between streaming and non-streaming responses.
835-835: LGTM! ImageTokens field addition.The optional
ImageTokensfield inChatCompletionTokensDetailsenables tracking image-related token usage for multimodal models.
Pratham-Mishra04
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
e8afa58 to
455b127
Compare
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
facc630 to
0822b25
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/providers/bedrock/responses.go (1)
478-482: Potential overwrite ofreasoning_configbyadditionalModelRequestFieldPaths.At line 480, if
additionalModelRequestFieldPathsexists inExtraParams, it completely replacesAdditionalModelRequestFieldswithorderedFields. This could discard thereasoning_configset at lines 466-469.Consider merging instead of replacing:
if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + // Merge orderedFields into existing AdditionalModelRequestFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = orderedFields + } else { + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } + } } }core/providers/anthropic/chat.go (1)
720-751: Chat → Anthropic streaming doesn’t yet propagate reasoning signatures
ToAnthropicChatStreamResponsemaps:
delta.Content→Textdeltas,delta.Reasoning→thinkingdeltas,- tool calls and finish reasons appropriately,
but ignores
delta.ReasoningDetails, which is where signature-only reasoning events will land on the Chat side.This means reasoning signatures carried in
ChatStreamResponseChoiceDelta.ReasoningDetails(with noReasoningtext) won’t be reflected back to Anthropic SSE.If you intend to support signature verification on the Chat streaming path (not just Responses), consider:
- Detecting
delta.ReasoningDetailswith a non-nilSignatureand emitting anAnthropicStreamDeltaof typesignaturewith that value.- Potentially also combining text+signature when both are present.
This would mirror the handling you already implemented in the Responses streaming bridge.
♻️ Duplicate comments (10)
core/providers/anthropic/errors.go (1)
21-25: Nil pointer dereference risk onbifrostErr.ErrorLine 24 accesses
bifrostErr.Error.Messagewithout checking ifbifrostErr.Erroris nil. This will panic if aBifrostErroris passed with a nilErrorfield.Apply this diff:
// Handle nested error fields with nil checks + message := "" + if bifrostErr.Error != nil { + message = bifrostErr.Error.Message + } + errorStruct := AnthropicMessageErrorStruct{ Type: errorType, - Message: bifrostErr.Error.Message, + Message: message, }core/providers/azure/azure.go (2)
452-458: Potential nil pointer dereference after error check.After checking
err != nil, the code directly accessesreqBody.ModelandreqBody.Streamat lines 456-457 without verifyingreqBodyis non-nil. IfToAnthropicChatRequestreturns(nil, nil), this will panic.func() (any, error) { reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } reqBody.Model = deployment reqBody.Stream = schemas.Ptr(true) return reqBody, nil },
627-632: Same nil check concern as ChatCompletionStream.The code accesses
reqBody.ModelandreqBody.Streamwithout a nil check after error handling.func() (any, error) { reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } + if reqBody == nil { + return nil, fmt.Errorf("anthropic responses request conversion returned nil") + } reqBody.Model = deployment reqBody.Stream = schemas.Ptr(true) return reqBody, nil },core/providers/cohere/chat.go (1)
126-137: ExtraParams can still override Reasoning-based thinking configuration.Lines 126-137 can overwrite
cohereReq.Thinkingset fromReasoning(lines 104-121). The past review flagged this but was marked as addressed. If this is intentional (ExtraParams takes precedence), consider adding a comment to document this behavior.plugins/semanticcache/utils.go (1)
669-671: ExtractReasoning.MaxTokensin cache metadata for accurate cache key generation.The nil-safety check is correct. However,
MaxTokensshould also be extracted since requests with identicalEffortbut differentMaxTokenscould incorrectly hit the same cache entry.if params.Reasoning != nil && params.Reasoning.Effort != nil { metadata["reasoning_effort"] = *params.Reasoning.Effort + if params.Reasoning.MaxTokens != nil { + metadata["reasoning_max_tokens"] = *params.Reasoning.MaxTokens + } }core/schemas/mux.go (1)
772-777: Reasoning.MaxTokens still not round-tripped in Chat ↔ Responses conversions
ChatReasoning.MaxTokensis still dropped when converting via mux:
ToResponsesRequestonly copiesEffortand is gated onReasoning.Effort != nil, so any configuredMaxTokensis ignored, and aReasoningobject with onlyMaxTokensis skipped entirely.ToChatRequestonly copiesEffortback fromResponsesParametersReasoning, soMaxTokenson the Responses side never makes it back intoChatReasoning.Since
MaxTokensis the field that ultimately drives Anthropic’s reasoning/thinking budget, this silently loses user config on any path that passes through mux.Recommend:
- Treat
MaxTokensas first-class in both directions, and- Gate on
(Effort != nil || MaxTokens != nil)so thatMaxTokensalone is enough to propagate reasoning.Example:
- // Handle Reasoning from reasoning_effort - if bcr.Params.Reasoning != nil && bcr.Params.Reasoning.Effort != nil { - brr.Params.Reasoning = &ResponsesParametersReasoning{ - Effort: bcr.Params.Reasoning.Effort, - } - } + // Handle structured Reasoning (Effort / MaxTokens) + if bcr.Params.Reasoning != nil && + (bcr.Params.Reasoning.Effort != nil || bcr.Params.Reasoning.MaxTokens != nil) { + brr.Params.Reasoning = &ResponsesParametersReasoning{ + Effort: bcr.Params.Reasoning.Effort, + MaxTokens: bcr.Params.Reasoning.MaxTokens, + } + }and:
- // Handle Reasoning from Reasoning - if brr.Params.Reasoning != nil { - bcr.Params.Reasoning = &ChatReasoning{ - Effort: brr.Params.Reasoning.Effort, - } - } + // Handle structured Reasoning from Responses -> Chat + if brr.Params.Reasoning != nil { + bcr.Params.Reasoning = &ChatReasoning{ + Effort: brr.Params.Reasoning.Effort, + MaxTokens: brr.Params.Reasoning.MaxTokens, + } + }Also applies to: 851-856
core/providers/bedrock/utils.go (2)
14-17: NilbifrostReq.Paramsshould be handled gracefully, not as an error
convertChatParametersnow returns an error whenbifrostReq.Params == nil. This is both:
- Inconsistent with other providers (which treat nil
Paramsas “no overrides”), and- A potential behavior break for callers that validly omit
paramsin chat requests.Given
convertInferenceConfig/convertToolConfigonly read fields off*ChatParameters, a more robust pattern is to synthesize a defaultChatParameterswhenParamsis nil instead of erroring, e.g.:func convertChatParameters(bifrostReq *schemas.BifrostChatRequest, bedrockReq *BedrockConverseRequest) error { - if bifrostReq.Params == nil { - return fmt.Errorf("bifrost request parameters are nil") - } + if bifrostReq.Params == nil { + bifrostReq.Params = &schemas.ChatParameters{ + ExtraParams: map[string]any{}, + } + }so downstream conversion can proceed with defaults and you only error on truly invalid nested fields (like missing
reasoning.max_tokenswhen reasoning is enabled).
27-47:reasoning_configcan still be overwritten byadditionalModelRequestFieldPathsThe new reasoning block correctly populates:
reasoning_config.type = "disabled"when Effort is"none", andreasoning_config.type = "enabled"withbudget_tokensvalidated againstanthropic.MinimumReasoningMaxTokens.However, later you do:
if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { bedrockReq.AdditionalModelRequestFields = orderedFields } }which overwrites
AdditionalModelRequestFieldsentirely, discarding the earlierreasoning_config.Instead, merge the extracted fields into the existing map:
- if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields - } + if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = orderedFields + } else { + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } + } + }so
reasoning_configand extra model fields can coexist.Also applies to: 70-75
core/providers/anthropic/chat.go (2)
99-117: Fix unsafe / redundantelse ifonbifrostReq.Params.ReasoningInside
ToAnthropicChatRequest:
- The main
if bifrostReq.Params != nil { ... }block already handles allReasoningcases, includingEffort == "none"and enabled reasoning with validatedMaxTokens.- The trailing
} else if bifrostReq.Params.Reasoning != nil && ...runs whenbifrostReq.Params == nil, but still dereferencesbifrostReq.Params.Reasoning, which will panic at runtime if ever evaluated. Logically,Reasoningcannot be non-nil whenParamsis nil anyway.This
else ifis both unnecessary and unsafe. You can simply remove it:- } else if bifrostReq.Params.Reasoning != nil && bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort == "none" { - anthropicReq.Thinking = &AnthropicThinking{ - Type: "disabled", - } - } + }The inside-Params branch already sets
Thinking.Type = "disabled"whenEffort == "none".Also applies to: 118-121
284-390: Anthropic → Bifrost chat response conversion is mostly solid; usage cache mapping still redundantThe new
(*AnthropicMessageResponse).ToBifrostChatResponse:
- Aggregates text into
ChatMessageContent(string when a single text block exists),- Converts tool_use blocks into
ChatAssistantMessage.ToolCalls,- Converts thinking blocks into
ChatAssistantMessage.ReasoningplusReasoningDetails(with signatures),- Maps finish reason via
ConvertAnthropicFinishReasonToBifrost,- Fills
BifrostLLMUsageincluding cached tokens.Two lingering issues in the usage block:
CacheCreationInputTokensis assigned twice toCompletionTokensDetails.CachedTokens(first in the struct literal, then again in theif CacheCreationInputTokens > 0block). The second block is redundant.- Semantically,
CacheCreationInputTokensreflects input-side cache creation; mapping it toCompletionTokensDetails.CachedTokens(completion-side) may be misleading. Treating it as prompt-side cached tokens (or documenting the choice) would better reflect its meaning.At minimum you can drop the second assignment:
- CompletionTokensDetails: &schemas.ChatCompletionTokensDetails{ - CachedTokens: response.Usage.CacheCreationInputTokens, - }, TotalTokens: response.Usage.InputTokens + response.Usage.OutputTokens, } - if response.Usage.CacheCreationInputTokens > 0 { - if bifrostResponse.Usage.CompletionTokensDetails == nil { - bifrostResponse.Usage.CompletionTokensDetails = &schemas.ChatCompletionTokensDetails{} - } - bifrostResponse.Usage.CompletionTokensDetails.CachedTokens = response.Usage.CacheCreationInputTokens - }and optionally revisit whether this field should instead live under
PromptTokensDetails.Also applies to: 417-436
🧹 Nitpick comments (6)
core/providers/perplexity/chat.go (1)
8-172: Consider aligning error handling with other providers in the stack.According to the broader PR context, other provider converters (e.g.,
ToAnthropicChatRequest,ToAnthropicResponsesRequest) now return(…, error)to propagate validation failures. This function still returns only*PerplexityChatRequestwithout an error.While the current changes don't introduce validation logic requiring errors, consider whether this function should adopt the same signature pattern for consistency and future extensibility—especially if reasoning validation (e.g., minimum token checks) might be added later.
Based on coding guidelines, reviewing in context of the whole stack.
core/providers/openai/types.go (1)
56-65: Clarify mutual exclusivity enforcement for embedded pointers.The comment warns that only one of
*ChatToolMessageor*OpenAIChatAssistantMessageshould be non-nil, but there's no runtime validation. Consider adding validation in the conversion functions to prevent accidental misuse.core/providers/cohere/chat.go (1)
592-599: Minor: Trailing newline in accumulated reasoning text.Line 598 appends
"\n"after each thinking block's text. This leaves a trailing newline inreasoningText. Consider usingstrings.TrimSuffixbefore assigning toassistantMessage.Reasoningat line 663.+import "strings" + if len(reasoningDetails) > 0 { if assistantMessage == nil { assistantMessage = &schemas.ChatAssistantMessage{} } assistantMessage.ReasoningDetails = reasoningDetails - assistantMessage.Reasoning = schemas.Ptr(reasoningText) + assistantMessage.Reasoning = schemas.Ptr(strings.TrimSuffix(reasoningText, "\n")) }core/schemas/chatcompletions.go (1)
634-667: Consider extracting common reasoning synthesis logic.The
UnmarshalJSONimplementations forChatAssistantMessage(lines 655-664) andChatStreamResponseChoiceDelta(lines 782-791) contain nearly identical logic for synthesizingReasoningDetailsfromReasoning. Consider extracting this into a helper function to reduce duplication.// Helper function to synthesize reasoning details from reasoning text func synthesizeReasoningDetails(reasoning *string, existingDetails []ChatReasoningDetails) []ChatReasoningDetails { if reasoning != nil && len(existingDetails) == 0 { text := *reasoning return []ChatReasoningDetails{{ Index: 0, Type: BifrostReasoningDetailsTypeText, Text: &text, }} } return existingDetails }Also applies to: 768-794
transports/bifrost-http/handlers/inference.go (1)
504-506: Redundant nil check.Since
ChatParametersis now initialized at line 477 before unmarshalling, this nil check is always false.- if req.ChatParameters == nil { - req.ChatParameters = &schemas.ChatParameters{} - } - extraParams, err := extractExtraParams(ctx.PostBody(), chatParamsKnownFields)core/providers/bedrock/chat.go (1)
58-60: Bedrock → Bifrost reasoning aggregation looks correctThe non-streaming path:
- Collects each
ReasoningContentblock intoChatReasoningDetails(index, typereasoning.text, text, signature),- Concatenates reasoning text into
reasoningTextand assigns it toChatAssistantMessage.Reasoning,- Falls back to
contentStrwhen there’s exactly one text block, preserving prior UX.This cleanly separates visible assistant output (content) from reasoning and keeps signatures attached to the corresponding reasoning entries.
Only very minor nit is the trailing newline in
reasoningText(due to+ "\n"), but that’s unlikely to matter; trimming can be deferred if it ever becomes an issue.Also applies to: 103-112, 117-120, 135-141
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (29)
core/providers/anthropic/anthropic.go(6 hunks)core/providers/anthropic/chat.go(10 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(4 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(3 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/semanticcache/utils.go(1 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (8)
- core/providers/openai/openai.go
- core/providers/cohere/types.go
- core/providers/anthropic/types.go
- plugins/jsonparser/utils.go
- core/providers/bedrock/bedrock.go
- ui/app/workspace/logs/views/logChatMessageView.tsx
- core/providers/vertex/vertex.go
- ui/lib/types/logs.ts
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
framework/streaming/chat.gocore/schemas/mux.gocore/providers/openai/utils.gocore/providers/perplexity/chat.goplugins/semanticcache/utils.gocore/schemas/responses.gocore/providers/bedrock/responses.gocore/providers/bedrock/utils.gocore/providers/openai/chat.gocore/providers/openai/types.gocore/providers/anthropic/responses.gotransports/bifrost-http/handlers/inference.gocore/schemas/chatcompletions.gocore/providers/anthropic/anthropic.gocore/providers/azure/azure.gocore/providers/bedrock/types.gocore/providers/bedrock/chat.gocore/providers/anthropic/errors.gocore/providers/cohere/chat.gocore/providers/cohere/cohere.gocore/providers/anthropic/chat.go
🧬 Code graph analysis (12)
framework/streaming/chat.go (3)
core/schemas/chatcompletions.go (1)
ChatAssistantMessage(626-632)core/utils.go (1)
Ptr(51-53)core/providers/gemini/types.go (1)
Type(778-778)
core/schemas/mux.go (3)
core/schemas/responses.go (3)
ResponsesParametersReasoning(232-237)BifrostResponsesStreamResponse(1424-1462)ResponsesStreamResponseTypeReasoningSummaryTextDelta(1390-1390)core/schemas/chatcompletions.go (1)
ChatReasoning(223-226)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/openai/utils.go (3)
core/providers/openai/types.go (2)
OpenAIMessage(56-65)OpenAIChatAssistantMessage(67-72)core/schemas/chatcompletions.go (3)
ChatMessage(469-478)ChatToolMessage(621-623)ChatAssistantMessage(626-632)ui/lib/types/logs.ts (1)
ChatMessage(115-124)
core/providers/bedrock/responses.go (4)
core/schemas/utils.go (4)
SafeExtractFromMap(519-525)SafeExtractString(280-297)SafeExtractInt(300-342)Ptr(16-18)core/schemas/responses.go (1)
ResponsesParametersReasoning(232-237)core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/openai/chat.go (4)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)
core/providers/openai/types.go (1)
core/schemas/chatcompletions.go (6)
ChatMessageRole(457-457)ChatMessageContent(524-527)ChatToolMessage(621-623)ChatAssistantMessageAnnotation(670-673)ChatAssistantMessageToolCall(686-692)ChatReasoning(223-226)
core/providers/anthropic/anthropic.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-282)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/bedrock/chat.go (1)
core/schemas/chatcompletions.go (8)
ChatReasoningDetails(723-730)BifrostReasoningDetailsTypeText(719-719)ChatContentBlockTypeText(582-582)ChatAssistantMessage(626-632)BifrostChatResponse(26-41)BifrostResponseChoice(704-712)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)
core/providers/anthropic/errors.go (2)
ui/lib/types/logs.ts (1)
BifrostError(233-239)core/providers/anthropic/types.go (2)
AnthropicMessageError(376-379)AnthropicMessageErrorStruct(382-385)
core/providers/cohere/chat.go (3)
core/schemas/chatcompletions.go (13)
BifrostChatRequest(12-19)ChatMessage(469-478)ChatReasoning(223-226)ChatNonStreamResponseChoice(745-748)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)ChatReasoningDetails(723-730)BifrostReasoningDetailsTypeText(719-719)ChatContentBlock(590-597)ChatAssistantMessageToolCall(686-692)ChatMessageContent(524-527)ChatAssistantMessageToolCallFunction(695-698)ChatAssistantMessage(626-632)core/providers/cohere/types.go (4)
CohereChatRequest(14-31)CohereThinking(170-173)ThinkingTypeDisabled(180-180)ThinkingTypeEnabled(179-179)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/cohere/cohere.go (2)
core/providers/cohere/chat.go (1)
ToCohereChatCompletionRequest(12-199)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/anthropic/chat.go (2)
core/schemas/chatcompletions.go (7)
BifrostChatRequest(12-19)ChatAssistantMessage(626-632)ChatAssistantMessageToolCall(686-692)ChatContentBlock(590-597)ChatReasoningDetails(723-730)ChatMessageContent(524-527)ChatMessage(469-478)core/providers/anthropic/types.go (5)
AnthropicMessageRequest(40-59)AnthropicThinking(65-68)MinimumReasoningMaxTokens(14-14)AnthropicContentBlock(147-159)AnthropicContentBlockTypeThinking(143-143)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (49)
core/providers/perplexity/chat.go (1)
30-36: No action needed—backwards compatibility is already handled.The deprecated
reasoning_effortfield is converted to the newReasoning.Effortstructure at the schema level duringUnmarshalJSONincore/schemas/chatcompletions.go(lines 212-216). When a request contains the legacy field, it is automatically mapped toReasoning.Effortbefore reaching this function, so the code correctly receives the converted nested structure.core/schemas/responses.go (2)
232-237: LGTM!The
ResponsesParametersReasoningstruct changes are well-structured. The newMaxTokensfield is correctly optional withomitempty, and the updatedEffortcomment clarifies the behavior that any value other than "none" enables reasoning.
322-325: Comment update is informative.The inline comment clarifying model-specific reasoning content behavior is helpful for maintainers.
core/providers/azure/azure.go (2)
353-360: Error propagation is correctly implemented.The error handling for
ToAnthropicChatRequestin the non-streaming path properly checks and propagates errors before using the request body.
529-534: Error propagation is correctly implemented.The error handling for
ToAnthropicResponsesRequestin the non-streaming Responses path properly propagates errors.core/providers/bedrock/responses.go (3)
349-367: Reasoning config extraction is well-implemented.The code correctly extracts
reasoning_configfromAdditionalModelRequestFieldsand maps it to the Bifrost schema with appropriate type checking usingSafeExtractFromMap,SafeExtractString, andSafeExtractInt.
452-472: Reasoning validation and conversion looks correct.The validation enforces that
max_tokensis required and meets the minimum threshold (MinimumReasoningMaxTokens = 1024). The disabled/enabled reasoning config mapping is properly structured.
656-660: Output conversion refactor is clean.Moving the output conversion to the top of
ToBifrostResponsesResponseimproves code organization.core/providers/anthropic/anthropic.go (6)
307-307: Error propagation correctly simplified.Removing the wrapping
nilerror return and directly returning fromToAnthropicChatRequestis cleaner and ensures errors propagate properly.
358-363: Streaming error propagation is correctly implemented.The error from
ToAnthropicChatRequestis now properly checked before settingStream = true, preventing use of an invalid request body.
551-560: Cached token tracking improves observability.Mapping
CacheReadInputTokenstoPromptTokensDetails.CachedTokensandCacheCreationInputTokenstoCompletionTokensDetails.CachedTokenssurfaces useful cache usage information.
651-651: Error propagation correctly simplified for Responses.Consistent with the ChatCompletion change.
700-704: Streaming error propagation is correctly implemented for Responses.The error from
ToAnthropicResponsesRequestis properly checked before settingStream = true.
893-902: Cached token tracking for Responses stream.Consistent with the chat completion streaming implementation. Maps cache tokens to the appropriate details fields.
core/providers/bedrock/types.go (2)
103-105: LGTM! Reasoning content types are well-structured.The new
BedrockReasoningContentandBedrockReasoningContentTexttypes correctly model Bedrock's reasoning blocks with the requiredtextand optionalsignaturefields for verification support.Also applies to: 152-159
438-440: LGTM! Delta structure properly supports reasoning streaming.The refactored
BedrockContentBlockDeltanow cleanly separates text, reasoning content, and tool use deltas, enabling proper streaming of reasoning content.core/providers/openai/types.go (1)
74-101: LGTM! MarshalJSON correctly transforms reasoning to reasoning_effort.The implementation properly shadows the embedded
Reasoningfield to exclude it from output and emitsreasoning_effortwhenReasoning.Effortis set. The type alias pattern correctly avoids infinite recursion.core/providers/cohere/chat.go (2)
104-121: LGTM! Reasoning validation is correctly implemented.The validation properly checks for
effort == "none"to disable thinking, requiresmax_tokenswhen reasoning is enabled, and enforces the minimum token threshold fromanthropic.MinimumReasoningMaxTokens.
423-446: LGTM! Streaming reasoning content is properly handled.The streaming handler correctly distinguishes between text content and thinking content, mapping thinking to both
ReasoningandReasoningDetailswith appropriate type markers.core/providers/cohere/cohere.go (2)
279-286: LGTM! Error propagation is correctly implemented for ChatCompletion.The request body builder now properly returns and propagates errors from
ToCohereChatCompletionRequest, allowing validation errors (like invalid reasoning tokens) to surface to callers.
331-338: LGTM! ChatCompletionStream correctly handles conversion errors.The streaming path properly checks for errors from
ToCohereChatCompletionRequestbefore setting the stream flag, ensuring validation failures are propagated.core/schemas/chatcompletions.go (5)
186-220: LGTM! Backward-compatible unmarshalling for reasoning_effort.The
UnmarshalJSONcorrectly handles the deprecatedreasoning_effortfield by converting it to the newReasoningstruct, with proper validation that both fields cannot coexist.
222-226: LGTM! ChatReasoning struct is well-designed.The struct cleanly separates
Effort(to enable/disable reasoning) fromMaxTokens(budget), with good documentation noting provider-specific requirements.
480-521: LGTM! ChatMessage unmarshalling correctly handles embedded types.The implementation properly handles the interplay with
ChatAssistantMessage's custom unmarshaller by unmarshalling each component separately and only setting embedded pointers when relevant fields are populated.
714-730: LGTM! ChatReasoningDetails provides comprehensive reasoning representation.The struct supports multiple reasoning types (summary, encrypted, text) with appropriate fields including
Signaturefor verification, enabling cross-provider reasoning interoperability.
835-835: LGTM! ImageTokens field addition.The new
ImageTokensfield properly extends token accounting to include image-related tokens.transports/bifrost-http/handlers/inference.go (4)
77-77: LGTM!The field name correctly updated from
"reasoning_effort"to"reasoning"to match the new nested reasoning structure.
169-199: LGTM!The custom
UnmarshalJSONproperly handles the interference between embeddedBifrostParamsand theChatParameterscustom unmarshaller. The three-stage unmarshalling approach correctly separates concerns.
271-280: LGTM!Consistent use of
sonic.Unmarshalaligns with the rest of the file.
476-478: LGTM!Initializing
ChatParametersbefore unmarshalling ensures the customUnmarshalJSONcan properly populate the struct.core/providers/openai/chat.go (3)
14-14: LGTM!Correctly uses the new
ConvertOpenAIMessagesToBifrostMessagesutility to handle the OpenAI-specific message structure including reasoning fields.
28-28: LGTM!Symmetric conversion using
ConvertBifrostMessagesToOpenAIMessagesensures proper round-trip handling of message structures.
63-65: LGTM!The nested nil-checks on
ReasoningandReasoning.Effortare correct. The transformation from "minimal" to "low" maintains compatibility with non-OpenAI providers.framework/streaming/chat.go (3)
39-43: LGTM!Correctly uses
bifrost.Ptrfor consistent pointer initialization when accumulating refusal content.
44-54: LGTM!Reasoning accumulation follows the same pattern as refusal handling, properly initializing
ChatAssistantMessageif nil and concatenating reasoning content across chunks.
55-105: LGTM!The index-based merging of
ReasoningDetailscorrectly handles streaming scenarios where details arrive across multiple chunks. The accumulation behavior (concatenate Text/Summary/Data, overwrite Signature/Type) aligns with expected streaming semantics where signatures are typically final values.core/providers/openai/utils.go (2)
5-24: LGTM!Clean conversion from OpenAI messages to Bifrost format. Correctly handles the nil check on
OpenAIChatAssistantMessagebefore creating the corresponding Bifrost struct.
26-45: LGTM!Symmetric conversion maintains consistency. Note that
ReasoningDetailsfromChatAssistantMessageis intentionally not copied toOpenAIChatAssistantMessagesince it's a Bifrost-specific accumulated field not part of the OpenAI API.core/schemas/mux.go (1)
1359-1365: Reasoning streaming delta mapping looks correctMapping
delta.ReasoningintoResponsesStreamResponseTypeReasoningSummaryTextDeltawithDeltaset to the reasoning text and a stableOutputIndexis consistent with the new Responses-side reasoning event type and with the Anthropic streaming bridge.No changes needed here.
core/providers/bedrock/utils.go (1)
27-47: Reasoning →reasoning_configmapping and validation look goodThe logic for Bedrock reasoning:
- Disables reasoning when
Effort == "none",- Requires
Reasoning.MaxTokensand enforces>= anthropic.MinimumReasoningMaxTokenswhen reasoning isn’t explicitly"none",- Sends
{type: "enabled", budget_tokens: max_tokens}toreasoning_config.This matches the Anthropic semantics and the PR’s minimum tokens requirement.
No changes needed here beyond the separate map-merge concern already raised.
core/providers/bedrock/chat.go (2)
38-40: Good: Bedrock chat now propagates parameter conversion errorsWrapping
convertChatParametersin:if err := convertChatParameters(...); err != nil { return nil, fmt.Errorf("failed to convert chat parameters: %w", err) }correctly surfaces validation failures (e.g., invalid reasoning.max_tokens) instead of continuing with a partially-initialized request.
293-341: Streaming reasoning text and signatures are wired through correctlyThe streaming handler:
- Emits
ChatStreamResponseChoiceDelta.Reasoningplus aReasoningDetailsentry whenReasoningContentDelta.Textis non-empty.- Emits a
ReasoningDetailsentry with onlySignaturewhenReasoningContentDelta.Signatureis present.This matches the new
ChatReasoningDetailsschema and ensures both reasoning text and signatures are available to downstream mux/UI layers.core/providers/anthropic/responses.go (1)
190-201: Thinking → Responses Reasoning mapping is aligned and usefulWhen
request.Thinkingis present:
"enabled"is mapped toResponsesParametersReasoning{ Effort: "auto", MaxTokens: BudgetTokens }, and"disabled"is mapped toResponsesParametersReasoning{ Effort: "none" }.This keeps Anthropic’s thinking config and the unified Responses
Reasoningobject in sync and is consistent with the broader reasoning model incore/schemas.core/providers/anthropic/chat.go (6)
13-16: Good: ToAnthropicChatRequest now validates nil request/inputReturning an error when
bifrostReq == nil || bifrostReq.Input == nilis reasonable; Anthropic chat requires message content, so this protects against constructing malformed provider requests.
211-220: ReasoningDetails → Anthropic thinking blocks mapping is soundWhen converting Bifrost messages to Anthropic:
- Each
ChatReasoningDetailsentry is turned into an Anthropicthinkingcontent block withThinkingandSignature,- These blocks are prepended before visible content, preserving reasoning plus attached signatures.
This is consistent with how Anthropic models thinking content and with the unified reasoning_details schema.
441-541: Bifrost Chat → Anthropic chat response reasoning/tool mapping looks correct
ToAnthropicChatResponse:
- Propagates usage and cached token details appropriately,
- Converts
FinishReason/StopStringback to AnthropicStopReason/StopSequence,- Translates
ChatAssistantMessage.ReasoningDetailsof typereasoning.textinto Anthropicthinkingcontent blocks with both text and signature,- Adds primary text content (string or blocks) and tool calls as
tool_useblocks.This keeps reasoning/signature and tool metadata intact across the round-trip.
636-655: Streaming thinking and signature deltas to Chat are correctly emittedIn
ToBifrostChatCompletionStream:
AnthropicStreamDeltaTypeThinkingis mapped toChatStreamResponseChoiceDelta.Reasoningplus aReasoningDetailsentry (typereasoning.text, text populated).AnthropicStreamDeltaTypeSignatureproduces a delta withReasoningDetailscarrying onlySignature.This matches the Chat streaming schema and aligns with the changes in mux/responses that consume
delta.ReasoningandReasoningDetails.Also applies to: 665-687
744-751: Reasoning text deltas → Anthropic thinking deltas look goodThe
else if delta.Reasoning != nilbranch correctly:
- Emits a
content_block_deltawithType: thinking,- Sets
Thinkingto the reasoning text.This aligns with Anthropic’s SSE format and matches how thinking deltas are handled elsewhere in the provider.
852-865: Chat stream error SSE formatting is consistent
ToAnthropicChatStreamErrorwrapsToAnthropicChatCompletionErrorinto anAnthropicStreamEvent{Type: error}and formats it as:event: error data: { ...json... }This matches standard Anthropic SSE error framing; no changes needed.
Pratham-Mishra04
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
455b127 to
8e37f9a
Compare
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
0822b25 to
05daf39
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/providers/cohere/chat.go (1)
104-137: ExtraParamsthinkingstill overwrites Reasoning-based thinking.A previous review flagged that
ExtraParams["thinking"](lines 126-137) can silently overwrite theThinkingconfiguration set fromReasoning(lines 104-121). The past review comment indicates this was addressed, but the current code still shows the override can occur.If both
ReasoningandExtraParams["thinking"]are provided, the ExtraParams version will win. Consider guarding the ExtraParams extraction:// Convert extra params if bifrostReq.Params.ExtraParams != nil { // Handle thinking parameter - if thinkingParam, ok := schemas.SafeExtractFromMap(bifrostReq.Params.ExtraParams, "thinking"); ok { + if cohereReq.Thinking == nil { + if thinkingParam, ok := schemas.SafeExtractFromMap(bifrostReq.Params.ExtraParams, "thinking"); ok { if thinkingMap, ok := thinkingParam.(map[string]interface{}); ok { thinking := &CohereThinking{} if typeStr, ok := schemas.SafeExtractString(thinkingMap["type"]); ok { thinking.Type = CohereThinkingType(typeStr) } if tokenBudget, ok := schemas.SafeExtractIntPointer(thinkingMap["token_budget"]); ok { thinking.TokenBudget = tokenBudget } cohereReq.Thinking = thinking } + } }
♻️ Duplicate comments (7)
core/providers/bedrock/utils.go (2)
14-17: NilParamsnow hard-fails Bedrock chat; verify this intentional vs other providersOther providers typically treat
Params == nilas “use defaults” rather than an error. This change makes Bedrock chat fail when callers omitparams, which is a behavior change and inconsistent with Gemini/Vertex/Perplexity patterns.If this isn’t a deliberate divergence, consider initializing a default
ChatParameters(withExtraParamsmap) instead of returning an error, and keep strict validation only for nested fields likereasoning.max_tokens.
28-47: Reasoning config can be dropped and uses inconsistent map typeTwo related issues:
Type mismatch for disabled reasoning config
Here you set:
bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ "type": "disabled", }but
ToBifrostResponsesRequest(and similar readers) expectreasoning_configasmap[string]interface{}and type‑assert accordingly. When this value comes from our own converter, the.(map[string]interface{})assertion will fail for the disabled case and we’ll never populateParams.Reasoning.Use
map[string]anyfor both enabled and disabled shapes:
- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{
"type": "disabled",- }
- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{
"type": "disabled",- }
additionalModelRequestFieldPathsoverwritesreasoning_configLater you do:
if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { bedrockReq.AdditionalModelRequestFields = orderedFields } }This replaces the entire map, discarding the previously-set
reasoning_config. If both need to coexist, merge into the existing map instead of overwriting:
- if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok {
bedrockReq.AdditionalModelRequestFields = orderedFields- }
- if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok {
if bedrockReq.AdditionalModelRequestFields == nil {bedrockReq.AdditionalModelRequestFields = orderedFields} else {for k, v := range orderedFields {bedrockReq.AdditionalModelRequestFields[k] = v}}- }
Same merge pattern should be applied anywhere else we combine
reasoning_configwith additional model request fields for Bedrock.Also applies to: 70-75
core/providers/azure/azure.go (1)
353-360: Still need defensive nil checks after Anthropic request conversionYou now propagate errors from
anthropic.ToAnthropic*Request, which is good, but several sites still assume the returned struct is non‑nil:
ChatCompletionStream:reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } reqBody.Model = deployment // potential nil deref reqBody.Stream = schemas.Ptr(true)
Responses/ResponsesStreamhave the same pattern withToAnthropicResponsesRequest.In non‑stream
ChatCompletion, you guard the assignment:if reqBody != nil { reqBody.Model = deployment }but if
reqBodyis nil anderris nil, you’ll serialize anilbody to JSON, producing"null"as the HTTP payload.For robustness (and to align with earlier review feedback), add a nil check after each conversion and return a clear error if it ever happens:
reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } + if reqBody == nil { + return nil, fmt.Errorf("anthropic chat request conversion returned nil") + } reqBody.Model = deployment reqBody.Stream = schemas.Ptr(true)Apply the same pattern to the non‑stream chat and both Responses paths.
Also applies to: 452-458, 529-535, 627-633
core/providers/anthropic/responses.go (1)
296-299: UnifyReasoningenable/disable semantics with chat pathIn
ToAnthropicResponsesRequest:if bifrostReq.Params.Reasoning != nil { if Reasoning.Effort != nil && *Effort != "none" { // require & validate MaxTokens, set Thinking enabled } else { // everything else (Effort==nil or "none") → Thinking.Type = "disabled" } }This treats
Effort == nilas “disabled”, even ifMaxTokensis set, whereas the chat converter previously treated “any non‑noneeffort or presence ofmax_tokens” as enabling thinking (with the same minimum‑tokens check). That means the sameReasoningpayload may enable thinking on chat but disable it on responses.Recommend extracting a small helper (e.g.
normalizeReasoningToThinking(reasoning *ResponsesParametersReasoning) (*AnthropicThinking, error)) and using it in both chat and responses so that:
- Reasoning is enabled when
(Effort != nil && *Effort != "none") || MaxTokens != nil,- Reasoning is disabled only when
Effort != nil && *Effort == "none"andMaxTokens == nil,- In the enabled case,
MaxTokensis required and checked againstMinimumReasoningMaxTokens.That keeps validation and behavior identical across APIs.
Also applies to: 325-344, 402-403
core/providers/bedrock/responses.go (1)
452-472: Alignreasoning_configmap type and avoid overwriting it with extra request fieldsTwo problems here mirror those in
convertChatParameters:
Disabled config type mismatch
Disabled reasoning is written as:
bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ "type": "disabled", }while
ToBifrostResponsesRequestabove expectsreasoning_configasmap[string]interface{}. When we round‑trip a disabled config through Bedrock, the type assertion will fail and we’ll loseParams.Reasoning.Use
map[string]anyfor both enabled and disabled cases:
- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{
"type": "disabled",- }
- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{
"type": "disabled",- }
ExtraParams request fields overwrite reasoning config
Later:
if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { bedrockReq.AdditionalModelRequestFields = orderedFields } }This replaces the whole map and drops the earlier
reasoning_config. For callers who specify both reasoning and additional request fields, reasoning will be silently disabled.Merge instead of replace, similar to the suggestion for
utils.go:
- if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok {
bedrockReq.AdditionalModelRequestFields = orderedFields- }
- if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok {
if bedrockReq.AdditionalModelRequestFields == nil {bedrockReq.AdditionalModelRequestFields = orderedFields} else {for k, v := range orderedFields {bedrockReq.AdditionalModelRequestFields[k] = v}}- }
Also applies to: 478-482
core/providers/anthropic/chat.go (2)
118-122: Critical: Dead code accessing nil Params - UNFIXED from previous review.This
else ifblock is unreachable and unsafe. The}on line 118 closes theif bifrostReq.Params != nilblock from line 25, placing thiselse ifin the branch whereParamsIS nil. AccessingbifrostReq.Params.Reasoningon line 118 would cause a panic.Remove this dead code block:
- } else if bifrostReq.Params.Reasoning != nil && bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort == "none" { - anthropicReq.Thinking = &AnthropicThinking{ - Type: "disabled", - } }If this logic is needed, move it inside the
if bifrostReq.Params != nilblock before line 117.
419-436: Minor: Redundant assignment and incorrect cache token mapping - UNFIXED from previous review.Two issues:
Duplicate assignment: Lines 425-427 and 430-435 both assign
CacheCreationInputTokenstoCompletionTokensDetails.CachedTokens. The conditional block (430-435) is redundant.Semantic mismatch:
CacheCreationInputTokensrepresents tokens used to create the input cache (a prompt/input operation), not completion. It should be mapped toPromptTokensDetails, notCompletionTokensDetails.Apply this fix:
bifrostResponse.Usage = &schemas.BifrostLLMUsage{ PromptTokens: response.Usage.InputTokens, PromptTokensDetails: &schemas.ChatPromptTokensDetails{ CachedTokens: response.Usage.CacheReadInputTokens, }, CompletionTokens: response.Usage.OutputTokens, - CompletionTokensDetails: &schemas.ChatCompletionTokensDetails{ - CachedTokens: response.Usage.CacheCreationInputTokens, - }, TotalTokens: response.Usage.InputTokens + response.Usage.OutputTokens, } - if response.Usage.CacheCreationInputTokens > 0 { - if bifrostResponse.Usage.CompletionTokensDetails == nil { - bifrostResponse.Usage.CompletionTokensDetails = &schemas.ChatCompletionTokensDetails{} - } - bifrostResponse.Usage.CompletionTokensDetails.CachedTokens = response.Usage.CacheCreationInputTokens + if response.Usage.CacheCreationInputTokens > 0 { + if bifrostResponse.Usage.PromptTokensDetails == nil { + bifrostResponse.Usage.PromptTokensDetails = &schemas.ChatPromptTokensDetails{} + } + bifrostResponse.Usage.PromptTokensDetails.CachedTokens += response.Usage.CacheCreationInputTokens }
🧹 Nitpick comments (3)
core/providers/cohere/chat.go (1)
592-599: Minor: Trailing newline in reasoning text.The
reasoningTextaccumulation appends"\n"after each thinking block (line 598), which will leave a trailing newline in the finalReasoningfield. Consider trimming:} else if block.Type == CohereContentBlockTypeThinking && block.Thinking != nil { reasoningDetails = append(reasoningDetails, schemas.ChatReasoningDetails{ Index: len(reasoningDetails), Type: schemas.BifrostReasoningDetailsTypeText, Text: block.Thinking, }) reasoningText += *block.Thinking + "\n" }Then at line 663:
- assistantMessage.Reasoning = schemas.Ptr(reasoningText) + assistantMessage.Reasoning = schemas.Ptr(strings.TrimSuffix(reasoningText, "\n"))This would require adding
"strings"to the imports.core/schemas/chatcompletions.go (1)
222-226: Consider adding validation forMaxTokensminimum.Based on the PR summary, there's a minimum value (1024) for reasoning
MaxTokens. Consider adding validation here or in a separate validator function to ensure requests don't specify values below the minimum.Example:
type ChatReasoning struct { Effort *string `json:"effort,omitempty"` // "none" | "minimal" | "low" | "medium" | "high" (any value other than "none" will enable reasoning) MaxTokens *int `json:"max_tokens,omitempty"` // Maximum number of tokens to generate for the reasoning output (required for anthropic) } + +// Validate checks if MaxTokens meets the minimum requirement +func (r *ChatReasoning) Validate() error { + if r.MaxTokens != nil && *r.MaxTokens < 1024 { + return fmt.Errorf("reasoning max_tokens must be at least 1024, got %d", *r.MaxTokens) + } + return nil +}Based on learnings, validation might be provider-specific and could be enforced in provider adapters instead.
core/providers/bedrock/chat.go (1)
103-112: Reasoning aggregation semantics: trailing newline and single stringThe reasoning handling correctly captures both plain text and signatures into
ReasoningDetailsand a single aggregatedReasoningstring. Note that you always append"\\n", soassistantMessage.Reasoningwill end with a trailing newline; if that’s undesirable in clients, considerstrings.TrimRight(reasoningText, "\n")before assigning.Also applies to: 135-141
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (29)
core/providers/anthropic/anthropic.go(4 hunks)core/providers/anthropic/chat.go(10 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(4 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(3 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/semanticcache/utils.go(1 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (11)
- core/providers/anthropic/errors.go
- ui/app/workspace/logs/views/logChatMessageView.tsx
- core/providers/bedrock/bedrock.go
- core/providers/anthropic/anthropic.go
- core/schemas/mux.go
- core/providers/bedrock/types.go
- core/providers/anthropic/types.go
- ui/lib/types/logs.ts
- core/providers/perplexity/chat.go
- core/schemas/responses.go
- plugins/semanticcache/utils.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/handlers/inference.gocore/providers/openai/chat.goplugins/jsonparser/utils.gocore/providers/cohere/types.gocore/providers/cohere/cohere.gocore/providers/bedrock/chat.gocore/providers/openai/types.gocore/providers/bedrock/responses.gocore/providers/openai/openai.gocore/providers/openai/utils.goframework/streaming/chat.gocore/providers/anthropic/responses.gocore/providers/bedrock/utils.gocore/providers/vertex/vertex.gocore/providers/anthropic/chat.gocore/providers/cohere/chat.gocore/providers/azure/azure.gocore/schemas/chatcompletions.go
🧬 Code graph analysis (11)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/chatcompletions.go (2)
ChatMessage(469-478)ChatParameters(155-184)ui/lib/types/logs.ts (1)
ChatMessage(115-124)
core/providers/openai/chat.go (4)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)
core/providers/cohere/cohere.go (2)
core/providers/cohere/chat.go (1)
ToCohereChatCompletionRequest(12-199)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/openai/types.go (1)
core/schemas/chatcompletions.go (6)
ChatMessageRole(457-457)ChatMessageContent(524-527)ChatToolMessage(621-623)ChatAssistantMessageAnnotation(670-673)ChatAssistantMessageToolCall(686-692)ChatReasoning(223-226)
core/providers/bedrock/responses.go (5)
core/schemas/utils.go (4)
SafeExtractFromMap(519-525)SafeExtractString(280-297)SafeExtractInt(300-342)Ptr(16-18)core/schemas/responses.go (1)
ResponsesParametersReasoning(232-237)ui/lib/types/logs.ts (1)
ResponsesParametersReasoning(519-526)core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/openai/openai.go (3)
plugins/mocker/main.go (1)
Usage(138-142)core/providers/perplexity/types.go (1)
Usage(69-78)ui/lib/types/logs.ts (1)
CompletionTokensDetails(197-202)
framework/streaming/chat.go (3)
core/schemas/chatcompletions.go (1)
ChatAssistantMessage(626-632)core/utils.go (1)
Ptr(51-53)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)
core/providers/anthropic/responses.go (3)
core/schemas/responses.go (1)
ResponsesParametersReasoning(232-237)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/types.go (3)
AnthropicMessageRequest(40-59)MinimumReasoningMaxTokens(14-14)AnthropicThinking(65-68)
core/providers/vertex/vertex.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-282)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/anthropic/chat.go (4)
core/schemas/chatcompletions.go (13)
BifrostChatRequest(12-19)ChatAssistantMessage(626-632)ChatAssistantMessageToolCall(686-692)ChatContentBlock(590-597)ChatReasoningDetails(723-730)ChatContentBlockTypeText(582-582)ChatToolTypeFunction(239-239)BifrostReasoningDetailsTypeText(719-719)ChatMessageContent(524-527)ChatMessage(469-478)ChatNonStreamResponseChoice(745-748)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)core/providers/anthropic/types.go (7)
AnthropicMessageRequest(40-59)AnthropicThinking(65-68)MinimumReasoningMaxTokens(14-14)AnthropicContentBlock(147-159)AnthropicContentBlockTypeThinking(143-143)AnthropicContentBlockTypeText(135-135)AnthropicContentBlockTypeToolUse(137-137)core/schemas/bifrost.go (1)
Anthropic(37-37)core/providers/anthropic/utils.go (1)
ConvertAnthropicFinishReasonToBifrost(27-32)
core/providers/cohere/chat.go (3)
core/schemas/chatcompletions.go (14)
BifrostChatRequest(12-19)ChatMessage(469-478)ChatReasoning(223-226)ChatNonStreamResponseChoice(745-748)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)ChatReasoningDetails(723-730)ChatContentBlock(590-597)ChatAssistantMessageToolCall(686-692)ChatInputImage(600-603)ChatMessageContent(524-527)ChatAssistantMessageToolCallFunction(695-698)ChatAssistantMessage(626-632)ChatMessageRole(457-457)core/providers/cohere/types.go (5)
CohereChatRequest(14-31)CohereThinking(170-173)ThinkingTypeDisabled(180-180)ThinkingTypeEnabled(179-179)CohereMessage(50-56)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (30)
core/providers/cohere/types.go (1)
521-524: LGTM!The
Thinkingfield addition toCohereStreamContentis well-structured with appropriate JSON tagging and aligns with the existingThinkingfield inCohereContentBlock(line 152). This enables proper propagation of reasoning content in streaming responses.core/providers/cohere/cohere.go (2)
279-286: LGTM!The error propagation from
ToCohereChatCompletionRequestis properly handled. The lambda correctly returns the error toCheckContextAndGetRequestBodywhich will handle it appropriately.
331-338: LGTM!Good defensive pattern - checking for errors before setting
Stream = true. This ensures the streaming flag is only set on a valid request object, and errors are propagated early.core/providers/cohere/chat.go (5)
242-256: LGTM!The reverse mapping from
CohereThinkingto BifrostChatReasoningis reasonable. UsingEffort="auto"as a default for enabled thinking is appropriate since Cohere'sThinkingTypeEnableddoesn't carry effort granularity.
319-346: LGTM!Clean refactoring to use the centralized
ToBifrostChatMessage()method for response conversion. This improves maintainability by consolidating message conversion logic.
401-447: LGTM!Good handling of both text and thinking deltas in streaming responses. The thinking content is properly mapped to both
ReasoningandReasoningDetailsfields for compatibility.Note: The
Index: 0is hardcoded inReasoningDetails, which is acceptable for single thinking blocks per chunk. If multiple thinking blocks could arrive in a single delta, this would need adjustment.
616-648: LGTM!Robust tool call conversion with proper nil checks:
- Skipping tool calls with nil
Function(lines 619-622)- Defaulting to empty string for nil function name (lines 630-633)
- Safe direct access to
Argumentssince it's a string, not pointerThis defensive approach prevents potential nil pointer dereferences.
556-571: LGTM!The
ToBifrostChatMessagemethod centralizes per-message conversion logic cleanly. The content type detection (string vs blocks, single text block optimization) is well-structured.plugins/jsonparser/utils.go (1)
304-323: LGTM! Field rename aligns with the broader refactoring.The update from
ThoughttoReasoningis consistent with the PR's goal of renaming reasoning-related fields across the codebase.framework/streaming/chat.go (2)
44-105: LGTM! Reasoning accumulation logic is well-structured.The implementation correctly handles streaming deltas for both the
Reasoningfield andReasoningDetails:
Reasoningtext is accumulated (concatenated) across chunksReasoningDetailsare merged byIndex, with text/summary/data accumulated and signature/type overwritten (appropriate for final values)The complexity is justified given the rich reasoning data model introduced in this PR.
38-43: Good consistency improvement with the Ptr helper.Using
bifrost.Ptrfor the initialRefusalassignment aligns with the pattern used forReasoningand other pointer fields in the codebase.core/schemas/chatcompletions.go (3)
186-220: Well-designed backward compatibility for deprecatedreasoning_effort.The custom
UnmarshalJSONcorrectly:
- Rejects requests with both
reasoning_effortandreasoning(mutual exclusivity)- Maps legacy
reasoning_effortto the new nestedReasoning.Effortstructure- Maintains forward compatibility with the new
reasoningobject
634-667: Verify the reasoning details synthesis behavior is intentional.The
UnmarshalJSONimplementation synthesizes areasoning_detailsentry of type "reasoning.text" wheneverReasoningis present butReasoningDetailsis empty.This implicit transformation may surprise API consumers who expect the response shape to match their input. Consider documenting this behavior or confirming it aligns with the API contract.
If this synthesis is provider-specific (e.g., for providers that return only
reasoningtext), consider moving the logic closer to those provider adapters rather than in the core schema unmarshaller.
765-794: Consistent synthesis logic in streaming delta.The same reasoning-to-reasoning_details synthesis logic is applied here for streaming deltas. The same documentation concern from the
ChatAssistantMessageunmarshaller applies: ensure this behavior is documented and intentional.transports/bifrost-http/handlers/inference.go (2)
169-199: Custom unmarshaller correctly handles embedded struct conflicts.The custom
UnmarshalJSONforChatRequestis necessary becauseChatParametershas its own custom unmarshaller that would interfere with sonic's handling of the embeddedBifrostParams. The three-step approach (unmarshal BifrostParams, then Messages, then ChatParameters) correctly assembles all fields.
476-478: Good defensive initialization before unmarshalling.Initializing
ChatParametersto a non-nil value before callingsonic.Unmarshalensures the customUnmarshalJSONmethod onChatParameterscan safely populate fields without nil pointer issues.core/providers/openai/openai.go (2)
515-520: LGTM! Token usage details correctly propagated in streaming.The implementation mirrors the existing pattern for basic token counts and correctly propagates the detailed token breakdowns (including reasoning tokens) before clearing
response.Usage.
981-986: LGTM! Token usage details correctly propagated in streaming.Consistent with the text completion streaming implementation, this correctly propagates token usage details including reasoning tokens.
core/providers/openai/chat.go (2)
63-65: LGTM! Correctly updated for nested Reasoning structure.The nil checks for
request.ChatParameters.Reasoningandrequest.ChatParameters.Reasoning.Effortappropriately handle the new nested structure, and the logic correctly maps "minimal" to "low" for non-OpenAI providers.
14-14: Good use of centralized conversion utilities.Delegating to
ConvertOpenAIMessagesToBifrostMessagesandConvertBifrostMessagesToOpenAIMessagesreduces duplication and ensures consistent message mapping across OpenAI-compatible providers.Also applies to: 28-28
core/providers/vertex/vertex.go (1)
283-286: LGTM! Proper error propagation from Anthropic converters.The updated calls to
ToAnthropicChatRequestandToAnthropicResponsesRequestnow correctly propagate conversion errors (e.g., validation failures for reasoning parameters) instead of silently ignoring them. This improves error visibility for misconfigured requests.Also applies to: 518-520, 688-690, 845-847
core/providers/bedrock/chat.go (1)
37-40: Error propagation fromconvertChatParameterslooks correctWiring
convertChatParametersto return an error and propagating it here makes Bedrock chat failures explicit and keeps the call pattern consistent with other providers.core/providers/bedrock/responses.go (2)
349-365: Reasoning extraction fromreasoning_configlooks structurally soundMapping
reasoning_config→ResponsesParameters.Reasoningwith:
"type": "enabled"→Effort: "auto"andMaxTokensfrombudget_tokens,- other types →
Effort: "none",is a reasonable default bridge from Bedrock’s config into the unified Responses schema. Behavior if
budget_tokensis missing/invalid is simply “no Reasoning”, which is acceptable as a conservative fallback.
656-660: Output conversion moved up; avoids duplicate workConverting
response.Output.MessagetobifrostResp.Outputonce at the top ofToBifrostResponsesResponsesimplifies the function and prevents accidental double-conversion. No issues here.core/providers/anthropic/responses.go (1)
190-201: Thinking → ResponsesReasoningbridge looks goodPopulating
Params.Reasoningfromrequest.Thinking(enabled →Effort: "auto", MaxTokens: BudgetTokens, else →Effort: "none") gives callers a unified view of Anthropic thinking in the Responses schema, while still preserving the rawThinkingobject inExtraParams["thinking"]. This is a sensible mapping.core/providers/openai/types.go (2)
56-72: LGTM: Well-documented embedded pointer pattern.The embedded pointer structs pattern with the warning comment appropriately documents the constraint that only one should be non-nil at a time. The structure correctly supports reasoning in assistant message responses.
74-101: LGTM: Correct implementation of backward-compatible marshalling.The custom marshalling correctly:
- Uses the alias pattern to prevent recursion
- Shadows the
Reasoningfield to omit it from JSON output- Maps
Reasoning.Efforttoreasoning_effortfor backward compatibility with the deprecated fieldThis is a sound approach for deprecating
reasoning_effortwhile maintaining API compatibility.core/providers/anthropic/chat.go (3)
13-16: LGTM: Appropriate error handling added.The signature change to return an error enables proper validation and error propagation, which is essential for the reasoning parameter validation introduced later.
211-220: LGTM: Proper reasoning details conversion.The conversion correctly:
- Checks for nil before accessing nested fields
- Maps Bifrost reasoning details to Anthropic thinking blocks
- Preserves signature information for verification
647-687: LGTM: Comprehensive streaming reasoning support.The streaming delta handling correctly:
- Maps thinking deltas to both
ReasoningandReasoningDetailsfields- Handles signature deltas separately with proper type and index
- Maintains consistency with the non-streaming conversion logic
Pratham-Mishra04
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
8e37f9a to
9e093fc
Compare
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
05daf39 to
6c16eed
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/providers/cohere/chat.go (1)
126-137: ExtraParamsthinkingstill silently overridesReasoning-based thinking.Despite the past comment being marked as addressed, the code still unconditionally overwrites
cohereReq.ThinkingfromExtraParams["thinking"]even if it was already set frombifrostReq.Params.Reasoning(lines 104-121). This means if both are provided,ExtraParamswins silently.Consider skipping the ExtraParams extraction if
Reasoningwas already processed:// Convert extra params if bifrostReq.Params.ExtraParams != nil { // Handle thinking parameter - if thinkingParam, ok := schemas.SafeExtractFromMap(bifrostReq.Params.ExtraParams, "thinking"); ok { + if cohereReq.Thinking == nil { + if thinkingParam, ok := schemas.SafeExtractFromMap(bifrostReq.Params.ExtraParams, "thinking"); ok { if thinkingMap, ok := thinkingParam.(map[string]interface{}); ok { thinking := &CohereThinking{} if typeStr, ok := schemas.SafeExtractString(thinkingMap["type"]); ok { thinking.Type = CohereThinkingType(typeStr) } if tokenBudget, ok := schemas.SafeExtractIntPointer(thinkingMap["token_budget"]); ok { thinking.TokenBudget = tokenBudget } cohereReq.Thinking = thinking } + } }core/schemas/chatcompletions.go (1)
155-185: Thereasoning_effort→Reasoningmapping will break existing callers that only setreasoning_effortwithoutmax_tokens.The
ChatParametersunmarshaller cleanly enforces mutual exclusivity betweenreasoningandreasoning_effortand convertsreasoning_effortintoChatReasoning{Effort: ...}. However,ToAnthropicChatRequestrequires that whenReasoningis non-nil,MaxTokensmust also be set (error: "reasoning.max_tokens is required for reasoning"). For legacy clients that previously sent onlyreasoning_effortwithoutreasoning.max_tokens, requests that used to work will now fail at this validation.If you intend to retain runtime compatibility for
reasoning_effort, consider one of:
- injecting a default
MaxTokenswhenReasoning.Effort != nilandReasoning.MaxTokens == nil(minimum is 1024), or- relaxing the validation to allow effort-only configs (with a provider-chosen default budget),
and documenting the new behavior clearly if you do intend this to be a hard breaking change.
♻️ Duplicate comments (6)
core/providers/bedrock/responses.go (2)
478-482: Same overwriting issue as in utils.go.Line 480 directly assigns
orderedFieldstoAdditionalModelRequestFields, which will discard thereasoning_configset at lines 452-472. This is the same issue flagged inbedrock/utils.go. Apply the same merge strategy here.
452-472: Type consistency issue with disabled reasoning config.At line 457, disabled reasoning is written as
map[string]string{"type": "disabled"}:bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ "type": "disabled", }However, the extraction logic at lines 351-352 in
ToBifrostResponsesRequestexpectsmap[string]interface{}. This type mismatch will cause the type assertion to fail, preventing disabled reasoning from being properly round-tripped.Use
map[string]anyfor consistency:- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ "type": "disabled", }core/providers/bedrock/utils.go (1)
26-47: Reasoning config may be overwritten byadditionalModelRequestFieldPaths.The
reasoning_configis correctly set inAdditionalModelRequestFieldsat lines 29-46. However, at line 73, ifadditionalModelRequestFieldPathsexists inExtraParams, the entireAdditionalModelRequestFieldsmap is replaced:bedrockReq.AdditionalModelRequestFields = orderedFields // Line 73 - overwrites!This silently discards the
reasoning_configset earlier. Merge instead of replacing:if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = orderedFields + } else { + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } + } } }Also applies to: 70-75
core/providers/azure/azure.go (1)
629-634: Same nil safety concern in ResponsesStream path.Similar to the Responses path, lines 633-634 access
reqBody.ModelandreqBody.Streamwithout checking ifreqBodyis nil after error propagation.Apply the same defensive check:
reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } + if reqBody != nil { reqBody.Model = deployment reqBody.Stream = schemas.Ptr(true) + } return reqBody, nilcore/providers/anthropic/responses.go (1)
190-201: Reasoning enabled/disabled semantics differ from chat path and ignore MaxTokens-only configs.In
ToBifrostResponsesRequestandToAnthropicResponsesRequest, reasoning is only considered “enabled” whenReasoning.Effort != nil && *Effort != "none". IfEffortisnilbutMaxTokensis set, you currently treat this as disabled and setThinking.Type = "disabled", whereasToAnthropicChatRequesttreatsEffort == nilwith non‑nilMaxTokensas enabled reasoning and enforces the token budget. This means the samereasoningpayload can behave differently depending on whether callers use Chat vs Responses, and MaxTokens-only configurations are silently discarded on the Responses path.Consider unifying the rule to something like:
- Enabled if
(Effort != nil && *Effort != "none") || MaxTokens != nil,- Disabled only if
Effort != nil && *Effort == "none"andMaxTokens == nil,- In the enabled case, always require and validate
MaxTokens >= MinimumReasoningMaxTokens,and factoring this into a shared helper used by both chat and responses converters to avoid drift.
Also applies to: 325-344
core/providers/anthropic/chat.go (1)
13-16: Chat reasoning validation is strict and currently diverges from responses path andreasoning_effortbehavior.
ToAnthropicChatRequestnow:
- rejects any request with
Params.Reasoning != nil,Effort != "none", andMaxTokens == nil, and- enforces
MaxTokens >= MinimumReasoningMaxTokenswhen reasoning is enabled.That part is good and matches Anthropic’s expectations. However:
- For Chat,
Effort == nil+ non‑nilMaxTokensis treated as “enabled reasoning with default effort”, while inToAnthropicResponsesRequestthe same shape is treated as disabled (see comment onresponses.go). This is a subtle semantic mismatch between chat and responses.- Combined with
ChatParameters.UnmarshalJSON, any legacyreasoning_effortusage that doesn’t also specifymax_tokenswill now produce an error instead of silently mapping to a default budget.If that’s intentional, documenting it as a hard breaking change would be useful. If not, consider aligning with the shared helper suggested in
responses.goand/or givingreasoning_efforta defaultMaxTokensto preserve prior behavior.Also applies to: 24-41, 99-117
🧹 Nitpick comments (7)
core/providers/cohere/chat.go (1)
7-7: Cross-package constant dependency.Using
anthropic.MinimumReasoningMaxTokenscouples Cohere's validation to Anthropic's package. If reasoning token minimums differ across providers in the future, consider moving this constant to a shared location (e.g.,schemaspackage) or defining a Cohere-specific constant.transports/bifrost-http/handlers/inference.go (1)
476-478: Optional: Remove redundant initialization.The
ChatParametersfield is pre-initialized here, but the customUnmarshalJSONmethod (lines 191-193) already handles the nil case and initializes it if needed. While harmless, this redundancy could be removed for cleaner code.Apply this diff if you prefer to remove the redundancy:
- req := ChatRequest{ - ChatParameters: &schemas.ChatParameters{}, - } + var req ChatRequestcore/providers/bedrock/utils.go (1)
14-17: Consider initializing defaults instead of erroring on nil params.Returning an error when
bifrostReq.Paramsis nil is inconsistent with other providers. For example, Gemini initializes defaultChatParameterswith an emptyExtraParamsmap, and Vertex/Perplexity process params conditionally without erroring.Following the Gemini pattern would improve consistency:
func convertChatParameters(bifrostReq *schemas.BifrostChatRequest, bedrockReq *BedrockConverseRequest) error { if bifrostReq.Params == nil { - return fmt.Errorf("bifrost request parameters are nil") + bifrostReq.Params = &schemas.ChatParameters{ + ExtraParams: make(map[string]interface{}), + } }This approach reserves errors for truly required nested fields (like
reasoning.max_tokenswhen reasoning is enabled).core/providers/anthropic/responses.go (1)
827-839: Signature streaming works but uses brittle heuristics and has a dead enrichment block.
- The
AnthropicStreamDeltaTypeSignaturebranch stores signatures instate.ReasoningSignaturesand reusesReasoningSummaryTextDeltafor both signatures and thinking, thenToAnthropicResponsesStreamResponseguesses whether a delta is a signature by length and base64‑ish characters. That’s workable but brittle; if you ever see short signatures or long/base64‑like reasoning text, it will misclassify. A dedicated stream response type for signatures (or an explicit flag) would be more robust long‑term.- At content_block_stop, the block that searches
responsesfor a reasoning item and assignsEncryptedContent(reasoningItem := ...) has no effect when there are no existing reasoning items inresponses(which is the common case for this branch). The actual signature propagation happens a few lines later when you attachEncryptedContenttodoneItem. That earlier block is effectively dead code and can be removed or wired to emit an explicit reasoning item event if you really want a separate message for signatures.These are behavioral nits rather than blockers, but cleaning them up would reduce surprise and make the signature path easier to reason about.
Also applies to: 969-989, 1001-1006, 1272-1293, 3057-3071
core/providers/anthropic/chat.go (3)
207-216: ReasoningDetails → thinking blocks mapping is lossy but acceptable for now.When building the Anthropic request, you prepend one
thinkingblock perChatReasoningDetails, preservingSignatureandTextbut ignoring other fields (e.g.,summaryvstext,data). That’s sufficient for Anthropic’s current API, which only accepts raw thinking text + optional signature, but it means callers can’t distinguish between different reasoning detail types once mapped.If you later rely on
reasoning.summaryor encrypted data distinctions, you may want to either encode those into the thinking text or extend AnthropicThinking/extra params to carry richer metadata.
472-483: Anthropic ← Bifrost reasoning round-trip only considers text-type details.In
ToAnthropicChatResponse, you rebuildthinkingblocks only fromChatAssistantMessage.ReasoningDetailsentries withType == BifrostReasoningDetailsTypeText. Any futurereasoning.summaryorreasoning.encryptedentries will be ignored here, even though the type system now allows them.If you plan to support richer reasoning detail types, consider:
- either treating all non‑nil
Textfields the same regardless ofType, or- adding a mapping from
reasoning.summary/reasoning.encryptedinto Anthropic content (e.g., one block per type) and usingSignatureto distinguish encrypted content.Also applies to: 503-521
626-645: Streaming reasoning/signature deltas are wired through, but usereasoning.textfor signatures.
- For
AnthropicStreamDeltaTypeThinking, you now emit bothdelta.Reasoningand a singleChatReasoningDetails{Type: reasoning.text, Text: ...}, which is a nice normalized shape.- For
AnthropicStreamDeltaTypeSignature, you emit aChatReasoningDetailswithSignatureset butTypestillreasoning.text. Given theChatReasoningDetailsTypeenum includesreasoning.encrypted, using that for signature‑only deltas would better match the public type contract and make it easier for UIs to distinguish encrypted/signature payloads from plain text reasoning.Switching the signature branch to
Type: BifrostReasoningDetailsTypeEncrypted(or similar) would be a low‑risk improvement.Also applies to: 655-677
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (30)
core/providers/anthropic/anthropic.go(4 hunks)core/providers/anthropic/chat.go(10 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(5 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(4 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/semanticcache/utils.go(2 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (9)
- core/providers/openai/openai.go
- ui/lib/types/logs.ts
- framework/streaming/chat.go
- core/providers/cohere/cohere.go
- core/providers/anthropic/anthropic.go
- ui/app/workspace/logs/views/logChatMessageView.tsx
- core/providers/perplexity/chat.go
- core/providers/bedrock/bedrock.go
- plugins/semanticcache/utils.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
core/providers/anthropic/errors.gotransports/bifrost-http/handlers/inference.gocore/providers/bedrock/chat.gocore/providers/vertex/vertex.goplugins/maxim/main.gocore/providers/bedrock/responses.gocore/providers/anthropic/types.goplugins/jsonparser/utils.gocore/schemas/mux.gocore/providers/openai/chat.gocore/schemas/responses.gocore/providers/bedrock/utils.gocore/providers/openai/types.gocore/providers/anthropic/responses.gocore/providers/openai/utils.gocore/providers/cohere/chat.gocore/providers/azure/azure.gocore/providers/cohere/types.gocore/providers/anthropic/chat.gocore/providers/bedrock/types.gocore/schemas/chatcompletions.go
🧠 Learnings (1)
📚 Learning: 2025-12-08T11:50:35.989Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 978
File: core/providers/openai/utils.go:26-45
Timestamp: 2025-12-08T11:50:35.989Z
Learning: OpenAI does not support the `ReasoningDetails` field. `ReasoningDetails` is a Bifrost-internal field and should not be included when converting messages to OpenAI format in the `ConvertBifrostMessagesToOpenAIMessages` function in `core/providers/openai/utils.go`.
Applied to files:
core/providers/openai/utils.go
🧬 Code graph analysis (14)
core/providers/anthropic/errors.go (1)
core/providers/anthropic/types.go (2)
AnthropicMessageError(376-379)AnthropicMessageErrorStruct(382-385)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/chatcompletions.go (2)
ChatMessage(469-478)ChatParameters(155-184)ui/lib/types/logs.ts (1)
ChatMessage(115-124)
core/providers/bedrock/chat.go (2)
core/schemas/chatcompletions.go (8)
ChatReasoningDetails(723-730)BifrostReasoningDetailsTypeText(719-719)ChatContentBlockTypeText(582-582)ChatAssistantMessage(626-632)BifrostChatResponse(26-41)BifrostResponseChoice(704-712)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)
core/providers/vertex/vertex.go (4)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/bedrock/responses.go (5)
core/schemas/utils.go (3)
SafeExtractFromMap(519-525)SafeExtractString(280-297)SafeExtractInt(300-342)core/schemas/responses.go (1)
ResponsesParametersReasoning(232-237)ui/lib/types/logs.ts (1)
ResponsesParametersReasoning(519-526)core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/schemas/mux.go (3)
core/schemas/responses.go (2)
ResponsesParametersReasoning(232-237)BifrostResponsesStreamResponse(1424-1462)core/schemas/chatcompletions.go (1)
ChatReasoning(223-226)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/openai/chat.go (4)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)
core/providers/bedrock/utils.go (3)
core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)OrderedMap(268-268)core/providers/bedrock/types.go (2)
BedrockConverseRequest(44-57)BedrockError(384-388)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/openai/types.go (1)
core/schemas/chatcompletions.go (6)
ChatMessageRole(457-457)ChatMessageContent(524-527)ChatToolMessage(621-623)ChatAssistantMessageAnnotation(670-673)ChatAssistantMessageToolCall(686-692)ChatReasoning(223-226)
core/providers/anthropic/responses.go (4)
core/schemas/responses.go (1)
ResponsesParametersReasoning(232-237)ui/lib/types/logs.ts (1)
ResponsesParametersReasoning(519-526)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/types.go (2)
MinimumReasoningMaxTokens(14-14)AnthropicThinking(65-68)
core/providers/openai/utils.go (2)
core/providers/openai/types.go (2)
OpenAIMessage(56-65)OpenAIChatAssistantMessage(67-72)core/schemas/chatcompletions.go (3)
ChatMessage(469-478)ChatToolMessage(621-623)ChatAssistantMessage(626-632)
core/providers/cohere/chat.go (3)
core/schemas/chatcompletions.go (10)
ChatMessage(469-478)ChatReasoning(223-226)ChatReasoningDetails(723-730)ChatContentBlock(590-597)ChatContentBlockTypeText(582-582)ChatContentBlockTypeImage(583-583)ChatInputImage(600-603)ChatMessageContent(524-527)ChatAssistantMessage(626-632)ChatMessageRole(457-457)core/providers/cohere/types.go (5)
CohereChatRequest(14-31)CohereThinking(170-173)ThinkingTypeDisabled(180-180)ThinkingTypeEnabled(179-179)CohereMessage(50-56)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/azure/azure.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/schemas/chatcompletions.go (1)
ui/lib/types/logs.ts (3)
ChatMessage(115-124)ChatMessageContent(113-113)ReasoningDetails(126-133)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (41)
core/providers/cohere/types.go (1)
520-524: LGTM! Thinking field addition for streaming content.The
Thinkingfield addition aligns with the reasoning support being added across providers and is consistent with the existingCohereContentBlockstruct which also has aThinkingfield (line 152).core/providers/cohere/chat.go (6)
104-121: Reasoning validation logic looks correct.The validation properly requires
MaxTokenswhen reasoning is enabled and enforces the minimum token budget. The early error returns provide clear feedback.One consideration: if
Effortis set to values other than"none"(e.g.,"low","medium","high"), all are treated as enabled. Verify this aligns with the unified Bifrost schema for reasoning effort levels.
242-256: LGTM! Reasoning conversion from Cohere to Bifrost format.The round-trip conversion properly maps
ThinkingTypeDisabledtoEffort: "none"and enabled thinking toEffort: "auto"with the token budget preserved.
423-446: Streaming thinking delta: hardcodedIndex: 0may lose ordering.For streaming reasoning content,
Indexis always set to0in each chunk. If the client accumulatesReasoningDetailsacross chunks, this could cause ordering issues or overwrites. Consider tracking and incrementing the index across streaming events if ordering matters for the downstream consumer.If the client is expected to append chunks without using the index for positioning, this is acceptable.
616-648: LGTM! Tool call conversion with proper nil safety.Good defensive handling of nil
Function(skip) and nilName(default to empty string). The index calculation usinglen(toolCalls)correctly assigns sequential indices.
214-221: Potential nil dereference ifreq.Messagescontains nil elements.Line 218 dereferences
message.ToBifrostChatMessage()which returnsnilif the receiver is nil. Ifreq.Messagesever contains nilCohereMessagepointers, this would panic.Verify that
req.Messagescannot contain nil elements, or add a guard:if req.Messages != nil { bifrostMessages := make([]schemas.ChatMessage, len(req.Messages)) for i, message := range req.Messages { + if msg := message.ToBifrostChatMessage(); msg != nil { - bifrostMessages[i] = *message.ToBifrostChatMessage() + bifrostMessages[i] = *msg + } } bifrostReq.Input = bifrostMessages }Note:
req.Messagesis[]CohereMessage(not[]*CohereMessage), so elements are values, not pointers. The receiver inToBifrostChatMessageis*CohereMessage, so Go takes the address of each value. This should be safe since value types can't be nil.
342-346: LGTM! Response message conversion.Clean integration of the
ToBifrostChatMessagemethod for converting the response message.transports/bifrost-http/handlers/inference.go (3)
77-77: LGTM!The field name change from
"reasoning_effort"to"reasoning"correctly aligns with the new unified reasoning support structure introduced in this PR.
169-199: Well-structured custom unmarshaller.The custom
UnmarshalJSONimplementation correctly handles the interaction between embeddedBifrostParamsandChatParameterswith its own custom unmarshaller. The three-step approach (BifrostParams → Messages → ChatParameters) is sound and properly avoids recursion using the type alias pattern.
270-280: This concern is unfounded. Sonic (github.com/bytedance/sonic) is explicitly designed as a drop-in replacement forencoding/jsonand fully supports all standard library types, includingjson.RawMessage. Usingsonic.Unmarshalwithjson.RawMessageis the standard pattern throughout this codebase for deferred JSON parsing—there are no compatibility issues or potential bugs from this approach.Likely an incorrect or invalid review comment.
core/schemas/responses.go (2)
232-237: LGTM! Schema additions for reasoning support are well-documented.The
MaxTokensfield addition and updated comment onEffortprovide clear guidance for consumers. The comment noting the Anthropic requirement is helpful for cross-provider compatibility.
323-324: Helpful clarification comment.The inline comment explaining behavioral differences between gpt-oss models and other OpenAI models regarding reasoning content blocks is useful for understanding model-specific behavior.
core/providers/anthropic/errors.go (1)
9-37: Well-implemented error conversion with proper nil safety.The function correctly:
- Guards against nil
bifrostErrinput- Safely extracts optional
Typefield- Safely extracts
Messagefrom nested error (addressing the past review comment)- Returns properly structured
AnthropicMessageErrorwithType: "error"core/providers/openai/chat.go (2)
14-14: Good refactor to use dedicated message conversion helpers.Using
ConvertOpenAIMessagesToBifrostMessagesensures consistent message transformation with proper handling of assistant message fields likeReasoning,Refusal, andToolCalls.
62-65: Correctly updated to use nested Reasoning structure.The filter now properly navigates the nested
Reasoning.Effortfield with appropriate nil checks, and the conversion from "minimal" to "low" aligns with the provider compatibility requirements.core/providers/vertex/vertex.go (4)
283-286: Good error propagation from Anthropic converter.The error handling pattern correctly captures and surfaces conversion errors early, preventing invalid request bodies from being sent to Vertex endpoints.
518-525: Consistent error handling in streaming path.The streaming path now properly handles errors from
ToAnthropicChatRequest. Note the nil check on line 522 (if reqBody != nil) provides additional safety before accessing fields.
689-693: Error propagation added for Responses API path.Errors from
ToAnthropicResponsesRequestare now properly captured and returned, maintaining consistency with the chat completion paths.
846-849: Error propagation added for ResponsesStream path.Completes the error handling improvements across all Vertex Anthropic integration points.
plugins/jsonparser/utils.go (1)
309-314: Field rename fromThoughttoReasoningaligns with schema changes.The shallow copy is appropriate here since the
Reasoningfield is not modified by the JSON parser plugin.core/schemas/mux.go (3)
773-778: Reasoning mapping now preserves MaxTokens in Chat→Responses conversion.The condition
(bcr.Params.Reasoning.Effort != nil || bcr.Params.Reasoning.MaxTokens != nil)ensures that reasoning configuration is mapped when either field is set, addressing the previous review concern about MaxTokens being dropped.
852-858: Complete Reasoning mapping in Responses→Chat conversion.Both
EffortandMaxTokensare now properly propagated fromResponsesParametersReasoningtoChatReasoning, ensuring round-trip consistency.
1361-1372: Streaming delta correctly usesReasoningfield.The streaming conversion now references
delta.Reasoningfor reasoning content, aligning with the renamed field structure.core/providers/anthropic/types.go (1)
12-15: Constant addition for reasoning validation is correct.The
MinimumReasoningMaxTokens = 1024value aligns with Anthropic's officially documented minimum thinking budget requirement. The constant is properly used for validation across Anthropic, Bedrock, and Cohere provider implementations to ensure reasoning.max_tokens meets the minimum threshold.core/providers/openai/utils.go (1)
5-45: LGTM! Conversion utilities correctly omitReasoningDetails.The conversion functions correctly handle the transformation between OpenAI and Bifrost message formats. Based on learnings, OpenAI does not support the
ReasoningDetailsfield, so it's properly omitted from both conversion directions while preserving the basicReasoningtext field that OpenAI does support.core/providers/bedrock/chat.go (3)
38-40: Good error propagation from parameter conversion.The function now correctly propagates errors from
convertChatParametersinstead of silently ignoring failures, improving error visibility for callers.
104-141: Reasoning content processing integrated correctly.The code properly collects
ReasoningContentblocks, accumulates reasoning text, and attaches bothReasoningDetailsandReasoningto the assistant message when present. The consolidation logic at lines 117-120 correctly simplifies single text blocks.
293-346: Streaming reasoning delta handling is correct.The early return at lines 298-300 when both
TextandSignatureare empty prevents the nil pointer issue flagged in previous reviews. The conditional construction at lines 303-344 ensures responses are only created when meaningful data exists.core/providers/bedrock/responses.go (2)
349-367: Reasoning extraction from Bedrock request is correctly implemented.The code properly extracts
reasoning_configfromAdditionalModelRequestFieldsand populatesParams.Reasoningwith the appropriate effort and max tokens based on the configuration type.
656-660: Output conversion is correctly implemented.The code properly converts
response.OutputtobifrostResp.Outputat the appropriate location in the function.core/providers/bedrock/types.go (2)
103-105: LGTM! Reasoning content types added correctly.The new
ReasoningContentfield inBedrockContentBlockand the associatedBedrockReasoningContentandBedrockReasoningContentTexttypes are well-structured and align with the reasoning support being added across the provider layer.Also applies to: 152-159
438-441: Delta structure correctly updated for streaming reasoning.The
BedrockContentBlockDeltanow includesText,ReasoningContent, andToolUsefields, properly supporting streaming of reasoning content deltas alongside existing text and tool use deltas.core/providers/openai/types.go (2)
42-43: LGTM! New message types enable provider-specific handling.The change from
[]schemas.ChatMessageto[]OpenAIMessageand the introduction ofOpenAIChatAssistantMessageprovide the necessary structure for handling OpenAI-specific message fields. The embedded pointer pattern correctly flattens assistant message fields into the top-level JSON object.Also applies to: 56-72
74-101: Custom marshalling correctly transforms reasoning for OpenAI compatibility.The
MarshalJSONimplementation cleverly shadows the embeddedReasoningfield (line 88) to prevent its emission, while extracting and emittingreasoning_effort(lines 96-98) fromReasoning.Effort. This pattern correctly transforms the internal Bifrost reasoning structure to OpenAI's expectedreasoning_effortfield without modifying the in-memory representation.core/providers/azure/azure.go (1)
531-536: The code is correct as written and does not require an additional nil check.
ToAnthropicResponsesRequestfollows Go's standard error-handling pattern: it either returns(nil, error)on all error paths (lines 298, 334, 342 in responses.go) or(*AnthropicMessageRequest, nil)on success (line 407 in responses.go). After theif err != nilcheck at lines 532-533,reqBodyis guaranteed to be non-nil, making the access at line 535 safe. This pattern is correct and idiomatic Go; the comparison to the ChatCompletion path is not applicable, asopenai.ToOpenAIResponsesRequestreturns only a pointer without an error value, requiring a nil check there.core/providers/anthropic/responses.go (1)
1428-1439: Nil-safe extraction for streaming errors looks good.Guarding
bifrostErr.Errorbefore readingMessageprevents nil dereferences inToAnthropicResponsesStreamErrorwhile still emitting a sensible SSE error payload.core/schemas/chatcompletions.go (2)
468-478: Custom unmarshalling for chat messages and reasoning details is sound and preserves invariants.
ChatMessage.UnmarshalJSONfirst decodes base fields, thenChatToolMessage, thenChatAssistantMessage, only attaching the assistant struct when it actually carries data; this avoids the previous interference from the assistant’s custom unmarshaller while keeping the embedded‑struct flattening behavior.ChatAssistantMessage.UnmarshalJSONandChatStreamResponseChoiceDelta.UnmarshalJSONboth synthesize a singleChatReasoningDetails{Type: reasoning.text}entry whenreasoningis present butreasoning_detailsis empty, which gives downstream code a uniform representation without overriding explicitreasoning_detailsfrom providers.Overall this looks correct and matches the UI
ReasoningDetailsshape.Also applies to: 480-521, 625-667, 755-794
714-731: Reasoning details andimage_tokenswiring look consistent with the new schema.The new
BifrostReasoningDetailsTypeconstants andChatReasoningDetailsstruct mirror the UI types and give you space for summary/encrypted variants, while the streaming delta struct now exposes bothreasoningandreasoning_detailsfor incremental updates. AddingImageTokensas an optional field onChatCompletionTokensDetailsis consistent with how other optional token counters (citations, search queries) are modeled.Also applies to: 755-763, 829-836
core/providers/anthropic/chat.go (3)
304-356: Non-stream response aggregation captures text, tools, reasoning, and usage correctly.
ToBifrostChatResponse’s aggregation logic:
- correctly collapses a single text block into
contentStrwhile usingChatContentBlocks for mixed content,- converts
tool_useblocks intoChatAssistantMessageToolCalls with JSON‑marshaled arguments, and- maps
thinkingblocks intoChatReasoningDetailsplus a concatenatedReasoningtext, while wiring Anthropic cache usage fields intoPromptTokensDetails.CachedTokensandCompletionTokensDetails.CachedTokens.Functionally this looks solid for Anthropic’s message API response shape.
Also applies to: 380-386, 413-426
710-741: Chat stream response reasoning mapping looks correct.
ToAnthropicChatStreamResponsenow mapsdelta.Reasoningback into an Anthropicthinkingdelta (Type: AnthropicStreamDeltaTypeThinking, Thinking: delta.Reasoning), which matches whatToBifrostChatCompletionStreamemits on the forward path and preserves reasoning in SSE form.
842-855: ReusingToAnthropicChatCompletionErrorfor SSE errors is a nice consolidation.
ToAnthropicChatStreamErrordelegating toToAnthropicChatCompletionErrorand then wrapping the result as anevent: errorSSE frame centralizes error shaping and avoids duplicating JSON structure here.
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
6c16eed to
1219509
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (6)
core/providers/bedrock/utils.go (1)
32-52:additionalModelRequestFieldPathsstill overwritesreasoning_configin AdditionalModelRequestFields.You correctly populate
AdditionalModelRequestFields["reasoning_config"]based onParams.Reasoning, but the later assignment:if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { bedrockReq.AdditionalModelRequestFields = orderedFields } }replaces the entire map, dropping any previously populated
reasoning_config. For requests that use both reasoning andadditionalModelRequestFieldPaths, Bedrock will never see the reasoning_config you validated.Merge, don’t replace, the map so
reasoning_configsurvives:- if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { - if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields - } - } + if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { + if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = orderedFields + } else { + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } + } + } + }This preserves
reasoning_configwhile still allowing additional fields to be injected.Also applies to: 94-99
core/providers/anthropic/chat.go (1)
431-523: Nil‑guard issue from earlier review is resolved; reasoning → thinking conversion looks safeThis version of
ToAnthropicChatResponsenow consistently checkschoice.ChatNonStreamResponseChoice != nilbefore accessing promoted fields likechoice.StopStringandchoice.Message, which addresses the earlier nil‑dereference risk in this function.The additional reasoning handling:
- Pulls
ReasoningDetailsfromchoice.Message.ChatAssistantMessageand turns"reasoning.text"entries intothinkingcontent blocks with optional signatures.- Leaves plain text + tool calls mapping unchanged.
One optional enhancement: for Bifrost responses that only set
ChatAssistantMessage.Reasoning(and notReasoningDetails), you might also want to emit athinkingblock by synthesizing a singleAnthropicContentBlockTypeThinkingfrom that string, similar to what the JSON unmarshallers do for the reverse direction.core/providers/openai/types.go (1)
42-44: Enforce or centralize the “exactly one payload” invariant onOpenAIMessage.
OpenAIMessagenow embeds both*schemas.ChatToolMessageand*OpenAIChatAssistantMessageand the comment calls out that only one should be non-nil. There’s still no runtime guard, so if a caller accidentally sets both, their fields will flatten into the same JSON object and may conflict.Given this is now the core request type for OpenAI chat, consider adding a small helper to make the invariant explicit:
func (m *OpenAIMessage) Validate() error { if m.ChatToolMessage != nil && m.OpenAIChatAssistantMessage != nil { return fmt.Errorf("OpenAIMessage cannot have both ChatToolMessage and OpenAIChatAssistantMessage set") } return nil }and then have the request-building code call
Validate()before marshalling, or expose constructors (e.g.NewToolMessage,NewAssistantMessage) that only set one of the embedded pointers.The rest of the struct layout and the
MarshalJSONlogic forreasoning_effortlook reasonable.Also applies to: 56-72
core/providers/cohere/chat.go (1)
104-121: Avoid silently overriding Reasoning-derivedThinkingwith ExtraParams["thinking"].Here you:
- First derive
cohereReq.Thinkingfrom the canonicalbifrostReq.Params.Reasoning(with validation and Anthropic-style minimum token checks).- Then, if
ExtraParams["thinking"]is present, you always parse it and assign a newCohereThinking, overwriting the Reasoning-based one.That means if both are supplied, the looser ExtraParams path silently wins over the validated Reasoning path.
Given Reasoning is the new unified field, consider only reading ExtraParams when
cohereReq.Thinkingis still nil, e.g.:- if bifrostReq.Params.ExtraParams != nil { - // Handle thinking parameter - if thinkingParam, ok := schemas.SafeExtractFromMap(bifrostReq.Params.ExtraParams, "thinking"); ok { + if bifrostReq.Params.ExtraParams != nil { + // Handle legacy thinking parameter only if Reasoning didn't already set Thinking + if cohereReq.Thinking == nil { + if thinkingParam, ok := schemas.SafeExtractFromMap(bifrostReq.Params.ExtraParams, "thinking"); ok { if thinkingMap, ok := thinkingParam.(map[string]interface{}); ok { thinking := &CohereThinking{} @@ - cohereReq.Thinking = thinking + cohereReq.Thinking = thinking } }This keeps ExtraParams as a backward-compatibility shim without weakening the new Reasoning-based validation.
Also applies to: 128-142
core/providers/bedrock/responses.go (1)
347-367: Fixreasoning_configtype mismatch and avoid clobbering it when applyingadditionalModelRequestFieldPaths.Two issues here:
ToBifrostResponsesRequestexpectsreasoning_configasmap[string]interface{}, but the disabled case inToBedrockResponsesRequestwritesmap[string]string. That type assertion will always fail, so Bedrock requests withtype: "disabled"never populateParams.Reasoningcorrectly on the way back.- When
additionalModelRequestFieldPathsis present, you replacebedrockReq.AdditionalModelRequestFieldswholesale withorderedFields, which discards any previously-setreasoning_config.Suggest:
- Use
map[string]anyfor both enabled and disabled configs.- Merge
orderedFieldsinto any existingAdditionalModelRequestFieldsrather than replacing it.@@ - // Convert additional model request fields to extra params + // Convert additional model request fields (currently only reasoning_config) to params if len(request.AdditionalModelRequestFields) > 0 { @@ if bifrostReq.Params.Reasoning != nil { if bedrockReq.AdditionalModelRequestFields == nil { bedrockReq.AdditionalModelRequestFields = make(schemas.OrderedMap) } if bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort == "none" { - bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ - "type": "disabled", - } + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ + "type": "disabled", + } } else { @@ - bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ "type": "enabled", "budget_tokens": *bifrostReq.Params.Reasoning.MaxTokens, } @@ - if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { - if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields - } - } + if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { + if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = orderedFields + } else { + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } + } + } + }Also applies to: 452-472, 474-482
core/providers/anthropic/responses.go (1)
296-303: Unify reasoning enablement semantics between Responses and Chat converters.
ToAnthropicResponsesRequestcurrently:
- Treats reasoning as enabled only when
Reasoning.Effort != nil && *Effort != "none"andMaxTokensis set/valid.- Treats
Effort == nil(with non-nil Reasoning) as disabled.But
ToAnthropicChatRequest(incore/providers/anthropic/chat.go) treats reasoning as enabled for any Reasoning whereEffort == nilorEffort != "none", and only disables whenEffort != nil && *Effort == "none".So the same
Reasoning{Effort: nil, MaxTokens: 2048}payload enables reasoning on Chat but disables it on Responses. That’s surprising for callers sharing a common schema.Consider aligning both paths on a single rule, e.g.:
- Disabled only when
Effort != nil && *Effort == "none"andMaxTokens == nil.- Enabled whenever
MaxTokens != nilor (Effort != nil && *Effort != "none"), and in the enabled case always require/validateMaxTokens >= MinimumReasoningMaxTokens.Ideally factor this into a shared helper used by both
ToAnthropicChatRequestandToAnthropicResponsesRequestso validation and error messages stay identical.Also applies to: 325-344, 402-402
🧹 Nitpick comments (6)
plugins/semanticcache/utils.go (1)
669-671: Reasoning metadata extraction looks consistent and nil‑safe.Using
params.Reasoningwith nested nil‑checks forEffortandMaxTokensaligns with the new Reasoning struct and avoids panics, while ensuring Responses cache keys now distinguish both effort and reasoning token budgets. If Chat’sReasoning.MaxTokensis also user‑visible and affects behavior, you may eventually want to mirror thereasoning_max_tokensextraction there for symmetry, but it’s not blocking for this PR.Also applies to: 752-754
core/providers/anthropic/chat.go (3)
13-118: Anthropic reasoning: strict max_tokens requirement may be too breaking when only reasoning_effort is setThe new reasoning block in
ToAnthropicChatRequestlooks consistent with Anthropic’s “thinking” API, and the nil checks +MinimumReasoningMaxTokensguard are good.However, combined with
ChatParameters.UnmarshalJSON, any request that specifies onlyreasoning_effort(e.g."medium") and targets Anthropic will now fail withreasoning.max_tokens is required for reasoning, whereas previously that extra field would effectively be ignored for Anthropic.That’s a behavior change from “ignored hint” → “hard error” for existing clients:
params.reasoning_effort = "medium"→cp.Reasoning.Effort = "medium", MaxTokens = nil- Anthropic path now errors unless
Reasoning.Effort == "none"orReasoning.MaxTokensis set and ≥MinimumReasoningMaxTokens.Please double-check that this stricter requirement is acceptable for Anthropic callers and clearly documented, or consider a softer fallback (e.g. defaulting
MaxTokenswhen only effort is present, or ignoring reasoning for Anthropic in that case rather than erroring).Also applies to: 207-216
280-429: Bifrost → Anthropic response mapping mostly looks good; consider settingObjectand reusing provider timestampsThe
ToBifrostChatResponseimplementation is well-structured: it correctly aggregates text, tool calls, and thinking blocks into a single assistant message, and maps cache read/create tokens intoPromptTokensDetails/CompletionTokensDetailsconsistently.Two non‑blocking improvements to consider:
Set
Objectexplicitly
For non‑stream Anthropic responses we usually wantObject: "chat.completion"onBifrostChatResponseto match other providers and the documented shape.Prefer provider timestamp if available
Createdis currentlytime.Now().Unix(). IfAnthropicMessageResponsecarries a created timestamp, using that instead would make logs/latency calculation more accurate.Neither is a correctness bug, but aligning these would make Anthropic responses more consistent with the rest of Bifrost.
710-741: Bifrost → Anthropic streaming drops signature-only reasoning details
ToAnthropicChatStreamResponsemapsdelta.Contentanddelta.Reasoningback into Anthropictextandthinkingdeltas, which is good. However, any information present only indelta.ReasoningDetails(for example, a signature chunk created byAnthropicStreamDeltaTypeSignatureon the way into Bifrost) is currently not emitted on the way back out.If this reverse path is expected to carry signatures (e.g., for an Anthropic‑compatible proxy endpoint), consider:
- Detecting a
ReasoningDetailsentry that has aSignaturebut noText, and- Emitting an Anthropic
content_block_deltaof typesignature(or the appropriate delta type) in that case.That would make the streaming conversion symmetric with the
ToBifrostChatCompletionStreamlogic.core/schemas/chatcompletions.go (1)
480-521: ChatMessage custom unmarshal correctly reconstructs flattened assistant/tool shapesThe new
ChatMessage.UnmarshalJSON:
- Separately unmarshals base fields (
name,role,content),- Unmarshals
ChatToolMessageto detecttool_call_id, and- Unmarshals
ChatAssistantMessage(including its own custom logic) and only attaches it when any relevant field is populated,is a reasonable workaround for the embedded pointer + custom unmarshal combination.
Triple‑unmarshalling is a bit heavier but acceptable given the complexity, and should preserve existing shapes for all roles (assistant/user/tool/system) without recursion issues.
core/providers/cohere/chat.go (1)
564-683:CohereMessage.ToBifrostChatMessageconversion looks correct; minor optional micro-optimization.Functionally:
- Handles string vs blocks content, including text and image blocks.
- Aggregates
thinkingblocks intoChatReasoningDetailsand a newline-joinedReasoningstring.- Converts tool calls with robust nil checks on
FunctionandFunction.Name.Only minor nit: you call
cm.Content.GetBlocks()multiple times inside the same branch. Caching once at the top of the blocks path would avoid repeated work:- } else if cm.Content.IsBlocks() { - for _, block := range cm.Content.GetBlocks() { + } else if cm.Content.IsBlocks() { + blocks := cm.Content.GetBlocks() + for _, block := range blocks {Not critical, but it slightly simplifies the loop.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (30)
core/providers/anthropic/anthropic.go(4 hunks)core/providers/anthropic/chat.go(11 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(5 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(4 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/semanticcache/utils.go(2 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (9)
- plugins/maxim/main.go
- ui/app/workspace/logs/views/logChatMessageView.tsx
- core/providers/azure/azure.go
- core/providers/anthropic/types.go
- core/providers/openai/utils.go
- core/providers/openai/openai.go
- core/providers/anthropic/errors.go
- transports/bifrost-http/handlers/inference.go
- core/providers/cohere/cohere.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
core/providers/cohere/types.goframework/streaming/chat.gocore/providers/openai/chat.goplugins/semanticcache/utils.gocore/providers/bedrock/utils.goplugins/jsonparser/utils.gocore/providers/perplexity/chat.goui/lib/types/logs.tscore/providers/anthropic/responses.gocore/schemas/mux.gocore/providers/openai/types.gocore/providers/bedrock/chat.gocore/schemas/responses.gocore/providers/anthropic/anthropic.gocore/providers/vertex/vertex.gocore/providers/cohere/chat.gocore/schemas/chatcompletions.gocore/providers/bedrock/bedrock.gocore/providers/bedrock/types.gocore/providers/bedrock/responses.gocore/providers/anthropic/chat.go
🧠 Learnings (2)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
core/providers/cohere/types.goframework/streaming/chat.gocore/providers/openai/chat.goplugins/semanticcache/utils.gocore/providers/bedrock/utils.goplugins/jsonparser/utils.gocore/providers/perplexity/chat.gocore/providers/anthropic/responses.gocore/schemas/mux.gocore/providers/openai/types.gocore/providers/bedrock/chat.gocore/schemas/responses.gocore/providers/anthropic/anthropic.gocore/providers/vertex/vertex.gocore/providers/cohere/chat.gocore/schemas/chatcompletions.gocore/providers/bedrock/bedrock.gocore/providers/bedrock/types.gocore/providers/bedrock/responses.gocore/providers/anthropic/chat.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/chat.gocore/providers/openai/types.go
🧬 Code graph analysis (8)
framework/streaming/chat.go (4)
core/schemas/chatcompletions.go (1)
ChatAssistantMessage(626-632)core/utils.go (1)
Ptr(51-53)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)core/providers/gemini/types.go (1)
Type(778-778)
core/providers/openai/chat.go (4)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)
core/providers/bedrock/utils.go (3)
core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)OrderedMap(268-268)core/providers/bedrock/types.go (2)
BedrockConverseRequest(48-62)BedrockError(390-394)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/schemas/mux.go (3)
core/schemas/responses.go (3)
ResponsesParametersReasoning(233-238)BifrostResponsesStreamResponse(1425-1463)ResponsesStreamResponseTypeReasoningSummaryTextDelta(1391-1391)core/schemas/chatcompletions.go (1)
ChatReasoning(223-226)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/anthropic/anthropic.go (4)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/vertex/vertex.go (5)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/schemas/models.go (1)
Model(109-129)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/schemas/chatcompletions.go (3)
core/providers/openai/types.go (1)
Alias(78-78)core/schemas/bifrost.go (2)
Alias(384-384)Alias(401-401)ui/lib/types/logs.ts (3)
ChatMessage(115-124)ChatMessageContent(113-113)ReasoningDetails(126-133)
core/providers/bedrock/responses.go (5)
core/schemas/utils.go (4)
SafeExtractFromMap(519-525)SafeExtractString(280-297)SafeExtractInt(300-342)Ptr(16-18)core/schemas/responses.go (1)
ResponsesParametersReasoning(233-238)ui/lib/types/logs.ts (1)
ResponsesParametersReasoning(519-526)core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (27)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (32)
core/providers/perplexity/chat.go (1)
30-36: LGTM - Reasoning effort mapping correctly updated.The nil checks are properly ordered to avoid nil pointer dereference, and the "minimal" → "low" mapping aligns with Perplexity's supported effort values. Note that
Reasoning.MaxTokensis not propagated to Perplexity, which appears intentional since Perplexity's reasoning configuration uses only effort levels.core/providers/openai/chat.go (2)
14-14: LGTM - Message conversion uses new utility functions.The change to use
ConvertOpenAIMessagesToBifrostMessagesproperly preserves OpenAI-specific assistant message fields (Refusal, Reasoning, Annotations, ToolCalls) during conversion, as shown in the relevant code snippet fromutils.go.
63-65: LGTM - Reasoning effort mapping correctly updated.The triple nil-check pattern (
Reasoning != nil && Reasoning.Effort != nil && *Effort == "minimal") safely handles the nested structure. The "minimal" → "low" mapping for non-OpenAI providers is consistent with the Perplexity provider implementation.core/schemas/responses.go (2)
233-238: LGTM - ResponsesParametersReasoning correctly extended.The addition of
MaxTokensfield aligns withChatReasoninginchatcompletions.go, enabling consistent reasoning configuration across Chat and Responses APIs. The comment update onEffortclarifies the enabling behavior.
324-325: LGTM - Helpful behavioral documentation added.The comment clarifies the difference between gpt-oss models (reasoning_text blocks only) and other OpenAI models (summaries + encrypted_content), which is valuable context for consumers of this struct.
core/schemas/mux.go (3)
773-778: LGTM - MaxTokens now properly preserved in Chat → Responses conversion.The condition
(bcr.Params.Reasoning.Effort != nil || bcr.Params.Reasoning.MaxTokens != nil)correctly addresses the previously flagged issue whereMaxTokenswas being dropped. Both fields are now propagated toResponsesParametersReasoning.
852-858: LGTM - MaxTokens properly preserved in Responses → Chat conversion.The conversion now maps both
EffortandMaxTokensfields, ensuring round-trip preservation of reasoning configuration. CreatingChatReasoningwheneverbrr.Params.Reasoningis non-nil is appropriate for maintaining structural symmetry.
1361-1372: LGTM - Streaming delta correctly uses Reasoning field.The change from
delta.Thoughttodelta.Reasoningaligns with the broader schema rename. TheResponsesStreamResponseTypeReasoningSummaryTextDeltaevent type correctly represents reasoning content in the Responses streaming format.core/providers/cohere/types.go (1)
535-539: LGTM - Thinking field added for streaming reasoning support.The optional
Thinkingfield enables Cohere streaming to capture thinking/reasoning deltas. Using Cohere's terminology ("Thinking") for the provider-specific struct is appropriate, as the conversion layer can map this to the unifiedReasoningfield in Bifrost schemas.plugins/jsonparser/utils.go (1)
309-314: LGTM - Field rename from Thought to Reasoning.The shallow copy of
Reasoningis appropriate since the JSON parser plugin modifies onlyContent, not reasoning data. This change aligns with the broader schema rename fromThoughttoReasoninginChatStreamResponseChoiceDelta.core/providers/bedrock/bedrock.go (1)
21-21: Stable stream IDs for Bedrock chat streaming are wired correctly.Generating a single UUID per Bedrock stream and reusing it for all chunks (including the final synthesized chunk) provides consistent IDs without introducing shared state, which matches expectations from other providers’ streaming behavior.
Also applies to: 771-773, 859-869, 882-885
framework/streaming/chat.go (1)
39-43: Reasoning and reasoning_details accumulation from chat stream chunks looks correct.The accumulator now safely initializes
ChatAssistantMessage, concatenatesReasoningtext across chunks, and mergesReasoningDetailsby index (accumulating text/summary/data and overwriting signature/type when provided), which is consistent with how deltas are emitted and should reconstruct a coherent final assistant message.Also applies to: 44-55, 56-105
core/providers/anthropic/anthropic.go (1)
303-311: Anthropic request conversion errors are now correctly propagated.Hooking
ToAnthropicChatRequest/ToAnthropicResponsesRequestintoCheckContextAndGetRequestBodyas real error‑returning closures ensures invalid inputs (e.g., bad reasoning config) surface asBifrostErrors instead of silently constructing bad request bodies, for both streaming and non‑streaming flows.Also applies to: 357-364, 656-663, 707-714
core/providers/bedrock/utils.go (1)
702-710: Bedrock error message extraction is safe and simplified.Copying
bifrostErr.Error.Messageinto a localmessage(with existing nil‑guard above) before constructingBedrockErroravoids any nested dereference issues and centralizes how the Bedrock error message is derived, without changing type/code mapping.core/providers/bedrock/types.go (1)
88-113: Reasoning content types for Bedrock are well‑structured and align with usage.Adding
ReasoningContenttoBedrockContentBlockplus theBedrockReasoningContent*structs, and mirroring that with aReasoningContentfield onBedrockContentBlockDelta, gives both non‑streaming and streaming paths a clear place to carry reasoning text + optional signature, which matches howToBifrostChatResponseand the streaming converter consume these fields.Also applies to: 157-164, 443-447
ui/lib/types/logs.ts (1)
115-124: Log types now expose reasoning content consistently with backend.Extending
ChatMessagewithreasoningandreasoning_detailsplus theReasoningDetailsinterface gives the UI enough structure to render both plain reasoning text and richer per‑segment metadata (summary/text/signature/data) coming from providers, without breaking existing tooling that reads ChatMessage.Also applies to: 126-133
core/providers/bedrock/chat.go (2)
24-41: Bedrock chat parameter conversion errors are now surfaced correctly.Wrapping
convertChatParametersand returning a contextual error when it fails ensures issues like invalid reasoning config (e.g., missing or too‑smallmax_tokens) stop the Bedrock call early instead of producing malformed requests.
59-61: Bedrock reasoning extraction and streaming deltas look correct and robust.Non‑streaming: you now safely handle
ReasoningContentby checkingReasoningTextfor nil, aggregatingreasoningDetailsandreasoningText, and attaching both toChatAssistantMessagealongside any tool calls, with a reasonable collapse to a single text block when applicable.Streaming: the
ReasoningContentdelta branch avoids returning a nil response pointer (early exit when both text and signature are absent) and only constructs a chunk when text and/or signature are present, emitting them viaReasoningandReasoningDetailsin a way that matches the accumulator logic inframework/streaming/chat.go.Also applies to: 121-133, 139-143, 157-163, 335-388
core/providers/anthropic/chat.go (2)
633-645: Streaming thinking/signature deltas: structure is consistent with new ReasoningDetails modelThe handling for
AnthropicStreamDeltaTypeThinkingandAnthropicStreamDeltaTypeSignaturecleanly maps Anthropic thinking/signature chunks into:
Delta.Reasoning+ a singleReasoningDetailsentry (type"reasoning.text") for thinking deltas.- A single
ReasoningDetailsentry carrying onlySignaturefor signature deltas.This matches the new
ChatReasoningDetailsschema, and should give downstreams both a simplereasoningstring (for thinking) and richer per‑chunk metadata.No changes requested here.
Also applies to: 655-677
842-855: Streaming error conversion is straightforward and consistent
ToAnthropicChatStreamErrordelegates toToAnthropicChatCompletionErrorand then wraps the JSON in an SSEevent: errorframe. The behavior is clear and aligned with the rest of the Anthropic SSE helpers.No issues from my side.
core/schemas/chatcompletions.go (5)
155-184: ChatParameters reasoning +reasoning_effortaliasing looks soundThe introduction of
Reasoning *ChatReasoningonChatParametersand the customUnmarshalJSONthat:
- Parses a legacy
reasoning_effortstring,- Rejects payloads that specify both
reasoningandreasoning_effort, and- Synthesizes
cp.Reasoningfromreasoning_effortwhen present,is a clean way to keep backward compatibility while moving to the richer
reasoningobject.The alias pattern with
sonicis idiomatic and avoids recursion; no structural issues spotted here.Also applies to: 186-220, 223-226
626-667: ChatAssistantMessage reasoning → ReasoningDetails synthesis is consistentThe updated
ChatAssistantMessageplusUnmarshalJSON:
- Adds first‑class
ReasoningandReasoningDetailsfields, and- Automatically synthesizes a single
"reasoning.text"ChatReasoningDetailsentry whenReasoningis set andReasoningDetailsis empty,nicely normalizes older “flat string” reasoning into the new structured representation.
This lines up with how the Anthropic adapter consumes
ReasoningDetails, and should make downstream UI and logging much easier to keep consistent.
714-731: ReasoningDetails type and constants align with UI expectations
BifrostReasoningDetailsTypeandChatReasoningDetails(withsummary,text,signature,data) match the UI’sReasoningDetailscontract (reasoning.summary/reasoning.encrypted/reasoning.text).This gives you enough flexibility to carry summaries, encrypted payloads, and signed reasoning without overloading a single string field. No issues here.
756-794: Streaming delta reasoning synthesis mirrors assistant message behaviorThe extended
ChatStreamResponseChoiceDeltaplus its customUnmarshalJSONapply the same pattern asChatAssistantMessage:
- Accepts both a flat
reasoningstring andreasoning_details,- When only
reasoningis present, synthesizes a single"reasoning.text"detail.That keeps streaming and non‑streaming representations aligned and ensures downstreams can always rely on
ReasoningDetailsbeing populated in at least the simple case.Looks good.
829-840: ImageTokens addition is fine; ensure providers populate itAdding
ImageTokens *inttoChatCompletionTokensDetailsis straightforward and matches how other token sub‑counters are modeled.Follow‑up (outside this file): double‑check that provider adapters that expose image token usage (e.g., OpenAI vision or other multimodal providers) are wired to set this field so it doesn’t remain permanently nil.
core/providers/bedrock/responses.go (1)
652-666: Output → Responses mapping looks correct.Mapping
response.Output.MessagethroughconvertBedrockMessageToResponsesMessagesintobifrostResp.Outputis straightforward and symmetric with the existing helpers; no issues spotted.core/providers/anthropic/responses.go (2)
190-201: Thinking → Params.Reasoning mapping is consistent.Deriving
Params.Reasoningfromrequest.Thinking(auto + budget_tokens vs none) while also keepingExtraParams["thinking"]preserves backwards compatibility and aligns Responses with the Anthropic thinking model.
1428-1439: Safer streaming error marshalling looks good.Extracting
messagedefensively frombifrostErr.Errorbefore building the Anthropic SSE error avoids nil dereferences and preserves any upstream error text.core/providers/vertex/vertex.go (1)
283-290: Anthropic conversion/error handling on Vertex paths looks consistent and safe.All Anthropic branches now:
- Delegate request shaping to
anthropic.ToAnthropicChatRequest/ToAnthropicResponsesRequestwith proper error propagation.- Only touch
reqBody.Model/reqBody.Streamafter the conversion, with an explicit nil guard in the non-stream Chat path.Given the updated converters always return a non-nil request when
err == nil, these sites won’t marshal anullbody anymore. No additional defensive checks seem necessary.Also applies to: 518-525, 689-695, 847-854
core/providers/openai/types.go (1)
74-101: Custom marshalling forreasoning_effortlooks correct.The alias + aux pattern cleanly:
- Suppresses the embedded
Reasoningfield in JSON, and- Emits
reasoning_effortwhenr.Reasoning != nil && r.Reasoning.Effort != nil.This preserves the legacy OpenAI wire format while allowing callers to use the new
ChatReasoningfield.core/providers/cohere/chat.go (2)
219-262: CohereThinking ↔ unified Reasoning + ExtraParams mapping looks good.
ToBifrostChatRequestmapsreq.ThinkingtoParams.Reasoning(Effort:"none"for disabled,"auto"plusMaxTokensfor enabled) and also mirrors it intoExtraParams["thinking"].- This matches the write side in
ToCohereChatCompletionRequestand keeps both the new Reasoning field and the legacythinkingparam available to callers.No functional issues spotted here.
Also applies to: 300-322
409-454: Streaming reasoning/thinking deltas are mapped appropriately.
StreamEventContentDeltanow distinguishes between text (Content...Text) and thinking (...Thinking), emitting:
Delta.Contentfor text chunks, andDelta.Reasoningplus aReasoningDetailsentry for thinking chunks.StreamEventToolPlanDeltasurfaces tool plans throughDelta.Reasoning, which is a reasonable choice for plan-like explanations.This aligns well with the unified streaming reasoning model and should play nicely with downstream consumers.
Also applies to: 457-467
TejasGhatte
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
982bc60 to
0f9320e
Compare
TejasGhatte
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
727dbd8 to
785a11f
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/providers/bedrock/utils.go (1)
94-98: Reasoning config can still be overwritten by ExtraParams handling.The reasoning_config set at lines 38-50 will be completely replaced if
additionalModelRequestFieldPathsexists in ExtraParams (line 97). This was flagged in a previous review but appears not to be fully addressed.Consider merging instead of replacing:
if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = orderedFields + } else { + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } + } } }core/providers/bedrock/responses.go (1)
478-481: Reasoning config can be overwritten by additionalModelRequestFieldPaths.Same issue as in
utils.go: the reasoning_config set at lines 453-471 will be lost ifadditionalModelRequestFieldPathsexists in ExtraParams. Apply the same merge pattern suggested forutils.go.
♻️ Duplicate comments (7)
core/providers/bedrock/utils.go (1)
10-10: Consider extractingMinimumReasoningMaxTokensto a shared location.Importing
anthropicjust for this constant creates a cross-provider dependency. This constant is used by Bedrock, Cohere, and Anthropic providers. Moving it tocore/schemasor a shared constants package would improve modularity.core/providers/bedrock/responses.go (1)
456-459: Type mismatch for disabled reasoning config.Line 457 uses
map[string]string{"type": "disabled"}but the reader at line 351 expectsmap[string]interface{}. This type assertion will fail when processing Bedrock responses with disabled reasoning.if bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort == "none" { - bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ "type": "disabled", }core/providers/anthropic/responses.go (1)
325-344: Reasoning enablement semantics still differ from Chat converter.
This treatsReasoning.Effort == nilas “disabled” (falls into theelse), whilecore/providers/anthropic/chat.gotreatsEffort == nilas enabled (unless explicitly"none") and requiresMaxTokens. This makes identical payloads behave differently between/responsesvs/chat/completions.core/providers/anthropic/chat.go (1)
415-426: Cache token mapping may still be semantically off (cache-creation is input-side).
CacheCreationInputTokensis being stored underCompletionTokensDetails.CachedTokens; if those tokens represent cache creation (prompt-side work), they likely belong with prompt/input cached tokens instead.core/providers/vertex/vertex.go (3)
518-525: Fix nilreqBodyhandling in Anthropic ChatCompletionStream (can panic on map write).
Ifanthropic.ToAnthropicChatRequest(request)returns(nil, nil),sonic.Unmarshallikely yields a nilmap, andrequestBody["anthropic_version"] = ...will panic; plus you’d send an invalid body. Return an error whenreqBody == niland only proceed after setting required fields.reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("chat completion input is not provided") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true)
847-854: Fix nilreqBodyhandling in Anthropic ResponsesStream (can panic on map write).
Same nil-body failure mode as above; fail fast and set fields unconditionally after the nil-check.reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("responses input is not provided") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true)
689-695: Fix nilreqBodyhandling in Anthropic Responses (can panic on map write).
Same issue:(nil, nil)conversion can later panic when settinganthropic_versionon a nil map (and would send an invalid request).reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - } + if reqBody == nil { + return nil, fmt.Errorf("responses input is not provided") + } + reqBody.Model = deployment
🧹 Nitpick comments (2)
core/schemas/chatcompletions.go (1)
835-835: Document the ImageTokens field usage.The new
ImageTokensfield is added without accompanying context. Consider adding a comment explaining when this field is populated and which providers support it, similar to the existing documentation for other token detail fields.core/providers/anthropic/responses.go (1)
190-201: Potential double-source for “thinking” config (ExtraParams + Params.Reasoning).
ToBifrostResponsesRequestwritesparams.ExtraParams["thinking"](Line 184-186) and also derivesparams.Reasoningfromrequest.Thinking(Line 190-201). If both are intended for backward compatibility, consider a short comment clarifying precedence to avoid consumers diverging.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (30)
core/providers/anthropic/anthropic.go(4 hunks)core/providers/anthropic/chat.go(11 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(5 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(4 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/semanticcache/utils.go(2 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (13)
- framework/streaming/chat.go
- core/providers/openai/utils.go
- core/providers/anthropic/types.go
- plugins/semanticcache/utils.go
- plugins/maxim/main.go
- core/providers/openai/chat.go
- core/schemas/responses.go
- core/providers/openai/openai.go
- core/providers/cohere/cohere.go
- plugins/jsonparser/utils.go
- core/providers/bedrock/bedrock.go
- core/providers/cohere/types.go
- ui/lib/types/logs.ts
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/handlers/inference.gocore/providers/bedrock/utils.gocore/providers/vertex/vertex.gocore/providers/anthropic/errors.gocore/providers/azure/azure.gocore/providers/bedrock/types.gocore/providers/bedrock/chat.gocore/providers/anthropic/responses.gocore/schemas/mux.gocore/providers/anthropic/anthropic.goui/app/workspace/logs/views/logChatMessageView.tsxcore/providers/cohere/chat.gocore/providers/perplexity/chat.gocore/providers/openai/types.gocore/providers/bedrock/responses.gocore/schemas/chatcompletions.gocore/providers/anthropic/chat.go
🧠 Learnings (2)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/inference.gocore/providers/bedrock/utils.gocore/providers/vertex/vertex.gocore/providers/anthropic/errors.gocore/providers/azure/azure.gocore/providers/bedrock/types.gocore/providers/bedrock/chat.gocore/providers/anthropic/responses.gocore/schemas/mux.gocore/providers/anthropic/anthropic.gocore/providers/cohere/chat.gocore/providers/perplexity/chat.gocore/providers/openai/types.gocore/providers/bedrock/responses.gocore/schemas/chatcompletions.gocore/providers/anthropic/chat.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/types.go
🧬 Code graph analysis (10)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/chatcompletions.go (2)
ChatMessage(469-478)ChatParameters(155-184)ui/lib/types/logs.ts (1)
ChatMessage(115-124)
core/providers/bedrock/utils.go (3)
core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)OrderedMap(268-268)core/providers/bedrock/types.go (2)
BedrockConverseRequest(48-62)BedrockError(390-394)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/vertex/vertex.go (5)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/schemas/models.go (1)
Model(109-129)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/anthropic/errors.go (2)
ui/lib/types/logs.ts (1)
BifrostError(233-239)core/providers/anthropic/types.go (2)
AnthropicMessageError(376-379)AnthropicMessageErrorStruct(382-385)
core/providers/azure/azure.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/bedrock/chat.go (2)
core/schemas/chatcompletions.go (4)
ChatReasoningDetails(723-730)BifrostReasoningDetailsTypeText(719-719)ChatContentBlockTypeText(582-582)ChatAssistantMessage(626-632)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)
core/providers/anthropic/responses.go (3)
core/schemas/responses.go (2)
ResponsesParametersReasoning(233-238)BifrostResponsesRequest(32-39)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/types.go (4)
AnthropicMessageRequest(40-59)MinimumReasoningMaxTokens(14-14)AnthropicThinking(65-68)AnthropicStreamError(397-400)
core/schemas/mux.go (3)
core/schemas/responses.go (3)
ResponsesParametersReasoning(233-238)BifrostResponsesStreamResponse(1425-1463)ResponsesStreamResponseTypeReasoningSummaryTextDelta(1391-1391)core/schemas/chatcompletions.go (1)
ChatReasoning(223-226)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/anthropic/anthropic.go (4)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/openai/types.go (2)
core/schemas/models.go (1)
Model(109-129)core/schemas/chatcompletions.go (10)
ChatMessageRole(457-457)ChatMessageContent(524-527)ChatToolMessage(621-623)ChatAssistantMessageAnnotation(670-673)ChatAssistantMessageToolCall(686-692)Alias(189-189)Alias(643-643)Alias(770-770)Alias(860-860)ChatReasoning(223-226)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (34)
ui/app/workspace/logs/views/logChatMessageView.tsx (1)
73-94: LGTM! Clean refactor from thought to reasoning.The changes correctly replace all references from
message.thoughttomessage.reasoningwith consistent rendering logic for both JSON and non-JSON content. The italic styling applied to reasoning text (line 89) effectively differentiates it from regular message content.core/providers/bedrock/types.go (1)
108-110: LGTM! Reasoning content types follow established patterns.The new reasoning content types mirror the existing guard content structure and integrate cleanly into both the content block and delta representations. The nullable pointers and omitempty tags are consistent with Bedrock's API conventions.
Also applies to: 157-164, 444-446
core/schemas/chatcompletions.go (1)
634-667: Reasoning details synthesis is consistent and well-designed.Both
ChatAssistantMessage.UnmarshalJSONandChatStreamResponseChoiceDelta.UnmarshalJSONapply the same synthesis logic: whenReasoningis non-nil butReasoningDetailsis empty, they automatically create a singlereasoning.textentry. This provides a unified interface for consumers while supporting both simple and detailed reasoning representations.Also applies to: 765-794
transports/bifrost-http/handlers/inference.go (2)
169-199: Custom unmarshaller correctly handles nested custom unmarshallers.The
ChatRequest.UnmarshalJSONmethod properly coordinates three separate unmarshal operations to avoid conflicts whenChatParametershas its own custom unmarshaller. The pattern of unmarshalling embedded structs separately and then combining them is the correct approach.
476-478: Good defensive initialization before unmarshalling.Initializing
ChatParametersto a non-nil value before callingsonic.Unmarshalensures the customChatParameters.UnmarshalJSONmethod has a valid target. This prevents potential nil pointer issues.core/providers/openai/types.go (1)
74-101: MarshalJSON correctly transforms reasoning structure.The custom marshaller properly:
- Shadows the
Reasoningfield to prevent it from being emitted- Safely extracts
Reasoning.Effortwith nil checks (line 96)- Emits
reasoning_effortinstead of the nestedreasoningobjectThis transformation is correct for providers that expect the flat
reasoning_effortfield.core/providers/cohere/chat.go (3)
104-121: Reasoning validation enforces minimum token requirements.The code correctly:
- Handles the special "none" effort case by setting
ThinkingTypeDisabled- Validates that
Reasoning.MaxTokensis present and meets the minimum threshold- References
anthropic.MinimumReasoningMaxTokens(1024) as a cross-provider standardThis ensures consistent reasoning behavior across providers.
431-454: Reasoning content properly synthesized in both streaming and non-streaming paths.Both paths correctly:
- Create
ReasoningDetailsentries with typeBifrostReasoningDetailsTypeText- Populate the
Reasoningfield with concatenated text- Handle nil-safety when accessing thinking content
The streaming path (lines 431-454) creates a single detail entry with the delta, while the non-streaming path (lines 600-609) aggregates thinking blocks with newlines (as addressed in the past review).
Also applies to: 600-609
564-683: Per-message conversion provides clean separation of concerns.The new
ToBifrostChatMessagemethod centralizes conversion logic with proper:
- Content type handling (string vs blocks)
- Tool call extraction with nil-safety (lines 630-633)
- Reasoning details synthesis from thinking blocks
- Assistant message construction when needed
This refactoring improves maintainability and reduces duplication.
core/providers/perplexity/chat.go (1)
30-36: LGTM! Reasoning field access updated with proper nil-safety.The code correctly navigates the nested
Reasoning.Effortpath with appropriate nil checks and maintains the special case mapping of "minimal" to "low" for Perplexity.core/providers/anthropic/errors.go (2)
9-37: Excellent nil-safety in error conversion.The function defensively handles all potential nil pointers:
- Input
bifrostErr(line 11)- Nested
bifrostErr.Type(lines 16-19)- Nested
bifrostErr.Error(lines 22-25)This ensures the converter never panics and always produces a valid
AnthropicMessageError, addressing the concerns from the past review.
42-48: Safe nested error population.The code ensures
bifrostErr.Erroris non-nil before populating its fields, preventing any nil pointer dereferences during error parsing.core/providers/anthropic/anthropic.go (2)
307-307: Proper error propagation added for chat completions.Both the non-streaming (line 307) and streaming (lines 358-364) paths now correctly handle errors from
ToAnthropicChatRequest. The streaming path checks for errors before enabling streaming, ensuring invalid requests fail fast.Also applies to: 358-364
659-659: Proper error propagation added for responses.Both the non-streaming (line 659) and streaming (lines 708-714) paths now correctly handle errors from
ToAnthropicResponsesRequest. This aligns with the chat completion path and ensures consistent error handling across all request types.Also applies to: 708-714
core/providers/azure/azure.go (4)
353-360: LGTM! Error handling correctly propagated for Anthropic chat requests.The conversion function now returns an error which is properly checked before proceeding. The nil check on line 357 provides additional safety.
452-460: LGTM! Consistent error handling in streaming path.The streaming path mirrors the non-streaming implementation with proper error propagation and nil safety checks.
531-538: LGTM! Responses API follows the same error handling pattern.
631-639: LGTM! ResponsesStream completes the consistent error handling pattern.All four Anthropic-related methods (ChatCompletion, ChatCompletionStream, Responses, ResponsesStream) now properly handle errors from the conversion functions with consistent nil safety checks.
core/schemas/mux.go (3)
773-778: LGTM! Reasoning.MaxTokens is now preserved in ToResponsesRequest.The condition at line 773 correctly checks for either
EffortorMaxTokensbeing set, ensuring the reasoning configuration is not silently dropped when onlyMaxTokensis provided. This addresses the previous review concern about MaxTokens being lost during Chat→Responses conversions.
852-858: LGTM! Reasoning config properly round-tripped in ToChatRequest.Both
EffortandMaxTokensare now copied fromResponsesParametersReasoningtoChatReasoning, ensuring complete round-trip preservation of reasoning configuration.
1361-1372: LGTM! Streaming delta correctly uses the new Reasoning field.The streaming conversion now properly handles
delta.Reasoningcontent and emits the appropriateResponsesStreamResponseTypeReasoningSummaryTextDeltaevent type.core/providers/bedrock/utils.go (2)
32-52: Reasoning config handling looks correct.The reasoning configuration is properly translated to Bedrock's format:
- Effort "none" maps to type "disabled"
- Otherwise requires and validates MaxTokens against
MinimumReasoningMaxTokens- Returns appropriate errors for missing or invalid MaxTokens
702-709: LGTM! Safe message extraction in ToBedrockError.The explicit nil check before accessing
bifrostErr.Error.Messageis defensive and clear, even though the earlier check at line 695 would return early ifErrorwere nil.core/providers/bedrock/responses.go (2)
349-367: LGTM! Reasoning config extraction from Bedrock request.The logic correctly extracts reasoning configuration from
AdditionalModelRequestFields:
- When type is "enabled" and budget_tokens is present, creates Reasoning with Effort "auto" and MaxTokens
- Otherwise sets Effort to "none"
662-666: LGTM! Output conversion placement improved.Moving the output message conversion to the top of the function ensures it's processed before other fields, improving code organization.
core/providers/bedrock/chat.go (5)
39-41: LGTM! Error handling for parameter conversion.The function now properly propagates errors from
convertChatParameterswith a descriptive error message.
120-133: LGTM! Nil-safe reasoning content collection.The code properly guards against nil
ReasoningText(lines 123-125) before accessing its fields, addressing the previous review concern about potential nil pointer dereference.
139-142: LGTM! Single text block optimization.Collapsing a single text block to
contentStrsimplifies the response structure for the common case of text-only responses.
157-163: LGTM! Reasoning properly attached to assistant message.The code correctly ensures an assistant message exists before attaching reasoning content, preserving both the detailed reasoning entries and the accumulated text summary.
335-388: LGTM! Streaming reasoning content handling.The implementation correctly:
- Returns early (lines 340-342) when neither Text nor Signature is set, preventing nil response issues
- Handles Text and Signature cases separately with appropriate
ReasoningDetailsentries- Addresses the previous review concern about potential nil response pointer
core/providers/anthropic/responses.go (1)
1428-1432: Nil-safe stream error message extraction looks good.
Prevents nil deref whenbifrostErr.Erroris absent.core/providers/anthropic/chat.go (3)
99-117: Reasoning validation + error-return upgrade is solid.
Failing fast on missing/too-smallreasoning.max_tokenswill prevent sending invalid Anthropic “thinking” configs downstream.
207-216: ReasoningDetails → thinking blocks mapping looks consistent with signature support.
Good to seeSignaturecarried through intoAnthropicContentBlock.
468-503: Nil-guards on promoted fields in ToAnthropicChatResponse are a good fix.
The addedchoice.ChatNonStreamResponseChoice != nilchecks prevent nil derefs when only stream choices are present.
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
785a11f to
ef2308c
Compare
Pratham-Mishra04
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
0f9320e to
14a7a70
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/providers/vertex/vertex.go (1)
689-709: Same nil-body hazard inResponsesAnthropic path.requestBody["anthropic_version"] = ...(Line 704-706) can panic ifreqBody == nilyields a nil map.Proposed fix:
reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - } + if reqBody == nil { + return nil, fmt.Errorf("responses input is not provided") + } + reqBody.Model = deploymentcore/providers/bedrock/chat.go (1)
66-83: Structured-output fast-path may drop other content (incl. reasoning) because it doesn’t requirelen(content)==1. Comment says “only one tool use and no other content”, but the condition only checkscontent[0].ToolUse != nil. If Bedrock returns additional blocks (text/reasoning), they’ll be ignored.Suggested tightening:
- if len(response.Output.Message.Content) > 0 && response.Output.Message.Content[0].ToolUse != nil && structuredOutputToolName == response.Output.Message.Content[0].ToolUse.Name { + if len(response.Output.Message.Content) == 1 && + response.Output.Message.Content[0].ToolUse != nil && + structuredOutputToolName == response.Output.Message.Content[0].ToolUse.Name {
♻️ Duplicate comments (15)
plugins/semanticcache/utils.go (1)
669-671: MissingMaxTokensextraction creates cache inconsistency with Responses API.The Chat API parameter extraction only captures
Reasoning.Effortbut omitsReasoning.MaxTokens, while the Responses API extraction (lines 752-754) correctly extracts both fields. This inconsistency means Chat API requests with identicalEffortbut differentMaxTokenscould incorrectly share cache entries, potentially serving responses with the wrong reasoning budget.The previous review comment identified this issue for lines 669-671 and was marked as addressed, but the fix was only applied to
extractResponsesParametersToMetadataand not toextractChatParametersToMetadata.Add
MaxTokensextraction to match the Responses API implementation:if params.Reasoning != nil { - metadata["reasoning_effort"] = *params.Reasoning.Effort + if params.Reasoning.Effort != nil { + metadata["reasoning_effort"] = *params.Reasoning.Effort + } + if params.Reasoning.MaxTokens != nil { + metadata["reasoning_max_tokens"] = *params.Reasoning.MaxTokens + } }core/schemas/chatcompletions.go (1)
165-220: Mutual-exclusivity error may be harsher than necessary during migration.
UnmarshalJSONhard-errors when bothreasoning_effortandreasoningare present (Line 207–209). If the intent is “compat + deprecation”, consider preferringreasoningand ignoringreasoning_effort(optionally logging) rather than rejecting the request outright. This matches a previously raised concern.- if aux.Alias != nil && aux.Alias.Reasoning != nil && aux.ReasoningEffort != nil { - return fmt.Errorf("both reasoning_effort and reasoning fields cannot be present at the same time") - } + // If both are set, prefer `reasoning` and ignore legacy `reasoning_effort`. + // (Optionally log a deprecation warning at the API boundary.)core/providers/cohere/chat.go (1)
104-143: Avoid silent override: ExtraParamsthinkingcan overwrite Reasoning-derivedThinking.
Right nowcohereReq.Thinkingcan be set fromParams.Reasoningand then overwritten byExtraParams["thinking"]with no warning. This matches a previously raised concern.- if thinkingParam, ok := schemas.SafeExtractFromMap(bifrostReq.Params.ExtraParams, "thinking"); ok { + if cohereReq.Thinking == nil { + if thinkingParam, ok := schemas.SafeExtractFromMap(bifrostReq.Params.ExtraParams, "thinking"); ok { if thinkingMap, ok := thinkingParam.(map[string]interface{}); ok { thinking := &CohereThinking{} @@ cohereReq.Thinking = thinking } } }core/providers/azure/azure.go (4)
351-361: Don’t allow nil reqBody on Anthropic conversion success path.
IfToAnthropicChatRequestever returns(nil, nil), this code returnsnilrequest body, which will likely serialize tonulland fail downstream (or behave unexpectedly). This matches a previously raised concern.if schemas.IsAnthropicModel(deployment) { reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - } - return reqBody, nil + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment + return reqBody, nil } else {
451-461: Same nil-reqBody guard needed in Anthropic streaming path.
You setreqBody.ModelandreqBody.Streamonly if non-nil; better to fail fast if conversion returns nil.
529-539: Same nil-reqBody guard needed for Anthropic Responses conversion.
Otherwise you can sendnulltoanthropic/v1/messages.
630-640: Same nil-reqBody guard needed for Anthropic ResponsesStream conversion.
Otherwise you may start an SSE request with an invalid/null body.core/providers/openai/types.go (1)
56-72: Enforce mutual exclusivity of embedded pointers to avoid JSON collisions.
OpenAIMessageembeds*schemas.ChatToolMessageand*OpenAIChatAssistantMessage, but nothing prevents both being non-nil (which can flatten conflicting fields into the same JSON object). This matches a previously raised concern.type OpenAIMessage struct { @@ *schemas.ChatToolMessage *OpenAIChatAssistantMessage } + +func (m *OpenAIMessage) Validate() error { + if m == nil { + return nil + } + if m.ChatToolMessage != nil && m.OpenAIChatAssistantMessage != nil { + return fmt.Errorf("OpenAIMessage: only one of ChatToolMessage or OpenAIChatAssistantMessage may be set") + } + return nil +}(Then ensure conversion paths call
Validate()before marshaling/sending.)core/providers/anthropic/chat.go (1)
413-426: Usage cache-token mapping still looks semantically inverted (cache creation is prompt-side).
CacheCreationInputTokensis input-side work but is being stored underCompletionTokensDetails.CachedTokens. This matches a previously raised concern, but the current code still shows the same mapping.bifrostResponse.Usage = &schemas.BifrostLLMUsage{ PromptTokens: response.Usage.InputTokens, PromptTokensDetails: &schemas.ChatPromptTokensDetails{ - CachedTokens: response.Usage.CacheReadInputTokens, + CachedTokens: response.Usage.CacheReadInputTokens + response.Usage.CacheCreationInputTokens, }, CompletionTokens: response.Usage.OutputTokens, - CompletionTokensDetails: &schemas.ChatCompletionTokensDetails{ - CachedTokens: response.Usage.CacheCreationInputTokens, - }, TotalTokens: response.Usage.InputTokens + response.Usage.OutputTokens, }(Or, if you want to keep them distinct, add a dedicated field rather than misclassifying it as completion-side cached tokens.)
core/providers/bedrock/utils.go (2)
32-99: Reasoning config can be lost + disabled type usesmap[string]string(incompatible with readers).
- You write disabled as
map[string]string(Line 38-40) but other codepaths typically type-assertmap[string]interface{}/map[string]any; this will fail to round-trip.- The later
additionalModelRequestFieldPathshandling assignsbedrockReq.AdditionalModelRequestFields = orderedFields(Line 97-98), which can silently overwritereasoning_configadded above.Proposed fix (merge + consistent map type):
- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ "type": "disabled", } ... - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = make(schemas.OrderedMap) + } + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + }
10-10: Avoid cross-provider coupling forMinimumReasoningMaxTokens. Importingcore/providers/anthropicjust for a shared min-tokens constant couples providers.Suggestion: move the constant to a shared package (e.g.,
core/schemasorcore/constants) and have both providers import that.Also applies to: 44-46
core/providers/bedrock/responses.go (2)
452-482:reasoning_configcan be overwritten byadditionalModelRequestFieldPaths. You setreasoning_config(Line 452-472), then later assignbedrockReq.AdditionalModelRequestFields = orderedFields(Line 480-481), discarding it.
Alsomap[string]stringfor disabled (Line 457-459) is inconsistent with the parsing side.Proposed fix:
- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ "type": "disabled", } ... - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = make(schemas.OrderedMap) + } + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + }
349-366: Round-trip break: disabledreasoning_configwon’t parse due to map type mismatch. Reader expectsmap[string]interface{}(Line 351), but writers currently usemap[string]stringin some paths (see Line 457-459).core/providers/vertex/vertex.go (2)
518-543: Potential panic:reqBody == nilcan lead to nilrequestBodymap then assignment. IfToAnthropicChatRequestreturns(nil, nil), you marshalnil, unmarshal intorequestBody(likely nil), then dorequestBody["anthropic_version"]=...(Line 537-539) which panics on a nil map.Proposed fix:
reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("chat completion input is not provided") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true)
847-871: Same nil-body hazard inResponsesStreamAnthropic path.Proposed fix:
reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("responses input is not provided") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true)
🧹 Nitpick comments (3)
core/providers/cohere/chat.go (1)
564-683: ToBifrostChatMessage: good nil-safety; consider setting ToolCall.Type when present.
You populate tool calls but don’t setType(unlike other converters that set"function"). If downstream expectsType, you may want to fill it for parity.core/providers/bedrock/utils.go (1)
702-710: Safe message extraction is good; consider non-empty fallback.message := ""can lead to blank"message"on the wire even when you had other error context.Suggestion: keep
"unknown error"whenbifrostErr.Error.Messageis empty.core/providers/bedrock/chat.go (1)
121-133: Reasoning extraction is safe; consider trimming trailing newline. You always append"\n"(Line 132-133), which makesassistantMessage.Reasoningend with a newline.Possible tweak:
strings.TrimSuffix(reasoningText, "\n")before assigning.Also applies to: 157-163
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (30)
core/providers/anthropic/anthropic.go(4 hunks)core/providers/anthropic/chat.go(11 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(5 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(4 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/semanticcache/utils.go(2 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (11)
- core/providers/openai/openai.go
- core/providers/perplexity/chat.go
- plugins/maxim/main.go
- core/providers/anthropic/types.go
- core/providers/anthropic/anthropic.go
- core/providers/cohere/types.go
- core/schemas/mux.go
- ui/app/workspace/logs/views/logChatMessageView.tsx
- transports/bifrost-http/handlers/inference.go
- plugins/jsonparser/utils.go
- core/providers/bedrock/bedrock.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
framework/streaming/chat.gocore/providers/cohere/cohere.gocore/providers/bedrock/utils.goplugins/semanticcache/utils.goui/lib/types/logs.tscore/schemas/responses.gocore/providers/anthropic/responses.gocore/providers/openai/utils.gocore/providers/bedrock/responses.gocore/providers/anthropic/errors.gocore/providers/openai/types.gocore/providers/bedrock/chat.gocore/providers/vertex/vertex.gocore/providers/azure/azure.gocore/providers/bedrock/types.gocore/providers/anthropic/chat.gocore/providers/openai/chat.gocore/providers/cohere/chat.gocore/schemas/chatcompletions.go
🧠 Learnings (3)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
framework/streaming/chat.gocore/providers/cohere/cohere.gocore/providers/bedrock/utils.goplugins/semanticcache/utils.gocore/schemas/responses.gocore/providers/anthropic/responses.gocore/providers/openai/utils.gocore/providers/bedrock/responses.gocore/providers/anthropic/errors.gocore/providers/openai/types.gocore/providers/bedrock/chat.gocore/providers/vertex/vertex.gocore/providers/azure/azure.gocore/providers/bedrock/types.gocore/providers/anthropic/chat.gocore/providers/openai/chat.gocore/providers/cohere/chat.gocore/schemas/chatcompletions.go
📚 Learning: 2025-12-08T11:50:35.996Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 978
File: core/providers/openai/utils.go:26-45
Timestamp: 2025-12-08T11:50:35.996Z
Learning: OpenAI does not support the `ReasoningDetails` field. `ReasoningDetails` is a Bifrost-internal field and should not be included when converting messages to OpenAI format in the `ConvertBifrostMessagesToOpenAIMessages` function in `core/providers/openai/utils.go`.
Applied to files:
core/providers/openai/utils.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/utils.gocore/providers/openai/types.gocore/providers/openai/chat.go
🧬 Code graph analysis (11)
framework/streaming/chat.go (3)
core/schemas/chatcompletions.go (1)
ChatAssistantMessage(626-632)core/utils.go (1)
Ptr(51-53)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)
core/providers/cohere/cohere.go (2)
core/providers/cohere/chat.go (1)
ToCohereChatCompletionRequest(12-204)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/bedrock/utils.go (3)
core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/bedrock/types.go (2)
BedrockConverseRequest(48-62)BedrockError(390-394)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/openai/utils.go (4)
core/providers/openai/types.go (2)
OpenAIMessage(56-65)OpenAIChatAssistantMessage(67-72)core/schemas/chatcompletions.go (3)
ChatMessage(469-478)ChatToolMessage(621-623)ChatAssistantMessage(626-632)ui/lib/types/logs.ts (1)
ChatMessage(115-124)core/providers/gemini/types.go (2)
Role(13-13)Content(922-930)
core/providers/anthropic/errors.go (1)
core/providers/anthropic/types.go (2)
AnthropicMessageError(376-379)AnthropicMessageErrorStruct(382-385)
core/providers/bedrock/chat.go (1)
core/schemas/chatcompletions.go (8)
ChatReasoningDetails(723-730)BifrostReasoningDetailsTypeText(719-719)ChatContentBlockTypeText(582-582)ChatAssistantMessage(626-632)BifrostChatResponse(26-41)BifrostResponseChoice(704-712)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)
core/providers/vertex/vertex.go (5)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/schemas/models.go (1)
Model(109-129)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/azure/azure.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/anthropic/chat.go (6)
core/schemas/chatcompletions.go (6)
BifrostChatRequest(12-19)ChatAssistantMessage(626-632)BifrostResponseChoice(704-712)ChatNonStreamResponseChoice(745-748)BifrostLLMUsage(812-819)ChatStreamResponseChoice(751-753)core/providers/anthropic/types.go (5)
AnthropicMessageRequest(40-59)AnthropicThinking(65-68)AnthropicContentBlock(147-159)AnthropicMessageResponse(273-282)AnthropicStreamDelta(347-355)core/schemas/models.go (1)
Model(109-129)core/schemas/bifrost.go (5)
BifrostResponseExtraFields(285-294)RequestType(83-83)ChatCompletionRequest(89-89)Anthropic(37-37)BifrostError(354-363)core/schemas/provider.go (1)
Provider(280-307)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/openai/chat.go (4)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)
core/schemas/chatcompletions.go (3)
core/providers/openai/types.go (1)
Alias(78-78)core/schemas/bifrost.go (2)
Alias(384-384)Alias(401-401)ui/lib/types/logs.ts (3)
ChatMessage(115-124)ChatMessageContent(113-113)ReasoningDetails(126-133)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (32)
framework/streaming/chat.go (2)
44-54: LGTM! Reasoning accumulation follows established patterns.The reasoning string accumulation correctly mirrors the existing refusal handling pattern with proper nil checks and initialization.
55-105: Reasoning details accumulation logic is correct.The index-based matching for streaming reasoning details is appropriate. A few observations:
- The O(n×m) search is acceptable given typical chunk sizes are small.
- Text, Summary, and Data are correctly accumulated (concatenated).
- Signature and Type are correctly overwritten (non-cumulative fields).
One minor consideration: if a provider ever sends duplicate indices within the same chunk, only the first match would be updated. This edge case seems unlikely in practice.
core/providers/anthropic/errors.go (1)
9-37: LGTM! Nil safety is properly handled.The function correctly guards against nil
bifrostErrand nilbifrostErr.Error, addressing the previous review concern. The defensive approach ensures robustness across all BifrostError sources.core/providers/openai/utils.go (2)
5-24: LGTM! Conversion correctly maps OpenAI messages to Bifrost format.The function properly handles the embedded
OpenAIChatAssistantMessagewith nil checks. As confirmed in learnings,ReasoningDetailsis intentionally omitted since OpenAI doesn't support this field.
26-45: LGTM! Bidirectional conversion maintains consistency.The reverse conversion correctly mirrors the OpenAI→Bifrost function.
ReasoningDetailsis appropriately excluded when converting to OpenAI format since it's a Bifrost-internal field. Based on learnings, this is the correct behavior.core/providers/openai/chat.go (3)
14-14: LGTM! Message conversion now uses dedicated utility.Using
ConvertOpenAIMessagesToBifrostMessagesensures consistent field mapping and proper handling of assistant-specific fields during the OpenAI→Bifrost conversion.
28-28: LGTM! Symmetrical conversion for outbound messages.The use of
ConvertBifrostMessagesToOpenAIMessagesmaintains consistency with the inbound conversion path.
62-65: LGTM! Reasoning effort handling updated for new schema structure.The triple nil-check chain (
Reasoning != nil && Effort != nil && value == "minimal") correctly guards against nil pointer dereferences when accessing the nested reasoning configuration. The"minimal"→"low"transformation ensures compatibility with providers that don't support the "minimal" effort level.ui/lib/types/logs.ts (1)
122-133: LGTM! Reasoning fields correctly added to ChatMessage.The new
reasoningandreasoning_detailsfields align with the backend schema changes. TheReasoningDetailsinterface correctly models the three reasoning detail types (reasoning.summary,reasoning.encrypted,reasoning.text) with appropriate optional fields for each variant.core/providers/openai/types.go (2)
42-44: Type change (Messages []OpenAIMessage) needs call-site audit for custom message fields.
This changes the request surface fromschemas.ChatMessagetoOpenAIMessage, so any code that relied onschemas.ChatMessageJSON shape (especially embedded assistant/tool fields) must now go through the converters consistently.
74-101: MarshalJSON behavior is consistent and nil-safe.
Omittingreasoningwhile emittingreasoning_effortfromReasoning.Effortis implemented cleanly; the nil checks onr.Reasoning/r.Reasoning.Effortare correct.core/providers/anthropic/chat.go (3)
13-17: Good: converter now returns error for nil request/input.
This aligns with the broader “conversion functions return errors” refactor and avoids downstream panics.
99-117: Reasoning validation looks correct, but effort semantics should be clarified.
Right now “enabled” is effectively “effort != none” (including nil effort), and Anthropic requiresreasoning.max_tokens. Ifeffortis optional in your API, consider documenting thatmax_tokensis mandatory for Anthropic wheneverreasoningis present and not explicitly"none".
710-750: Good: signature deltas now round-trip back to Anthropic SSE.
The added branch that mapsDelta.ReasoningDetails[0].SignaturetoAnthropicStreamDeltaTypeSignatureaddresses the “signature-only updates get dropped” issue.core/schemas/chatcompletions.go (2)
634-667: ReasoningDetails synthesis is sensible; watch index stability.
Synthesizing a singlereasoning_details[0]fromreasoningis fine, but if providers emit multi-part reasoning in-stream, ensure indices stay stable across chunks/messages (especially when appending provider-native details).
765-794: Delta ReasoningDetails synthesis mirrors message behavior (good).
Same note as above re: index stability and merging when bothReasoningDetailsandReasoningmight appear across partial chunks.core/providers/cohere/chat.go (3)
12-15: Nil-input guard is fine as defensive code.
Not rehashing the error-string wording given prior discussion.
219-226: Per-message conversion path looks cleaner.
UsingCohereMessage.ToBifrostChatMessage()reduces duplicated conversion logic and should help keep reasoning/tool-call handling consistent.
247-261: Reasoning ↔ Thinking mapping seems consistent.
Disabled maps toeffort: "none", enabled maps toeffort: "auto"plusmax_tokenswhen available.core/providers/cohere/cohere.go (2)
279-286: Good: error propagation from request conversion.
Passingfunc() (any, error) { return ToCohereChatCompletionRequest(request) }ensures conversion failures don’t get silently marshaled.
331-338: Good: stream request conversion now fails fast and setsStream=trueonly on success.
This avoids partially-initialized streaming requests.core/schemas/responses.go (1)
323-326: Helpful interoperability note (gpt-oss vs other OpenAI models).
This kind of “shape difference” comment is valuable given the custom parsing expectations elsewhere.core/providers/bedrock/utils.go (1)
15-18: Nil Params short-circuit is consistent with “params are optional” behavior.core/providers/bedrock/responses.go (1)
662-666: Output mapping moved to the top reads clearer and avoids duplication.core/providers/vertex/vertex.go (1)
283-291: ChatCompletion Anthropic path: nil body handled correctly.core/providers/bedrock/chat.go (3)
39-41: Good: errors from parameter conversion are now surfaced.
139-143: Single text block collapse is a nice normalization.
335-389: Streaming reasoning delta: safe “emit only when meaningful” behavior. The early return avoids producing(nil, nil, false)as a “response”.core/providers/anthropic/responses.go (4)
190-201: LGTM!The conversion from Anthropic's
Thinkingstructure to Bifrost'sReasoningstructure is correctly implemented. The mapping preserves the enabled/disabled state and budget tokens appropriately.
296-303: LGTM!Good defensive check for nil input with clear error message. The signature change to return an error tuple aligns with the broader refactoring to improve error handling across conversion functions.
402-403: LGTM!Return statement correctly updated to match the new
(*AnthropicMessageRequest, error)signature.
1428-1438: LGTM!Good defensive nil check before accessing nested
Error.Messagefield. This prevents potential nil pointer dereferences whenbifrostErr.Erroris nil.
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/providers/bedrock/utils.go (1)
15-52: Fix critical data loss when bothreasoningandadditionalModelRequestFieldPathsare present.At line 97 in
utils.go(and identically at line 480 inresponses.go), the assignmentbedrockReq.AdditionalModelRequestFields = orderedFieldscompletely replaces the map, discarding thereasoning_configthat was set at lines 38 or 47. When both parameters are present, the reasoning configuration is lost.Merge
reasoning_configfrom the existing map intoorderedFieldsbefore assignment, or preserve it by iterating throughorderedFieldswhile keeping the reasoning entry intact.core/providers/anthropic/chat.go (1)
637-677: Streaming reasoning/signature deltas must usechunk.Indexinstead of hardcoding 0.In
ToBifrostChatCompletionStream:
- thinking deltas emit
ReasoningDetails.Index: 0(line 641)- signature deltas emit
ReasoningDetails.Index: 0(line 668)However,
chunk.Index(*int) is available and the downstream accumulation logic inframework/streaming/chat.go:63matches reasoning details by Index to merge or update them. Hardcoding all indices to 0 causes all reasoning details to merge into a single entry, breaking the separation between thinking and signature blocks.Fix: Use
*chunk.Indexinstead of 0 when available. The signature field and TypeText type choice are correct by design.
♻️ Duplicate comments (9)
plugins/semanticcache/utils.go (1)
669-671: Critical: AddReasoning.MaxTokensextraction to Chat API cache metadata.The previous review comment flagging the missing
MaxTokensextraction was marked as addressed, but the fix was only applied toextractResponsesParametersToMetadata(lines 752-754), not here inextractChatParametersToMetadata. WithoutMaxTokensin the cache metadata, Chat API requests with identicalEffortbut differentMaxTokenswill incorrectly hit the same cache entry, leading to wrong responses being served from cache.Apply this diff to add
MaxTokensextraction alongsideEffort:if params.Reasoning != nil { if params.Reasoning.Effort != nil { metadata["reasoning_effort"] = *params.Reasoning.Effort } + if params.Reasoning.MaxTokens != nil { + metadata["reasoning_max_tokens"] = *params.Reasoning.MaxTokens + } }core/providers/bedrock/responses.go (1)
452-472: Fix type mismatch for disabled reasoning config and handle the map overwrite issue.Two critical issues:
Type mismatch: Line 457 creates
map[string]string{"type": "disabled"}, but the enabled case at line 466 usesmap[string]any. This inconsistency causes a type assertion failure inToBifrostResponsesRequest(line 351) which expectsmap[string]interface{}for the disabled case.Silent data loss: Lines 480-482 completely replace
bedrockReq.AdditionalModelRequestFields = orderedFieldswhenadditionalModelRequestFieldPathsexists. This discards thereasoning_configset at lines 457/466 unless the external paths explicitly include it—resulting in reasoning configuration being silently dropped.Fix the type mismatch by changing line 457 to:
- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{For the overwrite issue, merge the reasoning config into
orderedFieldsinstead of replacing the entire map:if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { + for key, value := range bedrockReq.AdditionalModelRequestFields { + orderedFields[key] = value + } bedrockReq.AdditionalModelRequestFields = orderedFields } }core/schemas/chatcompletions.go (1)
165-226:reasoning_effort+reasoningstrict exclusivity may be too breaking for a deprecation window.Right now you return an error if both are present. If you want a smoother migration, prefer
reasoningwhen both exist and ignorereasoning_effort(optionally add a warning elsewhere). This was raised earlier—calling out in case the stack intends “backward compatible”.core/providers/anthropic/responses.go (1)
325-344: Responses reasoning enablement semantics still differ from Chat converter (Effort==nil case).Here,
Effort==nilfalls into “disabled”, whilecore/providers/anthropic/chat.gotreatsEffort==nilas enabled unless explicitly"none"(and then requiresMaxTokens). This will make identicalReasoningpayloads behave differently across endpoints.This was raised previously; please ensure the stack didn’t reintroduce it and unify both converters (ideally via a shared helper) so:
- Disabled only when
Effort != nil && *Effort == "none".- Otherwise: require
MaxTokensand validate>= MinimumReasoningMaxTokens, then set thinking enabled.Confirm intended Anthropic “thinking” enable/disable semantics for Bifrost Reasoning: - Should Effort==nil with MaxTokens set enable thinking? - Should Effort==nil with MaxTokens nil be treated as disabled or as an error?core/providers/vertex/vertex.go (3)
518-525: Fail fast whenreqBody == nilin ChatCompletionStream (otherwise you can marshalnulland later panic).Right now,
reqBody == nilskips assignments but still gets marshalled/unmarshalled; later code writes intorequestBody(map) which can benil.reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true)
689-695: Fail fast whenreqBody == nilin Responses (same “marshal null” risk).reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment
847-854: Fail fast whenreqBody == nilin ResponsesStream (same “marshal null” risk).reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true)core/providers/openai/types.go (1)
42-44: Enforce mutual exclusivity of embedded message structs (prevents silent JSON field clobbering).
OpenAIMessagecan embed both*schemas.ChatToolMessageand*OpenAIChatAssistantMessage; if both are set, marshaling flattens/overwrites fields unpredictably. This was already raised—worth enforcing via aValidate()method (called before marshaling) or a customMarshalJSONonOpenAIMessage.Also applies to: 56-65
core/providers/anthropic/chat.go (1)
413-426: CacheCreationInputTokens should not populate completion cached tokens.You map
response.Usage.CacheCreationInputTokensintoCompletionTokensDetails.CachedTokens(Line 421-423). Cache creation is an input/prompt-side concern in Anthropic usage naming, so this likely belongs onPromptTokensDetails(or a dedicated field if you want to preserve “creation vs read” semantics). Also note this should align with the reverse mapping logic inToAnthropicChatResponseto avoid round-trip drift across the stack.
🧹 Nitpick comments (4)
core/providers/cohere/chat.go (1)
131-142: Clarify override behavior: ExtraParams["thinking"] overwrites Reasoning.Lines 131-142 unconditionally set
cohereReq.ThinkingfromExtraParams["thinking"], which can silently override theReasoning-based thinking set in lines 104-121. A past review comment noted this issue and was marked as addressed, but the current code still allows the override.If this is intentional (ExtraParams takes precedence), consider adding a comment explaining the precedence. Otherwise, add a guard:
if cohereReq.Thinking == nil { if thinkingParam, ok := schemas.SafeExtractFromMap(...); ok { // ... } }Based on learnings: A past review flagged that ExtraParams["thinking"] can overwrite Reasoning-based thinking. The issue was marked as addressed but the override behavior persists.
core/providers/bedrock/utils.go (1)
10-10: Consider extracting MinimumReasoningMaxTokens to a shared constants package.The constant is imported from the Anthropic provider but used by multiple providers (Bedrock, Cohere per past reviews), creating cross-provider dependencies. Moving this to a shared location like
core/constantsorcore/schemaswould reduce coupling.This is a minor architectural improvement and not blocking.
core/providers/anthropic/chat.go (2)
207-217: Only inject “thinking” blocks for assistant-role messages (and avoid empty details).Right now, any message with
ChatAssistantMessage.ReasoningDetailsgets converted into Anthropicthinkingblocks, regardless ofmsg.Role. Consider tightening tomsg.Role == schemas.ChatMessageRoleAssistantand skipping details where bothTextandSignatureare nil/empty to avoid emitting invalid/empty content blocks.
505-520: Tool-call argument unmarshal should beinterface{}(not forced object).
var input map[string]interface{}will discard/zero-out if arguments aren’t a JSON object. Safer to mirror the request-side handling:- var input map[string]interface{} + var input interface{} if toolCall.Function.Arguments != "" { if err := json.Unmarshal([]byte(toolCall.Function.Arguments), &input); err != nil { - input = map[string]interface{}{} + input = map[string]interface{}{} } } else { input = map[string]interface{}{} }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (30)
core/providers/anthropic/anthropic.go(4 hunks)core/providers/anthropic/chat.go(11 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(5 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(4 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/semanticcache/utils.go(2 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (8)
- core/providers/cohere/types.go
- core/providers/anthropic/types.go
- core/providers/openai/openai.go
- plugins/jsonparser/utils.go
- framework/streaming/chat.go
- core/schemas/mux.go
- core/providers/perplexity/chat.go
- core/schemas/responses.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
ui/app/workspace/logs/views/logChatMessageView.tsxplugins/semanticcache/utils.gotransports/bifrost-http/handlers/inference.gocore/providers/bedrock/bedrock.gocore/providers/anthropic/responses.gocore/providers/anthropic/errors.gocore/providers/bedrock/types.gocore/providers/bedrock/chat.gocore/providers/openai/chat.gocore/providers/cohere/chat.goui/lib/types/logs.tscore/providers/openai/utils.gocore/providers/cohere/cohere.gocore/providers/bedrock/responses.gocore/providers/bedrock/utils.gocore/providers/anthropic/anthropic.gocore/providers/vertex/vertex.gocore/providers/azure/azure.gocore/providers/openai/types.gocore/schemas/chatcompletions.gocore/providers/anthropic/chat.goplugins/maxim/main.go
🧠 Learnings (4)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
plugins/semanticcache/utils.gotransports/bifrost-http/handlers/inference.gocore/providers/bedrock/bedrock.gocore/providers/anthropic/responses.gocore/providers/anthropic/errors.gocore/providers/bedrock/types.gocore/providers/bedrock/chat.gocore/providers/openai/chat.gocore/providers/cohere/chat.gocore/providers/openai/utils.gocore/providers/cohere/cohere.gocore/providers/bedrock/responses.gocore/providers/bedrock/utils.gocore/providers/anthropic/anthropic.gocore/providers/vertex/vertex.gocore/providers/azure/azure.gocore/providers/openai/types.gocore/schemas/chatcompletions.gocore/providers/anthropic/chat.goplugins/maxim/main.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/inference.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/chat.gocore/providers/openai/utils.gocore/providers/openai/types.go
📚 Learning: 2025-12-08T11:50:35.996Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 978
File: core/providers/openai/utils.go:26-45
Timestamp: 2025-12-08T11:50:35.996Z
Learning: OpenAI does not support the `ReasoningDetails` field. `ReasoningDetails` is a Bifrost-internal field and should not be included when converting messages to OpenAI format in the `ConvertBifrostMessagesToOpenAIMessages` function in `core/providers/openai/utils.go`.
Applied to files:
core/providers/openai/utils.go
🧬 Code graph analysis (14)
ui/app/workspace/logs/views/logChatMessageView.tsx (2)
ui/lib/utils/validation.ts (2)
isJson(539-546)cleanJson(548-558)ui/app/workspace/logs/views/codeEditor.tsx (1)
CodeEditor(82-245)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/chatcompletions.go (2)
ChatMessage(469-478)ChatParameters(155-184)ui/lib/types/logs.ts (1)
ChatMessage(115-124)
core/providers/bedrock/bedrock.go (3)
core/providers/utils/utils.go (1)
CreateBifrostChatCompletionChunkResponse(687-716)core/schemas/bifrost.go (1)
ChatCompletionStreamRequest(90-90)core/schemas/models.go (1)
Model(109-129)
core/providers/anthropic/responses.go (3)
core/schemas/responses.go (2)
ResponsesParametersReasoning(233-238)BifrostResponsesRequest(32-39)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/types.go (4)
AnthropicMessageRequest(40-59)MinimumReasoningMaxTokens(14-14)AnthropicThinking(65-68)AnthropicStreamError(397-400)
core/providers/cohere/chat.go (3)
core/schemas/chatcompletions.go (3)
ChatMessage(469-478)ChatReasoning(223-226)ChatNonStreamResponseChoice(745-748)core/providers/cohere/types.go (5)
CohereChatRequest(14-32)CohereThinking(171-174)ThinkingTypeDisabled(181-181)ThinkingTypeEnabled(180-180)CohereMessage(51-57)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/openai/utils.go (3)
core/providers/openai/types.go (2)
OpenAIMessage(56-65)OpenAIChatAssistantMessage(67-72)core/schemas/chatcompletions.go (3)
ChatMessage(469-478)ChatToolMessage(621-623)ChatAssistantMessage(626-632)ui/lib/types/logs.ts (1)
ChatMessage(115-124)
core/providers/bedrock/responses.go (5)
core/schemas/utils.go (4)
SafeExtractFromMap(519-525)SafeExtractString(280-297)SafeExtractInt(300-342)Ptr(16-18)core/schemas/responses.go (1)
ResponsesParametersReasoning(233-238)ui/lib/types/logs.ts (1)
ResponsesParametersReasoning(519-526)core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/bedrock/utils.go (3)
core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)OrderedMap(268-268)core/providers/bedrock/types.go (2)
BedrockConverseRequest(48-62)BedrockError(390-394)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/vertex/vertex.go (4)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/azure/azure.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/openai/types.go (2)
core/schemas/models.go (1)
Model(109-129)core/schemas/chatcompletions.go (10)
ChatMessageRole(457-457)ChatMessageContent(524-527)ChatToolMessage(621-623)ChatAssistantMessageAnnotation(670-673)ChatAssistantMessageToolCall(686-692)Alias(189-189)Alias(643-643)Alias(770-770)Alias(860-860)ChatReasoning(223-226)
core/schemas/chatcompletions.go (1)
ui/lib/types/logs.ts (3)
ChatMessage(115-124)ChatMessageContent(113-113)ReasoningDetails(126-133)
core/providers/anthropic/chat.go (4)
core/schemas/chatcompletions.go (10)
BifrostChatRequest(12-19)ChatAssistantMessage(626-632)ChatAssistantMessageToolCall(686-692)ChatReasoningDetails(723-730)ChatMessageContent(524-527)ChatMessage(469-478)BifrostResponseChoice(704-712)ChatNonStreamResponseChoice(745-748)BifrostLLMUsage(812-819)ChatStreamResponseChoice(751-753)core/providers/anthropic/types.go (5)
AnthropicMessageRequest(40-59)AnthropicThinking(65-68)MinimumReasoningMaxTokens(14-14)AnthropicContentBlock(147-159)AnthropicContentBlockTypeThinking(143-143)core/schemas/bifrost.go (1)
Anthropic(37-37)core/schemas/utils.go (1)
Ptr(16-18)
plugins/maxim/main.go (1)
core/providers/gemini/types.go (1)
Type(778-778)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (44)
plugins/semanticcache/utils.go (1)
748-758: LGTM! Correct reasoning parameter extraction.The extraction of
Reasoning.Effort,Reasoning.MaxTokens, andReasoning.Summaryis correctly implemented with proper nil-safety checks. This ensures that the cache key for the Responses API properly differentiates requests with different reasoning configurations.plugins/maxim/main.go (1)
472-490: Consider usingnilinstead of&""for absent Code and Type fieldsThe nil-safety fix for
bifrostErr.Erroris solid. However, passingCode: &codeandType: &errorTypewill always create non-nil pointers (even to empty strings when no upstream value exists). If maxim-go'sGenerationError.CodeandGenerationError.Typeare pointer fields meant to represent optional values,nilbetter conveys "unset" than a pointer to an empty string.If the fields should truly be
nilwhen absent, consider:- code := "" - errorType := "" + var code *string + var errorType *string if bifrostErr.Error != nil { message = bifrostErr.Error.Message if bifrostErr.Error.Code != nil { - code = *bifrostErr.Error.Code + code = bifrostErr.Error.Code } if bifrostErr.Error.Type != nil { - errorType = *bifrostErr.Error.Type + errorType = bifrostErr.Error.Type } } genErr := logging.GenerationError{ Message: message, - Code: &code, - Type: &errorType, + Code: code, + Type: errorType, }core/providers/cohere/cohere.go (2)
282-283: LGTM! Function signature change aligns with error-handling pattern.The update to
ToCohereChatCompletionRequestto return(*CohereChatRequest, error)properly propagates conversion errors. The call site correctly handles the error return value.
332-337: LGTM! Streaming path correctly handles error return.The streaming conversion properly checks for errors from
ToCohereChatCompletionRequestand returns early if an error occurs, maintaining the error-handling contract.transports/bifrost-http/handlers/inference.go (3)
77-77: LGTM! "reasoning" field correctly added to known fields.The addition of
"reasoning": truetochatParamsKnownFieldsensures the new reasoning parameter is recognized and won't be treated as an extra parameter.
476-478: LGTM! Pre-initialization prevents nil pointer issues.Initializing
ChatParametersbefore unmarshalling ensures the customUnmarshalJSONonChatParametershas a non-nil target. This aligns with the approach documented in lines 169-199.
169-199: The bifrostAlias approach is correctly applied and necessary.The alias prevents infinite recursion: when ChatRequest implements UnmarshalJSON, calling sonic.Unmarshal directly on
*ChatRequestwould recursively invoke UnmarshalJSON. The alias (which is justBifrostParamswithout the method) allows sonic to unmarshal the fields without triggering the custom method. The three separate unmarshal calls are intentional—each handles a distinct part of the JSON structure (BifrostParams fields, Messages array, and ChatParameters with its own custom unmarshaller). No changes needed.ui/lib/types/logs.ts (2)
122-124: LGTM! Reasoning fields added to ChatMessage.The addition of
reasoning?: stringandreasoning_details?: ReasoningDetails[]aligns with the PR's goal of introducing reasoning support. The fields follow the existing optional pattern used in the interface.
126-133: Note: ReasoningDetails interface uses optional fields instead of discriminated union.As noted in past review comments, a discriminated union would provide stronger type safety (ensuring
type: "reasoning.summary"requires thesummaryfield). However, the maintainer has indicated this approach is acceptable for now.Based on learnings: A past review suggested using a discriminated union for type safety, but the maintainer indicated the current approach is acceptable.
core/providers/cohere/chat.go (2)
12-15: LGTM! Error return added for validation failures.Changing the signature to return
(*CohereChatRequest, error)enables proper error propagation for invalid reasoning configurations and nil inputs.
564-683: LGTM! Per-message conversion with comprehensive nil-safety.The new
ToBifrostChatMessage()method properly handles content types (string/blocks), reasoning details, and tool calls with appropriate nil checks throughout. The reasoning text concatenation (lines 606-609) correctly joins multiple thinking blocks with newlines.ui/app/workspace/logs/views/logChatMessageView.tsx (1)
73-94: LGTM! Terminology updated from "thought" to "reasoning".The changes consistently rename
message.thoughttomessage.reasoningand update the UI label from "Thought Process" to "Reasoning". This aligns with the backend changes introducing the new reasoning field structure.core/providers/bedrock/bedrock.go (1)
771-772: LGTM! UUID generation addresses missing Bedrock stream identifier.The comment clearly explains that Bedrock doesn't provide a unique stream identifier, so generating a UUID locally is the correct approach. The generated ID is used consistently throughout the streaming path (lines 859, 882).
core/providers/openai/chat.go (2)
63-65: LGTM! Reasoning effort mapping updated to nested structure.The updated code correctly accesses
request.ChatParameters.Reasoning.Effort(nested field) with proper nil checks, and preserves the "minimal" → "low" mapping for non-OpenAI providers.
14-14: BothConvertOpenAIMessagesToBifrostMessagesandConvertBifrostMessagesToOpenAIMessagesutility functions are present and properly implemented incore/providers/openai/utils.go. The functions are correctly used in lines 14 and 28 ofchat.go.core/providers/anthropic/errors.go (1)
9-37: LGTM! Error conversion with comprehensive nil-safety.The new
ToAnthropicChatCompletionErrorfunction properly handles nil inputs at multiple levels:
- Returns nil if
bifrostErris nil- Safely extracts
bifrostErr.Typewith default empty string- Guards against nil
bifrostErr.Error(lines 23-25) before accessing nestedMessagefieldThis addresses the concern raised in past reviews about nil pointer dereferences.
Based on learnings: A past review flagged the need for nil-safety when accessing
bifrostErr.Error. The current implementation includes the necessary guard.core/providers/openai/utils.go (1)
5-45: LGTM! Conversion functions correctly handle OpenAI-Bifrost message mapping.The converters correctly map the supported fields (Name, Role, Content, ChatToolMessage, and nested assistant message fields) between OpenAI and Bifrost formats. The code appropriately excludes
ReasoningDetailsfrom both conversions, consistent with learnings that OpenAI doesn't support this field.Based on learnings, OpenAI doesn't support the
ReasoningDetailsfield, so it should not be included in these conversion functions.core/providers/anthropic/anthropic.go (4)
303-311: LGTM! Error propagation from ToAnthropicChatRequest is correct.The non-streaming chat completion path now correctly propagates the error return from
ToAnthropicChatRequest, aligning with the updated function signature.
354-368: LGTM! Streaming path correctly handles conversion errors.The streaming path properly captures and returns errors from
ToAnthropicChatRequestbefore enabling streaming, ensuring errors surface before streaming begins.
655-663: LGTM! Error propagation from ToAnthropicResponsesRequest is correct.The responses path now correctly propagates the error return from
ToAnthropicResponsesRequest, consistent with the updated function signature.
704-718: LGTM! Streaming responses path correctly handles conversion errors.The streaming responses path properly captures and returns errors from
ToAnthropicResponsesRequestbefore enabling streaming.core/providers/azure/azure.go (4)
348-368: LGTM! ChatCompletion correctly handles conversion errors with nil safety.The function properly captures the error from
ToAnthropicChatRequest, returns early on error, and checksreqBody != nilbefore setting the Model field, preventing nil pointer dereferences.
448-465: LGTM! ChatCompletionStream correctly handles conversion errors with nil safety.The streaming path properly handles errors from
ToAnthropicChatRequestand includes a nil check onreqBodybefore accessing its fields.
526-550: LGTM! Responses correctly handles conversion errors with nil safety.The function properly captures the error from
ToAnthropicResponsesRequest, returns early on error, and checksreqBody != nilbefore setting the Model field.
627-644: LGTM! ResponsesStream correctly handles conversion errors with nil safety.The streaming responses path properly handles errors from
ToAnthropicResponsesRequestand includes a nil check onreqBodybefore accessing its fields.core/providers/bedrock/utils.go (1)
702-710: LGTM! Safe error message extraction.The code now safely checks that
bifrostErr.Error != nilbefore accessing theMessagefield, preventing nil pointer dereferences.core/providers/bedrock/responses.go (2)
349-367: LGTM! Reasoning config extraction from Bedrock to Bifrost is correct.The code properly extracts
reasoning_configfromAdditionalModelRequestFields, handles both enabled and disabled cases, and safely extracts the type and budget_tokens values.
662-666: LGTM! Output conversion is correctly positioned.The code now converts
response.OutputtobifrostResp.Outputat the top of the function, ensuring the output is populated in the final response.core/providers/bedrock/chat.go (4)
39-41: LGTM! Proper error handling for parameter conversion.The function now correctly handles and propagates errors from
convertChatParameters, ensuring invalid parameters are caught early.
59-60: LGTM! Reasoning content handling is safe and correct.The code properly:
- Initializes reasoning state variables
- Checks
ReasoningContent != nilandReasoningText != nilbefore accessing fields (lines 122-125), preventing nil pointer dereferences- Accumulates reasoning details with proper type and index
- Attaches reasoning data to the assistant message when present
Also applies to: 121-133, 157-163
139-142: LGTM! Content block collapsing optimization.The code correctly collapses a single text content block to a simple string, reducing unnecessary nesting while preserving the content.
335-388: LGTM! Streaming reasoning content delta handling is correct.The streaming path properly:
- Checks if both
Textis empty andSignatureis nil, returning early to avoid nil responses (lines 340-342)- Constructs appropriate responses for text deltas (lines 345-365)
- Constructs appropriate responses for signature deltas (lines 366-386)
- Ensures
ReasoningDetailsare properly populated in the deltaThis prevents the nil response pointer issue mentioned in past reviews.
core/providers/anthropic/responses.go (2)
296-299: Good: error-returning converter + nil input guard.Signature change to
(*AnthropicMessageRequest, error)and thebifrostReq == nilcheck make downstream error handling much safer.Also applies to: 402-402
1428-1432: Good: nil-safe stream error message extraction.Avoids nil deref when
bifrostErr.Erroris missing and still emits a valid SSE error payload.Also applies to: 1438-1439
core/providers/vertex/vertex.go (1)
283-289: Good: propagate conversion errors fromToAnthropicChatRequest.This aligns Vertex with the new error-returning Anthropic converters and avoids silent bad requests.
core/schemas/chatcompletions.go (4)
627-667: Good: synthesizereasoning_detailswhen onlyreasoningis provided.This keeps downstream consumers consistent (they can always look at
reasoning_details) while remaining tolerant to providers that only emit a singlereasoningstring.Also applies to: 757-794
714-730: LGTM: reasoning details typing covers summary/text/encrypted (+ signature).This seems like a solid cross-provider shape (and matches the UI typings you referenced).
835-836: LGTM:image_tokensaddition is safe (pointer + omitempty).
186-220: [rewritten comment]
[classification tag]core/providers/bedrock/types.go (1)
444-446: Schema difference is intentional and correct per AWS API.The code properly implements AWS Bedrock's intentional structural difference: non-delta
ReasoningContentuses a nestedreasoningTextwrapper, while streamingContentBlockDelta.ReasoningContentemits the structure directly withtextandsignaturefields. Each type correctly matches its corresponding AWS API response, and unmarshaling will succeed without issues.core/providers/anthropic/chat.go (4)
13-16: Nil/empty input guard is good; ensure callers handle the new error return.
ToAnthropicChatRequestnow returns(..., error); please ensure all call sites in the PR stack (#978 + downstack #979/#999) were updated to propagate/handle this (no silent ignored errors).
726-750: Good: signature deltas now round-trip to Anthropic SSE.The added
ReasoningDetails[0].Signature→AnthropicStreamDeltaTypeSignaturehandling fixes the “signature-only deltas dropped” path.
850-863: Rename toToAnthropicChatStreamErrorlooks fine; keep SSE formatting consistent.Looks consistent with the other SSE formatting. Please just sanity-check any external references to the old name in the PR stack (#978/#979/#999).
99-117: The stringly-typed effort check could use typed constants for clarity. The code at lines 99-117 correctly checksbifrostReq.Params.Reasoning, which is already normalized from the deprecatedreasoning_effortfield byChatParameters.UnmarshalJSON()during deserialization. No backward-compatibility risk here.However, replace the
*bifrostReq.Params.Reasoning.Effort == "none"string comparison with type-safe constants for the effort values ("none", "minimal", "low", "medium", "high") to avoid potential casing/typo issues.
TejasGhatte
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
ef2308c to
d592b74
Compare
TejasGhatte
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
14a7a70 to
0f9320e
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
framework/streaming/chat.go (1)
34-105: Avoid pointer aliasing in ReasoningDetails accumulation (copy strings before storing/mutating).
existingReasoningDetail.Text = reasoningDetail.Text(and similar for Summary/Data/Signature) stores pointers from the chunk directly into the final accumulated message, and later you mutate them via*existing += *incoming. Prefer taking ownership by copying into new pointers (like you do withbifrost.Ptr(...)for Refusal/Reasoning).Example pattern:
- existingReasoningDetail.Text = reasoningDetail.Text + v := *reasoningDetail.Text + existingReasoningDetail.Text = bifrost.Ptr(v)Apply similarly for
Summary,Data, and when appending a brand-newreasoningDetail(deep-copy its pointer fields before appending).core/providers/openai/chat.go (1)
7-17: Message converters likely dropReasoningDetails(reasoning signatures won’t round-trip).
ConvertOpenAIMessagesToBifrostMessages/ConvertBifrostMessagesToOpenAIMessagesappear to mapReasoningbut notReasoningDetails, so any per-chunk details/signatures may be lost when requests/responses pass through OpenAI provider plumbing.Suggested fix (in
core/providers/openai/utils.go), extend the assistant-message mapping to includeReasoningDetails:- Reasoning: message.OpenAIChatAssistantMessage.Reasoning, + Reasoning: message.OpenAIChatAssistantMessage.Reasoning, + ReasoningDetails: message.OpenAIChatAssistantMessage.ReasoningDetails,(and the reverse direction similarly).
Also applies to: 20-33
♻️ Duplicate comments (16)
core/providers/cohere/chat.go (1)
104-121: Critical: Still using Anthropic's minimum token constant for Cohere validation.Lines 112-113 validate
reasoning.max_tokens >= anthropic.MinimumReasoningMaxTokens(1024), but Cohere's thinking feature accepts token budgets as low as 500. This validation will incorrectly reject valid Cohere requests with token budgets between 500-1023.Define a Cohere-specific constant and use it for validation:
+const ( + CohereMinimumReasoningMaxTokens = 500 +) + func ToCohereChatCompletionRequest(bifrostReq *schemas.BifrostChatRequest) (*CohereChatRequest, error) { ... - } else if *bifrostReq.Params.Reasoning.MaxTokens < anthropic.MinimumReasoningMaxTokens { - return nil, fmt.Errorf("reasoning.max_tokens must be greater than or equal to %d", anthropic.MinimumReasoningMaxTokens) + } else if *bifrostReq.Params.Reasoning.MaxTokens < CohereMinimumReasoningMaxTokens { + return nil, fmt.Errorf("reasoning.max_tokens must be greater than or equal to %d", CohereMinimumReasoningMaxTokens)core/providers/bedrock/utils.go (2)
10-10: Avoid importing Anthropic provider just forMinimumReasoningMaxTokens(shared constant).
core/providers/bedrock/utils.goimportscore/providers/anthropiconly forMinimumReasoningMaxTokens. Consider moving that constant to a shared package (e.g.,core/schemasorcore/constants) to reduce cross-provider coupling.Also applies to: 42-46
32-52:additionalModelRequestFieldPathsoverwritesreasoning_config(merge instead of replace).You set
bedrockReq.AdditionalModelRequestFields["reasoning_config"] = ..., but later dobedrockReq.AdditionalModelRequestFields = orderedFields, which can silently discardreasoning_config.if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = make(schemas.OrderedMap) + } + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } } }Also applies to: 94-99
core/providers/bedrock/types.go (1)
108-165: Bedrock reasoning types likely don’t match AWS union semantics (missingredactedContent, delta “one-of” enforcement).If AWS returns
redactedContent(or emits signature as a separate delta member), current structs may fail to unmarshal or may misrepresent stream events. Consider modelingreasoningContentas a union that supports at least:
reasoningText: { text, signature? }redactedContent: <base64 bytes>
and for deltas, ensure only one member is present per event.AWS Bedrock Converse API JSON schema for message.content[].reasoningContent (reasoningText vs redactedContent) AWS Bedrock ConverseStream JSON schema for contentBlockDelta.delta.reasoningContent (union members per delta event)Also applies to: 444-447
core/providers/anthropic/responses.go (2)
190-201: Don’t setParams.Reasoningto enabled whenBudgetTokensis nil (invalid intermediate state).Right now
Thinking.Type=="enabled"can produceReasoning{Effort:"auto", MaxTokens:nil}. Prefer only setting reasoning whenBudgetTokens != nil(and otherwise leave it unset or setnone).if request.Thinking != nil { - if request.Thinking.Type == "enabled" { + if request.Thinking.Type == "enabled" && request.Thinking.BudgetTokens != nil { params.Reasoning = &schemas.ResponsesParametersReasoning{ Effort: schemas.Ptr("auto"), MaxTokens: request.Thinking.BudgetTokens, } - } else { + } else if request.Thinking.Type != "enabled" { params.Reasoning = &schemas.ResponsesParametersReasoning{ Effort: schemas.Ptr("none"), } } }
325-344: Unify reasoning enablement rules (Effort==nil currently disables reasoning).
ToAnthropicResponsesRequestcurrently enables reasoning only whenEffort != nil && *Effort != "none". If callers sendreasoning: { max_tokens: ... }(no effort), this will disable thinking. Consider enabling when(Effort == nil || *Effort != "none")and then requiring/validatingMaxTokensin the enabled case (to align with the general “explicit none disables” convention).core/providers/openai/types.go (2)
56-65: Enforce mutual exclusivity for embedded message variants (tool vs assistant) to avoid invalid JSON.Relying on a comment is risky because both embedded pointers flatten into the same JSON object. Consider adding
func (m OpenAIMessage) Validate() error(or custom MarshalJSON) that errors if both are non-nil, and call it in request building paths.
74-101: Add nil-receiver guard toOpenAIChatRequest.MarshalJSON; verifyreasoning_effortmapping matches current OpenAI API.func (r *OpenAIChatRequest) MarshalJSON() ([]byte, error) { + if r == nil { + return []byte("null"), nil + } type Alias OpenAIChatRequest ... }OpenAI Chat Completions request parameter "reasoning_effort": supported values and whether "reasoning" object should be omitted from requestscore/providers/vertex/vertex.go (3)
512-544: Fix nilreqBodypanic in Vertex Anthropic streaming request builder.
IfreqBodyis nil,requestBodybecomes nil and Line 537 will panic when indexing the map.@@ reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true) @@ var requestBody map[string]interface{} if err := sonic.Unmarshal(reqBytes, &requestBody); err != nil { return nil, fmt.Errorf("failed to unmarshal request body: %w", err) } + if requestBody == nil { + return nil, fmt.Errorf("failed to build anthropic request body") + }
679-710: Fix nilreqBodypanic in Vertex Anthropic Responses request builder.
Same pattern: marshal/unmarshal nil => nil map => Line 704 panics.@@ reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment @@ if err := sonic.Unmarshal(reqBytes, &requestBody); err != nil { return nil, fmt.Errorf("failed to unmarshal request body: %w", err) } + if requestBody == nil { + return nil, fmt.Errorf("failed to build anthropic request body") + }
831-872: Fix nilreqBodypanic in Vertex Anthropic ResponsesStream request builder.
Same issue as above; additionally setsStream.@@ reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true) @@ if err := sonic.Unmarshal(reqBytes, &requestBody); err != nil { return nil, fmt.Errorf("failed to unmarshal request body: %w", err) } + if requestBody == nil { + return nil, fmt.Errorf("failed to build anthropic request body") + }core/providers/bedrock/responses.go (1)
452-482: Don’t type-mismatch / overwritereasoning_configin Bedrock request construction.
- Disabled case uses
map[string]string, while other paths (and typical readers) expectmap[string]any.additionalModelRequestFieldPathsoverwritesAdditionalModelRequestFields, dropping the earlierreasoning_config.@@ if bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort == "none" { - bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ "type": "disabled", } } else { @@ if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = make(schemas.OrderedMap) + } + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } } }core/schemas/chatcompletions.go (2)
186-220: Avoid hard-breaking clients that send bothreasoning_effortandreasoningduring migration.
Line 207-209 rejects requests with both fields, which can break “dual-write” clients during rollout; preferreasoningand ignorereasoning_effort(optionally log a deprecation warning).@@ - if aux.Alias != nil && aux.Alias.Reasoning != nil && aux.ReasoningEffort != nil { - return fmt.Errorf("both reasoning_effort and reasoning fields cannot be present at the same time") - } + if aux.Alias != nil && aux.Alias.Reasoning != nil && aux.ReasoningEffort != nil { + // Prefer new field; ignore deprecated field for a non-breaking migration. + // TODO: emit deprecation warning via project logger, if available at this layer. + aux.ReasoningEffort = nil + } @@ if aux.ReasoningEffort != nil { cp.Reasoning = &ChatReasoning{ Effort: aux.ReasoningEffort, } }
480-521: Enforce ChatMessage embedded-type exclusivity + clear stale pointers.
This unmarshaller can set bothcm.ChatToolMessageandcm.ChatAssistantMessage(Line 504-518), violating the stated invariant; also, it never resets embedded pointers to nil, which can leak prior state ifcmis reused.func (cm *ChatMessage) UnmarshalJSON(data []byte) error { + // Clear embedded pointers to avoid stale state on reuse. + cm.ChatToolMessage = nil + cm.ChatAssistantMessage = nil @@ if toolMsg.ToolCallID != nil { cm.ChatToolMessage = (*ChatToolMessage)(&toolMsg) } @@ if assistantMsg.Refusal != nil || assistantMsg.Reasoning != nil || len(assistantMsg.ReasoningDetails) > 0 || len(assistantMsg.Annotations) > 0 || len(assistantMsg.ToolCalls) > 0 { cm.ChatAssistantMessage = &assistantMsg } + + if cm.ChatToolMessage != nil && cm.ChatAssistantMessage != nil { + return fmt.Errorf("chat message cannot be both tool and assistant message") + } return nil }core/providers/anthropic/chat.go (2)
207-216: Filter ReasoningDetails before converting to Anthropicthinkingblocks.
Currently every entry becomes a thinking block even ifTypeisreasoning.summary/reasoning.encryptedorTextis nil/empty, which can yield invalid/empty blocks.if msg.ChatAssistantMessage != nil && msg.ChatAssistantMessage.ReasoningDetails != nil { for _, reasoningDetail := range msg.ChatAssistantMessage.ReasoningDetails { - content = append(content, AnthropicContentBlock{ - Type: AnthropicContentBlockTypeThinking, - Signature: reasoningDetail.Signature, - Thinking: reasoningDetail.Text, - }) + if reasoningDetail.Type != schemas.BifrostReasoningDetailsTypeText || + reasoningDetail.Text == nil || *reasoningDetail.Text == "" { + continue + } + content = append(content, AnthropicContentBlock{ + Type: AnthropicContentBlockTypeThinking, + Signature: reasoningDetail.Signature, + Thinking: reasoningDetail.Text, + }) } }
373-387: Don’t drop signature-only reasoning details in non-stream responses.
assistantMessage.ReasoningDetailsis only assigned whenreasoningText != "", so signature-only blocks never surface.- if reasoningText != "" { + if len(reasoningDetails) > 0 { if assistantMessage == nil { assistantMessage = &schemas.ChatAssistantMessage{} } - assistantMessage.Reasoning = &reasoningText assistantMessage.ReasoningDetails = reasoningDetails + if reasoningText != "" { + assistantMessage.Reasoning = &reasoningText + } }
🧹 Nitpick comments (2)
core/providers/azure/azure.go (1)
348-366: Prefer failing fast if Anthropic request conversion returns nil (avoid sendingnullJSON).
Now thatanthropic.ToAnthropic*Requestreturns(req, error), it’s reasonable to treatreq == nilas an error to prevent accidentalnullrequest bodies.reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true) return reqBody, nil(Apply the same pattern in the non-stream ChatCompletion branch and in Responses / ResponsesStream.)
Also applies to: 448-462, 526-548, 620-642
core/providers/bedrock/chat.go (1)
59-63: Optional: trim trailing newline in aggregatedassistantMessage.Reasoning.
Right nowreasoningTextends with"\n", which can be noisy for consumers/UI.Also applies to: 157-163
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (30)
core/providers/anthropic/anthropic.go(4 hunks)core/providers/anthropic/chat.go(11 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(5 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(4 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/semanticcache/utils.go(2 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (12)
- core/providers/anthropic/types.go
- core/providers/openai/utils.go
- ui/lib/types/logs.ts
- core/providers/bedrock/bedrock.go
- plugins/semanticcache/utils.go
- core/providers/openai/openai.go
- core/schemas/responses.go
- core/providers/anthropic/anthropic.go
- plugins/jsonparser/utils.go
- core/schemas/mux.go
- core/providers/cohere/cohere.go
- transports/bifrost-http/handlers/inference.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/maxim/main.gocore/providers/bedrock/responses.gocore/providers/cohere/types.gocore/providers/perplexity/chat.gocore/providers/bedrock/utils.gocore/providers/azure/azure.gocore/providers/openai/types.goframework/streaming/chat.gocore/providers/anthropic/errors.gocore/providers/anthropic/responses.gocore/providers/cohere/chat.gocore/providers/bedrock/chat.gocore/providers/openai/chat.gocore/providers/bedrock/types.gocore/providers/anthropic/chat.goui/app/workspace/logs/views/logChatMessageView.tsxcore/providers/vertex/vertex.gocore/schemas/chatcompletions.go
🧠 Learnings (2)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
plugins/maxim/main.gocore/providers/bedrock/responses.gocore/providers/cohere/types.gocore/providers/perplexity/chat.gocore/providers/bedrock/utils.gocore/providers/azure/azure.gocore/providers/openai/types.goframework/streaming/chat.gocore/providers/anthropic/errors.gocore/providers/anthropic/responses.gocore/providers/cohere/chat.gocore/providers/bedrock/chat.gocore/providers/openai/chat.gocore/providers/bedrock/types.gocore/providers/anthropic/chat.gocore/providers/vertex/vertex.gocore/schemas/chatcompletions.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/types.gocore/providers/openai/chat.go
🧬 Code graph analysis (10)
core/providers/bedrock/responses.go (4)
core/schemas/utils.go (3)
SafeExtractFromMap(519-525)SafeExtractString(280-297)SafeExtractInt(300-342)core/schemas/responses.go (1)
ResponsesParametersReasoning(233-238)core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/azure/azure.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/openai/types.go (1)
core/schemas/chatcompletions.go (6)
ChatMessageRole(457-457)ChatMessageContent(524-527)ChatToolMessage(621-623)ChatAssistantMessageAnnotation(670-673)ChatAssistantMessageToolCall(686-692)ChatReasoning(223-226)
framework/streaming/chat.go (3)
core/schemas/chatcompletions.go (1)
ChatAssistantMessage(626-632)core/utils.go (1)
Ptr(51-53)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)
core/providers/anthropic/errors.go (2)
ui/lib/types/logs.ts (1)
BifrostError(233-239)core/providers/anthropic/types.go (2)
AnthropicMessageError(376-379)AnthropicMessageErrorStruct(382-385)
core/providers/cohere/chat.go (3)
core/schemas/chatcompletions.go (13)
BifrostChatRequest(12-19)ChatMessage(469-478)ChatReasoning(223-226)ChatReasoningDetails(723-730)ChatContentBlock(590-597)ChatAssistantMessageToolCall(686-692)ChatContentBlockTypeText(582-582)ChatContentBlockTypeImage(583-583)ChatInputImage(600-603)ChatMessageContent(524-527)ChatAssistantMessageToolCallFunction(695-698)ChatAssistantMessage(626-632)ChatMessageRole(457-457)core/providers/cohere/types.go (5)
CohereChatRequest(14-32)CohereThinking(171-174)ThinkingTypeDisabled(181-181)ThinkingTypeEnabled(180-180)CohereMessage(51-57)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/bedrock/chat.go (1)
core/schemas/chatcompletions.go (6)
ChatReasoningDetails(723-730)BifrostReasoningDetailsTypeText(719-719)ChatContentBlockTypeText(582-582)ChatAssistantMessage(626-632)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)
core/providers/openai/chat.go (4)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/schemas/utils.go (1)
Ptr(16-18)core/utils.go (1)
Ptr(51-53)
ui/app/workspace/logs/views/logChatMessageView.tsx (2)
ui/lib/utils/validation.ts (2)
isJson(539-546)cleanJson(548-558)ui/app/workspace/logs/views/codeEditor.tsx (1)
CodeEditor(82-245)
core/schemas/chatcompletions.go (4)
core/providers/openai/types.go (1)
Alias(78-78)core/schemas/bifrost.go (2)
Alias(384-384)Alias(401-401)ui/lib/types/logs.ts (3)
ChatMessage(115-124)ChatMessageContent(113-113)ReasoningDetails(126-133)core/providers/gemini/types.go (2)
Role(13-13)Content(922-930)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (16)
ui/app/workspace/logs/views/logChatMessageView.tsx (1)
73-94: No changes needed. The code is correct as written.
ChatMessage.reasoningis typed asreasoning?: string;in the type definition, so the guardmessage.reasoning &&ensures it's a non-empty string before use. The call toisJson(message.reasoning)is safe since it expects a string and receives one. The fallback rendering of{message.reasoning}is also safe since it's guaranteed to be a string. No type coercion or additional guards are required.core/providers/cohere/types.go (1)
536-538: TheThinkingfield addition is correct and properly integrated.The new
Thinkingfield inCohereStreamContentfollows the same pattern as the existingTextfield and aligns with the thinking support already present inCohereContentBlock(line 153). The processing code inchat.go(lines 431-444) explicitly handles the optionalThinkingfield, mapping it to the unifiedReasoningfield inBifrostChatResponsefor streaming reasoning deltas. This is consistent with how other providers like Anthropic implement thinking/reasoning streaming support. The test configuration indicates reasoning scenarios are covered.core/providers/cohere/chat.go (5)
219-226: LGTM: Clean refactor to per-message conversion.The message conversion now properly delegates to
ToBifrostChatMessage(), which provides better encapsulation and reusability across streaming and non-streaming paths.
246-261: LGTM: Reasoning conversion correctly maps Cohere thinking to Bifrost schema.The conversion appropriately handles both disabled (
effort: "none") and enabled ("effort: "auto"+ token budget) states. The mapping is consistent with Cohere's binary thinking model.
350-354: LGTM: Response conversion uses refactored message method.Consistent with the per-message conversion approach introduced in
ToBifrostChatRequest.
431-454: LGTM: Thinking deltas properly propagate reasoning content and details.The streaming path now correctly emits both
Reasoning(text) andReasoningDetails(with type and metadata) for thinking content deltas, maintaining consistency with the non-streaming response structure.
564-682: LGTM: Comprehensive per-message conversion with proper nil-safety and reasoning support.The implementation correctly handles:
- Content conversion across string, block, and thinking formats
- Tool call extraction with defensive nil checks
- Reasoning details aggregation with proper indexing
- Optimized single text block to string conversion
The thinking block concatenation (lines 606-609) properly prepends newlines only when needed, addressing the trailing newline concern from previous review.
plugins/maxim/main.go (1)
472-490: Nil-safety fix looks good; consider omitting Code/Type when unknown (nil vs empty string pointer).Right now
Code: &code/Type: &errorTypewill be non-nil even when empty; if Maxim treats nil differently from"", consider keeping these nil unless populated.- code := "" - errorType := "" + var code *string + var errorType *string if bifrostErr.Error != nil { message = bifrostErr.Error.Message if bifrostErr.Error.Code != nil { - code = *bifrostErr.Error.Code + code = bifrostErr.Error.Code } if bifrostErr.Error.Type != nil { - errorType = *bifrostErr.Error.Type + errorType = bifrostErr.Error.Type } } genErr := logging.GenerationError{ Message: message, - Code: &code, - Type: &errorType, + Code: code, + Type: errorType, }core/providers/anthropic/errors.go (1)
9-37: Robust Anthropic error conversion with nil-safety.core/providers/anthropic/responses.go (1)
1422-1440: Nil-safe streaming error conversion looks good.core/providers/openai/chat.go (1)
61-75: Effort normalization looks good (nested nil-guards).
TheReasoning != nil && Reasoning.Effort != nilguard prevents panics and keeps the normalization localized.core/providers/bedrock/chat.go (2)
38-42: Good: propagate convertChatParameters errors.
Prevents silently emitting malformed Bedrock requests.
121-163: Reasoning extraction + streaming delta handling look solid (nil-guard + no spurious nil response).
TheReasoningText == nilguard and the “only return response when text/signature exists” logic reduce NPEs and awkward(nil, nil, false)returns.Also applies to: 335-389
core/providers/anthropic/chat.go (1)
11-117: Reasoning budget validation and “disabled” handling look correct.
TheMinimumReasoningMaxTokenscheck and explicit disabled mode match the expected Anthropic “thinking” semantics.core/providers/perplexity/chat.go (2)
220-223: The reasoning tokens mapping is correct; Perplexity API does not return reasoning details or signatures.Perplexity's response structure includes only
ReasoningTokens(a token count), not reasoning details or signatures like other providers (Anthropic, Bedrock, Cohere). The code at lines 220–223 correctly handles the available field. No changes needed.
30-36: No issues found. The implementation is correct for Perplexity's API capabilities.Perplexity's API does not support a
reasoning_max_tokensparameter—it only exposesreasoning_effort(low/medium/high) for its Sonar deep-research models, withmax_tokenscontrolling overall completion length. The mapping of "minimal" to "low" is appropriate for Perplexity's supported values. The code is consistent with howresponses.gohandles the same transformation and correctly implements the nestedReasoning.Effortstructure from the unified schema.Likely an incorrect or invalid review comment.
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
transports/bifrost-http/handlers/inference.go (1)
62-89: Back-compat bug: keepreasoning_effortas a known field to avoid leaking it intoextra_params.
Even ifschemas.ChatParametersstill supports the deprecatedreasoning_effort, removing it fromchatParamsKnownFieldsmeans requests using the old field will likely get it echoed intoChatParameters.ExtraParams(and potentially forwarded downstream), causing duplicates/undefined behavior during the deprecation window (and across this PR stack).var chatParamsKnownFields = map[string]bool{ @@ "prompt_cache_key": true, + "reasoning_effort": true, // deprecated but still accepted; keep out of extra_params "reasoning": true, "response_format": true, @@ }core/providers/azure/azure.go (1)
351-365: Don’t allow Anthropic converter to return(nil, nil)—fail fast with an error.
In multiple places,reqBody == nilcurrently falls through and returnsreqBody, nil, which risks sendingnullJSON to the provider.Suggested fix (apply in all 4 call sites):
reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - } - return reqBody, nil + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment + return reqBody, nil(and similarly for
ToAnthropicResponsesRequest, and the streaming cases where you also setreqBody.Stream = schemas.Ptr(true)).Also applies to: 448-463, 529-547, 627-642
core/providers/bedrock/bedrock.go (1)
771-887: Fix inconsistentChunkIndexsemantics between intermediate and final chunks.The schema defines
ChunkIndexas starting from 0 for streaming responses (percore/schemas/bifrost.go), but the current implementation is inconsistent: intermediate chunks correctly use 0-based indexing (ChunkIndex = chunkIndex), while the final chunk usesCreateBifrostChatCompletionChunkResponsewhich internally adds 1 to the index (ChunkIndex = currentChunkIndex + 1). This causes the final chunk to have a 1-based index while earlier chunks are 0-based.The root cause is that
CreateBifrostChatCompletionChunkResponseunconditionally increments the chunk index. Either this helper should not add 1, or the final response should passchunkIndex - 1to maintain consistency. The current suggested fix of adding+ 1to intermediate chunks would break the 0-based semantic expected by the schema.
♻️ Duplicate comments (18)
core/providers/bedrock/types.go (1)
88-113: Bedrock reasoning content/delta still doesn’t model AWS union semantics (and omits redacted/encrypted variant).
This matches the earlier concern: Bedrock “reasoningContent” (especially deltas) is a union where each event carries one of {text, signature, redactedContent}, and responses may include provider-redacted/encrypted content. Current structs still look too “combined” and still lackredactedContent.Also applies to: 157-165, 442-447
core/providers/openai/types.go (2)
56-65: Embedded pointer flattening needs an invariant guard (tool vs assistant).
If both embedded pointers are ever non-nil, JSON output becomes ambiguous/overlapping; a smallValidate()(or constructor-only pattern) would make this safer.
74-101: MakeMarshalJSONnil-receiver safe.
CallingMarshalJSONon a nil*OpenAIChatRequestcan panic (the method dereferencesrwhen checkingr.Reasoning). A short early-return keeps it robust.core/providers/bedrock/utils.go (2)
1-12: DecoupleMinimumReasoningMaxTokensfromcore/providers/anthropic.
core/providers/bedrock/utils.goimportscore/providers/anthropicsolely foranthropic.MinimumReasoningMaxTokens, creating cross-provider coupling. Consider moving the constant to a shared package (e.g.,core/schemasor acore/constants) and importing that instead.Also applies to: 42-46
32-52: Preservereasoning_config: avoid overwritingAdditionalModelRequestFieldsand use consistent map types.
bedrockReq.AdditionalModelRequestFields = orderedFields(Line 97) can wipe the earlierreasoning_config. Merge instead.- Prefer
map[string]anyfor thereasoning_configobject even in the disabled case for consistency.- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ "type": "disabled", } if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = make(schemas.OrderedMap) + } + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } } }Also applies to: 73-99
core/providers/bedrock/responses.go (1)
347-367: Fixreasoning_configmap type + preventAdditionalModelRequestFieldsoverwrite (drops reasoning).
- Use
map[string]anyfor disabledreasoning_config(Line 457) to match how you read it (Line 351).- Merge
additionalModelRequestFieldPathsinto existingAdditionalModelRequestFieldsinstead of replacing it (Line 480).- bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ "type": "disabled", } if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = make(schemas.OrderedMap) + } + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } } }Also applies to: 452-482
core/providers/vertex/vertex.go (3)
689-709: Same nil-body hazard inResponsesAnthropic path.
Even if it “shouldn’t happen”, this is a cheap guardrail to keep behavior consistent withChatCompletion.reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic responses request conversion returned nil") + } + reqBody.Model = deployment
847-871: Same nil-body hazard inResponsesStreamAnthropic path.reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic responses request conversion returned nil") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true)
518-525: Fail fast on nilreqBodyin Anthropic streaming path (don’t marshalnull).
Right now if the converter ever returns(nil, nil), you’ll send anullrequest body to Vertex, which is harder to debug and can mask upstream issues.reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic chat request conversion returned nil") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true)core/providers/cohere/chat.go (2)
12-15: Signature change looks fine; error message is acceptable as a defensive guard.
104-121: Use Cohere-specific minimum token budget (don’t reuse Anthropic’s 1024).
This validation can reject valid Cohere requests (and the error message would be misleading).+const CohereMinimumThinkingTokenBudget = 500 // TODO: confirm from Cohere docs ... - } else if *bifrostReq.Params.Reasoning.MaxTokens < anthropic.MinimumReasoningMaxTokens { - return nil, fmt.Errorf("reasoning.max_tokens must be greater than or equal to %d", anthropic.MinimumReasoningMaxTokens) + } else if *bifrostReq.Params.Reasoning.MaxTokens < CohereMinimumThinkingTokenBudget { + return nil, fmt.Errorf("reasoning.max_tokens must be greater than or equal to %d", CohereMinimumThinkingTokenBudget) } else {core/providers/anthropic/responses.go (2)
190-201: Don’t setParams.Reasoningto enabled whenBudgetTokensis nil.
This can create an invalid intermediate Bifrost request (Effort="auto"butMaxTokens=nil).- if request.Thinking != nil { - if request.Thinking.Type == "enabled" { + if request.Thinking != nil { + if request.Thinking.Type == "enabled" && request.Thinking.BudgetTokens != nil { params.Reasoning = &schemas.ResponsesParametersReasoning{ Effort: schemas.Ptr("auto"), MaxTokens: request.Thinking.BudgetTokens, } - } else { + } else { params.Reasoning = &schemas.ResponsesParametersReasoning{ Effort: schemas.Ptr("none"), } } }
325-344: Unify reasoning enable/disable semantics with the chat converter.
Current condition enables reasoning only whenEffort != nil && Effort != "none", soEffort==nil+MaxTokens!=nilbecomes “disabled” here, which can diverge fromToAnthropicChatRequest. Consider enabling whenMaxTokens != nilas well, and disabling only whenEffort=="none".- if bifrostReq.Params.Reasoning != nil { - if bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort != "none" { - if bifrostReq.Params.Reasoning.MaxTokens != nil { + if bifrostReq.Params.Reasoning != nil { + // Disable only when explicitly requested. + if bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort == "none" { + anthropicReq.Thinking = &AnthropicThinking{Type: "disabled"} + } else if bifrostReq.Params.Reasoning.MaxTokens != nil { if *bifrostReq.Params.Reasoning.MaxTokens < MinimumReasoningMaxTokens { return nil, fmt.Errorf("reasoning.max_tokens must be greater than or equal to %d", MinimumReasoningMaxTokens) - } else { - anthropicReq.Thinking = &AnthropicThinking{ - Type: "enabled", - BudgetTokens: bifrostReq.Params.Reasoning.MaxTokens, - } - } - } else { - return nil, fmt.Errorf("reasoning.max_tokens is required for reasoning") - } - } else { - anthropicReq.Thinking = &AnthropicThinking{ - Type: "disabled", - } + } + anthropicReq.Thinking = &AnthropicThinking{ + Type: "enabled", + BudgetTokens: bifrostReq.Params.Reasoning.MaxTokens, + } + } else { + return nil, fmt.Errorf("reasoning.max_tokens is required for reasoning") } }core/schemas/chatcompletions.go (2)
186-220: Confirm the “both fields present” behavior is the intended contract.
Right now requests that send bothreasoning_effortandreasoningwill hard-fail. If you want a non-breaking migration, preferreasoningand ignorereasoning_effort(optionally with a warning).
480-521: Enforce exclusivity:ChatMessage.UnmarshalJSONcan set both embedded pointers.
This violates the “only one embedded struct non-nil” invariant and can cause ambiguous marshaling later.func (cm *ChatMessage) UnmarshalJSON(data []byte) error { @@ if toolMsg.ToolCallID != nil { cm.ChatToolMessage = (*ChatToolMessage)(&toolMsg) } @@ if assistantMsg.Refusal != nil || assistantMsg.Reasoning != nil || len(assistantMsg.ReasoningDetails) > 0 || len(assistantMsg.Annotations) > 0 || len(assistantMsg.ToolCalls) > 0 { cm.ChatAssistantMessage = &assistantMsg } + + // Enforce invariant: only one embedded pointer may be set + if cm.ChatToolMessage != nil && cm.ChatAssistantMessage != nil { + return fmt.Errorf("chat message cannot be both tool message (tool_call_id) and assistant message (tool_calls/reasoning/etc.)") + } return nil }core/providers/anthropic/chat.go (3)
207-216: Don’t emit invalid/empty Anthropicthinkingblocks fromReasoningDetails.
Right now everyReasoningDetailsentry becomes a thinking block even ifTypeisn’t text orTextis nil/empty, which can produce empty/invalid Anthropic content blocks.// First add reasoning details if msg.ChatAssistantMessage != nil && msg.ChatAssistantMessage.ReasoningDetails != nil { for _, reasoningDetail := range msg.ChatAssistantMessage.ReasoningDetails { + if reasoningDetail.Type != schemas.BifrostReasoningDetailsTypeText || + reasoningDetail.Text == nil || + *reasoningDetail.Text == "" { + continue + } content = append(content, AnthropicContentBlock{ Type: AnthropicContentBlockTypeThinking, Signature: reasoningDetail.Signature, Thinking: reasoningDetail.Text, }) } }
343-387: Signature-only reasoning details are dropped in non-stream Anthropic → Bifrost responses.
assistantMessage.ReasoningDetails = reasoningDetailsis only set whenreasoningText != "", so signature-only blocks never make it into the Bifrost response.- if reasoningText != "" { + if len(reasoningDetails) > 0 { if assistantMessage == nil { assistantMessage = &schemas.ChatAssistantMessage{} } - assistantMessage.Reasoning = &reasoningText assistantMessage.ReasoningDetails = reasoningDetails + if reasoningText != "" { + assistantMessage.Reasoning = &reasoningText + } }
413-426: Cache token attribution looks wrong (cache creation/read are input-side, not completion-side).
CacheCreationInputTokensandCacheReadInputTokensare “input” tokens; mappingCacheCreationInputTokensintoCompletionTokensDetails.CachedTokensis likely semantically incorrect and can break billing/telemetry semantics.if response.Usage != nil { bifrostResponse.Usage = &schemas.BifrostLLMUsage{ PromptTokens: response.Usage.InputTokens, PromptTokensDetails: &schemas.ChatPromptTokensDetails{ - CachedTokens: response.Usage.CacheReadInputTokens, + // Both are input-side cache tokens. + CachedTokens: response.Usage.CacheReadInputTokens + response.Usage.CacheCreationInputTokens, }, CompletionTokens: response.Usage.OutputTokens, - CompletionTokensDetails: &schemas.ChatCompletionTokensDetails{ - CachedTokens: response.Usage.CacheCreationInputTokens, - }, TotalTokens: response.Usage.InputTokens + response.Usage.OutputTokens, } }#!/bin/bash set -euo pipefail # Locate other providers’ cache token mapping for consistency. rg -n --type go 'Cache(Read|Creation)InputTokens|PromptTokensDetails|CompletionTokensDetails' -C2 core/providers
🧹 Nitpick comments (10)
transports/bifrost-http/handlers/inference.go (1)
267-288: Consider logging (at debug) when skipping unknown fields due to unmarshal errors.
Right nowextractExtraParamssilently drops fields it can’t unmarshal (continue), which can make debugging provider-specific params painful.framework/streaming/chat.go (1)
34-105: Consider deep-copying pointer fields when mergingReasoningDetailsto avoid aliasing.
When initializingexistingReasoningDetail.Text/Summary/Data/SignaturefromreasoningDetail.*, prefer copying the string value into a new pointer (similar to howRefusal/Reasoningare handled), instead of assigning the incoming pointer directly.Example pattern:
- existingReasoningDetail.Text = reasoningDetail.Text + existingReasoningDetail.Text = bifrost.Ptr(*reasoningDetail.Text)(similarly for
Summary,Data,Signature).core/providers/bedrock/chat.go (1)
59-61: Avoid trailing newline in aggregatedassistantMessage.Reasoning.
reasoningText += ... + "\n"leaves a trailing newline in the final value. Consider joining without a tail newline for more stable UI/log rendering.- reasoningText += contentBlock.ReasoningContent.ReasoningText.Text + "\n" + if len(reasoningText) > 0 { + reasoningText += "\n" + } + reasoningText += contentBlock.ReasoningContent.ReasoningText.TextAlso applies to: 121-133, 157-163
core/providers/vertex/vertex.go (1)
283-295: Good: handleToAnthropicChatRequesterror and set deployment model.
One tweak: ifreqBody == nil, the error message should reflect “conversion returned nil” (not “input not provided”), sincerequestis already present here.- if reqBody == nil { - return nil, fmt.Errorf("chat completion input is not provided") - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic chat request conversion returned nil") + }core/providers/cohere/chat.go (3)
247-261: Only setReasoning.MaxTokenswhenTokenBudgetis present.
Right now you can emitEffort="auto"withMaxTokens=nil, which is an odd “enabled but unspecified” state.} else { bifrostReq.Params.Reasoning = &schemas.ChatReasoning{ Effort: schemas.Ptr("auto"), } - if req.Thinking.TokenBudget != nil { - bifrostReq.Params.Reasoning.MaxTokens = req.Thinking.TokenBudget - } + bifrostReq.Params.Reasoning.MaxTokens = req.Thinking.TokenBudget }
413-454: Stream deltas: consider emittingReasoningDetailsconsistently (ToolPlanDelta too).
content.thinkingemits bothReasoningandReasoningDetails, buttool_planemits onlyReasoning. If downstream/UI logic prefersreasoning_details, you’ll lose structure here.case StreamEventToolPlanDelta: if chunk.Delta != nil && chunk.Delta.Message != nil && chunk.Delta.Message.ToolPlan != nil { streamResponse := &schemas.BifrostChatResponse{ Object: "chat.completion.chunk", Choices: []schemas.BifrostResponseChoice{{ Index: 0, ChatStreamResponseChoice: &schemas.ChatStreamResponseChoice{ Delta: &schemas.ChatStreamResponseChoiceDelta{ Reasoning: chunk.Delta.Message.ToolPlan, + ReasoningDetails: []schemas.ChatReasoningDetails{{ + Index: 0, + Type: schemas.BifrostReasoningDetailsTypeText, + Text: chunk.Delta.Message.ToolPlan, + }}, }, }, }}, }Also applies to: 466-467
564-683: SettoolCall.Typefor generated tool calls to match the rest of the API.
schemas.ChatAssistantMessageToolCall.Typeis populated elsewhere as"function"; leaving it nil can cause downstream assumptions/tests to diverge.bifrostToolCall := schemas.ChatAssistantMessageToolCall{ Index: uint16(len(toolCalls)), + Type: schemas.Ptr("function"), ID: toolCall.ID, Function: schemas.ChatAssistantMessageToolCallFunction{ Name: functionName, Arguments: functionArguments, }, }core/providers/anthropic/chat.go (3)
99-117: Reasoning validation: consider clearer/typed errors and compatibility expectations.
Returning rawfmt.Errorf(...)here may undermine the “error handling standardization” target branch goals; consider routing through a provider-typed error helper (if available) so callers can map to consistent HTTP status / error codes.
637-677: Streaming:ReasoningDetails.Indexis always0(may break multi-block thinking/signature).
If Anthropic emits multiple thinking/signature blocks over time, hardcoding index0can collapse distinct blocks.
742-749: Streaming signature deltas: don’t assume signature is always atReasoningDetails[0].
If multiple reasoning details exist, emitting the first can miss the actual signature update.- } else if len(delta.ReasoningDetails) > 0 && delta.ReasoningDetails[0].Signature != nil && *delta.ReasoningDetails[0].Signature != "" { + } else if len(delta.ReasoningDetails) > 0 { // Handle signature deltas + var sig *string + for i := range delta.ReasoningDetails { + if delta.ReasoningDetails[i].Signature != nil && *delta.ReasoningDetails[i].Signature != "" { + sig = delta.ReasoningDetails[i].Signature + break + } + } + if sig == nil { + break + } streamResp.Type = "content_block_delta" streamResp.Index = &choice.Index streamResp.Delta = &AnthropicStreamDelta{ Type: AnthropicStreamDeltaTypeSignature, - Signature: delta.ReasoningDetails[0].Signature, + Signature: sig, } - } else if len(delta.ToolCalls) > 0 { + } else if len(delta.ToolCalls) > 0 {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (30)
core/providers/anthropic/anthropic.go(4 hunks)core/providers/anthropic/chat.go(11 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(5 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(4 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/semanticcache/utils.go(2 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (8)
- core/schemas/responses.go
- ui/app/workspace/logs/views/logChatMessageView.tsx
- core/providers/anthropic/anthropic.go
- core/providers/perplexity/chat.go
- plugins/maxim/main.go
- core/providers/openai/openai.go
- core/providers/openai/utils.go
- plugins/semanticcache/utils.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/handlers/inference.gocore/schemas/mux.gocore/providers/cohere/cohere.gocore/providers/cohere/types.gocore/providers/vertex/vertex.goframework/streaming/chat.gocore/providers/anthropic/types.gocore/providers/openai/chat.gocore/providers/openai/types.gocore/providers/bedrock/utils.gocore/providers/azure/azure.gocore/providers/anthropic/errors.gocore/providers/bedrock/bedrock.gocore/schemas/chatcompletions.gocore/providers/bedrock/responses.goplugins/jsonparser/utils.goui/lib/types/logs.tscore/providers/bedrock/chat.gocore/providers/cohere/chat.gocore/providers/anthropic/responses.gocore/providers/anthropic/chat.gocore/providers/bedrock/types.go
🧠 Learnings (3)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/inference.gocore/schemas/mux.gocore/providers/cohere/cohere.gocore/providers/cohere/types.gocore/providers/vertex/vertex.goframework/streaming/chat.gocore/providers/anthropic/types.gocore/providers/openai/chat.gocore/providers/openai/types.gocore/providers/bedrock/utils.gocore/providers/azure/azure.gocore/providers/anthropic/errors.gocore/providers/bedrock/bedrock.gocore/schemas/chatcompletions.gocore/providers/bedrock/responses.goplugins/jsonparser/utils.gocore/providers/bedrock/chat.gocore/providers/cohere/chat.gocore/providers/anthropic/responses.gocore/providers/anthropic/chat.gocore/providers/bedrock/types.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/inference.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/chat.gocore/providers/openai/types.go
🧬 Code graph analysis (13)
core/providers/cohere/cohere.go (2)
core/providers/cohere/chat.go (1)
ToCohereChatCompletionRequest(12-204)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/vertex/vertex.go (4)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/utils.go (1)
Ptr(51-53)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
framework/streaming/chat.go (4)
core/schemas/chatcompletions.go (1)
ChatAssistantMessage(626-632)core/utils.go (1)
Ptr(51-53)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)core/providers/gemini/types.go (1)
Type(778-778)
core/providers/openai/chat.go (4)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/utils.go (1)
Ptr(51-53)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/openai/types.go (1)
core/schemas/chatcompletions.go (10)
ChatMessageRole(457-457)ChatMessageContent(524-527)ChatToolMessage(621-623)ChatAssistantMessageAnnotation(670-673)ChatAssistantMessageToolCall(686-692)Alias(189-189)Alias(643-643)Alias(770-770)Alias(860-860)ChatReasoning(223-226)
core/providers/bedrock/utils.go (3)
core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)OrderedMap(268-268)core/providers/bedrock/types.go (2)
BedrockConverseRequest(48-62)BedrockError(390-394)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/azure/azure.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-403)
core/providers/anthropic/errors.go (1)
core/providers/anthropic/types.go (2)
AnthropicMessageError(376-379)AnthropicMessageErrorStruct(382-385)
core/providers/bedrock/bedrock.go (3)
core/providers/utils/utils.go (1)
CreateBifrostChatCompletionChunkResponse(687-716)core/schemas/bifrost.go (1)
ChatCompletionStreamRequest(90-90)core/schemas/models.go (1)
Model(109-129)
core/schemas/chatcompletions.go (4)
core/providers/openai/types.go (1)
Alias(78-78)core/schemas/bifrost.go (2)
Alias(384-384)Alias(401-401)ui/lib/types/logs.ts (3)
ChatMessage(115-124)ChatMessageContent(113-113)ReasoningDetails(126-133)core/providers/gemini/types.go (2)
Role(13-13)Content(922-930)
core/providers/bedrock/responses.go (5)
core/schemas/utils.go (3)
SafeExtractFromMap(519-525)SafeExtractString(280-297)SafeExtractInt(300-342)core/schemas/responses.go (1)
ResponsesParametersReasoning(233-238)ui/lib/types/logs.ts (1)
ResponsesParametersReasoning(519-526)core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/bedrock/chat.go (1)
core/schemas/chatcompletions.go (4)
ChatReasoningDetails(723-730)BifrostReasoningDetailsTypeText(719-719)ChatContentBlockTypeText(582-582)ChatAssistantMessage(626-632)
core/providers/anthropic/responses.go (3)
core/schemas/responses.go (2)
ResponsesParametersReasoning(233-238)BifrostResponsesRequest(32-39)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/types.go (3)
MinimumReasoningMaxTokens(14-14)AnthropicThinking(65-68)AnthropicStreamError(397-400)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (31)
core/providers/openai/chat.go (3)
8-17: Good: avoids leaking OpenAI-specific message struct into Bifrost schema.
Switching toConvertOpenAIMessagesToBifrostMessages(request.Messages)keepsschemas.BifrostChatRequest.Inputconsistent with the rest of the core pipeline and preserves assistant/tool metadata viaChatAssistantMessage/ChatToolMessage.
21-33: Good: request conversion now round-trips via the new OpenAI message wrapper.
ConvertBifrostMessagesToOpenAIMessages(bifrostReq.Input)should reduce shape drift as more provider-specific assistant fields get added.
62-65: Sanity-check provider intent for"minimal" → "low"coercion.
This looks correct for the “OpenAI-shaped request to non-OpenAI provider” paths (Gemini/Mistral/Vertex) where"minimal"may be unsupported, but worth confirming it shouldn’t happen for true OpenAI outbound requests.core/providers/cohere/types.go (1)
534-539: Looks right: streaming content can now carry thinking deltas.
AddingThinking *stringaligns withCohereContentBlockTypeThinkingand keeps the struct flexible for both content-start and delta events.transports/bifrost-http/handlers/inference.go (2)
169-199: Good: custom unmarshal makes embedded + custom-unmarshal fields deterministic under sonic.
This is a pragmatic fix givenChatParameters’ custom unmarshalling and the embeddedBifrostParamsfield flattening.
475-482: Good: ensuresChatParametersis non-nil before unmarshal paths rely on it.
Helps avoid nil edge cases whenChatParameters.UnmarshalJSONexpects a receiver.ui/lib/types/logs.ts (1)
115-133: Type additions are consistent with the backend “reasoning” rollout.
No concerns with addingreasoning/reasoning_detailstoChatMessagefor log surfacing.core/providers/anthropic/types.go (2)
11-15: Good: shared minimum makes reasoning budget validation consistent across providers.
ExposingMinimumReasoningMaxTokensis a clean way to centralize the Anthropic “thinking budget” floor used elsewhere in this stack.
39-59: No concerns withOutputFormatwiring here.
Field presence is gated by the beta header comment, and the change is non-invasive.core/providers/cohere/cohere.go (2)
269-287: Good: don’t swallow request-conversion errors.
PropagatingToCohereChatCompletionRequesterrors prevents hard-to-debug nil/partial requests later in the call chain.
327-339: Good: only markStream=trueafter successful request build.
Prevents sending a malformed streaming request body when conversion fails.core/providers/anthropic/errors.go (1)
9-48: LGTM: defensive nil-handling in Anthropic error conversion.
This properly avoids panics whenbifrostError nestedbifrostErr.Erroris nil, and enriches parsed errors consistently.plugins/jsonparser/utils.go (1)
303-323: LGTM: deep-copy updated forDelta.Reasoning.core/schemas/mux.go (1)
710-869: LGTM: reasoning round-trip now preservesMaxTokensand streaming usesDelta.Reasoning.Also applies to: 1361-1372
core/providers/bedrock/chat.go (3)
39-41: Good: propagateconvertChatParameterserrors to caller.
Keeps Bedrock request conversion consistent with the “conversion can fail” refactor across providers.
139-143: Nice: normalize single-textcontentBlocksback tocontentStr.
This keeps the response payload minimal and consistent with other provider converters.
333-388: Streaming reasoning delta handling looks correct (no nil response leaks).
The early return when bothTextandSignatureare absent prevents accidental(nil, nil, false)responses with ambiguous meaning.core/providers/anthropic/responses.go (3)
296-299: Good: nil request now returns an error.
Prevents silentnildereferences and matches the broader “conversion can fail” approach.
402-403: Good: updated return signature is correctly wired.
1428-1439: Good: nil-safe extraction of streaming error message.
Avoids panics whenbifrostErr.Erroris nil.core/providers/cohere/chat.go (4)
203-204: Good: request converter now returns(req, err)and exits cleanly.
219-226: Good: reuse per-message conversion instead of duplicating mapping logic.
339-341: Good: initializeChatNonStreamResponseChoiceexplicitly.
351-354: Good: response message conversion is centralized now.core/schemas/chatcompletions.go (6)
165-166: Good:ChatParameters.Reasoningis now first-class.
222-226:ChatReasoningshape looks good for cross-provider needs.
627-667: Good: synthesizereasoning_detailswhen onlyreasoningis provided.
This makes downstream rendering/normalization simpler.
714-730: Reasoning details type/constants look consistent with UI typing.
757-794: Good: streaming delta also synthesizesreasoning_details.
835-836: Good:ImageTokensaddition is additive and optional.core/providers/anthropic/chat.go (1)
13-16: All call sites forToAnthropicChatRequesthave been correctly updated with proper error handling.The signature change to return
(*AnthropicMessageRequest, error)has been consistently implemented across the codebase. All six call sites inazure.go,anthropic.go, andvertex.goproperly handle the error return value by either checking for errors and propagating them or returning them directly through callback functions.
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
d592b74 to
4c3f30d
Compare
Pratham-Mishra04
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
0f9320e to
14a7a70
Compare
TejasGhatte
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
4c3f30d to
b584c84
Compare
TejasGhatte
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
14a7a70 to
0f9320e
Compare
Pratham-Mishra04
force-pushed
the
12-01-refactor_core_providers_error_handling_standardization
branch
from
0f9320e to
be363a1
Compare
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
b584c84 to
0ea65ac
Compare
TejasGhatte
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
0ea65ac to
e3cbb41
Compare
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
e3cbb41 to
0ea65ac
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (12)
transports/bifrost-http/handlers/inference.go (2)
62-89: Don’t forward deprecatedreasoning_effortasExtraParams.
Right nowreasoning_effortis not inchatParamsKnownFields, so legacy clients will have it copied intoschemas.ChatParameters.ExtraParamsviaextractExtraParams(), even thoughschemas.ChatParameters.UnmarshalJSONalready consumes it. This can leak a deprecated OpenAI-ish field into provider requests unexpectedly.var chatParamsKnownFields = map[string]bool{ "model": true, "messages": true, "fallbacks": true, "stream": true, @@ "prompt_cache_key": true, "reasoning": true, + // Deprecated alias; keep as known so it doesn't land in ExtraParams. + "reasoning_effort": true, "response_format": true, @@ }Also applies to: 267-288
980-1071: Pass request type tohandleStreamingResponseto avoid sending[DONE]for Responses API when stream closes without chunks.Responses streams can complete without emitting chunks (edge case). The current logic infers
includeEventTypefrom the last chunk received; if none arrive, it remainsfalseand sends[DONE], which violates the Responses API spec. Per documented behavior inrouter.go: "OpenAI 'responses' API and Anthropic messages API: they signal completion by simply closing the stream, not sending[DONE]."Refactor
handleStreamingResponseto accept a request type parameter and decide DONE-vs-no-DONE by endpoint type, not by last chunk presence.core/providers/anthropic/errors.go (1)
39-50: GuardparseAnthropicErroragainst a nilbifrostErrreturn.
IfproviderUtils.HandleProviderAPIErrorcan ever returnnil, thebifrostErr.Error == nilcheck will panic. Consider a defensive fallback.func parseAnthropicError(resp *fasthttp.Response) *schemas.BifrostError { var errorResp AnthropicError bifrostErr := providerUtils.HandleProviderAPIError(resp, &errorResp) + if bifrostErr == nil { + bifrostErr = &schemas.BifrostError{} + } if errorResp.Error != nil { if bifrostErr.Error == nil { bifrostErr.Error = &schemas.ErrorField{} } bifrostErr.Error.Type = &errorResp.Error.Type bifrostErr.Error.Message = errorResp.Error.Message } return bifrostErr }core/providers/azure/azure.go (4)
348-366: Don’t allow Anthropic request bodies to silently become JSONnull.
IfreqBodyis nil witherr == nil, return an explicit error (otherwise you’ll POSTnull).reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment return reqBody, nil
448-466: Same: fail early if Anthropic streamingreqBodyis nil.
Otherwise streaming path can also POSTnull.reqBody, err := anthropic.ToAnthropicChatRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true) return reqBody, nil
526-548: Same issue forResponses: nil body should be an error.reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment return reqBody, nil
620-645: Same issue forResponsesStream: nil body should be an error.reqBody, err := anthropic.ToAnthropicResponsesRequest(request) if err != nil { return nil, err } - if reqBody != nil { - reqBody.Model = deployment - reqBody.Stream = schemas.Ptr(true) - } + if reqBody == nil { + return nil, fmt.Errorf("anthropic request conversion returned nil") + } + reqBody.Model = deployment + reqBody.Stream = schemas.Ptr(true) return reqBody, nilcore/providers/bedrock/responses.go (1)
452-499: Fix disabledreasoning_configtype + avoid overwriting it from ExtraParams.
- Writing
map[string]stringbreaks the earlier.(map[string]interface{})read.additionalModelRequestFieldPathscan replace the whole map, droppingreasoning_config.if bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort == "none" { - bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]string{ + bedrockReq.AdditionalModelRequestFields["reasoning_config"] = map[string]any{ "type": "disabled", } } else { ... } if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = make(schemas.OrderedMap) + } + for k, v := range orderedFields { + // Don't clobber reasoning_config set from Params.Reasoning unless explicitly intended. + if k == "reasoning_config" { + continue + } + bedrockReq.AdditionalModelRequestFields[k] = v + } } }core/providers/vertex/vertex.go (3)
512-545: Streaming Anthropic path should also fail fast whenreqBody == nil.
Right now, nil would flow into marshal/unmarshal with a confusing error. Mirror the non-stream path’s explicit nil check.
679-710: Same:ResponsesAnthropic path should reject nilreqBody.
Otherwisesonic.Unmarshalintomap[string]interface{}can fail on"null".
831-872: Same:ResponsesStreamAnthropic path should reject nilreqBody.core/providers/anthropic/responses.go (1)
184-201: Don’t setparams.Reasoningto “auto” whenthinking.type=="enabled"butbudget_tokensis nilRight now (Line 191-195) you create
Effort:"auto"withMaxTokens:nil, which violates your own schema comment (“required for anthropic”) and can break round-trips/logging. Prefer: only setparams.ReasoningwhenBudgetTokens != nil; otherwise leave it unset (or setEffort:"none"only when explicitly disabled).- if request.Thinking != nil { - if request.Thinking.Type == "enabled" { - params.Reasoning = &schemas.ResponsesParametersReasoning{ - Effort: schemas.Ptr("auto"), - MaxTokens: request.Thinking.BudgetTokens, - } - } else { - params.Reasoning = &schemas.ResponsesParametersReasoning{ - Effort: schemas.Ptr("none"), - } - } - } + if request.Thinking != nil { + if request.Thinking.Type == "enabled" { + if request.Thinking.BudgetTokens != nil { + params.Reasoning = &schemas.ResponsesParametersReasoning{ + Effort: schemas.Ptr("auto"), + MaxTokens: request.Thinking.BudgetTokens, + } + } + } else { + params.Reasoning = &schemas.ResponsesParametersReasoning{ + Effort: schemas.Ptr("none"), + } + } + }
♻️ Duplicate comments (10)
core/providers/bedrock/utils.go (2)
3-12: Avoid importingcore/providers/anthropicjust for a constant.
This keeps an unnecessary cross-provider dependency alive (anthropic.MinimumReasoningMaxTokens). Consider moving the constant to a shared package (e.g.,core/schemasorcore/constants) and importing that from providers.Search in the repo: where is `MinimumReasoningMaxTokens` referenced across providers, and is there already a shared constants package used by multiple providers?Also applies to: 42-46
32-52:reasoning_configcan still be clobbered by ExtraParams assignment.
You setbedrockReq.AdditionalModelRequestFields["reasoning_config"]first, but later overwrite the whole map whenadditionalModelRequestFieldPathsis present. This silently drops the reasoning configuration.if bifrostReq.Params != nil && bifrostReq.Params.ExtraParams != nil { if requestFields, exists := bifrostReq.Params.ExtraParams["additionalModelRequestFieldPaths"]; exists { if orderedFields, ok := schemas.SafeExtractOrderedMap(requestFields); ok { - bedrockReq.AdditionalModelRequestFields = orderedFields + if bedrockReq.AdditionalModelRequestFields == nil { + bedrockReq.AdditionalModelRequestFields = make(schemas.OrderedMap) + } + for k, v := range orderedFields { + bedrockReq.AdditionalModelRequestFields[k] = v + } } }Also applies to: 73-100
core/providers/bedrock/types.go (1)
88-113: Bedrock reasoning-content modeling looks incomplete for real-world payloads (missingredactedContent/ delta union semantics).
If AWS emits redacted/encrypted reasoning or signature-only deltas, current structs may fail to represent (and in some cases may not decode cleanly). Suggest aligning with AWS API/SDK union members and addingredactedContentsupport at least.type BedrockReasoningContent struct { ReasoningText *BedrockReasoningContentText `json:"reasoningText,omitempty"` + // Provider-encrypted reasoning payloads. + RedactedContent *string `json:"redactedContent,omitempty"` } type BedrockReasoningContentText struct { - Text string `json:"text"` + Text string `json:"text"` Signature *string `json:"signature,omitempty"` }AWS Bedrock Converse/ConverseStream: what is the exact JSON shape of `reasoningContent` for (1) non-streaming ContentBlock and (2) streaming ContentBlockDelta? Specifically: are `text`, `signature`, and `redactedContent` siblings in one object, or union members sent separately?Also applies to: 157-165, 442-447
core/schemas/chatcompletions.go (3)
154-220: Backward-compat: don’t error when bothreasoningandreasoning_effortare present.
Given the deprecation/migration goal, rejecting requests that include both fields is likely an avoidable breaking change. Preferreasoning(new) and ignorereasoning_effort(old) when both are provided.- if aux.Alias != nil && aux.Alias.Reasoning != nil && aux.ReasoningEffort != nil { - return fmt.Errorf("both reasoning_effort and reasoning fields cannot be present at the same time") - } + // If both are present, prefer `reasoning` and ignore the deprecated alias. if aux.ReasoningEffort != nil { - cp.Reasoning = &ChatReasoning{ - Effort: aux.ReasoningEffort, - } + if cp.Reasoning == nil { + cp.Reasoning = &ChatReasoning{Effort: aux.ReasoningEffort} + } } - // ExtraParams etc. are already handled by the alias + // Note: ExtraParams is json:"-" and must be populated elsewhere. return nil
218-219: Fix misleading comment aboutExtraParams.
ExtraParamsisjson:"-", so it is not populated by the alias unmarshal.
480-521: Enforce the “only one embedded pointer” invariant (and clear old values).
This unmarshal can still set bothChatToolMessageandChatAssistantMessagefor malformed inputs, and it never clears previous embedded pointers if the same struct instance is reused.func (cm *ChatMessage) UnmarshalJSON(data []byte) error { // Unmarshal the base fields directly + cm.ChatToolMessage = nil + cm.ChatAssistantMessage = nil @@ if toolMsg.ToolCallID != nil { cm.ChatToolMessage = (*ChatToolMessage)(&toolMsg) } @@ if assistantMsg.Refusal != nil || assistantMsg.Reasoning != nil || len(assistantMsg.ReasoningDetails) > 0 || len(assistantMsg.Annotations) > 0 || len(assistantMsg.ToolCalls) > 0 { cm.ChatAssistantMessage = &assistantMsg } + + if cm.ChatToolMessage != nil && cm.ChatAssistantMessage != nil { + return fmt.Errorf("invalid chat message: both tool_call_id and assistant fields are set") + } return nil }core/providers/openai/types.go (1)
56-65: Add runtime enforcement for the “only one embedded pointer” invariantThe comment (Line 61-64) warns about collisions, but nothing prevents
ChatToolMessageandOpenAIChatAssistantMessagebeing set together. Add aValidate()(or customMarshalJSON) and call it before serialization.core/providers/anthropic/chat.go (2)
474-487: Bug: signature-onlyReasoningDetailsare still dropped (guard requiresText != nil)The condition at Line 477 currently prevents emitting thinking blocks when only
Signatureis present. Remove thereasoningDetail.Text != nilguard.- if reasoningDetail.Type == schemas.BifrostReasoningDetailsTypeText && reasoningDetail.Text != nil && - ((reasoningDetail.Text != nil && *reasoningDetail.Text != "") || - (reasoningDetail.Signature != nil && *reasoningDetail.Signature != "")) { + if reasoningDetail.Type == schemas.BifrostReasoningDetailsTypeText && + ((reasoningDetail.Text != nil && *reasoningDetail.Text != "") || + (reasoningDetail.Signature != nil && *reasoningDetail.Signature != "")) { content = append(content, AnthropicContentBlock{ Type: AnthropicContentBlockTypeThinking, Thinking: reasoningDetail.Text, Signature: reasoningDetail.Signature, }) }
415-428: Usage mapping:CacheCreationInputTokensshould not be stored under completion cached tokens
CacheCreationInputTokensis an input/prompt cache-creation metric; putting it inCompletionTokensDetailsmakes dashboards/consumers misinterpret it. Consider mapping both cache-read and cache-create intoPromptTokensDetails(with separate fields if you have them), or at least don’t attribute creation to completion.bifrostResponse.Usage = &schemas.BifrostLLMUsage{ PromptTokens: response.Usage.InputTokens, PromptTokensDetails: &schemas.ChatPromptTokensDetails{ CachedTokens: response.Usage.CacheReadInputTokens, }, CompletionTokens: response.Usage.OutputTokens, - CompletionTokensDetails: &schemas.ChatCompletionTokensDetails{ - CachedTokens: response.Usage.CacheCreationInputTokens, - }, TotalTokens: response.Usage.InputTokens + response.Usage.OutputTokens, } + if response.Usage.CacheCreationInputTokens > 0 { + // If you only have one cached token bucket, treat cache-creation as prompt-side. + bifrostResponse.Usage.PromptTokensDetails.CachedTokens += response.Usage.CacheCreationInputTokens + }core/providers/anthropic/responses.go (1)
325-346: Fix reasoning enablement semantics: current logic dropsmax_tokens-only reasoning and differs fromanthropic/chat.go
ToAnthropicChatRequesttreatsEffort == nilas “enabled unless explicitly none” and requiresMaxTokens; this converter (Line 326-345) only does anything whenEffort != nil, so{ reasoning: { max_tokens: 1024 } }becomes “no thinking” here but “thinking enabled” in chat. Unify them.- if bifrostReq.Params.Reasoning != nil { - if bifrostReq.Params.Reasoning.Effort != nil { - if *bifrostReq.Params.Reasoning.Effort != "none" { - if bifrostReq.Params.Reasoning.MaxTokens != nil { - if *bifrostReq.Params.Reasoning.MaxTokens < MinimumReasoningMaxTokens { - return nil, fmt.Errorf("reasoning.max_tokens must be greater than or equal to %d", MinimumReasoningMaxTokens) - } else { - anthropicReq.Thinking = &AnthropicThinking{ - Type: "enabled", - BudgetTokens: bifrostReq.Params.Reasoning.MaxTokens, - } - } - } else { - return nil, fmt.Errorf("reasoning.max_tokens is required for reasoning") - } - } else { - anthropicReq.Thinking = &AnthropicThinking{ - Type: "disabled", - } - } - } - } + if bifrostReq.Params.Reasoning != nil { + // Disable only when explicitly requested. + if bifrostReq.Params.Reasoning.Effort != nil && *bifrostReq.Params.Reasoning.Effort == "none" { + anthropicReq.Thinking = &AnthropicThinking{Type: "disabled"} + } else { + if bifrostReq.Params.Reasoning.MaxTokens == nil { + return nil, fmt.Errorf("reasoning.max_tokens is required for reasoning") + } + if *bifrostReq.Params.Reasoning.MaxTokens < MinimumReasoningMaxTokens { + return nil, fmt.Errorf("reasoning.max_tokens must be greater than or equal to %d", MinimumReasoningMaxTokens) + } + anthropicReq.Thinking = &AnthropicThinking{ + Type: "enabled", + BudgetTokens: bifrostReq.Params.Reasoning.MaxTokens, + } + } + }
🧹 Nitpick comments (5)
transports/bifrost-http/handlers/inference.go (1)
169-199: ChatRequest custom unmarshal is reasonable; consider reducing repeated decoding if it shows up in profiles.
Triplesonic.Unmarshalper request (params, messages, then ChatParameters) is probably fine, but it’s a hot path. If p99 matters, consider a single pass into a temp struct that embedsBifrostParams+[]ChatMessage+schemas.ChatParametersand then assigns.Also applies to: 474-513
core/providers/bedrock/utils.go (1)
32-52: Consider treating empty{ "reasoning": {} }as “no-op” instead of error.
CurrentlyReasoning != nilwith neithereffortnormax_tokenswill error. If clients send an empty object during migrations, this becomes a sharp edge.core/providers/cohere/types.go (1)
534-539: Enforce delta invariants fortextvsthinking.
WithThinkingadded, consider validating (at decode/convert time) that only one ofText/Thinkingis set per delta (and ideally thatTypematches), to prevent ambiguous chunk handling.core/providers/openai/utils.go (1)
5-45: Good utility split; consider enforcing “only one embedded message struct” invariant.
To avoid producing invalidschemas.ChatMessage/openai.OpenAIMessagewhen inputs are malformed, consider only copyingChatToolMessageforrole=tool, and only copyingOpenAIChatAssistantMessage/ChatAssistantMessageforrole=assistant. (Not mappingReasoningDetailshere is correct, per repo learnings.)core/providers/bedrock/chat.go (1)
59-63: Reasoning extraction is now nil-safe; consider avoiding a trailing newline inreasoningText
Right now you always append"\n"(Line 132-133). If consumers display this verbatim, it’ll render with a trailing blank line.Also applies to: 121-133, 157-163
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (30)
core/providers/anthropic/anthropic.go(4 hunks)core/providers/anthropic/chat.go(11 hunks)core/providers/anthropic/errors.go(1 hunks)core/providers/anthropic/responses.go(5 hunks)core/providers/anthropic/types.go(2 hunks)core/providers/azure/azure.go(4 hunks)core/providers/bedrock/bedrock.go(4 hunks)core/providers/bedrock/chat.go(5 hunks)core/providers/bedrock/responses.go(4 hunks)core/providers/bedrock/types.go(3 hunks)core/providers/bedrock/utils.go(4 hunks)core/providers/cohere/chat.go(10 hunks)core/providers/cohere/cohere.go(2 hunks)core/providers/cohere/types.go(1 hunks)core/providers/openai/chat.go(3 hunks)core/providers/openai/openai.go(2 hunks)core/providers/openai/types.go(2 hunks)core/providers/openai/utils.go(1 hunks)core/providers/perplexity/chat.go(1 hunks)core/providers/vertex/vertex.go(4 hunks)core/schemas/chatcompletions.go(7 hunks)core/schemas/mux.go(3 hunks)core/schemas/responses.go(2 hunks)framework/streaming/chat.go(1 hunks)plugins/jsonparser/utils.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/semanticcache/utils.go(2 hunks)transports/bifrost-http/handlers/inference.go(5 hunks)ui/app/workspace/logs/views/logChatMessageView.tsx(1 hunks)ui/lib/types/logs.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (8)
- core/providers/perplexity/chat.go
- ui/app/workspace/logs/views/logChatMessageView.tsx
- plugins/maxim/main.go
- core/providers/openai/openai.go
- plugins/semanticcache/utils.go
- core/providers/anthropic/types.go
- core/schemas/mux.go
- core/providers/cohere/cohere.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
core/providers/cohere/types.gocore/providers/anthropic/anthropic.gocore/schemas/responses.gocore/providers/openai/utils.goui/lib/types/logs.tstransports/bifrost-http/handlers/inference.gocore/providers/anthropic/errors.goframework/streaming/chat.gocore/providers/openai/chat.gocore/providers/bedrock/responses.gocore/providers/anthropic/responses.gocore/providers/bedrock/types.gocore/providers/bedrock/utils.gocore/schemas/chatcompletions.gocore/providers/bedrock/bedrock.gocore/providers/vertex/vertex.gocore/providers/cohere/chat.gocore/providers/azure/azure.gocore/providers/openai/types.goplugins/jsonparser/utils.gocore/providers/bedrock/chat.gocore/providers/anthropic/chat.go
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
core/providers/cohere/types.gocore/providers/anthropic/anthropic.gocore/schemas/responses.gocore/providers/openai/utils.gotransports/bifrost-http/handlers/inference.gocore/providers/anthropic/errors.goframework/streaming/chat.gocore/providers/openai/chat.gocore/providers/bedrock/responses.gocore/providers/anthropic/responses.gocore/providers/bedrock/types.gocore/providers/bedrock/utils.gocore/schemas/chatcompletions.gocore/providers/bedrock/bedrock.gocore/providers/vertex/vertex.gocore/providers/cohere/chat.gocore/providers/azure/azure.gocore/providers/openai/types.goplugins/jsonparser/utils.gocore/providers/bedrock/chat.gocore/providers/anthropic/chat.go
📚 Learning: 2025-12-08T11:50:35.996Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 978
File: core/providers/openai/utils.go:26-45
Timestamp: 2025-12-08T11:50:35.996Z
Learning: OpenAI does not support the `ReasoningDetails` field. `ReasoningDetails` is a Bifrost-internal field and should not be included when converting messages to OpenAI format in the `ConvertBifrostMessagesToOpenAIMessages` function in `core/providers/openai/utils.go`.
Applied to files:
core/providers/openai/utils.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/utils.gocore/providers/openai/chat.gocore/providers/openai/types.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/inference.go
📚 Learning: 2025-12-12T10:28:54.988Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 978
File: core/providers/anthropic/chat.go:207-216
Timestamp: 2025-12-12T10:28:54.988Z
Learning: In core/providers/anthropic/chat.go, within ToAnthropicChatRequest, ensure the mapping from ReasoningDetails to Anthropic thinking blocks preserves all fields from ReasoningDetails, including when Text is nil or empty. Do not drop or coerce missing Text; propagate nil/empty values to maintain complete bidirectional mapping of Anthropic events in Bifrost. Add tests to verify that ReasoningDetails with Text nil or "" round-trip without loss, and update code to handle nullable Text without dropping data. If there are multiple blocks, preserve order and all fields during mapping.
Applied to files:
core/providers/anthropic/chat.go
🧬 Code graph analysis (16)
core/providers/anthropic/anthropic.go (4)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/utils.go (1)
Ptr(51-53)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-405)
core/providers/openai/utils.go (2)
core/providers/openai/types.go (2)
OpenAIMessage(56-65)OpenAIChatAssistantMessage(67-72)core/schemas/chatcompletions.go (3)
ChatMessage(469-478)ChatToolMessage(621-623)ChatAssistantMessage(626-632)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/chatcompletions.go (2)
ChatMessage(469-478)ChatParameters(155-184)ui/lib/types/logs.ts (1)
ChatMessage(115-124)
core/providers/anthropic/errors.go (1)
core/providers/anthropic/types.go (2)
AnthropicMessageError(376-379)AnthropicMessageErrorStruct(382-385)
framework/streaming/chat.go (2)
core/schemas/chatcompletions.go (1)
ChatAssistantMessage(626-632)core/utils.go (1)
Ptr(51-53)
core/providers/openai/chat.go (4)
core/providers/openai/utils.go (2)
ConvertOpenAIMessagesToBifrostMessages(5-24)ConvertBifrostMessagesToOpenAIMessages(26-45)core/schemas/chatcompletions.go (1)
ChatParameters(155-184)core/utils.go (1)
Ptr(51-53)core/schemas/utils.go (1)
Ptr(16-18)
core/providers/bedrock/responses.go (5)
core/schemas/utils.go (5)
SafeExtractFromMap(519-525)SafeExtractString(280-297)SafeExtractInt(300-342)Ptr(16-18)IsAnthropicModel(1043-1045)core/schemas/responses.go (1)
ResponsesParametersReasoning(233-238)ui/lib/types/logs.ts (1)
ResponsesParametersReasoning(519-526)core/schemas/chatcompletions.go (1)
OrderedMap(268-268)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/anthropic/responses.go (4)
core/schemas/responses.go (2)
ResponsesParametersReasoning(233-238)BifrostResponsesRequest(32-39)ui/lib/types/logs.ts (1)
ResponsesParametersReasoning(519-526)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/types.go (3)
AnthropicMessageRequest(40-59)AnthropicThinking(65-68)AnthropicStreamError(397-400)
core/providers/bedrock/utils.go (4)
core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)OrderedMap(268-268)core/providers/bedrock/types.go (2)
BedrockConverseRequest(48-62)BedrockError(390-394)core/schemas/utils.go (1)
IsAnthropicModel(1043-1045)core/providers/anthropic/types.go (1)
MinimumReasoningMaxTokens(14-14)
core/providers/bedrock/bedrock.go (2)
core/providers/utils/utils.go (1)
CreateBifrostChatCompletionChunkResponse(687-716)core/schemas/bifrost.go (1)
ChatCompletionStreamRequest(90-90)
core/providers/vertex/vertex.go (5)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/schemas/models.go (1)
Model(109-129)core/utils.go (1)
Ptr(51-53)core/schemas/utils.go (1)
Ptr(16-18)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-405)
core/providers/cohere/chat.go (2)
core/schemas/chatcompletions.go (15)
ChatMessage(469-478)ChatReasoning(223-226)BifrostChatResponse(26-41)BifrostResponseChoice(704-712)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)ChatReasoningDetails(723-730)ChatContentBlock(590-597)ChatAssistantMessageToolCall(686-692)ChatContentBlockTypeText(582-582)ChatContentBlockTypeImage(583-583)ChatInputImage(600-603)ChatMessageContent(524-527)ChatAssistantMessage(626-632)ChatMessageRole(457-457)core/providers/cohere/types.go (4)
CohereChatRequest(14-32)CohereThinking(171-174)ThinkingTypeEnabled(180-180)CohereMessage(51-57)
core/providers/azure/azure.go (2)
core/providers/anthropic/chat.go (1)
ToAnthropicChatRequest(13-278)core/providers/anthropic/responses.go (1)
ToAnthropicResponsesRequest(296-405)
core/providers/openai/types.go (2)
core/schemas/models.go (1)
Model(109-129)core/schemas/chatcompletions.go (10)
ChatMessageRole(457-457)ChatMessageContent(524-527)ChatToolMessage(621-623)ChatAssistantMessageAnnotation(670-673)ChatAssistantMessageToolCall(686-692)Alias(189-189)Alias(643-643)Alias(770-770)Alias(860-860)ChatReasoning(223-226)
core/providers/bedrock/chat.go (2)
core/schemas/chatcompletions.go (6)
ChatReasoningDetails(723-730)BifrostReasoningDetailsTypeText(719-719)ChatContentBlockTypeText(582-582)ChatAssistantMessage(626-632)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)ui/lib/types/logs.ts (1)
ReasoningDetails(126-133)
core/providers/anthropic/chat.go (3)
core/schemas/chatcompletions.go (12)
BifrostChatRequest(12-19)ChatAssistantMessage(626-632)ChatAssistantMessageToolCall(686-692)ChatContentBlock(590-597)ChatReasoningDetails(723-730)ChatAssistantMessageToolCallFunction(695-698)ChatToolTypeFunction(239-239)ChatMessageContent(524-527)ChatMessage(469-478)ChatNonStreamResponseChoice(745-748)ChatStreamResponseChoice(751-753)ChatStreamResponseChoiceDelta(756-763)core/providers/anthropic/types.go (8)
AnthropicMessageRequest(40-59)AnthropicThinking(65-68)MinimumReasoningMaxTokens(14-14)AnthropicContentBlock(147-159)AnthropicContentBlockTypeThinking(143-143)AnthropicMessageResponse(273-282)AnthropicContentBlockTypeText(135-135)AnthropicContentBlockTypeToolUse(137-137)core/providers/anthropic/utils.go (1)
ConvertAnthropicFinishReasonToBifrost(27-32)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (16)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (36)
ui/lib/types/logs.ts (1)
115-133: LGTM: UI log types align with backendreasoning+reasoning_details.
Field names and shapes match the new API surface and keep optionality appropriate for logs.core/providers/bedrock/utils.go (1)
702-710: LGTM: safer nested error message extraction.
This avoids nil deref and keeps Bedrock error payload stable.core/schemas/chatcompletions.go (2)
627-667: LGTM: synthesizingreasoning_detailsfromreasoningreduces downstream branching.
This makes logs/stream aggregation consistent when providers emit only a raw reasoning string.Also applies to: 756-794
829-840: LGTM:ImageTokensis additive and backwards-safe.
No concerns with the optional pointer field.core/providers/bedrock/bedrock.go (1)
771-772: LGTM: UUID generation for Bedrock stream identification.Bedrock doesn't provide a unique stream identifier, so generating one locally using
uuid.New().String()is a reasonable approach. The comment clearly documents this design decision, and the ID is used consistently for both per-chunk responses and the final chunk.core/schemas/responses.go (2)
233-238: LGTM: Reasoning configuration properly documented.The
ResponsesParametersReasoningstruct now includes:
Effortwith clear documentation of allowed values including "none"MaxTokensfor Anthropic reasoning budget tokensThe comment clarifies that
MaxTokensis required for Anthropic, which aligns with the validation logic incore/providers/anthropic/responses.gothat enforcesMinimumReasoningMaxTokens.
324-326: LGTM: Model-specific reasoning content block documentation.The comment clarifies the distinction between gpt-oss models (reasoning_text content blocks) and other OpenAI models (summaries + encrypted_content). This helps consumers understand provider-specific behavior.
core/providers/anthropic/anthropic.go (4)
307-311: LGTM: Error propagation from ToAnthropicChatRequest.The converter function now returns
(*AnthropicMessageRequest, error), and this change properly propagates any conversion errors (such as reasoning validation failures) to the caller. This aligns with the updated signature incore/providers/anthropic/chat.go.
357-364: LGTM: Streaming request conversion with error handling.The lambda now properly:
- Handles the error from
ToAnthropicChatRequest- Returns early on error
- Sets
Stream = trueonly on successThis pattern ensures validation errors from reasoning configuration are surfaced before the streaming request is made.
659-663: LGTM: Error propagation from ToAnthropicResponsesRequest.Consistent with the ChatCompletion change, the Responses endpoint now propagates conversion errors properly.
707-714: LGTM: ResponsesStream conversion with proper error handling.The streaming responses path follows the same pattern as ChatCompletionStream, correctly handling errors from the converter before setting the Stream flag.
core/providers/openai/chat.go (3)
14-14: LGTM: Message conversion using helper function.Using
ConvertOpenAIMessagesToBifrostMessagesproperly maps OpenAI-specific message fields (includingOpenAIChatAssistantMessagewith Reasoning, Refusal, Annotations, ToolCalls) to the BifrostChatMessageschema.
28-28: LGTM: Bidirectional message conversion.Using
ConvertBifrostMessagesToOpenAIMessagesensures consistent round-trip conversion and properly mapsChatAssistantMessagefields back to OpenAI format.
63-65: LGTM: Updated reasoning parameter handling with proper nil checks.The nested nil checks for
ReasoningandReasoning.Effortprevent nil pointer dereferences. The "minimal" to "low" conversion maintains compatibility with providers that don't support the "minimal" effort level (which is OpenAI-specific).framework/streaming/chat.go (3)
39-43: LGTM: Refusal handling with proper pointer initialization.Using
bifrost.Ptr(*chunk.Delta.Refusal)correctly creates a new pointer for the initial refusal value, avoiding aliasing issues with the original chunk data.
44-54: LGTM: Reasoning accumulation across stream chunks.The pattern correctly:
- Initializes
ChatAssistantMessageif nil- Creates a new pointer for the first reasoning chunk
- Accumulates subsequent reasoning text via string concatenation
This mirrors the established pattern for Refusal handling.
55-105: LGTM: ReasoningDetails accumulation with index-based merging.The implementation correctly handles streaming reasoning details:
- Text/Summary/Data: Accumulated via string concatenation across chunks
- Signature: Overwritten (correct since signatures are final values)
- Type: Overwritten on each update (correct since type should be consistent)
- New entries: Appended when index doesn't exist
The index-based approach ensures that partial updates for the same reasoning block are properly merged while allowing multiple distinct reasoning blocks to coexist.
core/providers/anthropic/errors.go (1)
9-37: Nil-safe error conversion looks solid.
This avoids panics onbifrostErr.Type/bifrostErr.Errorand returns a stable Anthropic-shaped error.plugins/jsonparser/utils.go (1)
303-323: Rename alignment (Thought → Reasoning) looks correct.
No additional concerns in this copy path given the plugin only mutatesContent.core/providers/bedrock/responses.go (2)
347-368:reasoning_configextraction logic is fine, but depends on consistent map types.
Looks good assumingreasoning_configis always amap[string]any/map[string]interface{}.
658-666: Output conversion placement is an improvement.
Moving output mapping up front makes the response assembly easier to reason about and reduces duplicated conversion risks.core/providers/vertex/vertex.go (1)
281-299: Nice: explicit nil check avoids hidden marshal failures.
This is the right pattern for the Anthropic converter call-site.core/providers/anthropic/responses.go (2)
296-299: Good:ToAnthropicResponsesRequestnow nil-checks and returns an error
This prevents silent panics at call-sites and aligns with the broader stack refactor towards error-returning converters.
1430-1441: Good: nil-safe message extraction for SSE error formatting
Avoids nil deref whenbifrostErr.Erroris nil.core/providers/openai/types.go (2)
42-44: Watch for downstream breakage:OpenAIChatRequest.Messagesis now[]OpenAIMessage
Call-sites building requests must populate the newOpenAIMessageshape (esp. tool messages / assistant extras) rather than plainschemas.ChatMessage.
74-104:MarshalJSONcorrectly emitsreasoning_effortand omitsreasoning(plus nil-safety)
Looks solid and keeps request payload compatible while still allowing internalReasoningusage.core/providers/bedrock/chat.go (2)
39-41: Good:convertChatParameterserrors are now propagated
This matches the stack-wide move to error-returning converters and avoids silently sending malformed requests.
335-389: Good: reasoning delta emission no longer returns an unexpected nil response pointer
The early-return guard (Line 339-342) fixes the prior “nil response” footgun.core/providers/cohere/chat.go (4)
11-14: Good: request converter now returns an error instead of constructing invalid requests
Matches the stack’s standardized error handling.Also applies to: 200-201
103-118: Confirm Cohere semantics: do we really want to requirereasoning.max_tokensto enable thinking?
Cohere’sthinking.token_budgetis typically optional; this enforces a stricter contract at the Bifrost layer. If intentional, great—otherwise consider allowing enabled-without-budget and letting Cohere decide defaults.
410-451: Good: streaming now forwards Cohere “thinking” intoReasoning+ReasoningDetails
This keeps the UI/logging path consistent with other providers.Also applies to: 463-464
561-672: Nice:ToBifrostChatMessagepreserves thinking blocks into structuredReasoningDetails(without trailing newline)
The join-style concatenation avoids the trailing newline issue seen elsewhere.core/providers/anthropic/chat.go (4)
13-16: Good: converter now returns an error for nil request/input instead of panicking
Fits the stack’s “converter functions return errors” direction.Also applies to: 277-278
99-117: Reasoning validation looks right for Anthropic, but keep chat/responses semantics aligned
This treatsEffort == nilas “enabled unless explicitly none,” which is the behaviorresponses.goshould mirror too (currently it doesn’t).
207-216: Preserving allReasoningDetailsinto Anthropic thinking blocks is correct here
This matches the repo learning to keep bidirectional fidelity even for nil/emptyText. Based on learnings.
630-682: Good: streaming now carries reasoning text + signature deltas in both directions
The added delta mappings preserve verification signatures instead of dropping them.Also applies to: 714-754
Merge activity
|
Pratham-Mishra04
changed the base branch from
12-01-refactor_core_providers_error_handling_standardization
to
graphite-base/978
Pratham-Mishra04
force-pushed
the
12-01-feat_added_reasoning_support_to_chat_completions
branch
from
0ea65ac to
bfbcf49
Compare
Pratham-Mishra04
deleted the
12-01-feat_added_reasoning_support_to_chat_completions
branch
Summary
Added support for reasoning capabilities across providers, with a focus on Anthropic's thinking feature. This PR introduces a unified reasoning interface in the Bifrost schema and implements provider-specific mappings.
Changes
reasoningfield toChatParametersto replace the deprecatedreasoning_effortfieldType of change
Affected areas
How to test
Test reasoning capabilities with Anthropic models:
Screenshots/Recordings
N/A
Breaking changes
The
reasoning_effortfield is now deprecated in favor of the newreasoningobject. Existing code usingreasoning_effortwill continue to work but should be updated to use the new structure.Related issues
N/A
Security considerations
Reasoning content is properly sanitized and displayed in the UI.
Checklist
docs/contributing/README.mdand followed the guidelines