feat: send back raw request in extra fields #1010
Conversation
This stack of pull requests is managed by Graphite. Learn more about stacking. |
|
Caution Review failedThe pull request is closed. 📝 WalkthroughSummary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings. WalkthroughAdds a per-provider SendBackRawRequest flag and end-to-end wiring to capture, persist, and optionally surface raw request payloads across provider handlers, utils, streaming finalization, logging, DB migrations, HTTP transport, and UI; introduces ExtraFields.RawRequest and a runtime/context override key. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Client
participant HTTP as "Bifrost HTTP"
participant Config as "ConfigStore/DB"
participant Provider
participant Utils as "providerUtils"
participant Stream as "Streaming Framework"
participant Logs as "LogStore/Plugins"
Client->>HTTP: API request
HTTP->>Config: load provider config (includes SendBackRawRequest)
HTTP->>Provider: forward request (context may include override)
Provider->>Utils: HandleProviderResponse(responseBody, response, requestBody, sendBackRawRequest, sendBackRawResponse)
Utils-->>Provider: (rawRequest?, rawResponse?, error)
Provider->>Stream: finalize response (attach ExtraFields.RawRequest when present)
Stream->>Client: emit final ProcessedStreamResponse (includes RawRequest if final)
Provider->>Logs: persist raw_request when enabled
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (45)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (8)
core/providers/utils/utils.go (1)
344-407: Critical bug:rawRequestis populated fromresponseBodyinstead ofrequestBody.The function signature accepts
requestBody []byte(line 344), but it's never used. Instead,rawRequestis populated by unmarshallingresponseBody(line 359):rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) // Wrong: uses responseBodyThis defeats the purpose of the "send back raw request" feature. The raw request should contain the actual request payload sent to the provider, not the response.
Apply this diff to fix the logic:
go func() { defer wg.Done() if sendBackRawRequest { - rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) + rawRequestErr = sonic.Unmarshal(requestBody, &rawRequest) } }()Additionally, callers need to pass the actual request body instead of
nil:-rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &elevenlabsResponse, nil, provider.sendBackRawRequest, ...) +rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &elevenlabsResponse, jsonData, provider.sendBackRawRequest, ...)core/providers/mistral/mistral.go (1)
105-118: Same issue:listModelsByKeypopulatesRawRequestincorrectly for GET requests.Similar to the Elevenlabs provider,
HandleProviderResponseis called withnilfor the request body, but the current implementation (as noted inutils.go) unmarshals the response body intorawRequest. For aListModelsGET request, there is no request body, soRawRequestshould benilor the logic should skip setting it entirely.This is a consequence of the bug in
HandleProviderResponsenoted earlier.Once
HandleProviderResponseis fixed to userequestBody, this code should naturally handle the case correctly (passingnilwould result in norawRequestbeing set).core/providers/bedrock/bedrock.go (1)
452-468: Same issue:listModelsByKeypopulatesRawRequestincorrectly.Like other providers,
HandleProviderResponseis called withnilfor the request body parameter. Due to the bug inHandleProviderResponse, this will incorrectly populateRawRequestfrom the response body for this GET request.This will be resolved when the
HandleProviderResponsebug is fixed.core/providers/elevenlabs/elevenlabs.go (1)
103-120: Critical bug inHandleProviderResponse:rawRequestis populated from response body instead of request body.The utility function at
core/providers/utils/utils.go:344has a semantic error where therequestBodyparameter is accepted but never used. Instead, line 359 unmarshal'sresponseBodyintorawRequest:if sendBackRawRequest { rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) // Bug: should use requestBody }This means:
- For GET requests like
ListModels,RawRequestwill incorrectly contain the response data instead of beingnilor empty.- For all POST/PUT requests,
RawRequestwill contain the response data instead of the actual request body that was sent.- The
requestBodyparameter is dead code and should be used instead ofresponseBody.This affects all providers using
HandleProviderResponse(30+ call sites across elevenlabs, gemini, openai, anthropic, cohere, bedrock, vertex, and others). Fix the utility function to userequestBodyforrawRequestandresponseBodyforrawResponse.core/providers/cohere/cohere.go (1)
546-569: Critical: Duplicate code blocks setting raw request/response.Lines 546-569 contain the same logic repeated three times. This appears to be a copy-paste error. The raw request and raw response should only be set once each.
Apply this diff to remove the duplicate code:
// Set raw request if enabled if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { bifrostResponse.ExtraFields.RawRequest = rawRequest } - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - bifrostResponse.ExtraFields.RawResponse = rawResponse - } - - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - bifrostResponse.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - bifrostResponse.ExtraFields.RawResponse = rawResponse - } - // Set raw response if enabled if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { bifrostResponse.ExtraFields.RawResponse = rawResponse }core/providers/vertex/vertex.go (1)
820-830: Duplicate raw response assignment.
RawResponseis set twice in this block - once at lines 821-823 and again at lines 828-830. Remove the duplicate assignment.// Set raw response if enabled if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { response.ExtraFields.RawResponse = rawResponse } if request.Model != deployment { response.ExtraFields.ModelDeployment = deployment } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } - return response, nilcore/providers/openrouter/openrouter.go (1)
100-119: Same bug:HandleProviderResponseunmarshals response body as raw request.Similar to the Vertex provider,
HandleProviderResponseis called withnilas the request body parameter, but the utility function unmarshalsresponseBody(notrequestBody) when building the raw request. The returnedrawRequestwill contain the response data, not the actual request payload.Additionally, calling
ShouldSendBackRawRequesttwice (once at line 100, once at line 112) is redundant - consider storing the result in a variable.+ shouldSendRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + shouldSendRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + var openrouterResponse schemas.BifrostListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &openrouterResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &openrouterResponse, nil, shouldSendRawRequest, shouldSendRawResponse) if bifrostErr != nil { return nil, bifrostErr } for i := range openrouterResponse.Data { openrouterResponse.Data[i].ID = string(schemas.OpenRouter) + "/" + openrouterResponse.Data[i].ID } openrouterResponse.ExtraFields.Latency = latency.Milliseconds() // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if shouldSendRawRequest { openrouterResponse.ExtraFields.RawRequest = rawRequest } // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if shouldSendRawResponse { openrouterResponse.ExtraFields.RawResponse = rawResponse }core/providers/azure/azure.go (1)
197-218: Fix HandleProviderResponse to unmarshal requestBody for rawRequest, not responseBody.The
HandleProviderResponsefunction at line 359 ofcore/providers/utils/utils.goincorrectly unmarshalsresponseBodyintorawRequest. This should unmarshalrequestBodyinstead, otherwiseRawRequestwill contain response data rather than the actual request payload. While this specific call at line 197 passesnilforrequestBody(since it's a GET request), the bug affects other callers that pass actual request data, such as line 274 in the same file.
🧹 Nitpick comments (9)
framework/changelog.md (1)
2-2: Consider refining the changelog entry to emphasize the user-facing feature.The entry "feat: support raw request configuration and logging" accurately describes the implementation aspects but could be clearer about the primary user-facing capability: sending raw requests back in API response extra fields. This would align better with the PR title and objectives.
Consider updating the entry to:
-feat: support raw request configuration and logging +feat: send back raw request in extra fields and support loggingThis makes it explicit that the primary feature is returning raw requests to callers, with logging as a supporting capability.
core/providers/bedrock/bedrock.go (1)
718-731: MissingRawRequestin non-streamingChatCompletionandResponsespaths.The non-streaming
ChatCompletion(lines 718-731) andResponses(lines 952-967) methods setRawResponsewhen enabled but don't setRawRequest. For consistency with the streaming paths and the feature's intent, these should also include the raw request whenShouldSendBackRawRequestis true.Add raw request handling to non-streaming paths:
// Set raw response if enabled if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { var rawResponse interface{} if err := sonic.Unmarshal(responseBody, &rawResponse); err == nil { bifrostResponse.ExtraFields.RawResponse = rawResponse } } + +// Set raw request if enabled +if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + providerUtils.ParseAndSetRawRequest(&bifrostResponse.ExtraFields, jsonData) +}Also applies to: 952-967
core/providers/anthropic/anthropic.go (1)
204-204: Consider calling Should helper functions before HandleProviderResponse to avoid unnecessary work.*Across multiple methods (listModelsByKey, TextCompletion, ChatCompletion, Responses),
provider.sendBackRawRequestandprovider.sendBackRawResponseare passed directly to HandleProviderResponse, but then the Should* helper functions are checked again before setting ExtraFields (e.g., lines 214-216, 287-290).This means HandleProviderResponse may unnecessarily unmarshal raw request/response data even when the context indicates they shouldn't be sent back. For efficiency and consistency with the context-override mechanism, consider calling the Should* functions before HandleProviderResponse.
Example for ChatCompletion (similar changes apply to other methods):
+ shouldSendRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + shouldSendRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, response, jsonData, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, response, jsonData, shouldSendRawRequest, shouldSendRawResponse) if bifrostErr != nil { return nil, bifrostErr } // ... - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if shouldSendRawRequest { bifrostResponse.ExtraFields.RawRequest = rawRequest } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if shouldSendRawResponse { bifrostResponse.ExtraFields.RawResponse = rawResponse }Also applies to: 274-274, 335-335, 689-689
core/providers/openai/openai.go (4)
171-180: Inconsistent raw response checking pattern.For
RawRequest, the code usesproviderUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest)which allows context-based overrides. However, forRawResponse, the code usessendBackRawResponsedirectly at line 178, bypassing context overrides.Compare with other methods in this file (e.g., lines 1810, 1901, 2082) that use
providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)for the same check. This inconsistency means per-request overrides via context will work differently for raw request vs raw response.- // Set raw response if enabled - if sendBackRawResponse { + // Set raw response if enabled + if providerUtils.ShouldSendBackRawResponse(ctx, sendBackRawResponse) { response.ExtraFields.RawResponse = rawResponse }
302-310: Same inconsistency in HandleOpenAITextCompletionRequest.The pattern uses
ShouldSendBackRawRequestfor raw request but direct valuesendBackRawResponsefor raw response, which is inconsistent with the context-override pattern.- // Set raw response if enabled - if sendBackRawResponse { + // Set raw response if enabled + if providerUtils.ShouldSendBackRawResponse(ctx, sendBackRawResponse) { response.ExtraFields.RawResponse = rawResponse }
703-711: Inconsistency continues in HandleOpenAIChatCompletionRequest.Same pattern issue - using helper for raw request but direct check for raw response.
- // Set raw response if enabled - if sendBackRawResponse { - response.ExtraFields.RawResponse = rawResponse - } + // Set raw response if enabled + if providerUtils.ShouldSendBackRawResponse(ctx, sendBackRawResponse) { + response.ExtraFields.RawResponse = rawResponse + }
1550-1558: HandleOpenAIEmbeddingRequest follows the same pattern.Consider applying the same consistency fix here to use the helper for raw response checking.
- // Set raw response if enabled if sendBackRawResponse { + // Set raw response if enabled + if providerUtils.ShouldSendBackRawResponse(ctx, sendBackRawResponse) { response.ExtraFields.RawResponse = rawResponse }core/providers/gemini/gemini.go (2)
164-167: Inconsistent parameter passing to HandleProviderResponse.This call pre-evaluates
ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)and passes the result, while other providers (Azure, OpenAI) pass the raw boolean flag and check afterward. While functionally equivalent, this creates an inconsistent pattern across providers.For consistency with other providers, consider passing the flag directly:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse)
274-277: Same pattern inconsistency in ChatCompletion.Pre-evaluating
ShouldSendBackRawResponsebefore passing toHandleProviderResponsediffers from other providers.- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, response, jsonData, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, response, jsonData, provider.sendBackRawRequest, provider.sendBackRawResponse)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(6 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(5 hunks)core/providers/gemini/gemini.go(10 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(3 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/logging/main.gocore/schemas/provider.goui/lib/types/logs.tsplugins/logging/changelog.mdtransports/bifrost-http/lib/account.goframework/logstore/migrations.goui/lib/types/schemas.tstransports/changelog.mdframework/streaming/types.gocore/providers/groq/groq.goui/lib/types/config.tsframework/configstore/tables/provider.goplugins/logging/operations.goframework/changelog.mdcore/providers/ollama/ollama.goui/app/workspace/logs/views/logDetailsSheet.tsxframework/streaming/chat.goframework/configstore/clientconfig.goframework/streaming/transcription.gocore/providers/parasail/parasail.gocore/changelog.mdcore/providers/utils/utils.goframework/logstore/tables.goframework/configstore/rdb.gocore/providers/bedrock/bedrock.gocore/providers/cohere/cohere.gocore/providers/cerebras/cerebras.gotransports/bifrost-http/lib/config.goui/app/workspace/providers/page.tsxframework/streaming/audio.gocore/providers/sgl/sgl.goui/app/workspace/providers/fragments/performanceFormFragment.tsxcore/providers/elevenlabs/elevenlabs.gocore/providers/perplexity/perplexity.goframework/configstore/migrations.goframework/streaming/responses.gocore/providers/vertex/vertex.gotransports/bifrost-http/handlers/providers.gocore/providers/openrouter/openrouter.gocore/schemas/bifrost.gocore/providers/azure/azure.gocore/providers/mistral/mistral.gocore/providers/anthropic/anthropic.gocore/providers/openai/openai.gocore/providers/gemini/gemini.go
🧬 Code graph analysis (21)
core/providers/groq/groq.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/ollama/ollama.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
ui/app/workspace/logs/views/codeEditor.tsx (1)
CodeEditor(82-245)
core/providers/parasail/parasail.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/utils/utils.go (3)
core/schemas/bifrost.go (4)
BifrostError(355-364)ErrorField(373-380)BifrostResponseExtraFields(285-295)BifrostContextKeySendBackRawRequest(118-118)ui/lib/types/logs.ts (2)
BifrostError(226-232)ErrorField(217-224)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/cerebras/cerebras.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
framework/streaming/audio.go (1)
framework/streaming/types.go (2)
ProcessedStreamResponse(116-124)StreamType(10-10)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (2)
ui/components/ui/form.tsx (4)
FormItem(161-161)FormLabel(162-162)FormControl(163-163)FormMessage(165-165)ui/components/ui/switch.tsx (1)
Switch(36-36)
core/providers/elevenlabs/elevenlabs.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
framework/configstore/migrations.go (3)
framework/configstore/tables/virtualkey.go (4)
TableVirtualKeyProviderConfig(23-36)TableVirtualKeyProviderConfig(39-41)TableVirtualKeyProviderConfigKey(12-15)TableVirtualKeyProviderConfigKey(18-20)framework/migrator/migrator.go (3)
New(131-149)DefaultOptions(100-106)Migration(62-69)framework/configstore/tables/provider.go (2)
TableProvider(13-38)TableProvider(41-41)
framework/streaming/responses.go (1)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeResponses(17-17)
core/providers/vertex/vertex.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ShouldSendBackRawResponse(518-523)
core/providers/openrouter/openrouter.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
core/providers/mistral/mistral.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/openai/openai.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(510-515)HandleProviderResponse(344-407)ParseAndSetRawRequest(409-416)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (48)
transports/changelog.md (2)
1-2: Verify annotation consistency with enriched summary.The enriched summary states that Line 1 ("raw response accumulation for streaming") is a retained entry from prior work, but the annotation marks it with
~(indicating a new/modified addition). Clarify whether:
- Line 1 is genuinely a new addition to this PR (contradicting the summary), or
- Line 1 was already present and only Line 2 is new (matching the summary claim).
If Line 1 is truly pre-existing and retained, the
~annotation should be removed to accurately reflect that it's unchanged.
2-2: Good changelog documentation of the new feature.Line 2 accurately captures the key capability introduced in this PR: raw request logging and inclusion in responses. The entry complements Line 1 by documenting both raw response and raw request capabilities, which aligns with the PR's goal to enhance debugging and observability.
When verified, ensure the full changelog accurately reflects the complete feature scope across the stack (sendBackRawRequest configuration, provider implementations, UI updates, and database migrations).
transports/bifrost-http/lib/account.go (1)
108-109: PropagatingSendBackRawRequestalongsideSendBackRawResponselooks correctThis cleanly mirrors the existing
SendBackRawResponseplumbing and ensures the per-provider flag from the store is available in the finalProviderConfig. No additional validation or defaults seem necessary at this layer; higher-level logic can decide how to act on these flags.core/changelog.md (1)
1-1: LGTM!The changelog entry clearly documents the new feature.
ui/lib/types/logs.ts (1)
282-282: LGTM!The
raw_requestfield is properly added as optional and mirrors the existingraw_responsefield pattern.ui/lib/types/schemas.ts (2)
372-372: LGTM!The
send_back_raw_requestfield is properly added as optional across configuration schemas, mirroring the existingsend_back_raw_responsepattern.Also applies to: 388-388, 405-405, 416-416
486-487: LGTM!The required boolean fields in
performanceFormSchemaare consistent with the form's treatment ofsend_back_raw_response. Forms require explicit boolean values rather than optional fields.core/providers/parasail/parasail.go (2)
21-21: LGTM!The
sendBackRawRequestfield is properly added and initialized from the provider config, following the same pattern assendBackRawResponse.Also applies to: 52-52
72-72: LGTM!The
sendBackRawRequestflag is correctly propagated through all provider request handlers using theShouldSendBackRawRequesthelper function.Also applies to: 99-99, 123-123
ui/lib/types/config.ts (1)
166-166: LGTM!The
send_back_raw_requestfield is properly added as optional to all relevant configuration interfaces, maintaining consistency with the existingsend_back_raw_responsepattern.Also applies to: 190-190, 201-201
core/schemas/bifrost.go (2)
118-118: LGTM!The context key
BifrostContextKeySendBackRawRequestfollows the established naming convention and will properly control raw request inclusion.
292-292: LGTM!The
RawRequestfield is properly added toBifrostResponseExtraFieldswith appropriate type and JSON tags, mirroring the existingRawResponsefield.core/providers/ollama/ollama.go (2)
22-22: LGTM!The
sendBackRawRequestfield is properly added and initialized, following the established pattern used forsendBackRawResponse.Also applies to: 59-59
82-82: LGTM!The
sendBackRawRequestflag is correctly propagated through all provider endpoints (ListModels, TextCompletion, TextCompletionStream, ChatCompletion, ChatCompletionStream, and Embedding) using theShouldSendBackRawRequesthelper function.Also applies to: 98-98, 115-115, 133-133, 153-153, 200-200
core/providers/elevenlabs/elevenlabs.go (1)
432-435: LGTM: Correct use of request body for raw request in streaming.The
SpeechStreamimplementation correctly usesjsonBody(the actual request payload) withParseAndSetRawRequestto populate the raw request field. This is the correct pattern for capturing the raw request data.core/providers/utils/utils.go (2)
409-416: LGTM:ParseAndSetRawRequesthelper correctly implemented.This helper function correctly takes the request body bytes (
jsonBody) and unmarshals them intoRawRequest. The soft failure behavior (logging a warning instead of returning an error) is appropriate for this diagnostic feature.
510-515: LGTM: Consistent implementation withShouldSendBackRawResponse.The function follows the same pattern as
ShouldSendBackRawResponse, allowing per-request overrides via context while falling back to the provider's default configuration.core/providers/groq/groq.go (2)
20-21: LGTM: Consistent implementation ofsendBackRawRequestfield.The
GroqProvidercorrectly adds thesendBackRawRequestfield and initializes it from configuration, following the same pattern as other providers in this PR.Also applies to: 56-57
76-77: Consistent propagation of raw request flag to shared handlers.The raw request and response flags are correctly evaluated using
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)andShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)and passed to the shared OpenAI-compatible handlers forListModels,ChatCompletion, andChatCompletionStream. All three handlers (HandleOpenAIListModelsRequest,HandleOpenAIChatCompletionRequest,HandleOpenAIChatCompletionStreaming) correctly implement the raw request feature by conditionally populating theRawRequestandRawResponsefields in the response based on these flags.core/providers/mistral/mistral.go (1)
161-162: LGTM: Correct propagation to shared OpenAI handlers.The raw request flag is correctly passed to
HandleOpenAIChatCompletionRequest,HandleOpenAIChatCompletionStreaming, andHandleOpenAIEmbeddingRequestusing theShouldSendBackRawRequesthelper.Also applies to: 185-186, 234-235
core/providers/bedrock/bedrock.go (1)
892-895: LGTM: Streaming paths correctly capture raw request.Both
ChatCompletionStreamandResponsesStreamcorrectly useParseAndSetRawRequestwithjsonData(the actual request body) to populate the raw request in the final streaming response.Also applies to: 1044-1047
plugins/logging/changelog.md (1)
1-1: LGTM!Clear changelog entry documenting the new raw request logging feature.
framework/streaming/types.go (1)
123-123: LGTM!The new RawRequest field is consistent with the existing RawResponse handling pattern and appropriately typed as
*interface{}to support flexible JSON-serializable content.plugins/logging/main.go (1)
45-45: LGTM!The RawRequest field integration into the logging plugin follows the established pattern for RawResponse and correctly respects the
disableContentLoggingflag to prevent logging sensitive data when content logging is disabled.Also applies to: 500-502
transports/bifrost-http/lib/config.go (1)
360-360: LGTM!The SendBackRawRequest field is correctly propagated through the configuration loading and redaction paths, mirroring the established pattern for SendBackRawResponse. The boolean configuration flag itself is not sensitive and appropriately included in redacted configurations.
Also applies to: 1379-1379
framework/configstore/tables/provider.go (1)
20-20: LGTM!The SendBackRawRequest boolean field is appropriately added to the TableProvider model and logically positioned next to the related SendBackRawResponse field. As a simple boolean, no additional serialization hooks are needed.
framework/configstore/clientconfig.go (1)
52-52: LGTM!The SendBackRawRequest configuration field is well-documented and consistently positioned alongside the related SendBackRawResponse field. The naming and structure align with established patterns in the codebase.
ui/app/workspace/providers/page.tsx (1)
65-76: Default provider now seedssend_back_raw_requestfieldAdding
send_back_raw_request: undefinedin the 404 initialization path keeps the provider shape consistent with the new backend config flag and mirrors the existingsend_back_raw_responsebehavior. The ButtononClickbrace tweak is purely formatting with no behavioral impact.Also applies to: 215-225
core/schemas/provider.go (1)
31-34: ProviderConfig raw‑request flag and error constant look consistentThe added
ErrProviderRawRequestUnmarshalandSendBackRawRequestflag are aligned with the existing raw‑response counterparts and integrate cleanly intoProviderConfigwithout requiring additional defaulting logic.Also applies to: 236-244
framework/logstore/migrations.go (1)
43-45: Newraw_requestlogs migration is consistent with existing patternsThe
migrationAddRawRequestColumnwiring and implementation match the established migration style (transactional options,HasColumnguard, and rollback). Adding it after the timestamp migration is safe, and the ID is unique within this file.Also applies to: 645-674
framework/logstore/tables.go (1)
106-109: RawRequest field on Log model is wired consistentlyAdding
RawRequest string 'gorm:"type:text" json:"raw_request"'alongsideRawResponseis consistent with how other opaque text fields are stored and exposed. Leaving it out ofContentSummaryand the parsed fields is sensible given the sensitive/debugging nature of raw payloads.framework/configstore/migrations.go (1)
91-93: Providersend_back_raw_requestDB migration matches table and stackThe new
migrationAddSendBackRawRequestColumnscorrectly adds thesend_back_raw_requestcolumn toTableProviderwith an idempotentHasColumncheck and rollback. Wiring it intotriggerMigrationsat the end is compatible with the existing stack of provider migrations and aligns with theSendBackRawRequestfield already present onTableProvider.Also applies to: 1284-1311
core/providers/sgl/sgl.go (1)
22-22: LGTM! Consistent implementation of raw request support.The
sendBackRawRequestfield is properly added to the struct, initialized from config, and propagated through all OpenAI-compatible request handlers (ListModels, TextCompletion, TextCompletionStream, ChatCompletion, ChatCompletionStream, Embedding). The pattern matches other providers in the stack.Also applies to: 59-59, 79-79, 95-95, 112-112, 130-130, 150-150, 197-197
framework/streaming/audio.go (1)
174-182: LGTM for RawRequest propagation in final response.The
RawRequestfield is correctly set in theProcessedStreamResponsestruct, aligning with the pattern used in other streaming handlers (chat, responses, transcription).framework/streaming/responses.go (1)
782-790: LGTM for RawRequest propagation in both final response paths.The
RawRequestfield is correctly set inProcessedStreamResponsefor both OpenAI-compatible and non-OpenAI provider paths, maintaining consistency with the feature implementation.Also applies to: 871-879
core/providers/cerebras/cerebras.go (1)
20-20: LGTM! Consistent implementation of raw request support.The
sendBackRawRequestfield is properly added, initialized from config, and propagated through all supported OpenAI-compatible request handlers. The pattern is consistent with other providers in this PR stack.Also applies to: 51-51, 71-71, 89-89, 111-111, 129-129, 153-153
core/providers/cohere/cohere.go (2)
71-71: LGTM for struct field and initialization.The
sendBackRawRequestfield is properly added to the struct and initialized from config.Also applies to: 110-110
223-241: LGTM for raw request handling in other methods.The raw request support is correctly implemented in
listModelsByKey,ChatCompletion,ChatCompletionStream,ResponsesStream, andEmbeddingmethods. The pattern is consistent with other providers.Also applies to: 305-326, 482-485, 736-739, 789-810
core/providers/vertex/vertex.go (1)
61-62: LGTM: NewsendBackRawRequestfield added consistently.The field addition follows the existing pattern for
sendBackRawResponseand is properly wired through the constructor and all relevant methods.ui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
128-156: LGTM: Raw request toggle follows existing patterns.The new "Include Raw Request" toggle is implemented consistently with the existing "Include Raw Response" toggle, including proper form binding, validation triggering, and clear user-facing documentation.
transports/bifrost-http/handlers/providers.go (1)
61-62: LGTM:SendBackRawRequestfield properly added to API responses.The new field is correctly added to
ProviderResponseand properly wired through all provider CRUD operations (add, update, get, list) with consistent handling matching the existingSendBackRawResponsepattern.framework/configstore/rdb.go (1)
213-214: LGTM: Database persistence correctly wired forSendBackRawRequest.The new field is properly persisted and retrieved across all provider configuration paths (create, update, batch update, get), following the same pattern as
SendBackRawResponse.core/providers/openrouter/openrouter.go (1)
22-23: LGTM: NewsendBackRawRequestfield added consistently.The field addition and initialization follow the established pattern.
core/providers/azure/azure.go (1)
26-27: Consistent implementation of the raw request feature.The
sendBackRawRequestfield is properly wired from config and consistently applied across all API methods. The pattern of usingShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)before settingExtraFields.RawRequestcorrectly supports both provider-level defaults and per-request context overrides.Also applies to: 51-52
core/providers/openai/openai.go (2)
1189-1197: HandleOpenAIResponsesRequest uses consistent pattern.This method correctly uses
ShouldSendBackRawRequestfor raw request andsendBackRawResponsedirect check for raw response. Consider aligning all methods to use the helper function for both.
596-603: Streaming finalization correctly includes raw request.The streaming handlers properly use
ParseAndSetRawRequestto include the raw request data in the final response when enabled. This aligns with the feature's intent to expose request payloads for debugging.core/providers/gemini/gemini.go (2)
637-643: Streaming finalization correctly implements raw request.The
SpeechStreamgoroutine properly usesParseAndSetRawRequestto attach the raw request data to the final response, consistent with the OpenAI streaming implementation.
904-910: TranscriptionStream finalization follows the correct pattern.Raw request is properly set on the final streaming response when enabled.
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
2 times, most recently
from
4ab2a0a to
10060d1
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (14)
plugins/logging/main.go (1)
34-47: Pool reset incomplete: RawRequest and RawResponse must be cleared.The
putUpdateLogData()function inplugins/logging/pool.goresets 9 fields but omitsRawRequestandRawResponse. SinceUpdateLogDatainstances are reused viaupdateDataPool, stale request/response data from one request can leak into subsequent requests when these interface{} pointers are not zeroed. Adddata.RawRequest = nilanddata.RawResponse = nilto the reset logic inputUpdateLogData().core/providers/elevenlabs/elevenlabs.go (1)
103-120: Inconsistent use of context override for raw request flag.At line 103,
provider.sendBackRawRequestis passed directly toHandleProviderResponse, whileShouldSendBackRawResponseis used for the response flag. This means per-request context overrides won't work correctly for raw requests.If a request sets the context key to enable raw request but the provider default is
false,HandleProviderResponsewon't unmarshal the raw request, but the check at line 113 will pass, resulting innilbeing assigned toRawRequest.Apply this diff to fix the inconsistency:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &elevenlabsResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &elevenlabsResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/bedrock/bedrock.go (1)
452-473: Inconsistent use of context override for raw request flag in listModelsByKey.Same issue as in elevenlabs.go - line 452 passes
provider.sendBackRawRequestdirectly toHandleProviderResponse, but line 466 checksShouldSendBackRawRequest(ctx, ...). Per-request context overrides won't work correctly.Apply this diff:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, bedrockResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, bedrockResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/mistral/mistral.go (1)
105-123: Inconsistent use of context override for raw request flag in listModelsByKey.Same pattern issue - line 105 passes
provider.sendBackRawRequestdirectly, but line 116 checksShouldSendBackRawRequest(ctx, ...).Apply this diff:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/vertex/vertex.go (2)
443-471: Inconsistent use of context override in ChatCompletion HandleProviderResponse calls.Both Anthropic (line 443) and non-Anthropic (line 478) paths pass raw provider flags directly instead of using
ShouldSendBackRawRequest/ShouldSendBackRawResponse, but then check context overrides when setting the fields.Apply these diffs:
For Anthropic path (line 443):
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), anthropicResponse, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), anthropicResponse, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))For non-Anthropic path (line 478):
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), response, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), response, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))Also applies to: 478-499
798-831: Inconsistent context check and duplicate RawResponse assignment.Two issues:
- Line 798 passes raw provider flags directly instead of using context helpers
- Lines 820-823 and 828-830 both set
RawResponsewhen enabled - this is duplicate codeApply this diff to fix both issues:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), anthropicResponse, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), anthropicResponse, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))And remove the duplicate RawResponse assignment (lines 828-830):
if request.Model != deployment { response.ExtraFields.ModelDeployment = deployment } - - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } return response, nilcore/providers/anthropic/anthropic.go (1)
170-224: Non‑streaming raw_request handling is incorrect and ignores per‑request overrides.Two coupled issues across these call sites:
Wrong source for
rawRequest:
HandleProviderResponsecurrently unmarshals the response body intorawRequestand never uses therequestBodyargument, so when you later set:
response.ExtraFields.RawRequest = rawRequestbifrostResponse.ExtraFields.RawRequest = rawRequest
theraw_requestfield will actually contain the provider response, not the original request payload.Context overrides not honored for parsing:
These calls passprovider.sendBackRawRequest/provider.sendBackRawResponsedirectly intoHandleProviderResponse, but the extra fields are gated withShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)/ShouldSendBackRawResponse(...).
If a request-level override in the context enables raw data while the provider default is false,Should*will be true and you’ll attempt to setExtraFields.RawRequest/RawResponse, butHandleProviderResponsewill never have parsed them (flags were false), so they staynil.Consider:
- Updating
HandleProviderResponseto unmarshalrequestBody(when non‑nil) intorawRequest, and- Passing
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)/ShouldSendBackRawResponse(...)intoHandleProviderResponseso parsing and ExtraFields gating use the same effective flags.This will make
raw_requestactually reflect the outbound JSON and ensure per‑request overrides behave as advertised.Also applies to: 246-298, 307-359, 661-714
core/providers/utils/utils.go (1)
340-407:HandleProviderResponseignoresrequestBodyand mislabels raw_request.Right now this helper:
- Always unmarshals
responseBodyinto:
- the structured
response,rawRequest,rawResponse,- Never uses the
requestBodyargument.That means:
- Every provider calling
HandleProviderResponse(..., requestBody, sendBackRawRequest, ...)will end up withraw_requestrepresenting the response JSON, not the original request payload.- For list‑models or other GETs where requestBody is nil, calling with
sendBackRawRequest=truewill still try to parse the response as “raw request”.Additionally, when both
sendBackRawRequestandsendBackRawResponseare true, you never checkrawResponseErr, so a failure to unmarshal the raw response is silently ignored in that mode.Suggested fix:
- Use
requestBodyfor the raw-request path, and guard against empty bodies:func HandleProviderResponse[T any]( - responseBody []byte, response *T, requestBody []byte, - sendBackRawRequest bool, sendBackRawResponse bool, -) (interface{}, interface{}, *schemas.BifrostError) { + responseBody []byte, + response *T, + requestBody []byte, + sendBackRawRequest bool, + sendBackRawResponse bool, +) (interface{}, interface{}, *schemas.BifrostError) { var rawRequest interface{} var rawResponse interface{} var wg sync.WaitGroup var structuredErr, rawRequestErr, rawResponseErr error wg.Add(3) go func() { defer wg.Done() structuredErr = sonic.Unmarshal(responseBody, response) }() go func() { defer wg.Done() - if sendBackRawRequest { - rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) - } + if sendBackRawRequest && len(requestBody) > 0 { + rawRequestErr = sonic.Unmarshal(requestBody, &rawRequest) + } }() go func() { defer wg.Done() if sendBackRawResponse { rawResponseErr = sonic.Unmarshal(responseBody, &rawResponse) } }() wg.Wait() if structuredErr != nil { return nil, nil, &schemas.BifrostError{ IsBifrostError: true, Error: &schemas.ErrorField{ Message: schemas.ErrProviderResponseUnmarshal, Error: structuredErr, }, } } if sendBackRawRequest { if rawRequestErr != nil { return nil, nil, &schemas.BifrostError{ IsBifrostError: true, Error: &schemas.ErrorField{ Message: schemas.ErrProviderRawRequestUnmarshal, Error: rawRequestErr, }, } } return rawRequest, rawResponse, nil } if sendBackRawResponse { if rawResponseErr != nil { return nil, nil, &schemas.BifrostError{ IsBifrostError: true, Error: &schemas.ErrorField{ Message: schemas.ErrProviderRawResponseUnmarshal, Error: rawResponseErr, }, } } return rawRequest, rawResponse, nil } return nil, nil, nil }Once callers start passing
ShouldSendBackRawRequest/Response(...)instead of provider defaults, this will correctly tie parsing cost and behavior to the effective flags.core/providers/cohere/cohere.go (2)
171-244: Non‑streaming Cohere raw_request handling should userequestBodyand context‑aware flags.Across
listModelsByKey,ChatCompletion,Responses, andEmbeddingyou have patterns like:rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( bodyOrResponseBody, responseStructPtr, jsonBodyOrNil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), ) ... if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { ExtraFields.RawRequest = rawRequest } if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { ExtraFields.RawResponse = rawResponse }Given the current
HandleProviderResponseimplementation:
rawRequestis derived from the response body, notjsonBody, soRawRequestwill actually reflect the provider response, not the original request.- Parsing is gated by
provider.sendBackRawRequest/Response, but ExtraFields are gated byShouldSendBackRawRequest/Response; if a context override enables raw data while the provider default is false, you’ll attempt to set RawRequest/RawResponse even thoughHandleProviderResponsenever parsed them (they remain nil).Once
HandleProviderResponseis fixed to userequestBodyfor the raw-request path, please also update these calls to:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( - body, &cohereResponse, nil, provider.sendBackRawRequest, - providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), - ) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + body, &cohereResponse, nil, + providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), + providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), + )(and similarly for the
jsonBody-using sites), so that:
raw_requesttruly reflects the outbound payload when there is one, and- parsing work is done only when the effective flags (including per‑request overrides) say it is needed.
Also applies to: 280-329, 505-572, 762-813
531-569: Duplicate RawRequest/RawResponse assignments inResponsesshould be cleaned up.In
Responses, theExtraFields.RawRequest/ExtraFields.RawResponseassignments appear three times under the sameShouldSendBackRaw*conditions. This is harmless but confusing copy‑paste noise.You can safely keep a single pair of blocks:
- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - bifrostResponse.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - bifrostResponse.ExtraFields.RawResponse = rawResponse - } - - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - bifrostResponse.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - bifrostResponse.ExtraFields.RawResponse = rawResponse - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - bifrostResponse.ExtraFields.RawResponse = rawResponse - } + // Set raw request/response if enabled + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + bifrostResponse.ExtraFields.RawRequest = rawRequest + } + if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + bifrostResponse.ExtraFields.RawResponse = rawResponse + }This keeps behavior unchanged but makes the intent clearer.
core/providers/gemini/gemini.go (1)
220-309: Gemini ChatCompletion: align raw_request parsing with effective flags.Here you have:
rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( body, response, jsonData, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), ) ... if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { response.ExtraFields.RawRequest = rawRequest }Given
HandleProviderResponsecurrently parsesresponseBodyintorawRequestand ignoresjsonData,RawRequestwill actually contain the response JSON, not the OpenAI‑style request you built. Additionally, parsing is gated byprovider.sendBackRawRequestwhile the extra field usesShouldSendBackRawRequest(...), so a request-level override enabling raw_request won’t cause the body to be parsed.Once the helper is fixed to use
requestBodyfor raw_request, consider:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( - body, response, jsonData, - provider.sendBackRawRequest, - providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), - ) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + body, response, jsonData, + providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), + providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), + )so parsing and flag usage stay in sync.
core/providers/openrouter/openrouter.go (1)
99-119: Use context-aware send-back flags consistently when parsing and exposing raw dataIn
listModelsByKey,HandleProviderResponseis called with:rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( responseBody, &openrouterResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), )but
ExtraFields.RawRequest/RawResponseare later gated byShouldSendBackRawRequest/Response. If the provider default isfalseyet the context setsBifrostContextKeySendBackRawRequestorBifrostContextKeySendBackRawResponsetotrue,HandleProviderResponsewill never capture the raw data, and you end up assigningnilintoExtraFields.Consider computing effective flags once and reusing them:
- var openrouterResponse schemas.BifrostListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( - responseBody, &openrouterResponse, nil, - provider.sendBackRawRequest, - providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), - ) + var openrouterResponse schemas.BifrostListModelsResponse + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, &openrouterResponse, nil, + sendBackRawRequest, + sendBackRawResponse, + ) @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + // Set raw request if enabled + if sendBackRawRequest { openrouterResponse.ExtraFields.RawRequest = rawRequest } @@ - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if sendBackRawResponse { openrouterResponse.ExtraFields.RawResponse = rawResponse }This keeps parsing and exposure in sync with the per-request override semantics.
core/providers/azure/azure.go (1)
22-28: Align non-streaming Azure paths with context-aware raw-request/response flagsFor non-streaming calls (
listModelsByKey,TextCompletion,ChatCompletion,Responses,Embedding), you currently do:rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( responseBody, response, jsonDataOrNil, provider.sendBackRawRequest, provider.sendBackRawResponse, ) // ... if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { response.ExtraFields.RawRequest = rawRequest } if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { response.ExtraFields.RawResponse = rawResponse }This means:
- If the provider default flags are false but the caller sets
BifrostContextKeySendBackRawRequestand/orBifrostContextKeySendBackRawResponseto true,HandleProviderResponsenever captures raw data (both remain nil).- Yet
ShouldSendBackRawRequest/Responsecan still return true, so you end up exposing nilRawRequest/RawResponseeven though the request asked for them.To keep parsing and exposure in sync with per-request overrides (and to match the streaming paths), compute effective flags once per call and reuse them, e.g. in
TextCompletion:- response := &schemas.BifrostTextCompletionResponse{} - - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( - responseBody, response, jsonData, - provider.sendBackRawRequest, - provider.sendBackRawResponse, - ) + response := &schemas.BifrostTextCompletionResponse{} + + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, response, jsonData, + sendBackRawRequest, + sendBackRawResponse, + ) @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + // Set raw request if enabled + if sendBackRawRequest { response.ExtraFields.RawRequest = rawRequest } @@ - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + // Set raw response if enabled + if sendBackRawResponse { response.ExtraFields.RawResponse = rawResponse }and apply the same pattern in
listModelsByKey,ChatCompletion,Responses, andEmbedding(replacingresponsewith the appropriate type).This preserves provider defaults while honoring per-request overrides and avoids returning “empty” raw data when the caller has explicitly requested it.
Also applies to: 47-53, 195-221, 240-296, 352-440, 537-625, 719-777
core/providers/openai/openai.go (1)
1566-1632: Raw request support missing for Speech and Transcription endpoints.The Speech, SpeechStream, Transcription, and TranscriptionStream handlers lack raw request population in
ExtraFields, unlike all other endpoint types (TextCompletion, ChatCompletion, Embedding, Responses, ListModels) which consistently populate theRawRequestfield when thesendBackRawRequestflag is enabled. These endpoints construct JSON requests that could be captured the same way other handlers do, so this appears to be an oversight rather than a justified omission.
♻️ Duplicate comments (4)
core/providers/perplexity/perplexity.go (1)
18-25: Align Perplexity ChatCompletion raw_request handling with context overrides and fixed helper.In
ChatCompletionyou currently have:rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( responseBody, &response, jsonBody, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), )Two problems here:
HandleProviderResponse(as currently implemented) ignoresjsonBodyand unmarshals the response body intorawRequest, soExtraFields.RawRequestwill not reflect the outbound request.- You’re passing
provider.sendBackRawRequestinstead of the context‑awareShouldSendBackRawRequest(...), while later you gateExtraFields.RawRequestwithShouldSendBackRawRequest(ctx, provider.sendBackRawRequest). If a per‑request override enables raw_request while the provider default is false, you’ll try to setRawRequestbutHandleProviderResponsewill never have parsed it (flag was false), and it will remain nil.Once
HandleProviderResponseis updated to actually userequestBodyfor raw_request, please also switch this call to:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( - responseBody, &response, jsonBody, - provider.sendBackRawRequest, - providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), - ) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, &response, jsonBody, + providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), + providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), + )This keeps parsing cost aligned with the effective flags and makes per‑request overrides behave correctly.
Also applies to: 50-56, 127-169
core/providers/gemini/gemini.go (1)
131-188: GeminilistModelsByKey: fix raw_request parsing and remove duplicate RawResponse assignment.Two issues in this block:
HandleProviderResponse misuse / context mismatch:
As with other providers,HandleProviderResponsecurrently ignores itsrequestBodyargument and unmarshals the response body intorawRequest, soRawRequestwill mirror the response instead of the request. In addition, you passprovider.sendBackRawRequestinto the helper but gateExtraFields.RawRequest/RawResponsewithShouldSendBackRawRequest/Response, so per‑request overrides can turn on the flag without having actually parsed the raw data.Once
HandleProviderResponseis fixed, this call should be updated to:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( - resp.Body(), &geminiResponse, nil, - provider.sendBackRawRequest, - providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), - ) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), &geminiResponse, nil, + providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), + providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), + )Duplicate RawResponse assignment (previously flagged):
You setresponse.ExtraFields.RawResponse = rawResponsetwice under the sameShouldSendBackRawResponsecondition:if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { response.ExtraFields.RawResponse = rawResponse } if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { response.ExtraFields.RawResponse = rawResponse }The second block is redundant and can be removed.
framework/streaming/transcription.go (1)
184-187: Fix potential nil deref and makeRawRequestpointer truly optional in final transcription responseHere
result.TranscriptionStreamResponseis accessed without checking thatresultandresult.TranscriptionStreamResponseare non‑nil, andRawRequestis always set to a non‑nil pointer even when no raw request exists.This can panic on error-only final chunks (when
resultis nil) and also prevents callers from distinguishing “no raw request” (nil pointer) from “present but null”.Consider:
- var rawRequest interface{} - if result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.TranscriptionStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && + result.TranscriptionStreamResponse != nil && + result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.TranscriptionStreamResponse.ExtraFields.RawRequest) + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeTranscription, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis matches the safer pattern used elsewhere in the stack and avoids panics on error final chunks.
Also applies to: 188-196
framework/streaming/responses.go (1)
723-726: Guardresultbefore readingResponsesStreamResponseand return a nilRawRequestpointer when absentIn both final-chunk paths:
- OpenAI-compatible providers (
if provider == schemas.OpenAI || ...):result.ResponsesStreamResponse.ExtraFields.RawRequestis read at lines 723-726 without checkingresult/result.ResponsesStreamResponsefor nil.- Non-OpenAI path later in the function does the same at lines 866-869.
In both cases you also use:
var rawRequest interface{} ... RawRequest: &rawRequest,which always sets a non-nil
*interface{}even when no raw request exists, making it impossible to treatnilas “no raw request”.This can panic on error-only final chunks (where
resultis nil) and misleads consumers ofProcessedStreamResponse.Suggested fix (OpenAI-compatible branch):
- if isFinalChunk { - var rawRequest interface{} - if result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + if isFinalChunk { + var rawRequest *interface{} + if result != nil && + result.ResponsesStreamResponse != nil && + result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.ResponsesStreamResponse.ExtraFields.RawRequest) + rawRequest = &v + } @@ - return &ProcessedStreamResponse{ + return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeResponses, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilAnd similarly for the non-OpenAI branch:
- if shouldProcess { - data, processErr := a.processAccumulatedResponsesStreamingChunks(requestID, bifrostErr, isFinalChunk) + if shouldProcess { + data, processErr := a.processAccumulatedResponsesStreamingChunks(requestID, bifrostErr, isFinalChunk) if processErr != nil { ... } - var rawRequest interface{} - if result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && + result.ResponsesStreamResponse != nil && + result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.ResponsesStreamResponse.ExtraFields.RawRequest) + rawRequest = &v + } @@ return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeResponses, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis avoids nil dereferences and makes
RawRequesttruly optional.Also applies to: 782-790, 866-879
🧹 Nitpick comments (5)
framework/changelog.md (1)
1-2: Changelog entries are clear and well-formatted.Both entries accurately capture the features being added across the stack. The "feat:" format is consistent and concise.
Optional: Consider documenting security implications.
Given the PR objectives note that raw requests may contain sensitive information (API keys, user data), a brief note in the changelog might help users understand the implications of enabling this feature. This is not critical for this release but could be helpful for future reference or in a security/breaking-changes section if one exists.
framework/logstore/migrations.go (1)
645-674:migrationAddRawRequestColumnmatches existing migration patterns and is idempotentID is unique, the AddColumn path is properly guarded with
HasColumn, and the rollback shape (unconditional DropColumn via the migrator) is consistent with the other “add column” migrations. This should safely backfill theraw_requestcolumn for existing installations while being a no-op on fresh databases wherelogs_initalready created the column from the updatedLogstruct.If you find yourself adding more columns of this kind, consider a small helper (e.g.,
addColumnIfNotExists(tx, &Log{}, "raw_request")) to reduce duplication across these migration functions.core/providers/utils/utils.go (1)
409-416:ParseAndSetRawRequestandShouldSendBackRawRequestlook good; ensure callers use them consistently.
ParseAndSetRawRequest’s best‑effort parse + warn pattern is appropriate for streaming end‑of‑stream usage and matchesParseAndSetRawResponsestyle.ShouldSendBackRawRequestmirrorsShouldSendBackRawResponseand supports per‑request enablement via context, falling back to provider defaults.The remaining work is mostly at call sites: for non‑streaming handlers, prefer passing
ShouldSendBackRawRequest(ctx, default)/ShouldSendBackRawResponse(...)intoHandleProviderResponseso that parsing and ExtraFields gating use the same effective flags and avoid unnecessary unmarshalling.Also applies to: 510-515
core/providers/gemini/gemini.go (1)
327-343: Gemini streaming raw_request handling looks solid; consider extending to non‑streaming Speech/Transcription.
- ChatCompletionStream, SpeechStream, and TranscriptionStream all:
- Use
ShouldSendBackRawRequest/Responseto compute effective flags.- Attach
RawResponseper chunk when enabled.- On final “done” chunks, call
ParseAndSetRawRequest(&response.ExtraFields, jsonBody)whenShouldSendBackRawRequestis true, giving you a structuredraw_requestthat matches the outbound body.That’s consistent with the new utils helpers and other providers.
One thing to consider for completeness (optional):
- The non‑streaming
SpeechandTranscriptionmethods currently only setRawResponseviacompleteRequest; if you also wantRawRequestthere, you could mirror the streaming pattern by callingParseAndSetRawRequest(&response.ExtraFields, jsonData)underShouldSendBackRawRequest(...).This would make raw_request availability symmetric between streaming and non‑streaming Gemini endpoints.
Also applies to: 373-392, 434-647, 687-914
core/providers/openai/openai.go (1)
292-305: Consider optimizing raw request pre-computation pattern.The current pattern calls
HandleProviderResponsewith the provider-levelsendBackRawRequestconfig, which pre-computesrawRequesteven if a context override might disable it. ThenShouldSendBackRawRequestis called to check the context before actually settingExtraFields.RawRequest.This wastes one JSON unmarshal operation when context overrides disable the feature. You could optimize by calling
ShouldSendBackRawRequestfirst, but this adds complexity and the overhead is minimal (only when provider-level flag is enabled).Example optimized pattern:
shouldSendRaw := providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, response, jsonData, shouldSendRaw, sendBackRawResponse) if bifrostErr != nil { return nil, bifrostErr } // ... set ExtraFields ... if shouldSendRaw { response.ExtraFields.RawRequest = rawRequest }Also applies to: 693-706, 1179-1192, 1540-1553
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(8 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(5 hunks)core/providers/gemini/gemini.go(10 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(3 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
✅ Files skipped from review due to trivial changes (1)
- core/changelog.md
🚧 Files skipped from review as they are similar to previous changes (19)
- ui/lib/types/logs.ts
- framework/configstore/clientconfig.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- plugins/logging/operations.go
- framework/streaming/chat.go
- core/providers/ollama/ollama.go
- plugins/logging/changelog.md
- core/schemas/provider.go
- core/schemas/bifrost.go
- ui/lib/types/config.ts
- core/providers/sgl/sgl.go
- ui/lib/types/schemas.ts
- framework/streaming/audio.go
- framework/logstore/tables.go
- ui/app/workspace/providers/fragments/performanceFormFragment.tsx
- transports/changelog.md
- framework/streaming/types.go
- transports/bifrost-http/lib/config.go
- ui/app/workspace/providers/page.tsx
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/logging/main.gocore/providers/cohere/cohere.gocore/providers/perplexity/perplexity.goframework/streaming/transcription.goframework/configstore/migrations.goframework/changelog.mdframework/logstore/migrations.gocore/providers/vertex/vertex.goframework/configstore/tables/provider.gocore/providers/elevenlabs/elevenlabs.goframework/configstore/rdb.gocore/providers/parasail/parasail.gocore/providers/azure/azure.gocore/providers/utils/utils.gocore/providers/anthropic/anthropic.gocore/providers/mistral/mistral.gotransports/bifrost-http/lib/account.goframework/streaming/responses.gocore/providers/groq/groq.gocore/providers/openrouter/openrouter.gocore/providers/openai/openai.gocore/providers/cerebras/cerebras.gotransports/bifrost-http/handlers/providers.gocore/providers/bedrock/bedrock.gocore/providers/gemini/gemini.go
🧬 Code graph analysis (18)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
framework/streaming/transcription.go (1)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeTranscription(16-16)
framework/configstore/migrations.go (2)
framework/migrator/migrator.go (3)
New(131-149)DefaultOptions(100-106)Migration(62-69)framework/configstore/tables/provider.go (2)
TableProvider(13-38)TableProvider(41-41)
framework/logstore/migrations.go (2)
framework/migrator/migrator.go (3)
DefaultOptions(100-106)New(131-149)Migration(62-69)framework/logstore/tables.go (2)
Log(74-137)Log(140-142)
core/providers/vertex/vertex.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ShouldSendBackRawResponse(518-523)
core/providers/elevenlabs/elevenlabs.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/parasail/parasail.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/azure/azure.go (3)
core/providers/utils/utils.go (2)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)core/schemas/utils.go (1)
IsAnthropicModel(1043-1045)core/providers/anthropic/anthropic.go (2)
AcquireAnthropicMessageResponse(46-50)ReleaseAnthropicMessageResponse(53-57)
core/providers/utils/utils.go (3)
core/schemas/bifrost.go (4)
BifrostError(355-364)ErrorField(373-380)BifrostResponseExtraFields(285-295)BifrostContextKeySendBackRawRequest(118-118)ui/lib/types/logs.ts (2)
BifrostError(226-232)ErrorField(217-224)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/mistral/mistral.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
core/providers/groq/groq.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/openrouter/openrouter.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
core/providers/openai/openai.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(510-515)HandleProviderResponse(344-407)ParseAndSetRawRequest(409-416)
core/providers/cerebras/cerebras.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (21)
framework/logstore/migrations.go (1)
43-45: Wiring newraw_requestmigration intotriggerMigrationslooks correctThe new call is placed alongside the other schema-only migrations and will run for all dialects (unlike the sqlite-only timestamp update), which is appropriate for adding the
raw_requestcolumn everywhere. No issues from an ordering/idempotency perspective.plugins/logging/main.go (1)
498-505: Content-gated RawRequest logging looks correct; double-check upstream gating.Copying
extraFields.RawRequestintoupdateData.RawRequestonly whendisableContentLoggingis unset/false mirrors the existingRawResponsebehavior and avoids logging raw requests when content logging is disabled. Please just verify that upstream providers only populateextraFields.RawRequestwhen the new per-providerSendBackRawRequest(or equivalent) flag is enabled, so log storage remains aligned with that configuration across the stack.core/providers/parasail/parasail.go (1)
21-22: LGTM - Consistent implementation of sendBackRawRequest.The new
sendBackRawRequestfield is properly added, initialized from config, and threaded through all API call paths usingShouldSendBackRawRequesthelper. The pattern mirrors the existingsendBackRawResponsehandling and correctly considers context overrides.Also applies to: 52-53, 72-73, 99-100, 123-124
core/providers/elevenlabs/elevenlabs.go (1)
432-435: LGTM - Correct raw request handling in stream finalization.The
SpeechStreamfunction correctly checksShouldSendBackRawRequestbefore callingParseAndSetRawRequestto attach the raw request to the final stream response.core/providers/bedrock/bedrock.go (1)
725-728: LGTM - Correct raw request handling in completion and streaming paths.The
ChatCompletion,ChatCompletionStream,Responses, andResponsesStreammethods all correctly:
- Pass
jsonBody/jsonDatatoHandleProviderResponsefor request body capture- Check
ShouldSendBackRawRequestbefore callingParseAndSetRawRequestThe streaming finalization patterns attach raw requests to final responses appropriately.
Also applies to: 897-900, 966-969, 1054-1057
core/providers/mistral/mistral.go (1)
161-165: LGTM - Correct raw request flag propagation to shared handlers.The
ChatCompletion,ChatCompletionStream, andEmbeddingmethods correctly useShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)when calling the shared OpenAI-compatible handlers.Also applies to: 185-186, 234-235
core/providers/vertex/vertex.go (1)
604-605: LGTM - Correct raw request flag propagation in streaming handlers.The
ChatCompletionStream(both Anthropic and OpenAI paths) andResponsesStreamcorrectly useShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)when calling shared streaming handlers.Also applies to: 680-681, 950-951
core/providers/anthropic/anthropic.go (2)
21-29: AnthropicProvider: newsendBackRawRequestwiring looks consistent.Field addition and constructor assignment from
config.SendBackRawRequestmatch existingsendBackRawResponsepattern and align with the new feature flag.Also applies to: 76-111
398-411: Streaming raw_request integration looks correct and context‑aware.For both chat and responses streaming paths you precompute
sendBackRawRequest/sendBackRawResponsevia theShould*helpers, use those booleans in the shared streaming handlers, and on the final chunk attachRawRequestviaParseAndSetRawRequest(&response.ExtraFields, jsonBody)when enabled. This is consistent with the new utilities and with other providers.Also applies to: 414-655, 750-763, 769-999
core/providers/groq/groq.go (1)
16-22: Groq raw_request flag wiring is clean and context‑aware.
sendBackRawRequestis correctly initialized fromconfig.SendBackRawRequest, and all OpenAI‑compatible handlers receiveproviderUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)/ShouldSendBackRawResponse(...), so per‑request overrides will work uniformly without extra parsing when disabled.Also applies to: 27-59, 66-80, 157-171, 173-199
core/providers/cerebras/cerebras.go (1)
16-22: Cerebras raw_request plumbing is consistent with the shared OpenAI pattern.
sendBackRawRequestis correctly initialized from config and all relevant operations threadShouldSendBackRawRequest/Responseinto the OpenAI-compatible handlers, so context overrides and provider defaults will be honored without extra unmarshalling when disabled.Also applies to: 27-54, 61-75, 77-93, 95-118, 120-134, 136-162
core/providers/cohere/cohere.go (2)
66-74: CohereProvider: new raw_request flag wiring looks good.
sendBackRawRequestis correctly added and initialized fromconfig.SendBackRawRequest, mirroringsendBackRawResponseand respecting the existing provider config structure.Also applies to: 79-113
407-503: Streaming Cohere raw_request handling is consistent and usesParseAndSetRawRequest.For both chat and responses streaming:
- You correctly compute the effective flags with
ShouldSendBackRawRequest/Responseand pass them down.- On the final chunk, you attach
RawRequestviaParseAndSetRawRequest(&response.ExtraFields, jsonBody)when enabled, and setLatencyto total duration.This matches the shared utils behavior and other providers; once
HandleProviderResponseis fixed for non‑streaming paths, Cohere’s streaming and non‑streaming raw_request behavior will be aligned.Also applies to: 574-760, 762-813
core/providers/gemini/gemini.go (1)
23-30: GeminiProvider: new raw_request flag wiring is correct.
sendBackRawRequestis initialized fromconfig.SendBackRawRequestand stored alongsidesendBackRawResponse, matching the pattern used by other providers.Also applies to: 35-63
transports/bifrost-http/lib/account.go (1)
80-116: BaseAccount now correctly propagatesSendBackRawRequestinto ProviderConfig.Assigning
providerConfig.SendBackRawRequest = config.SendBackRawRequestalongside the existingSendBackRawResponsewiring ensures providers see the new flag from stored configuration without altering any other behavior.framework/configstore/migrations.go (1)
91-93:send_back_raw_requestmigration wiring looks consistent and safeThe new migration is correctly hooked into
triggerMigrations, uses the same pattern as existing column migrations, and operates ontables.TableProviderwith the expectedsend_back_raw_requestcolumn name. Fresh installs get the column frommigrationInit, and existing installs are updated idempotently viaHasColumnchecks; rollback mirrors other column migrations.Also applies to: 1143-1153, 1284-1311
framework/configstore/tables/provider.go (1)
20-22:SendBackRawRequestcolumn addition onTableProvideris aligned with usageThe new
SendBackRawRequestbool cleanly extendsTableProvider, with a JSON tag matching the API and a default GORM column name (send_back_raw_request) that matches the migration and RDB wiring. No extra serialization hooks are needed.framework/configstore/rdb.go (1)
207-216: Raw-request flag is correctly threaded through RDB provider config paths
SendBackRawRequestis now persisted and loaded alongside existing provider fields inUpdateProvidersConfig,UpdateProvider,AddProvider, andGetProvidersConfig, matching the new column onTableProvider.Note that
UpdateProviderbuilds a freshProviderConfigand only assigns these flags from the HTTP payload; given the documented “no partial updates” behavior in the handler, API callers should continue to send complete provider configs (including both send_back_* flags) on every update to avoid unintentionally resetting them.Also applies to: 323-330, 432-441, 623-630
transports/bifrost-http/handlers/providers.go (1)
55-65: HTTP provider handlers propagatesend_back_raw_requestconsistentlyThe new
send_back_raw_requestflag is:
- Exposed on
ProviderResponse(list/get).- Accepted on create/update via optional
*boolfields and correctly mapped intoconfigstore.ProviderConfig.- Returned via
getProviderResponseFromConfigbased on stored config.This mirrors existing
send_back_raw_responsewiring and keeps the REST surface in sync with the underlying config store. Given the “full config” semantics ofupdateProvider, clients should continue to send both send_back_* flags on every update to preserve their values.Also applies to: 173-182, 245-253, 315-323, 357-365, 419-441, 458-465, 842-861
core/providers/openai/openai.go (2)
597-599: LGTM: Streaming raw request capture implemented correctly.The streaming handlers correctly capture raw requests at the end of the stream using
ParseAndSetRawRequest. This approach:
- Avoids redundancy by setting raw request only on the final chunk
- Uses consistent utility for error handling and logging
- Properly checks the
sendBackRawRequestflag before capturingThe pattern is consistent across TextCompletionStreaming, ChatCompletionStreaming, and ResponsesStreaming.
Also applies to: 978-980, 1084-1086, 1429-1431
96-112: LGTM: Function signatures updated consistently.All handler functions have been updated to accept the
sendBackRawRequestparameter. The parameter positioning is consistent (always beforesendBackRawResponse), and all call sites correctly pass the flag through usingproviderUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest).Also applies to: 186-211, 233-245, 342-357, 632-644, 747-764, 1117-1129, 1230-1246, 1477-1490
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (8)
core/providers/openai/openai.go (1)
116-183: Non‑stream handlers depend on HandleProviderResponse correctly parsing raw request.All non‑stream handlers now call
providerUtils.HandleProviderResponse(..., jsonData, sendBackRawRequest, sendBackRawResponse)and then copyrawRequest/rawResponseintoExtraFieldswhen the corresponding flags are enabled. This is structurally sound, but it assumesHandleProviderResponseactually uses therequestBodyargument for the raw request. Per the shared snippet incore/providers/utils/utils.go,requestBodyis currently unused andrawRequestis unmarshalled fromresponseBody, which would makeRawRequesteffectively mirrorRawResponse.I’d recommend updating
HandleProviderResponseto unmarshalrequestBodyintorawRequest(when non‑nil) so these call sites return the real raw request payload instead of the response JSON.Also applies to: 235-313, 633-714, 1117-1199, 1478-1560
core/providers/anthropic/anthropic.go (1)
170-224: Non‑stream raw request/response handling relies on shared utils correctness.For list‑models, text, chat, and responses, you now use:
HandleProviderResponse(..., provider.sendBackRawRequest, provider.sendBackRawResponse)and- conditional copies of
rawRequest/rawResponseintoExtraFieldsguarded byShouldSendBackRawRequest/ShouldSendBackRawResponse.This is structurally fine, but as in other providers, it assumes
HandleProviderResponseunmarshals the raw request from the suppliedjsonData. Per the shared utils snippet, it currently unmarshalsresponseBodyfor both structured response andrawRequest, ignoringrequestBody, which would makeRawRequestactually mirror the response.Consider fixing
HandleProviderResponseto use therequestBodyargument forrawRequestso these Anthropic call sites expose the true request payload.Also applies to: 246-298, 307-359, 661-714
core/providers/azure/azure.go (1)
135-221: Non‑stream Azure handlers correctly plumb rawRequest/rawResponse but depend on utils fix.For list‑models, text, chat, responses, and embeddings you:
- Pass
provider.sendBackRawRequest/provider.sendBackRawResponseintoHandleProviderResponse, and- Conditionally assign
ExtraFields.RawRequest/RawResponseusingShouldSendBackRawRequest/ShouldSendBackRawResponse.This is consistent with other providers. However, as noted elsewhere,
HandleProviderResponsecurrently unmarshalsresponseBodyintorawRequestand ignoresrequestBody, so these paths would expose the response JSON asRawRequest. OnceHandleProviderResponseis fixed to use therequestBodyargument for the raw request, these Azure handlers should work as intended without further changes.Also applies to: 236-296, 349-440, 533-626, 716-777
core/providers/gemini/gemini.go (1)
220-309: ChatCompletion raw request/response handling is correct but depends on utils’ semantics.
ChatCompletionnow:
- Calls
HandleProviderResponse(body, response, jsonData, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)), and- Copies
rawRequest/rawResponseintoExtraFieldsunderShouldSendBackRawRequest/ShouldSendBackRawResponse.The wiring and gating are sound. Just be aware that
HandleProviderResponsecurrently unmarshalsresponseBodyintorawRequestrather thanrequestBody(per the utils snippet), so until that is fixed,RawRequesthere will actually mirror the response JSON.core/providers/cohere/cohere.go (1)
66-74: Align raw-request flags with context overrides and deduplicateResponsesraw fieldsTwo issues here:
Flag mismatch for
HandleProviderResponsevs. ExtraFields gating
In
listModelsByKey,ChatCompletion,Responses, andEmbedding, you call:rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( bodyOrResponseBody, &cohereResponse, jsonBodyOrNil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), )but later gate
ExtraFields.RawRequestusing:if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { response.ExtraFields.RawRequest = rawRequest }If the provider config has
SendBackRawRequest = falsebut the context setsBifrostContextKeySendBackRawRequest = true,ShouldSendBackRawRequest(...)returnstrue, yetHandleProviderResponseran withsendBackRawRequest = false, sorawRequestremainsnil. The context override never actually produces araw_requestpayload.Suggested pattern (applies to all these call sites):
sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( responseBody, &cohereResponse, jsonBody, sendBackRawRequest, sendBackRawResponse, ) ... if sendBackRawRequest { response.ExtraFields.RawRequest = rawRequest } if sendBackRawResponse { response.ExtraFields.RawResponse = rawResponse }This keeps the behavior of config + context overrides consistent across parsing and ExtraFields population.
Duplicated
RawRequest/RawResponseassignment inResponses
In
Responses, lines 546–569 repeatRawRequestandRawResponseassignment blocks multiple times, plus an additional finalRawResponseblock. All are conditioned on the sameShouldSendBackRawRequest/Responsechecks and the samerawRequest/rawResponsevalues.This is benign but noisy and confusing for future readers.
Consolidate to a single block:
- // Set raw request if enabled
- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) {
bifrostResponse.ExtraFields.RawRequest = rawRequest- }
- // Set raw response if enabled
- if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) {
bifrostResponse.ExtraFields.RawResponse = rawResponse- }
- // Set raw request if enabled
- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) {
bifrostResponse.ExtraFields.RawRequest = rawRequest- }
- // Set raw response if enabled
- if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) {
bifrostResponse.ExtraFields.RawResponse = rawResponse- }
- // Set raw response if enabled
- if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) {
bifrostResponse.ExtraFields.RawResponse = rawResponse- }
- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) {
bifrostResponse.ExtraFields.RawRequest = rawRequest- }
- if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) {
bifrostResponse.ExtraFields.RawResponse = rawResponse- }
Also applies to: 105-113, 171-244, 281-329, 505-572, 574-603, 643-759
core/providers/utils/utils.go (1)
340-407: FixHandleProviderResponseto actually userequestBodyfor raw_request and validate both raw pathsThe current implementation of
HandleProviderResponsehas a few correctness problems:
requestBodyis ignored andrawRequestis parsed from the responsefunc HandleProviderResponse[T any](responseBody []byte, response *T, requestBody []byte, sendBackRawRequest bool, sendBackRawResponse bool) ...
- Inside,
rawRequestis unmarshaled fromresponseBody, notrequestBody, soraw_requesteffectively mirrorsraw_responseat call sites.- This contradicts the function signature and the PR’s goal of exposing the actual request payload.
Both raw paths aren’t validated when both flags are true
- When
sendBackRawRequestandsendBackRawResponseare bothtrue, onlyrawRequestErris checked;rawResponseErris ignored ifrawRequestsucceeds, which can silently drop raw_response parsing errors.GET/list-models callers may have no
requestBody
- Call sites like list‑models pass
requestBody == nil. If we start parsing fromrequestBody, we must not treat “no body” as an error; in that case,raw_requestshould just remainnil.A concrete fix that addresses all three:
-func HandleProviderResponse[T any](responseBody []byte, response *T, requestBody []byte, sendBackRawRequest bool, sendBackRawResponse bool) (interface{}, interface{}, *schemas.BifrostError) { - var rawRequest interface{} - var rawResponse interface{} +func HandleProviderResponse[T any](responseBody []byte, response *T, requestBody []byte, sendBackRawRequest bool, sendBackRawResponse bool) (interface{}, interface{}, *schemas.BifrostError) { + var rawRequest interface{} + var rawResponse interface{} var wg sync.WaitGroup - var structuredErr, rawRequestErr, rawResponseErr error - - wg.Add(3) + var structuredErr, rawRequestErr, rawResponseErr error + + wg.Add(3) go func() { defer wg.Done() structuredErr = sonic.Unmarshal(responseBody, response) }() go func() { defer wg.Done() - if sendBackRawRequest { - rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) - } + // Only attempt to parse raw request when explicitly requested and we actually have a body. + if sendBackRawRequest && len(requestBody) > 0 { + rawRequestErr = sonic.Unmarshal(requestBody, &rawRequest) + } }() go func() { defer wg.Done() if sendBackRawResponse { rawResponseErr = sonic.Unmarshal(responseBody, &rawResponse) } }() wg.Wait() if structuredErr != nil { return nil, nil, &schemas.BifrostError{ IsBifrostError: true, Error: &schemas.ErrorField{ Message: schemas.ErrProviderResponseUnmarshal, Error: structuredErr, }, } } - if sendBackRawRequest { - if rawRequestErr != nil { - return nil, nil, &schemas.BifrostError{ - IsBifrostError: true, - Error: &schemas.ErrorField{ - Message: schemas.ErrProviderRawRequestUnmarshal, - Error: rawRequestErr, - }, - } - } - return rawRequest, rawResponse, nil - } - - if sendBackRawResponse { - if rawResponseErr != nil { - return nil, nil, &schemas.BifrostError{ - IsBifrostError: true, - Error: &schemas.ErrorField{ - Message: schemas.ErrProviderRawResponseUnmarshal, - Error: rawResponseErr, - }, - } - } - return rawRequest, rawResponse, nil - } - - return nil, nil, nil + if sendBackRawRequest && rawRequestErr != nil { + return nil, nil, &schemas.BifrostError{ + IsBifrostError: true, + Error: &schemas.ErrorField{ + Message: schemas.ErrProviderRawRequestUnmarshal, + Error: rawRequestErr, + }, + } + } + + if sendBackRawResponse && rawResponseErr != nil { + return nil, nil, &schemas.BifrostError{ + IsBifrostError: true, + Error: &schemas.ErrorField{ + Message: schemas.ErrProviderRawResponseUnmarshal, + Error: rawResponseErr, + }, + } + } + + // If neither raw path was requested, both rawRequest/rawResponse will be nil, which is fine. + return rawRequest, rawResponse, nil }
- This change:
- Makes
requestBodymeaningful and ensuresraw_requestis the original JSON payload when available.- Validates both raw paths independently.
- Avoids errors when there is legitimately no request body (e.g., list‑models GETs).
ParseAndSetRawRequestandShouldSendBackRawRequestare otherwise fine and align with this behavior.Also applies to: 409-416, 510-515
core/providers/elevenlabs/elevenlabs.go (1)
23-30: Fix raw-request flag mismatch inlistModelsByKeyand consider parity for non-stream speech
HandleProviderResponseflag mismatch
In
listModelsByKeyyou call:rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( resp.Body(), &elevenlabsResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), )but later gate
ExtraFields.RawRequestwith:if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { response.ExtraFields.RawRequest = rawRequest }If
SendBackRawRequestisfalsein config but the context setsBifrostContextKeySendBackRawRequest = true,ShouldSendBackRawRequest(...)istrueyetHandleProviderResponsenever attempted to populaterawRequest(because it sawsendBackRawRequest == false), soraw_requestwill always benull.Same fix as other providers: compute the effective flags once, and feed them consistently into both
HandleProviderResponseand the ExtraFields:sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( resp.Body(), &elevenlabsResponse, nil, sendBackRawRequest, sendBackRawResponse, ) ... if sendBackRawRequest { response.ExtraFields.RawRequest = rawRequest } if sendBackRawResponse { response.ExtraFields.RawResponse = rawResponse }This keeps context overrides behaving as expected.
Optional: non-stream
Speechcould also exposeraw_request
The non-stream
Speechmethod already buildsjsonDatausingCheckContextAndGetRequestBodyand sets upExtraFields.For symmetry with
SpeechStream, you could optionally add:if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { providerUtils.ParseAndSetRawRequest(&bifrostResponse.ExtraFields, jsonData) }after setting the base
ExtraFields, gated onwithTimestampsRequestas appropriate.Not required for correctness, but it would make raw-request behavior consistent between streaming and non-stream speech.
Also applies to: 35-63, 70-123, 287-441, 501-535
core/providers/bedrock/bedrock.go (1)
28-36: Align Bedrock list-models raw-request flags and consider Embedding parity
listModelsByKeyuses inconsistent flags for raw_request
You call:
rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( responseBody, bedrockResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), ) ... if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { response.ExtraFields.RawRequest = rawRequest }As with other providers, if the context sets
BifrostContextKeySendBackRawRequest = truebut the provider’sSendBackRawRequestisfalse,ShouldSendBackRawRequest(...)returnstruebutHandleProviderResponsenever parsedrawRequest, soraw_requestwill always be nil.Recommended change (pattern applies here exactly as in Cohere/Elevenlabs):
sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( responseBody, bedrockResponse, nil, sendBackRawRequest, sendBackRawResponse, ) ... if sendBackRawRequest { response.ExtraFields.RawRequest = rawRequest } if sendBackRawResponse { response.ExtraFields.RawResponse = rawResponse }This ensures context overrides actually control both parsing and ExtraFields population.
Embedding path currently only sets
RawResponse
In
Embedding, you already:
- Build
jsonDataviaCheckContextAndGetRequestBodyper model type.- Capture
rawResponsebytes and optionally unmarshal intoRawResponsewhen enabled.For feature parity with Chat/Responses, consider also exposing
raw_request:
Track the last
jsonDatayou used within the switch, and after creatingbifrostResponse:if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { providerUtils.ParseAndSetRawRequest(&bifrostResponse.ExtraFields, jsonData) }(If you have multiple
jsonDatavariables in the switch, factor one common variable scoped above the switch.)Not required for correctness, but it keeps Bedrock’s embedding behavior aligned with other request types.
Also applies to: 59-80, 329-476, 673-739, 741-907, 909-980, 982-1063, 1165-1257
♻️ Duplicate comments (2)
framework/streaming/chat.go (1)
279-291: Critical: Guard against nilresultand fix pointer semantics forRawRequest.Two critical issues remain from the previous review:
Nil pointer dereference: Line 280 accesses
result.ChatResponse.ExtraFields.RawRequestwithout verifying thatresultorresult.ChatResponseare non-nil. This code executes outside the earlierelse if result != nil && result.ChatResponse != nilblock (line 223), so when a final chunk arrives with an error (bifrostErr != nilandresult == nil), this will panic.Always non-nil pointer: Line 290 always assigns
&rawRequest, creating a non-nil pointer even whenrawRequestholds a nilinterface{}. This makes it impossible for consumers to distinguish "no raw request" from "raw request with nil value."Apply this diff to fix both issues:
- var rawRequest interface{} - if result.ChatResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ChatResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { + v := result.ChatResponse.ExtraFields.RawRequest + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: streamType, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis prevents the panic when
resultis nil and ensuresRawRequestis truly optional (nil when absent).core/providers/gemini/gemini.go (1)
131-188: Fix duplicate RawResponse assignment and verify raw‑request decoding.In
listModelsByKey:
- You correctly call
HandleProviderResponseand gateExtraFields.RawRequest/RawResponseassignments behindShouldSendBackRawRequest/ShouldSendBackRawResponse.- However,
RawResponseis set twice under the same condition (Lines 178‑181 and 183‑185), which is a leftover duplication from a prior review.You can safely remove the second
if providerUtils.ShouldSendBackRawResponse(...)block:// Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } - - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + response.ExtraFields.RawResponse = rawResponse + }Also, as with other providers, this path assumes
HandleProviderResponseuses the (currentlynil) request body for the raw request. Given list‑models is a GET without a JSON body, you may want to leavesendBackRawRequesteffectively disabled here, or extend your design if you ever want to surface non‑body request details.
🧹 Nitpick comments (8)
framework/configstore/migrations.go (1)
1284-1311: Consider migration naming and transaction usage.The migration implementation follows established patterns and correctly adds the
send_back_raw_requestcolumn. However, note:
- The migration ID uses plural "columns" but only adds a single column. Consider
"add_send_back_raw_request_column"for consistency.- Unlike the corresponding migration in
framework/logstore/migrations.go(line 647), this migration usesmigrator.DefaultOptionswhich hasUseTransaction: false. Confirm whether both migrations should use transactions for consistency, especially since schema changes benefit from atomic operations.Apply this diff if you want to align the naming:
- ID: "add_send_back_raw_request_columns", + ID: "add_send_back_raw_request_column",core/providers/openai/openai.go (1)
345-407: Streaming raw‑request handling aligns with existing streaming patterns.The streaming handlers now accept a
sendBackRawRequestflag and, at end‑of‑stream, callproviderUtils.ParseAndSetRawRequest(&response.ExtraFields, jsonBody)when the flag is true. This mirrors the raw‑response behavior (only final chunk carries aggregated metadata) and avoids per‑chunk bloat. GivenjsonBodyis the original request body, this is the right place to attachRawRequest.You might optionally normalize usage so all providers consistently call
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)at the outermost entry points and pass the resulting bool through, rather than sometimes re‑callingShouldSendBackRawRequestinside inner helpers.Also applies to: 436-607, 749-1094, 1231-1451
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
128-157: Raw request toggle UI is consistent with existing raw response control.The new
FormField/Switchfor “Include Raw Request” is implemented the same way assend_back_raw_response, includingform.triggeron change and explanatory helper text, so it should behave as expected and be intuitive for users.If you later add security guidance to the UI (e.g., warning about sensitive data in raw requests), this is the right spot to surface it.
core/providers/anthropic/anthropic.go (1)
361-412: Streaming end‑of‑stream RawRequest injection is aligned with design.
HandleAnthropicChatCompletionStreamingandHandleAnthropicResponsesStreamnow acceptsendBackRawRequestand, on the terminal event, callParseAndSetRawRequest(&response.ExtraFields, jsonBody)before emitting the final chunk. This matches how OpenAI streaming does it and should give clients a single, canonical raw request per stream.For consistency with other providers, you might later consider using
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)at the outermost entry points (ChatCompletionStream,ResponsesStream) and pass that bool down, instead of relying only on the provider‑level default.Also applies to: 416-429, 476-656, 716-781, 828-999
transports/bifrost-http/handlers/providers.go (1)
173-182: addProvider correctly persists SendBackRawRequest with sensible default.
addProvidernow acceptssend_back_raw_requestas an optional bool and storesconfig.SendBackRawRequestaspayload.SendBackRawRequest != nil && *payload.SendBackRawRequest. This makesfalsethe default when omitted and respects explicittrue, matching howSendBackRawResponseis handled. The fallback redacted‑config path also copiesSendBackRawRequestback into the response.If you later need to distinguish “unset” from “explicit false”, you may want to store a tri‑state in config instead of a plain bool, but for now this is adequate.
Also applies to: 245-253, 275-283
core/providers/azure/azure.go (1)
298-347: Streaming Azure→OpenAI/Anthropic helpers correctly forward sendBackRawRequest.In
TextCompletionStream,ChatCompletionStream, andResponsesStream, the computedsendBackRawRequest(providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)) is passed through to the shared OpenAI/Anthropic streaming handlers. This ensures those helpers can attachRawRequestat the final stream event when the flag is enabled for Azure calls.You may later want to align non‑stream call sites to also use
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)when callingHandleProviderResponse, for symmetry with the streaming paths.Also applies to: 442-531, 628-714
core/providers/gemini/gemini.go (1)
315-343: Streaming and embedding paths forward sendBackRawRequest consistently.
ChatCompletionStreamandEmbeddingnow forwardproviderUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)into the shared OpenAI streaming/embedding helpers. That keeps Gemini’s behavior aligned with OpenAI/Azure and ensures a single final stream chunk can carry the raw request when enabled.You might later standardize on passing
ShouldSendBackRawRequestintoHandleProviderResponseas well (instead ofprovider.sendBackRawRequestdirectly) for full symmetry between streaming and non‑streaming flows.Also applies to: 373-392
core/providers/vertex/vertex.go (1)
988-999: Optional: addRawRequestsupport for Embedding responses for consistency.
Embeddingcurrently only exposesExtraFields.RawResponse, even though the provider now has asendBackRawRequestflag and other methods attachRawRequestwhen enabled. If you want parity across operations, you can addRawRequesthere with minimal changes:@@ func (provider *VertexProvider) Embedding(ctx context.Context, key schemas.Key, request *schemas.BifrostEmbeddingRequest) (*schemas.BifrostEmbeddingResponse, *schemas.BifrostError) { // Use centralized Vertex converter bifrostResponse := vertexResponse.ToBifrostEmbeddingResponse() @@ bifrostResponse.ExtraFields.RequestType = schemas.EmbeddingRequest bifrostResponse.ExtraFields.Latency = latency.Milliseconds() @@ if bifrostResponse.ExtraFields.ModelRequested != deployment { bifrostResponse.ExtraFields.ModelDeployment = deployment } - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - var rawResponseMap map[string]interface{} - if err := sonic.Unmarshal(resp.Body(), &rawResponseMap); err != nil { - return nil, providerUtils.NewBifrostOperationError(schemas.ErrProviderRawResponseUnmarshal, err, providerName) - } - bifrostResponse.ExtraFields.RawResponse = rawResponseMap - } + // Set raw request/response if enabled + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + var rawRequestMap map[string]interface{} + if err := sonic.Unmarshal(jsonBody, &rawRequestMap); err != nil { + return nil, providerUtils.NewBifrostOperationError(schemas.ErrProviderRawRequestUnmarshal, err, providerName) + } + bifrostResponse.ExtraFields.RawRequest = rawRequestMap + } + + if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + var rawResponseMap map[string]interface{} + if err := sonic.Unmarshal(resp.Body(), &rawResponseMap); err != nil { + return nil, providerUtils.NewBifrostOperationError(schemas.ErrProviderRawResponseUnmarshal, err, providerName) + } + bifrostResponse.ExtraFields.RawResponse = rawResponseMap + }This keeps the existing
RawResponsebehavior and addsRawRequestonly when enabled, reusing the already-availablejsonBody.Also applies to: 1090-1097
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(8 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(5 hunks)core/providers/gemini/gemini.go(10 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(3 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
✅ Files skipped from review due to trivial changes (3)
- plugins/logging/changelog.md
- framework/changelog.md
- transports/changelog.md
🚧 Files skipped from review as they are similar to previous changes (22)
- ui/lib/types/logs.ts
- framework/streaming/responses.go
- plugins/logging/main.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- plugins/logging/operations.go
- core/providers/cerebras/cerebras.go
- framework/configstore/rdb.go
- core/providers/perplexity/perplexity.go
- core/providers/ollama/ollama.go
- core/changelog.md
- framework/configstore/tables/provider.go
- ui/lib/types/config.ts
- core/providers/openrouter/openrouter.go
- ui/lib/types/schemas.ts
- framework/configstore/clientconfig.go
- transports/bifrost-http/lib/account.go
- core/providers/mistral/mistral.go
- framework/logstore/tables.go
- core/schemas/provider.go
- framework/streaming/transcription.go
- framework/streaming/types.go
- core/providers/groq/groq.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
ui/app/workspace/providers/page.tsxframework/streaming/audio.gotransports/bifrost-http/lib/config.gocore/providers/anthropic/anthropic.gocore/providers/utils/utils.gocore/providers/vertex/vertex.goframework/configstore/migrations.gocore/providers/parasail/parasail.gocore/providers/elevenlabs/elevenlabs.gocore/providers/openai/openai.goframework/logstore/migrations.gotransports/bifrost-http/handlers/providers.gocore/providers/azure/azure.gocore/providers/gemini/gemini.goui/app/workspace/providers/fragments/performanceFormFragment.tsxcore/providers/sgl/sgl.gocore/providers/bedrock/bedrock.gocore/providers/cohere/cohere.goframework/streaming/chat.gocore/schemas/bifrost.go
🧬 Code graph analysis (15)
framework/streaming/audio.go (1)
framework/streaming/types.go (2)
ProcessedStreamResponse(116-124)StreamType(10-10)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/utils/utils.go (3)
core/schemas/bifrost.go (4)
BifrostError(355-364)ErrorField(373-380)BifrostResponseExtraFields(285-295)BifrostContextKeySendBackRawRequest(118-118)ui/lib/types/logs.ts (2)
BifrostError(226-232)ErrorField(217-224)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/vertex/vertex.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ShouldSendBackRawResponse(518-523)
framework/configstore/migrations.go (3)
framework/configstore/tables/virtualkey.go (4)
TableVirtualKeyProviderConfig(23-36)TableVirtualKeyProviderConfig(39-41)TableVirtualKeyProviderConfigKey(12-15)TableVirtualKeyProviderConfigKey(18-20)framework/migrator/migrator.go (3)
New(131-149)DefaultOptions(100-106)Migration(62-69)framework/configstore/tables/provider.go (2)
TableProvider(13-38)TableProvider(41-41)
core/providers/parasail/parasail.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/elevenlabs/elevenlabs.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/openai/openai.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(510-515)HandleProviderResponse(344-407)ParseAndSetRawRequest(409-416)
framework/logstore/migrations.go (2)
framework/migrator/migrator.go (3)
DefaultOptions(100-106)New(131-149)Migration(62-69)framework/logstore/tables.go (2)
Log(74-137)Log(140-142)
core/providers/azure/azure.go (1)
core/providers/utils/utils.go (2)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (2)
ui/components/ui/form.tsx (4)
FormItem(161-161)FormLabel(162-162)FormControl(163-163)FormMessage(165-165)ui/components/ui/switch.tsx (1)
Switch(36-36)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
framework/streaming/chat.go (1)
framework/streaming/types.go (2)
ProcessedStreamResponse(116-124)StreamType(10-10)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (18)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (22)
framework/configstore/migrations.go (2)
91-93: LGTM!The migration call is properly placed at the end of the migration chain with appropriate error handling.
1143-1154: Verify the relevance of these modifications.These changes to the join table setup in
migrationMoveKeysToProviderConfigappear unrelated to the raw request feature described in the PR objectives. Confirm whether these modifications are intentional or if they were inadvertently included.framework/logstore/migrations.go (2)
43-45: LGTM!The migration call is properly placed at the end of the migration chain with appropriate error handling.
645-674: LGTM!The migration implementation is clean and follows established patterns:
- Properly checks for column existence before adding
- Uses transactions for atomicity
- Includes a complete rollback function
- Error messages are descriptive
core/providers/parasail/parasail.go (4)
16-23: Provider-levelsendBackRawRequestflag is wired correctlyAdding
sendBackRawRequesttoParasailProviderand initializing it fromconfig.SendBackRawRequestmirrors the existingsendBackRawResponsepattern and keeps the per-provider default in one place. This looks correct and consistent with the rest of the provider design.Also applies to: 48-54
62-76: ListModels now respects the raw-request flag end‑to‑endPassing
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)intoHandleOpenAIListModelsRequestalongside the existing raw-response flag cleanly extends the behavior for list-models without changing other concerns. This is aligned with the utils helper semantics and other providers’ pattern.
90-104: ChatCompletion correctly propagatessendBackRawRequestThe additional
ShouldSendBackRawRequestargument in theHandleOpenAIChatCompletionRequestcall cleanly threads the new flag through the non‑streaming chat path. Ordering relative to the raw-response flag is intuitive and consistent, so this should behave as expected once the shared OpenAI helper is updated.
110-131: Streaming chat path now also supports raw-request inclusionWiring
ShouldSendBackRawRequestintoHandleOpenAIChatCompletionStreamingensures the streaming Parasail path matches the non‑streaming behavior for raw requests, avoiding divergence between the two chat entry points. The change is minimal and consistent with howsendBackRawResponseis handled.core/providers/openai/openai.go (1)
24-31: OpenAIProvider sendBackRawRequest wiring looks consistent.Adding
sendBackRawRequesttoOpenAIProvider, initializing it fromconfig.SendBackRawRequest, and threadingproviderUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)through public entry points (ListModels,TextCompletion,ChatCompletion,Responses,Embedding, and streaming variants) is coherent and matches howsendBackRawResponseis already handled.Also applies to: 36-69, 81-112
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
26-38: Form data wiring forsend_back_raw_requestis correct and symmetric.Including
send_back_raw_requestindefaultValues,form.reseton provider change, andupdatedProviderinonSubmitmirrors the existingsend_back_raw_responsebehavior and should round‑trip the new config flag cleanly through the UI/API.Also applies to: 44-53, 57-67
core/providers/anthropic/anthropic.go (1)
21-29: AnthropicProvider sendBackRawRequest wiring is straightforward.Adding
sendBackRawRequesttoAnthropicProviderand initializing it fromconfig.SendBackRawRequestpreserves the existing pattern forsendBackRawResponseand keeps configuration centralized.Also applies to: 76-111
transports/bifrost-http/handlers/providers.go (2)
55-65: Public HTTP surface now exposes send_back_raw_request cleanly.Adding
SendBackRawRequesttoProviderResponseand wiring it fromconfig.SendBackRawRequestingetProviderResponseFromConfigkeeps the HTTP API in sync with the internal config and mirrors the existingSendBackRawResponsebehavior.Also applies to: 842-861
315-323: updateProvider flow applies send_back_raw_request consistently.
updateProvidernow:
- Accepts optional
send_back_raw_request,- Applies it to
config.SendBackRawRequestonly when non‑nil, and- Ensures the redacted‑config fallback response also surfaces the updated value.
This keeps the flag’s behavior consistent across create/update flows and avoids unintentionally resetting it when the field is omitted.
Also applies to: 357-365, 419-437, 454-466
core/providers/azure/azure.go (1)
22-28: AzureProvider now tracks sendBackRawRequest alongside sendBackRawResponse.Introducing
sendBackRawRequestand initializing it fromconfig.SendBackRawRequestkeeps Azure’s provider config aligned with OpenAI/Anthropic/Gemini and centralizes the raw‑data behavior at the provider level.Also applies to: 33-54
core/providers/gemini/gemini.go (2)
23-30: GeminiProvider sendBackRawRequest integration is in line with other providers.Adding
sendBackRawRequesttoGeminiProviderand wiring it fromconfig.SendBackRawRequestmirrors the existingsendBackRawResponsebehavior and keeps all Gemini operations governed by the same config.Also applies to: 35-63
395-432: Speech/Transcription streaming RawRequest injection at stream end looks good.For speech and transcription streaming, you:
- Use the existing
completeRequesthelper for non‑streaming operations, attaching onlyRawResponsewhen enabled, and- At stream end in
SpeechStream/TranscriptionStream, callParseAndSetRawRequest(&response.ExtraFields, jsonBody)whenShouldSendBackRawRequestis true.This gives clients a single, authoritative raw request payload for the entire stream without inflating per‑chunk events, which is a good trade‑off.
Also applies to: 435-647, 650-685, 687-914
core/schemas/bifrost.go (1)
105-121: New context key andRawRequestfield look consistentThe added
BifrostContextKeySendBackRawRequestandRawRequestfield inBifrostResponseExtraFieldsare typed and tagged consistently with existing context keys andRawResponse; no issues from this file alone.Also applies to: 284-295
core/providers/sgl/sgl.go (1)
17-24: SGL raw-request wiring looks goodThe
sendBackRawRequestflag is correctly sourced from config and consistently passed asShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)into the shared OpenAI helpers alongside the raw‑response flag. This keeps config and per‑request overrides aligned.Also applies to: 29-61, 69-83, 85-101, 104-119, 121-135, 141-159, 187-201
transports/bifrost-http/lib/config.go (1)
335-363: SendBackRawRequest is correctly wired through load and redactionMapping
dbProvider.SendBackRawRequestinto the in-memoryProviderConfigand includingSendBackRawRequestinGetProviderConfigRedactedkeeps the new flag consistent across persistence and API exposure; no issues here.Also applies to: 1374-1382
ui/app/workspace/providers/page.tsx (1)
64-77: UI default forsend_back_raw_requestlooks appropriateInitializing unknown providers with
send_back_raw_request: undefined(alongsidesend_back_raw_response) in the 404 path is consistent with the rest of the config shape and lets the form control it explicitly; no concerns.Also applies to: 215-225
core/providers/vertex/vertex.go (2)
56-63: Config-level wiring forsendBackRawRequestlooks correct.Struct field and constructor wiring for
sendBackRawRequestmirror the existingsendBackRawResponsepattern and correctly useconfig.SendBackRawRequestas the provider default. No changes needed here.Also applies to: 78-84
597-606: Streaming paths correctly propagate raw request/response flags.Passing
ShouldSendBackRawRequest/Response(ctx, provider.sendBackRaw*)into the Anthropic and OpenAI streaming helpers (HandleAnthropicChatCompletionStreaming,HandleOpenAIChatCompletionStreaming,HandleAnthropicResponsesStream) ensures per-request overrides are respected for streaming just like non-streaming calls. This is consistent with the overall design of the new flags.Also applies to: 672-681, 942-952
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
10060d1 to
d6466cb
Compare
Pratham-Mishra04
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
37ebe6b to
cbd9ab9
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/providers/utils/utils.go (1)
344-407:HandleProviderResponsebuildsRawRequestfrom the response body and ignoresrequestBody(breaks feature semantics).
HandleProviderResponsecurrently does:
- Unmarshals the structured response from
responseBody(correct).- Unmarshals both
rawRequestandrawResponsefromresponseBody(Lines 356‑365).- Never uses the
requestBodyargument at all.As a result:
ExtraFields.RawRequestwill actually contain the provider response, not the request payload you sent.- The new
requestBodyparameter is dead, and callers passingjsonDataget no benefit.- Once this is relied on by multiple providers, returned raw request data will be misleading.
You can fix semantics and simplify the code by unmarshalling sequentially and using
requestBodyfor the raw request, e.g.:-func HandleProviderResponse[T any](responseBody []byte, response *T, requestBody []byte, sendBackRawRequest bool, sendBackRawResponse bool) (interface{}, interface{}, *schemas.BifrostError) { - var rawRequest interface{} - var rawResponse interface{} - - var wg sync.WaitGroup - var structuredErr, rawRequestErr, rawResponseErr error - - wg.Add(3) - go func() { - defer wg.Done() - structuredErr = sonic.Unmarshal(responseBody, response) - }() - go func() { - defer wg.Done() - if sendBackRawRequest { - rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) - } - }() - go func() { - defer wg.Done() - if sendBackRawResponse { - rawResponseErr = sonic.Unmarshal(responseBody, &rawResponse) - } - }() - wg.Wait() - - if structuredErr != nil { - return nil, nil, &schemas.BifrostError{ - IsBifrostError: true, - Error: &schemas.ErrorField{ - Message: schemas.ErrProviderResponseUnmarshal, - Error: structuredErr, - }, - } - } - - if sendBackRawRequest { - if rawRequestErr != nil { - return nil, nil, &schemas.BifrostError{ - IsBifrostError: true, - Error: &schemas.ErrorField{ - Message: schemas.ErrProviderRawRequestUnmarshal, - Error: rawRequestErr, - }, - } - } - return rawRequest, rawResponse, nil - } - - if sendBackRawResponse { - if rawResponseErr != nil { - return nil, nil, &schemas.BifrostError{ - IsBifrostError: true, - Error: &schemas.ErrorField{ - Message: schemas.ErrProviderRawResponseUnmarshal, - Error: rawResponseErr, - }, - } - } - return rawRequest, rawResponse, nil - } - - return nil, nil, nil -} +func HandleProviderResponse[T any]( + responseBody []byte, + response *T, + requestBody []byte, + sendBackRawRequest bool, + sendBackRawResponse bool, +) (interface{}, interface{}, *schemas.BifrostError) { + var rawRequest interface{} + var rawResponse interface{} + + // Parse structured response first + if err := sonic.Unmarshal(responseBody, response); err != nil { + return nil, nil, &schemas.BifrostError{ + IsBifrostError: true, + Error: &schemas.ErrorField{ + Message: schemas.ErrProviderResponseUnmarshal, + Error: err, + }, + } + } + + // Optionally parse raw request from the actual request body (if present) + if sendBackRawRequest && len(requestBody) > 0 { + if err := sonic.Unmarshal(requestBody, &rawRequest); err != nil { + return nil, nil, &schemas.BifrostError{ + IsBifrostError: true, + Error: &schemas.ErrorField{ + Message: schemas.ErrProviderRawRequestUnmarshal, + Error: err, + }, + } + } + } + + // Optionally parse raw response from the provider response body + if sendBackRawResponse { + if err := sonic.Unmarshal(responseBody, &rawResponse); err != nil { + return nil, nil, &schemas.BifrostError{ + IsBifrostError: true, + Error: &schemas.ErrorField{ + Message: schemas.ErrProviderRawResponseUnmarshal, + Error: err, + }, + } + } + } + + return rawRequest, rawResponse, nil +}This:
- Makes
RawRequestreflect the real JSON body you sent (requestBody), aligning with the new feature’s intent.- Avoids trying to parse a non‑existent request body (e.g., list‑models GETs) by guarding on
len(requestBody) > 0, so enabling raw‑request support doesn’t break those endpoints.- Keeps raw‑response behavior unchanged, still surfacing
ErrProviderRawResponseUnmarshalon failure.
ParseAndSetRawRequestalready usesjsonBodycorrectly; this change brings the non‑streaming helper in line with that behavior.core/providers/cohere/cohere.go (1)
546-569: Remove duplicated RawRequest/RawResponse assignments in Cohere ResponsesIn
CohereProvider.Responsesyou setbifrostResponse.ExtraFields.RawRequestandRawResponsethree times in a row:// Set raw request if enabled if ShouldSendBackRawRequest(...) { RawRequest = rawRequest } // Set raw response if enabled if ShouldSendBackRawResponse(...) { RawResponse = rawResponse } // Set raw request if enabled if ShouldSendBackRawRequest(...) { RawRequest = rawRequest } // Set raw response if enabled if ShouldSendBackRawResponse(...) { RawResponse = rawResponse } // Set raw response if enabled if ShouldSendBackRawResponse(...) { RawResponse = rawResponse }This is redundant, adds noise, and makes the logic harder to read without changing behavior.
You can collapse this to a single pair of checks:
- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - bifrostResponse.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - bifrostResponse.ExtraFields.RawResponse = rawResponse - } - - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - bifrostResponse.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - bifrostResponse.ExtraFields.RawResponse = rawResponse - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - bifrostResponse.ExtraFields.RawResponse = rawResponse - } + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + bifrostResponse.ExtraFields.RawRequest = rawRequest + } + if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + bifrostResponse.ExtraFields.RawResponse = rawResponse + }This keeps behavior identical while improving readability.
♻️ Duplicate comments (9)
framework/streaming/chat.go (1)
279-291: Critical: Guard against nilresultand fix pointer semantics forRawRequest.Two critical issues remain from the previous review:
Nil pointer dereference: Lines 280-281 access
result.ChatResponse.ExtraFields.RawRequestwithout verifying thatresultorresult.ChatResponseare non-nil. This code executes outside the earlierelse if result != nil && result.ChatResponse != nilblock (line 223), so when a final chunk arrives with an error (bifrostErris non-nil whileresultis nil), this will panic.Always non-nil pointer: Line 290 always assigns
&rawRequest, creating a non-nil pointer even whenrawRequestholds a nilinterface{}. This makes it impossible for consumers to distinguish "no raw request" from "raw request with nil value."Apply this diff to fix both issues:
- var rawRequest interface{} - if result.ChatResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ChatResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { + v := result.ChatResponse.ExtraFields.RawRequest + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: streamType, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis prevents the panic when
resultis nil and ensuresRawRequestis truly optional (nil when absent).framework/streaming/audio.go (1)
170-173: Critical: Guard against nilresult.SpeechStreamResponseand fix pointer semantics forRawRequest.Two critical issues remain from the previous review:
Nil pointer dereference: Line 171 accesses
result.SpeechStreamResponse.ExtraFields.RawRequestwithout verifying thatresultorresult.SpeechStreamResponseare non-nil. This code executes outside the earlierelse if result != nil && result.SpeechStreamResponse != nilblock (line 129), so when a final chunk arrives with an error orSpeechStreamResponseis nil, this will panic.Always non-nil pointer: Line 181 always assigns
&rawRequest, creating a non-nil pointer even whenrawRequestholds a nilinterface{}. This makes it impossible for consumers to distinguish "no raw request" from "raw request with nil value."Apply this diff to fix both issues:
- var rawRequest interface{} - if result.SpeechStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.SpeechStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.SpeechStreamResponse != nil && result.SpeechStreamResponse.ExtraFields.RawRequest != nil { + v := result.SpeechStreamResponse.ExtraFields.RawRequest + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeAudio, Model: model, Provider: provider, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis prevents the panic when
resultorSpeechStreamResponseis nil and ensuresRawRequestis truly optional (nil when absent).Also applies to: 181-181
framework/streaming/responses.go (2)
723-726: Critical: Guard against nilresult.ResponsesStreamResponsein OpenAI provider path.Line 724 accesses
result.ResponsesStreamResponse.ExtraFields.RawRequestwithout first ensuringresultandresult.ResponsesStreamResponseare non-nil. While there's a check at line 747 (if result != nil && result.ResponsesStreamResponse != nil), the access at line 724 happens BEFORE that check in theif isFinalChunkblock. Additionally, line 789 always assigns&rawRequest, creating a non-nil pointer even when no raw request exists.Apply this diff to fix both issues:
if isFinalChunk { - var rawRequest interface{} - if result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + v := result.ResponsesStreamResponse.ExtraFields.RawRequest + rawRequest = &v + } shouldProcess := false // Get the accumulator to check if processing has already been triggered accumulator := a.getOrCreateStreamAccumulator(requestID)And at line 789:
Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis prevents nil pointer panics and ensures
RawRequestis truly optional.Also applies to: 789-789
866-870: Critical: Guard against nilresult.ResponsesStreamResponsein non-OpenAI provider path.Lines 867-868 access
result.ResponsesStreamResponse.ExtraFields.RawRequestwithout checking thatresultandresult.ResponsesStreamResponseare non-nil. This occurs in the final chunk processing path whereresultcan be nil during error cases. Additionally, line 878 always assigns&rawRequest, preventing consumers from distinguishing "no raw request" from "raw request with nil value."Apply this diff to fix both issues:
- var rawRequest interface{} - if result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + v := result.ResponsesStreamResponse.ExtraFields.RawRequest + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeResponses, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis prevents panics and ensures correct pointer semantics for optional fields.
Also applies to: 878-878
plugins/logging/operations.go (1)
163-179: Raw request logging ignoresdisableContentLoggingin non‑streaming path (and minor precedence bug).In
updateLogEntry,raw_requestis always persisted (lines 172‑179) even whendisableContentLoggingis true, while all other content fields (and the streaming path) respect the content‑logging flag. This can unexpectedly log sensitive request payloads despite content logging being disabled, and is inconsistent with the streaming branch where raw request/response live inside the content‑logging guard.Also, the condition at Line 163:
if p.disableContentLogging == nil || !*p.disableContentLogging && data.RawResponse != nil {relies on
&&precedence and will evaluate true whendisableContentLogging == nileven ifdata.RawResponseis nil, resulting in serializing a nil response.Recommend:
- Gate both
raw_responseandraw_requestunder the same content‑logging conditional.- Fix the precedence around
data.RawResponse != nil.For example:
- if p.disableContentLogging == nil || !*p.disableContentLogging && data.RawResponse != nil { - rawResponseBytes, err := sonic.Marshal(data.RawResponse) - if err != nil { - p.logger.Error("failed to marshal raw response: %v", err) - } else { - updates["raw_response"] = string(rawResponseBytes) - } - } - - if data.RawRequest != nil { - rawRequestBytes, err := sonic.Marshal(data.RawRequest) - if err != nil { - p.logger.Error("failed to marshal raw request: %v", err) - } else { - updates["raw_request"] = string(rawRequestBytes) - } - } + if p.disableContentLogging == nil || !*p.disableContentLogging { + if data.RawResponse != nil { + rawResponseBytes, err := sonic.Marshal(data.RawResponse) + if err != nil { + p.logger.Error("failed to marshal raw response: %v", err) + } else { + updates["raw_response"] = string(rawResponseBytes) + } + } + + if data.RawRequest != nil { + rawRequestBytes, err := sonic.Marshal(data.RawRequest) + if err != nil { + p.logger.Error("failed to marshal raw request: %v", err) + } else { + updates["raw_request"] = string(rawRequestBytes) + } + } + }This makes non‑streaming behavior match the streaming block (lines 256‑316) and honors the content‑logging contract for raw requests as well.
Also applies to: 256-316
core/providers/perplexity/perplexity.go (1)
145-145: Inconsistent usage of ShouldSendBackRawRequest.The past review comment flagged that
provider.sendBackRawRequestis passed directly toHandleProviderResponsewhileShouldSendBackRawResponseuses the context-aware helper. The comment was marked as addressed, but the current code still shows the direct flag usage.For consistency with context-based overrides:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/gemini/gemini.go (1)
178-185: Duplicate code block setting raw response twice.Lines 178-181 and 183-185 are identical — both conditionally set
RawResponseunder the same condition. This was flagged in a previous review but appears to still be present.// Set raw request if enabled if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { response.ExtraFields.RawRequest = rawRequest } // Set raw response if enabled if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { response.ExtraFields.RawResponse = rawResponse } - - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } return response, nilcore/providers/openai/openai.go (1)
161-183: Fix HandleProviderResponse raw‑request capture and guard GET list‑models from nil bodiesAll the non‑stream OpenAI handlers you’ve wired (
listModelsByKey,HandleOpenAITextCompletionRequest,HandleOpenAIChatCompletionRequest,HandleOpenAIResponsesRequest,HandleOpenAIEmbeddingRequest) now rely onproviderUtils.HandleProviderResponseto capturerawRequestusing thesendBackRawRequestflag.Per the current implementation in
core/providers/utils/utils.go(and as already flagged in the earlier review):if sendBackRawRequest { rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) // uses responseBody }This means:
ExtraFields.RawRequestwill actually contain the response, not the request, for all non‑stream paths usingHandleProviderResponse.- The new
sendBackRawRequestplumbing in this file will not behave as advertised until the helper is corrected.You should update the helper to unmarshal from the request body, and also avoid treating a nil/empty request body as an error (important for GET endpoints like list‑models):
go func() { defer wg.Done() if sendBackRawRequest { - rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) + if len(requestBody) == 0 { + // No structured request body to capture (e.g., GET list-models); skip raw request parsing + return + } + rawRequestErr = sonic.Unmarshal(requestBody, &rawRequest) } }()Once this is fixed, be aware that
listModelsByKeycurrently passesrequestBody == nilbut may havesendBackRawRequest == true(through the flags passed intolistModelsByKey). Without thelen(requestBody) == 0guard (or explicitly disabling raw‑request capture for list‑models), that combination would start failing with a raw‑request unmarshal error.Aside from this helper bug, the way you propagate
sendBackRawRequest/sendBackRawResponsefrom the provider entry points into the handlers (and then intoExtraFields) looks consistent and should behave correctly once the utility is fixed.Also applies to: 292-310, 693-711, 1179-1197, 1540-1558
core/providers/vertex/vertex.go (1)
61-63: Vertex raw payload handling still ignores context overrides and duplicates RawResponse assignmentThe concerns from the earlier review on this file appear unresolved:
- Context‑override mismatch in non‑streaming paths
listModelsByKey, AnthropicChatCompletion, and AnthropicResponsescallHandleProviderResponse(..., provider.sendBackRawRequest, provider.sendBackRawResponse)but later gateExtraFields.RawRequest/RawResponseviaShouldSendBackRawRequest/Response(ctx, provider.sendBackRaw*).- If the provider defaults are
falsebut a caller enables raw payloads via context,HandleProviderResponsenever captures them, andExtraFields.Raw*ends up nil even thoughShould*returns true.listModelsByKeyalso appendsrawRequest/rawResponseinto slices without nil checks, so you may accumulate[]interface{}{nil, ...}.You can fix this by computing effective flags once and reusing them everywhere, as previously suggested, e.g.:
@@ func (provider *VertexProvider) listModelsByKey(...) - var allModels []VertexModel - var rawRequests []interface{} - var rawResponses []interface{} + var allModels []VertexModel + var rawRequests []interface{} + var rawResponses []interface{} + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) @@ - var vertexResponse VertexListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &vertexResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + var vertexResponse VertexListModelsResponse + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + &vertexResponse, + nil, // GET: no JSON body + false, // or sendBackRawRequest if you decide to support raw-request capture here + sendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - rawRequests = append(rawRequests, rawRequest) - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - rawResponses = append(rawResponses, rawResponse) - } + if sendBackRawRequest && rawRequest != nil { + rawRequests = append(rawRequests, rawRequest) + } + if sendBackRawResponse && rawResponse != nil { + rawResponses = append(rawResponses, rawResponse) + } @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequests - } - - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponses - } + if sendBackRawRequest && len(rawRequests) > 0 { + response.ExtraFields.RawRequest = rawRequests + } + if sendBackRawResponse && len(rawResponses) > 0 { + response.ExtraFields.RawResponse = rawResponses + }Apply the same “compute once, reuse” pattern in Anthropic
ChatCompletionandResponseswhen callingHandleProviderResponseand when populatingExtraFields.
- Redundant RawResponse assignment in Anthropic
ResponsesIn the Anthropic
Responsesbranch you setresponse.ExtraFields.RawResponseboth inside theif ShouldSendBackRawResponseblock around lines 820–823 and again around 828–830. The second assignment is redundant and can be safely removed:- // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } @@ - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - }Finally, note that
listModelsByKeypassesrequestBody == nilintoHandleProviderResponsewhile potentially enabling raw‑request capture. OnceHandleProviderResponseis corrected to use the request body for rawRequest, this GET path will start failing unless you disable raw‑request capture here or make the helper treat nil bodies as “no raw request”.Also applies to: 79-84, 148-236, 438-502, 795-833
🧹 Nitpick comments (4)
core/providers/azure/azure.go (1)
22-28: Unify raw‑data flags betweenHandleProviderResponseandExtraFieldsto honor context overrides.Right now the non‑streaming methods pass
provider.sendBackRawRequest/provider.sendBackRawResponseintoHandleProviderResponse, but they populateExtraFields.RawRequest/RawResponsebased onShouldSendBackRawRequest/Response(ctx, provider.sendBackRaw*). If a caller flips the context flag totruewhile the config default isfalse,ShouldSendBackRaw*can returntruebutHandleProviderResponsewill never have populated the correspondingraw*value (it staysnil).To make context overrides effective and avoid confusing
nilraw fields, consider computing the effective flags once and reusing them, e.g.:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, + azureResponse, + nil, // or jsonData where applicable + sendBackRawRequest, + sendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if sendBackRawRequest { response.ExtraFields.RawRequest = rawRequest } @@ - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if sendBackRawResponse { response.ExtraFields.RawResponse = rawResponse }The same pattern applies to
TextCompletion,ChatCompletion,Responses, andEmbedding.Also applies to: 47-53, 197-218, 274-294, 404-437, 590-624, 754-774
core/providers/mistral/mistral.go (1)
105-105: Raw request unmarshalling will fail for GET requests with no body.
HandleProviderResponseis called withnilforrequestBody, butprovider.sendBackRawRequestis passed directly. Looking at the util function, whensendBackRawRequestis true butrequestBodyis nil, it will attemptsonic.Unmarshal(responseBody, &rawRequest)— this unmarshals the response body intorawRequest, not the actual request payload.For consistency with other providers and to avoid confusion, consider using the context-aware helper:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))Note: Since this is a GET request with no body, enabling raw request here may not provide meaningful data. Consider whether this flag should be skipped entirely for list operations.
core/providers/elevenlabs/elevenlabs.go (1)
103-103: Consider using context-aware helper for consistency.Similar to other providers, the
sendBackRawRequestflag is passed directly toHandleProviderResponseinstead of using the context-aware helper. For consistency with the conditional check at line 113:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &elevenlabsResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &elevenlabsResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/gemini/gemini.go (1)
164-164: Consider using context-aware helper for consistency.Both
listModelsByKeyandChatCompletionpassprovider.sendBackRawRequestdirectly toHandleProviderResponseinstead of usingShouldSendBackRawRequest(ctx, ...). For consistency with the conditional checks and other methods likeChatCompletionStreamandEmbedding:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))Also applies to: 274-274
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(8 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(5 hunks)core/providers/gemini/gemini.go(10 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(3 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (17)
- plugins/logging/main.go
- framework/configstore/tables/provider.go
- transports/bifrost-http/lib/config.go
- core/schemas/bifrost.go
- core/providers/cerebras/cerebras.go
- core/providers/bedrock/bedrock.go
- framework/streaming/transcription.go
- ui/lib/types/schemas.ts
- plugins/logging/changelog.md
- transports/bifrost-http/handlers/providers.go
- ui/app/workspace/providers/page.tsx
- ui/lib/types/logs.ts
- core/changelog.md
- framework/configstore/migrations.go
- transports/changelog.md
- framework/changelog.md
- ui/app/workspace/logs/views/logDetailsSheet.tsx
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
core/providers/groq/groq.gocore/schemas/provider.goframework/configstore/rdb.goplugins/logging/operations.goframework/streaming/types.gocore/providers/mistral/mistral.gocore/providers/sgl/sgl.gocore/providers/ollama/ollama.gocore/providers/utils/utils.gocore/providers/gemini/gemini.goui/lib/types/config.tscore/providers/parasail/parasail.goframework/logstore/tables.goframework/streaming/audio.goui/app/workspace/providers/fragments/performanceFormFragment.tsxframework/streaming/responses.gocore/providers/vertex/vertex.goframework/configstore/clientconfig.gocore/providers/elevenlabs/elevenlabs.gocore/providers/cohere/cohere.goframework/logstore/migrations.goframework/streaming/chat.gocore/providers/perplexity/perplexity.gocore/providers/anthropic/anthropic.gotransports/bifrost-http/lib/account.gocore/providers/azure/azure.gocore/providers/openrouter/openrouter.gocore/providers/openai/openai.go
🧬 Code graph analysis (17)
core/providers/groq/groq.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/mistral/mistral.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/ollama/ollama.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/utils/utils.go (2)
core/schemas/bifrost.go (4)
BifrostError(355-364)ErrorField(373-380)BifrostResponseExtraFields(285-295)BifrostContextKeySendBackRawRequest(118-118)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/parasail/parasail.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
framework/streaming/audio.go (1)
framework/streaming/types.go (3)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (2)
ui/components/ui/form.tsx (4)
FormItem(161-161)FormLabel(162-162)FormControl(163-163)FormMessage(165-165)ui/components/ui/switch.tsx (1)
Switch(36-36)
framework/streaming/responses.go (1)
framework/streaming/types.go (3)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)
core/providers/vertex/vertex.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ShouldSendBackRawResponse(518-523)
framework/configstore/clientconfig.go (2)
core/schemas/provider.go (1)
CustomProviderConfig(216-222)ui/lib/types/config.ts (1)
CustomProviderConfig(153-158)
core/providers/elevenlabs/elevenlabs.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
framework/streaming/chat.go (1)
framework/streaming/types.go (3)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/openrouter/openrouter.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (21)
framework/streaming/types.go (1)
123-123: LGTM! Field type is appropriate for optional raw request data.The
RawRequest *interface{}field correctly uses a pointer to allow distinguishing "no raw request" (nil pointer) from "raw request with value" (non-nil pointer), andinterface{}provides the flexibility needed to hold various request payload types.transports/bifrost-http/lib/account.go (1)
80-115: WiringSendBackRawRequestthroughGetConfigForProviderlooks correct.
providerConfig.SendBackRawRequest = config.SendBackRawRequestmirrors the existingSendBackRawResponsepath and ensures providers see the configured flag. No further changes needed here.core/schemas/provider.go (1)
21-34: New raw‑request error and ProviderConfig flag are consistent with existing raw‑response wiring.
ErrProviderRawRequestUnmarshalandProviderConfig.SendBackRawRequestfollow the existing patterns for raw responses (naming, comments, JSON tags). Given the semantics are enforced in the utils/providers layer, this schema change looks good as‑is.Also applies to: 232-244
framework/configstore/clientconfig.go (1)
28-55: Config‑store representation forsend_back_raw_requestmatches core schemas.
framework/configstore.ClientConfig.ProviderConfig.SendBackRawRequestmirrorsSendBackRawResponseand aligns withcore/schemas.ProviderConfig. This should round‑trip correctly through configstore and into providers.framework/logstore/tables.go (1)
72-109:Log.RawRequestschema matches existingRawResponsepattern.Adding
RawRequest stringwithgorm:"type:text"and JSON tagraw_requestis consistent with howRawResponseis stored and exposed. No additional serialization hooks are required since these fields are treated as opaque JSON strings.framework/logstore/migrations.go (1)
11-47: Raw‑request column migration is wired correctly and follows existing patterns.
triggerMigrationsinvokesmigrationAddRawRequestColumn, and the migration itself conditionally addsraw_requesttologs, with a straightforward rollback. This mirrors howraw_responseand other columns are added and should be safe across upgrades and fresh installs.Also applies to: 645-674
ui/lib/types/config.ts (1)
166-167: LGTM! Type definitions are consistent with the existing pattern.The new
send_back_raw_requestoptional boolean fields correctly mirror the existingsend_back_raw_responsepattern across all three interfaces (ModelProviderConfig,AddProviderRequest,UpdateProviderRequest), maintaining consistency with Go backend structures.Also applies to: 190-191, 201-202
core/providers/groq/groq.go (2)
20-21: LGTM! Consistent implementation of raw request tracking.The
sendBackRawRequestfield is properly added to the struct and initialized from config, following the established pattern forsendBackRawResponse.Also applies to: 56-57
76-77: LGTM! Raw request flag correctly propagated to handlers.The
ShouldSendBackRawRequestutility is consistently passed to all OpenAI-compatible handlers (ListModels,ChatCompletion,ChatCompletionStream), allowing both provider-level defaults and per-request context overrides.Also applies to: 166-167, 190-191
core/providers/parasail/parasail.go (2)
21-22: LGTM! Consistent with other provider implementations.The Parasail provider correctly follows the same pattern as other providers for adding raw request tracking support.
Also applies to: 52-53
72-73: LGTM! Raw request flag propagation is consistent.The
ShouldSendBackRawRequestcall is properly added toListModels,ChatCompletion, andChatCompletionStreamhandlers.Also applies to: 99-100, 123-124
core/providers/ollama/ollama.go (2)
22-23: LGTM! Raw request field properly added to Ollama provider.Field addition and initialization follow the established pattern.
Also applies to: 59-60
82-83: LGTM! Comprehensive coverage across all supported request types.The
ShouldSendBackRawRequestflag is correctly propagated to all six OpenAI-compatible handlers that Ollama supports:ListModels,TextCompletion,TextCompletionStream,ChatCompletion,ChatCompletionStream, andEmbedding.Also applies to: 98-99, 115-116, 133-134, 153-154, 200-201
core/providers/sgl/sgl.go (2)
22-23: LGTM! SGL provider correctly implements raw request tracking.Field addition and initialization are consistent with the other providers in this PR.
Also applies to: 59-60
79-80: LGTM! All handler invocations updated consistently.The
ShouldSendBackRawRequestflag is properly propagated to all six supported handlers, matching the Ollama provider's implementation pattern.Also applies to: 95-96, 112-113, 130-131, 150-151, 197-198
core/providers/mistral/mistral.go (1)
21-22: LGTM — raw request wiring is correctly implemented.The
sendBackRawRequestfield is properly added to the struct, initialized from config, and consistently propagated to OpenAI-compatible handlers usingShouldSendBackRawRequestfor ChatCompletion, ChatCompletionStream, and Embedding operations.Also applies to: 57-58, 161-161, 185-185, 234-234
core/providers/elevenlabs/elevenlabs.go (1)
27-27: LGTM — raw request handling properly integrated.The
sendBackRawRequestfield is correctly wired and the SpeechStream final response appropriately usesParseAndSetRawRequestto capture the raw request body.Also applies to: 60-60, 112-115, 432-435
core/providers/perplexity/perplexity.go (1)
23-23: LGTM — raw request field and conditional logic are correct.The struct field, constructor wiring, ExtraFields assignment, and streaming handler parameter are all properly implemented.
Also applies to: 54-54, 158-161, 193-193
framework/configstore/rdb.go (1)
213-214: LGTM — SendBackRawRequest correctly persisted across all CRUD operations.The new field is properly wired through
AddProvider,UpdateProvider,UpdateProvidersConfig, andGetProvidersConfig, following the same pattern as the existingSendBackRawResponsefield.Also applies to: 327-328, 438-439, 627-628
core/providers/gemini/gemini.go (1)
27-27: LGTM — raw request handling properly integrated across Gemini operations.The field is correctly wired and raw request visibility is properly propagated to ChatCompletion, ChatCompletionStream, Embedding, SpeechStream, and TranscriptionStream operations.
Also applies to: 60-60, 298-306, 334-334, 388-388, 637-640, 904-907
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
31-37: Raw‑request toggle wiring is consistent with existing raw‑response behavior
send_back_raw_requestis correctly threaded through form defaults, resets, submit payload, and the newSwitchfield, mirroringsend_back_raw_response. The UX and state handling look coherent.Also applies to: 45-53, 65-67, 128-156
Pratham-Mishra04
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
cbd9ab9 to
feb7ebe
Compare
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
d6466cb to
1cb0123
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/providers/gemini/gemini.go (1)
131-181: Honor context-based raw-request/response flags when unmarshalling, not just when setting ExtraFieldsIn both
listModelsByKeyandChatCompletion,HandleProviderResponseis called withprovider.sendBackRawRequest/provider.sendBackRawResponse, whileShouldSendBackRawRequest/Responseis only used later when populatingExtraFields. If a caller enables raw payloads via context while the provider defaults are false, raw request/response will never be captured (they will staynil), yet the laterShouldSendBackRaw*checks will still assign thosenilvalues intoExtraFields.Compute the effective flags once using
ShouldSendBackRawRequest/Response, pass them intoHandleProviderResponse, and reuse the same booleans when attachingExtraFieldsto keep capture and exposure consistent and to support context overrides:func (provider *GeminiProvider) listModelsByKey(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { @@ - // Parse Gemini's response - var geminiResponse GeminiListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + // Parse Gemini's response + var geminiResponse GeminiListModelsResponse + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + &geminiResponse, + nil, // GET has no request body + sendBackRawRequest, + sendBackRawResponse, + ) @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + // Set raw request if enabled + if sendBackRawRequest { response.ExtraFields.RawRequest = rawRequest } // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if sendBackRawResponse { response.ExtraFields.RawResponse = rawResponse }func (provider *GeminiProvider) ChatCompletion(ctx context.Context, key schemas.Key, request *schemas.BifrostChatRequest) (*schemas.BifrostChatResponse, *schemas.BifrostError) { @@ - response := &schemas.BifrostChatResponse{} - - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, response, jsonData, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + response := &schemas.BifrostChatResponse{} + + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + body, + response, + jsonData, + sendBackRawRequest, + sendBackRawResponse, + ) @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + // Set raw request if enabled + if sendBackRawRequest { response.ExtraFields.RawRequest = rawRequest } // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if sendBackRawResponse { response.ExtraFields.RawResponse = rawResponse }This keeps the behavior consistent with streaming and embedding paths (which already pass the effective booleans) and ensures context overrides actually control raw payload capture.
Also applies to: 268-303
core/providers/utils/utils.go (1)
344-407: HandleProviderResponse ignores requestBody and silently mislabels the “raw request”
HandleProviderResponsetakesrequestBody []byteand is used by providers expectingRawRequestto reflect the actual JSON request (e.g., passingjsonBodyfromCheckContextAndGetRequestBody), but the implementation currently:
- Completely ignores
requestBodyand instead unmarshalsresponseBodyintorawRequest, soRawRequestandRawResponseeffectively duplicate the response.- Always
wg.Add(3)even when only a subset of unmarshals will run.- When both
sendBackRawRequestandsendBackRawResponseare true, it only validatesrawRequestErrand can return a non-nilrawResponseErras if everything succeeded.This breaks the feature semantics (“send back raw request”) and can hide raw-response unmarshal failures.
Use
requestBodyfor the request path, only spawn goroutines for the enabled flags, and validate both error paths. For example:-func HandleProviderResponse[T any](responseBody []byte, response *T, requestBody []byte, sendBackRawRequest bool, sendBackRawResponse bool) (interface{}, interface{}, *schemas.BifrostError) { - var rawRequest interface{} - var rawResponse interface{} - - var wg sync.WaitGroup - var structuredErr, rawRequestErr, rawResponseErr error - - wg.Add(3) - go func() { - defer wg.Done() - structuredErr = sonic.Unmarshal(responseBody, response) - }() - go func() { - defer wg.Done() - if sendBackRawRequest { - rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) - } - }() - go func() { - defer wg.Done() - if sendBackRawResponse { - rawResponseErr = sonic.Unmarshal(responseBody, &rawResponse) - } - }() - wg.Wait() +func HandleProviderResponse[T any](responseBody []byte, response *T, requestBody []byte, sendBackRawRequest bool, sendBackRawResponse bool) (interface{}, interface{}, *schemas.BifrostError) { + var rawRequest interface{} + var rawResponse interface{} + + var wg sync.WaitGroup + var structuredErr, rawRequestErr, rawResponseErr error + + // Always parse the typed response + wg.Add(1) + go func() { + defer wg.Done() + structuredErr = sonic.Unmarshal(responseBody, response) + }() + + attemptRawRequest := sendBackRawRequest && len(requestBody) > 0 + if attemptRawRequest { + wg.Add(1) + go func() { + defer wg.Done() + rawRequestErr = sonic.Unmarshal(requestBody, &rawRequest) + }() + } + + if sendBackRawResponse { + wg.Add(1) + go func() { + defer wg.Done() + rawResponseErr = sonic.Unmarshal(responseBody, &rawResponse) + }() + } + + wg.Wait() @@ if structuredErr != nil { return nil, nil, &schemas.BifrostError{ @@ - if sendBackRawRequest { - if rawRequestErr != nil { + if attemptRawRequest { + if rawRequestErr != nil { return nil, nil, &schemas.BifrostError{ @@ - if sendBackRawResponse { - if rawResponseErr != nil { + if sendBackRawResponse { + if rawResponseErr != nil { return nil, nil, &schemas.BifrostError{ @@ - return rawRequest, rawResponse, nil + return rawRequest, rawResponse, nil } return nil, nil, nil }This ensures:
RawRequestactually reflects the original request JSON when available.- Context-controlled
sendBackRawRequest/Responseflags affect both capture and exposure consistently.- Raw-response unmarshal failures surface as proper
BifrostErrors.
♻️ Duplicate comments (12)
core/providers/perplexity/perplexity.go (1)
145-145: Inconsistent usage of ShouldSendBackRawRequest at capture vs exposure time persists.This issue was flagged in a previous review and marked as addressed, but the code still passes
provider.sendBackRawRequestdirectly toHandleProviderResponseat capture time (line 145), while usingShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)at exposure time (lines 159-161).Apply the suggested fix from the previous review:
-rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) +rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/openrouter/openrouter.go (1)
99-119: Raw request capture/exposure inconsistency and nil requestBody handling issue persists.This was flagged in a previous review at lines 22-24. The issues remain:
provider.sendBackRawRequestis passed directly at capture time (line 100) butShouldSendBackRawRequestis used at exposure time (lines 112-114), causing context overrides to be ignored at capture.- For this GET endpoint,
requestBodyisnil, which could cause issues ifHandleProviderResponseis updated to unmarshal the request body for raw request capture.Apply the comprehensive fix suggested in the previous review to compute effective flags once and use them consistently.
core/providers/vertex/vertex.go (1)
147-236: Vertex still ignores context-based raw-request/response overrides when capturing (and appends nils)The non-streaming Vertex paths mirror the earlier issue that was called out:
listModelsByKeycallsHandleProviderResponsewithprovider.sendBackRawRequest/provider.sendBackRawResponse, but later decides whether to append torawRequests/rawResponsesand setExtraFieldsusingShouldSendBackRawRequest/Response(ctx, ...).ChatCompletion(both Anthropic and non-Anthropic branches) and the AnthropicResponsespath do the same.Consequences:
- If a caller enables raw payloads via context while the provider defaults are false,
HandleProviderResponsenever captures rawRequest/rawResponse, soExtraFields.RawRequest/RawResponsewill end upnildespite the override.- In
listModelsByKey,rawRequests/rawResponsescan accumulatenilentries, which are then assigned wholesale intoresponse.ExtraFields.Compute effective flags once per call and pass them into both unmarshalling and ExtraFields wiring, and skip appending nils, for example:
func (provider *VertexProvider) listModelsByKey(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { @@ - // Accumulate all models from paginated requests - var allModels []VertexModel - var rawRequests []interface{} - var rawResponses []interface{} + // Accumulate all models from paginated requests + var allModels []VertexModel + var rawRequests []interface{} + var rawResponses []interface{} @@ - // Loop through all pages until no nextPageToken is returned + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + + // Loop through all pages until no nextPageToken is returned @@ - // Parse Vertex's response - var vertexResponse VertexListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &vertexResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + // Parse Vertex's response + var vertexResponse VertexListModelsResponse + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + &vertexResponse, + nil, // GET has no request body + sendBackRawRequest, + sendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - rawRequests = append(rawRequests, rawRequest) - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - rawResponses = append(rawResponses, rawResponse) - } + if sendBackRawRequest && rawRequest != nil { + rawRequests = append(rawRequests, rawRequest) + } + if sendBackRawResponse && rawResponse != nil { + rawResponses = append(rawResponses, rawResponse) + } @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequests - } - - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponses - } + if sendBackRawRequest && len(rawRequests) > 0 { + response.ExtraFields.RawRequest = rawRequests + } + if sendBackRawResponse && len(rawResponses) > 0 { + response.ExtraFields.RawResponse = rawResponses + }func (provider *VertexProvider) ChatCompletion(ctx context.Context, key schemas.Key, request *schemas.BifrostChatRequest) (*schemas.BifrostChatResponse, *schemas.BifrostError) { @@ - deployment := provider.getModelDeployment(key, request.Model) + deployment := provider.getModelDeployment(key, request.Model) + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) @@ - if schemas.IsAnthropicModel(deployment) { + if schemas.IsAnthropicModel(deployment) { @@ - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), anthropicResponse, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + anthropicResponse, + jsonBody, + sendBackRawRequest, + sendBackRawResponse, + ) @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse { + response.ExtraFields.RawResponse = rawResponse + } @@ - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), response, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + response, + jsonBody, + sendBackRawRequest, + sendBackRawResponse, + ) @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse { + response.ExtraFields.RawResponse = rawResponse + }func (provider *VertexProvider) Responses(ctx context.Context, key schemas.Key, request *schemas.BifrostResponsesRequest) (*schemas.BifrostResponsesResponse, *schemas.BifrostError) { @@ - // Create response object from pool - anthropicResponse := anthropic.AcquireAnthropicMessageResponse() - defer anthropic.ReleaseAnthropicMessageResponse(anthropicResponse) - - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), anthropicResponse, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + // Create response object from pool + anthropicResponse := anthropic.AcquireAnthropicMessageResponse() + defer anthropic.ReleaseAnthropicMessageResponse(anthropicResponse) + + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + anthropicResponse, + jsonBody, + sendBackRawRequest, + sendBackRawResponse, + ) @@ - response.ExtraFields.ModelRequested = request.Model - - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + response.ExtraFields.ModelRequested = request.Model + if sendBackRawRequest { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse { + response.ExtraFields.RawResponse = rawResponse + }This keeps provider defaults intact, correctly supports per-request overrides, and avoids filling arrays with
nilentries.Also applies to: 438-502, 795-827
ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
364-388: Raw Request viewer now correctly falls back to the request stringThe new Raw Request block is wired correctly and the parse fallback now returns
log.raw_requestinstead oflog.raw_response, so the “Raw Request sent to …” section always shows request data, parsed when JSON and raw otherwise.plugins/logging/operations.go (1)
163-179: Make raw_request logging respectdisableContentLoggingand fix the RawResponse conditionTwo issues remain in the non-streaming logging path:
disableContentLoggingis ignored for raw_request –data.RawRequestis always marshalled and stored, even when content logging is disabled, unlike other content fields and unlike the streaming path.- RawResponse condition mixes
||/&&incorrectly –
if p.disableContentLogging == nil || !*p.disableContentLogging && data.RawResponse != nil
evaluates as(disable == nil) || (!*disable && data.RawResponse != nil), so whendisableContentLoggingisnilyou’ll log raw responses even ifdata.RawResponseisnil(marshallingnilto"null").For consistency with the rest of the plugin and with the streaming path, gate both raw_response and raw_request behind a single “logging enabled” flag and a non-nil value:
- if p.disableContentLogging == nil || !*p.disableContentLogging && data.RawResponse != nil { - rawResponseBytes, err := sonic.Marshal(data.RawResponse) - if err != nil { - p.logger.Error("failed to marshal raw response: %v", err) - } else { - updates["raw_response"] = string(rawResponseBytes) - } - } - - if data.RawRequest != nil { - rawRequestBytes, err := sonic.Marshal(data.RawRequest) - if err != nil { - p.logger.Error("failed to marshal raw request: %v", err) - } else { - updates["raw_request"] = string(rawRequestBytes) - } - } + loggingEnabled := p.disableContentLogging == nil || !*p.disableContentLogging + if loggingEnabled && data.RawResponse != nil { + rawResponseBytes, err := sonic.Marshal(data.RawResponse) + if err != nil { + p.logger.Error("failed to marshal raw response: %v", err) + } else { + updates["raw_response"] = string(rawResponseBytes) + } + } + + if loggingEnabled && data.RawRequest != nil { + rawRequestBytes, err := sonic.Marshal(data.RawRequest) + if err != nil { + p.logger.Error("failed to marshal raw request: %v", err) + } else { + updates["raw_request"] = string(rawRequestBytes) + } + }The streaming
updateStreamingLogEntryblock already respectsdisableContentLoggingfor both raw_request and raw_response, so no change is needed there beyond what you’ve added. This brings non-streaming behavior in line with streaming and with the plugin’s content-logging contract.Also applies to: 304-312
framework/streaming/audio.go (1)
170-182: MakeProcessedStreamResponse.RawRequesttruly optional (avoid always returning a non-nil pointer)You’ve added a guard around
result.SpeechStreamResponse(good), but the pointer semantics are still off:var rawRequest interface{} if result != nil && result.SpeechStreamResponse != nil && result.SpeechStreamResponse.ExtraFields.RawRequest != nil { rawRequest = result.SpeechStreamResponse.ExtraFields.RawRequest } ... RawRequest: &rawRequest,Even when no raw request exists,
RawRequestis a non-nil pointer to a nilinterface{}, so downstream code cannot distinguish “no raw request” from “raw request present but nil”.Align with how other streaming paths were fixed by making the pointer itself optional:
- var rawRequest interface{} - if result != nil && result.SpeechStreamResponse != nil && result.SpeechStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.SpeechStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.SpeechStreamResponse != nil && result.SpeechStreamResponse.ExtraFields.RawRequest != nil { + v := result.SpeechStreamResponse.ExtraFields.RawRequest + rawRequest = &v + } @@ Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest,This keeps the nil-safety while allowing consumers to reliably check for
nilRawRequest to mean “no request captured”.framework/streaming/transcription.go (1)
184-187: MakeRawRequesttruly optional instead of always returning a non‑nil pointerYou now guard against
nilresult, which fixes the panic risk, butRawRequestis still always set to a non‑nil*interface{}(it just holds a nil value when there’s no raw request). That makes it hard for consumers to distinguish “no raw request captured” from “explicit null”.Consider the same pattern suggested earlier:
- var rawRequest interface{} - if result != nil && result.TranscriptionStreamResponse != nil && result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.TranscriptionStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.TranscriptionStreamResponse != nil && result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.TranscriptionStreamResponse.ExtraFields.RawRequest) + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeTranscription, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis keeps the panic protection and makes
RawRequestnil when no raw request exists.Also applies to: 188-196
framework/streaming/chat.go (1)
279-282: Align chat streamingRawRequesthandling with optional pointer semanticsAs with transcription,
RawRequestis always set to&rawRequesthere, so callers see a non‑nil pointer even when no raw request was captured.You can keep the nil‑safety and make
RawRequesttruly optional:- var rawRequest interface{} - if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ChatResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.ChatResponse.ExtraFields.RawRequest) + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: streamType, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis keeps behavior consistent with other streaming paths.
Also applies to: 283-291
framework/streaming/responses.go (2)
723-726: Make OpenAI responses streamingRawRequestpointer optional and remove redundant checkCurrent code both redundantly checks
result.ResponsesStreamResponse != niland always sets a non‑nilRawRequestpointer, even when no raw request exists.You can simplify and fix the semantics:
- var rawRequest interface{} - if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.ResponsesStreamResponse.ExtraFields.RawRequest) + rawRequest = &v + } … - RawRequest: &rawRequest, + RawRequest: rawRequest,This avoids duplicate checks and keeps
RawRequestnil when absent.Also applies to: 782-790
866-869: Apply the same optionalRawRequestpattern in the generic responses-stream finalizationThe generic (non‑OpenAI) final path has the same “always non‑nil
RawRequestpointer” pattern.Recommend mirroring the fix:
- var rawRequest interface{} - if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.ResponsesStreamResponse.ExtraFields.RawRequest) + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeResponses, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilKeeps finalization consistent with other streaming handlers and preserves “optional” semantics.
Also applies to: 871-879
core/providers/anthropic/anthropic.go (2)
204-221: Inconsistent raw request flag evaluation in non-streaming paths.The code passes
provider.sendBackRawRequestdirectly toHandleProviderResponse(line 204), but then usesShouldSendBackRawRequest(ctx, ...)to gate the assignment (line 214). This means context overrides won't affect whetherHandleProviderResponseactually captures the raw request.For GET requests like
listModelsByKey, this is benign sincerequestBodyisnilanyway. However, for consistency with streaming paths (which correctly compute the effective flag once and pass it through), consider computing the effective flag upfront:+ effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &anthropicResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &anthropicResponse, nil, false, effectiveSendBackRawResponse) // ... - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + // Note: GET request has no body, so RawRequest is intentionally omittedThis aligns with the pattern in streaming paths and makes the intent clearer for GET endpoints.
274-295: Context override inconsistency applies to POST endpoints as well.The same pattern issue exists here:
HandleProviderResponsereceives the provider default (provider.sendBackRawRequest), but the assignment is gated byShouldSendBackRawRequest(ctx, ...). If a caller setsBifrostContextKeySendBackRawRequest=truein context while the provider default isfalse, the raw request won't be captured becauseHandleProviderResponseseesfalse.The past review comment provides a comprehensive fix pattern for this.
🧹 Nitpick comments (9)
core/providers/mistral/mistral.go (1)
103-123: Inconsistent context override handling for raw request capture vs exposure.At line 105,
provider.sendBackRawRequestis passed directly toHandleProviderResponse, but at lines 116-118,ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)is used for exposure. This creates a mismatch:
- If context override is
truebut provider default isfalse, capture won't happen (flag isfalse), but exposure check passes →rawRequestwill benil.- For consistency with raw response handling (which uses context-aware helper at capture), use the helper at capture time too.
Additionally, for this GET endpoint,
requestBodyisnil. IfHandleProviderResponseis later updated to unmarshal the request body (not just response body) for raw request capture, this could cause issues.var elevenlabsResponse ElevenlabsListModelsResponse -rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) +effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) +effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) +// Note: rawRequest will be nil for GET endpoints (no request body) +rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, effectiveSendBackRawRequest, effectiveSendBackRawResponse) if bifrostErr != nil { return nil, bifrostErr } ... // Set raw request if enabled -if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { +if effectiveSendBackRawRequest && rawRequest != nil { response.ExtraFields.RawRequest = rawRequest } // Set raw response if enabled -if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { +if effectiveSendBackRawResponse && rawResponse != nil { response.ExtraFields.RawResponse = rawResponse }core/providers/elevenlabs/elevenlabs.go (1)
103-120: Same context override inconsistency in listModelsByKey as other providers.At line 103,
provider.sendBackRawRequestis passed directly toHandleProviderResponse, while lines 113-115 use the context-awareShouldSendBackRawRequesthelper. This inconsistency means context overrides won't be honored at capture time.Consider computing effective flags once:
var elevenlabsResponse ElevenlabsListModelsResponse -rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &elevenlabsResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) +effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) +effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) +rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &elevenlabsResponse, nil, effectiveSendBackRawRequest, effectiveSendBackRawResponse) ... // Set raw request if enabled -if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { +if effectiveSendBackRawRequest && rawRequest != nil { response.ExtraFields.RawRequest = rawRequest }core/providers/bedrock/bedrock.go (1)
450-456: Clarify raw-request handling for BedrockListModelsand align withHandleProviderResponsesemanticsIn
listModelsByKeyyou call:rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( responseBody, bedrockResponse, nil, // requestBody provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), )and later gate assignment with
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest).Two concerns:
Nil request body when raw requests are enabled
If
HandleProviderResponseattempts to unmarshal the raw request wheneversendBackRawRequestis true, passing anil/empty body here may cause anErrProviderRawRequestUnmarshaland turn on a feature that can’t actually be satisfied for this GET endpoint.Per-request overrides are ignored for parsing
The decision to parse the raw request is based on
provider.sendBackRawRequest, but exposure is gated byShouldSendBackRawRequest(ctx, provider.sendBackRawRequest). That means a per-request override (context flag) can ask for a raw request, butHandleProviderResponsewill never populate it if the provider default is false.Consider:
- Either disabling raw-request parsing for this GET endpoint (always pass
falseforsendBackRawRequestand rely only on raw response), or- Passing a real serialized representation of the request (e.g., marshalled from
request) together with asendBackRawRequestboolean derived fromShouldSendBackRawRequest.This will avoid unexpected unmarshalling errors and keep per-request overrides effective.
Also applies to: 463-473
core/providers/azure/azure.go (1)
429-437: Same context override pattern issue as other providers.Azure's non-streaming paths have the same inconsistency where
HandleProviderResponsereceives the provider default but assignment is gated by context-aware check. This should be addressed stack-wide for consistency.core/providers/openai/openai.go (5)
171-180: Redundant context check in listModelsByKey.The
sendBackRawRequestparameter already incorporates context overrides (computed by caller viaShouldSendBackRawRequest). The check on line 173 redundantly evaluatesShouldSendBackRawRequestagain. Compare with line 178 which correctly usessendBackRawResponsedirectly.// Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { + if sendBackRawRequest { response.ExtraFields.RawRequest = rawRequest }
302-310: Same redundant context check pattern in text completion.The handler receives an already-computed
sendBackRawRequestflag but re-evaluatesShouldSendBackRawRequest. Should use the parameter directly likesendBackRawResponseon line 308.
703-711: Inconsistent flag handling between raw request and raw response.
sendBackRawRequestis re-evaluated viaShouldSendBackRawRequest(line 704), butsendBackRawResponseis used directly (line 709). Both parameters should be treated consistently.
1189-1197: Responses handler has same inconsistency.Same pattern:
sendBackRawRequestis re-checked butsendBackRawResponseis used directly.
1550-1558: Embedding handler follows same pattern.Same redundant context check for raw request but direct use for raw response.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(9 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(7 hunks)core/providers/gemini/gemini.go(12 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(3 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
✅ Files skipped from review due to trivial changes (1)
- core/changelog.md
🚧 Files skipped from review as they are similar to previous changes (20)
- transports/bifrost-http/lib/account.go
- plugins/logging/main.go
- core/providers/parasail/parasail.go
- ui/app/workspace/providers/fragments/performanceFormFragment.tsx
- core/schemas/provider.go
- framework/configstore/tables/provider.go
- core/providers/groq/groq.go
- framework/logstore/tables.go
- framework/configstore/rdb.go
- core/schemas/bifrost.go
- framework/changelog.md
- core/providers/cohere/cohere.go
- ui/lib/types/logs.ts
- core/providers/ollama/ollama.go
- plugins/logging/changelog.md
- ui/app/workspace/providers/page.tsx
- framework/streaming/types.go
- ui/lib/types/schemas.ts
- transports/changelog.md
- framework/configstore/clientconfig.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
framework/configstore/migrations.gocore/providers/cerebras/cerebras.goplugins/logging/operations.goframework/streaming/audio.goframework/logstore/migrations.gocore/providers/gemini/gemini.gocore/providers/perplexity/perplexity.goui/lib/types/config.tscore/providers/vertex/vertex.gotransports/bifrost-http/lib/config.gocore/providers/openrouter/openrouter.gocore/providers/mistral/mistral.goframework/streaming/chat.gocore/providers/azure/azure.gocore/providers/anthropic/anthropic.gotransports/bifrost-http/handlers/providers.gocore/providers/utils/utils.gocore/providers/bedrock/bedrock.goframework/streaming/responses.gocore/providers/elevenlabs/elevenlabs.gocore/providers/sgl/sgl.goframework/streaming/transcription.gocore/providers/openai/openai.goui/app/workspace/logs/views/logDetailsSheet.tsx
🧬 Code graph analysis (18)
core/providers/cerebras/cerebras.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
framework/streaming/audio.go (1)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeAudio(15-15)
framework/logstore/migrations.go (2)
framework/migrator/migrator.go (3)
DefaultOptions(100-106)New(131-149)Migration(62-69)framework/logstore/tables.go (2)
Log(74-137)Log(140-142)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
core/providers/vertex/vertex.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ShouldSendBackRawResponse(518-523)
core/providers/openrouter/openrouter.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
core/providers/mistral/mistral.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
framework/streaming/chat.go (1)
framework/streaming/types.go (3)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)
core/providers/azure/azure.go (3)
core/providers/utils/utils.go (2)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)core/schemas/utils.go (1)
IsAnthropicModel(1043-1045)core/providers/anthropic/anthropic.go (2)
AcquireAnthropicMessageResponse(46-50)ReleaseAnthropicMessageResponse(53-57)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/utils/utils.go (3)
core/schemas/bifrost.go (4)
BifrostError(355-364)ErrorField(373-380)BifrostResponseExtraFields(285-295)BifrostContextKeySendBackRawRequest(118-118)ui/lib/types/logs.ts (2)
BifrostError(226-232)ErrorField(217-224)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
framework/streaming/responses.go (1)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeResponses(17-17)
core/providers/elevenlabs/elevenlabs.go (2)
core/providers/utils/utils.go (5)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)NewBifrostOperationError(478-489)core/schemas/provider.go (1)
ErrProviderRawResponseUnmarshal(32-32)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
framework/streaming/transcription.go (1)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeTranscription(16-16)
ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
ui/app/workspace/logs/views/codeEditor.tsx (1)
CodeEditor(82-245)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (28)
core/providers/cerebras/cerebras.go (1)
20-21: LGTM!The
sendBackRawRequestfield is correctly added, initialized from config, and consistently propagated usingproviderUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)across all handler calls. The pattern aligns with the broader PR's approach for OpenAI-compatible providers.Also applies to: 51-52, 71-72, 89-90, 111-112, 129-130, 153-154
core/providers/mistral/mistral.go (1)
161-162: LGTM for delegated handler calls.The
ChatCompletion,ChatCompletionStream, andEmbeddingmethods correctly useproviderUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)when delegating to OpenAI-compatible handlers.Also applies to: 185-186, 234-235
core/providers/perplexity/perplexity.go (1)
23-24: LGTM for field initialization and streaming handler.The
sendBackRawRequestfield is correctly added, initialized from config, and properly used with context-aware helper inChatCompletionStream.Also applies to: 54-55, 158-161, 193-194
core/providers/openrouter/openrouter.go (1)
146-147: LGTM for delegated handler calls.All delegated handler calls (
TextCompletion,TextCompletionStream,ChatCompletion,ChatCompletionStream,Responses,ResponsesStream) correctly useproviderUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest).Also applies to: 167-168, 185-186, 209-210, 229-230, 249-250
core/providers/elevenlabs/elevenlabs.go (1)
256-258: LGTM for Speech and SpeechStream raw request handling.Both methods correctly check
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)and useParseAndSetRawRequestto set the raw request inExtraFields.Also applies to: 436-439
core/providers/utils/utils.go (1)
409-416: Raw-request helpers look good and match the broader feature design
ParseAndSetRawRequestandShouldSendBackRawRequestare straightforward, side-effect-only helpers that:
- Safely attempt to parse the JSON request body into
ExtraFields.RawRequest, logging a warning on failure without breaking the flow.- Respect a per-request context override (
BifrostContextKeySendBackRawRequest) while defaulting to the provider’s configuration.These are consistent with existing
ShouldSendBackRawResponsesemantics and with how providers useParseAndSetRawRequestin speech/transcription paths.Also applies to: 510-515
core/providers/sgl/sgl.go (1)
18-24: Raw-request wiring for SGL is consistent and correct
sendBackRawRequestis initialized from config and every OpenAI-compatible path correctly passesShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)into the shared handlers, mirroring the existing raw-response pattern. No issues here.Also applies to: 55-61, 69-83, 85-99, 101-119, 121-135, 137-159, 187-201
framework/logstore/migrations.go (1)
43-45: Raw-request column migration is consistent with existing migrations
migrationAddRawRequestColumnfollows the same pattern as prior logstore migrations (transactional,HasColumnguard, symmetric rollback) and is correctly wired intotriggerMigrations. Looks good.Also applies to: 645-674
transports/bifrost-http/lib/config.go (1)
339-364: SendBackRawRequest wiring from store to in‑memory and redacted configs looks consistentThe new
SendBackRawRequestfield is correctly copied from the DB provider config and surfaced in the redacted view alongsideSendBackRawResponse; no additional redaction logic is needed for this non‑sensitive boolean.Also applies to: 1376-1383
ui/lib/types/config.ts (1)
160-169: TS config types correctly mirror new backendsend_back_raw_requestflagAdding
send_back_raw_request?: booleanalongside the existingsend_back_raw_responseon provider config and add/update requests keeps the UI types in sync with the Go structs and preserves optional/default semantics.Also applies to: 184-193, 196-204
framework/configstore/migrations.go (1)
85-95: Newsend_back_raw_requestmigration is correctly wired into the chain
migrationAddSendBackRawRequestColumnsfollows the existing migrator pattern (HasColumn guard, AddColumn/DropColumn) and is correctly invoked fromtriggerMigrations, so the new provider flag will be created once and is rollback‑safe.Also applies to: 1254-1311
core/providers/bedrock/bedrock.go (2)
28-36: Provider-levelsendBackRawRequestflag is plumbed correctly into BedrockProviderThe new
sendBackRawRequestfield is initialized fromProviderConfigand stored onBedrockProvideralongsidesendBackRawResponse, giving all Bedrock operations access to the configured default.Also applies to: 72-79
551-555: Raw-request capture viaParseAndSetRawRequestis consistent across Bedrock non-GET callsUsing
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)plusParseAndSetRawRequest(&ExtraFields, jsonData)in:
TextCompletion,ChatCompletion,ChatCompletionStreamfinalization, andResponses/ResponsesStreamfinalizationkeeps behavior consistent with other providers and avoids hard failures when raw-request unmarshalling fails (it just logs). This looks good.
Also applies to: 730-734, 971-975, 1059-1062
core/providers/anthropic/anthropic.go (3)
397-411: Streaming paths correctly handle context overrides.The streaming entry points (
ChatCompletionStream,ResponsesStream) correctly compute the effective flag usingShouldSendBackRawRequest(ctx, ...)and pass it through to the streaming handlers. This ensures context overrides are respected.
656-659: Raw request injection in streaming final chunk looks correct.The streaming handler correctly uses
ParseAndSetRawRequestto inject the raw request into the final chunk'sExtraFieldswhen the flag is enabled. ThejsonBodyparameter contains the original request payload.
991-999: ResponsesStream raw request handling is consistent.Same pattern as chat completion streaming - raw request is correctly injected into the final response chunk when enabled.
core/providers/azure/azure.go (2)
26-28: Provider field addition is consistent with other providers.The
sendBackRawRequestfield follows the established pattern.
495-501: Streaming paths correctly handle context overrides for both model types.Both Anthropic and OpenAI model paths in streaming correctly compute and pass the effective
sendBackRawRequestflag.transports/bifrost-http/handlers/providers.go (5)
61-62: Response struct field addition is consistent.The
SendBackRawRequestfield follows the same pattern asSendBackRawResponse.
179-181: Payload field uses pointer type for optionality.Using
*boolallows distinguishing between "not provided" (nil) and explicitly "false", correctly defaulting tofalsewhen not specified.
250-252: Config field assignment handles nil correctly.The expression
payload.SendBackRawRequest != nil && *payload.SendBackRawRequestsafely defaults tofalsewhen the field is not provided.
431-436: Update logic correctly preserves existing value when not specified.The nil check ensures that
SendBackRawRequestis only updated when explicitly provided in the payload, preserving the existing value otherwise. This is consistent withSendBackRawResponsehandling.
856-857: Response mapping includes the new field.The
getProviderResponseFromConfigfunction correctly includesSendBackRawRequestin the response.core/providers/openai/openai.go (5)
28-31: Provider field addition follows established pattern.
96-111: ListModels correctly computes effective flag at entry point.Unlike some other providers' non-streaming paths, OpenAI's
ListModelscorrectly usesShouldSendBackRawRequest(ctx, ...)before passing to handlers, ensuring context overrides are respected throughout.
596-603: Streaming path correctly uses flag directly.Unlike non-streaming paths, the streaming handler uses
sendBackRawRequestdirectly without redundant context evaluation.
977-984: Responses-to-chat fallback correctly handles raw request.The fallback path that converts chat completions to responses format correctly injects the raw request when the
completedevent is reached.
1471-1473: Overall raw request feature implementation is functionally correct.The OpenAI provider correctly implements the raw request feature:
- Entry points compute effective flags using
ShouldSendBackRawRequest- Handlers capture raw requests via
HandleProviderResponse- Streaming paths inject raw requests in final chunks via
ParseAndSetRawRequestThe minor inconsistency (redundant context checks in some handlers) doesn't affect correctness since the flag already incorporates context overrides from callers.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (5)
core/providers/gemini/gemini.go (2)
164-176: Context-based raw request override not honored during capture.
HandleProviderResponseat line 164 receivesprovider.sendBackRawRequestdirectly, but lines 174-176 useShouldSendBackRawRequest(ctx, ...)for the conditional assignment. If a caller enables raw request via context while the provider default is false,HandleProviderResponsewon't capturerawRequest, resulting innilbeing assigned toExtraFields.RawRequest.Apply this diff to ensure context overrides work correctly:
// Parse Gemini's response var geminiResponse GeminiListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))
270-297: Same context override issue in ChatCompletion.Same issue as
listModelsByKey:HandleProviderResponseat line 270 usesprovider.sendBackRawRequestdirectly, but lines 295-297 use the context-aware helper. Context-based overrides won't work for raw request capture.Apply this diff:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, response, jsonData, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, response, jsonData, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/utils/utils.go (1)
344-407: Critical:rawRequestunmarshalsresponseBodyinstead ofrequestBody.The
HandleProviderResponsefunction acceptsrequestBody []byteas a parameter (line 344), but at line 359, it unmarshalsresponseBodyintorawRequestinstead ofrequestBody. This meansrawRequestwill contain the response data, not the request data, completely defeating the purpose of the raw request feature.Apply this diff to fix:
go func() { defer wg.Done() if sendBackRawRequest { - rawRequestErr = sonic.Unmarshal(responseBody, &rawRequest) + rawRequestErr = sonic.Unmarshal(requestBody, &rawRequest) } }()Additionally, consider adding a nil check for
requestBodyto avoid unmarshalling errors when no request body is provided:go func() { defer wg.Done() - if sendBackRawRequest { + if sendBackRawRequest && len(requestBody) > 0 { rawRequestErr = sonic.Unmarshal(requestBody, &rawRequest) } }()core/providers/bedrock/bedrock.go (1)
452-468: Potential issue:listModelsByKeypasses nil requestBody withsendBackRawRequestenabled
HandleProviderResponsereceivesnilas therequestBodyparameter (3rd argument) butprovider.sendBackRawRequestmay betrue. For this GET endpoint, there's no request body to capture. If the helper attempts to unmarshal a nil byte slice when the flag is enabled, it could return an error.Consider explicitly disabling raw request capture for this GET endpoint or ensuring
HandleProviderResponsegracefully handles nil request bodies:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, bedrockResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, bedrockResponse, nil, false, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))Alternatively, compute effective flags once and pass
falsefor raw request on GET endpoints.core/providers/mistral/mistral.go (1)
105-118: Same issue:listModelsByKeypasses nil requestBody with raw request flagSimilar to Bedrock, this GET endpoint passes
nilas therequestBodytoHandleProviderResponsewhileprovider.sendBackRawRequestmay betrue. Consider explicitly passingfalsefor raw request capture on GET endpoints:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, false, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))
♻️ Duplicate comments (9)
core/providers/perplexity/perplexity.go (1)
145-166: Inconsistent use ofShouldSendBackRawRequestbetweenHandleProviderResponseandExtraFieldsassignment.At line 145,
provider.sendBackRawRequestis passed directly toHandleProviderResponse, but at line 159,ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)is used to gate the assignment. If context overrides the raw request flag totruebutprovider.sendBackRawRequestisfalse,HandleProviderResponsewon't unmarshal the raw request (returningnil), yet the conditional at line 159 would pass, assigningniltoExtraFields.RawRequest.For consistency with the streaming path (line 193) and to respect context overrides correctly:
var response PerplexityChatResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))Note: This same pattern should be verified across other providers with similar non-streaming paths.
core/providers/vertex/vertex.go (3)
443-471: Context-based overrides not honored in ChatCompletion for both Anthropic and OpenAI paths.Both
HandleProviderResponsecalls (lines 443 and 478) passprovider.sendBackRawRequestandprovider.sendBackRawResponsedirectly, bypassing context overrides. The conditional checks at lines 464 and 492 useShouldSendBackRawRequest/ShouldSendBackRawResponse, creating an inconsistency.Apply this diff for both paths:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), anthropicResponse, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), anthropicResponse, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), response, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), response, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))Also applies to: 478-499
799-824: Context-based overrides not honored in Responses (Anthropic path).Same issue:
HandleProviderResponseat line 799 passes direct values instead of using context-aware helpers.
203-212: Context-based raw request override still not honored in listModelsByKey.Despite past review comments addressing this issue (marked as resolved in commits 34ba1e5 to d6466cb),
HandleProviderResponseat line 203 still receivesprovider.sendBackRawRequestdirectly rather than usingShouldSendBackRawRequest(ctx, ...). This means context-based overrides won't capture the raw request data.Apply this diff to fix:
// Parse Vertex's response var vertexResponse VertexListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &vertexResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &vertexResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))Also applies to: 230-236
framework/streaming/transcription.go (1)
184-196: Nil deref risk addressed, but pointer nullability issue remains.The nil checks on line 185 now properly guard against nil dereference (addressing the first concern from the previous review). However, using
&rawRequeston line 195 still yields a non-nil pointer even whenrawRequestis nil, makingRawRequestalways non-nil regardless of whether raw request data exists.Apply this diff to make
RawRequesttruly optional:- var rawRequest interface{} - if result != nil && result.TranscriptionStreamResponse != nil && result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.TranscriptionStreamResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.TranscriptionStreamResponse != nil && result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.TranscriptionStreamResponse.ExtraFields.RawRequest) + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeTranscription, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilframework/streaming/chat.go (1)
279-291:RawRequestpointer is always non-nil even when no raw request existsThe nil-dereference guard at line 280 addresses the previous crash concern. However, line 290 still assigns
&rawRequest, which creates a non-nil pointer to a nilinterface{}value when no raw request is present. This differs from the more idiomatic pattern where a nil pointer means "field absent."Consider assigning a pointer only when raw request data actually exists:
- var rawRequest interface{} - if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ChatResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { + v := result.ChatResponse.ExtraFields.RawRequest + rawRequest = &v + } return &ProcessedStreamResponse{ // ... - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilThis keeps
RawRequesttruly optional (nil when absent vs non-nil pointing to nil).core/providers/openrouter/openrouter.go (1)
100-119: Context overrides not honored at capture time forlistModelsByKeyThis matches the previous review concern:
HandleProviderResponsereceivesprovider.sendBackRawRequestdirectly, but the exposure check usesShouldSendBackRawRequest(ctx, ...). If the provider default isfalsebut the context flag istrue, capture never happens yet the exposure check returnstrue.Additionally, passing
nilasrequestBodyfor this GET endpoint may cause issues if raw request capture is enabled.Consider computing effective flags once and using them consistently:
+ effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) var openrouterResponse schemas.BifrostListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &openrouterResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &openrouterResponse, nil, false, effectiveSendBackRawResponse) // ... - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if effectiveSendBackRawRequest && rawRequest != nil { openrouterResponse.ExtraFields.RawRequest = rawRequest }core/providers/cohere/cohere.go (1)
223-241: Context override inconsistency persists in non-streaming paths.The
HandleProviderResponsecall at line 223 receivesprovider.sendBackRawRequestdirectly, while theExtraFields.RawRequestassignment at lines 234-236 usesShouldSendBackRawRequest(ctx, provider.sendBackRawRequest). This means:
- If context override enables raw request but provider default is
false,HandleProviderResponsewon't capture the raw request, yet the condition at line 234 will be true, resulting innilbeing assigned.- For
listModelsByKey, passingnilas the request body toHandleProviderResponsewhile potentially enabling raw request capture could cause issues.Compute the effective flag once and use it consistently:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, &cohereResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, &cohereResponse, nil, false, effectiveSendBackRawResponse) // GET: no request body ... - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + // No raw request for GET endpointsApply the same pattern to
ChatCompletion,Responses, andEmbeddingmethods.core/providers/anthropic/anthropic.go (1)
204-221: Non-streaming paths still have context override inconsistency.Same issue as Cohere:
HandleProviderResponsereceivesprovider.sendBackRawRequestdirectly whileExtraFieldsassignment usesShouldSendBackRawRequest. ForlistModelsByKey(GET),nilis passed as request body.The past review comment was marked as addressed, but the pattern in the code still shows the inconsistency. Either the fix hasn't been applied yet or there's a disconnect.
#!/bin/bash # Check if there are commits that addressed this issue after the review git log --oneline -10 --all -- core/providers/anthropic/anthropic.go
🧹 Nitpick comments (6)
core/providers/azure/azure.go (1)
197-218: Same context override pattern issue in listModelsByKey.Follows the same pattern as Cohere and Anthropic. GET endpoint passes
nilrequest body withprovider.sendBackRawRequest, then checksShouldSendBackRawRequestfor assignment. Consider passingfalsefor the request flag since GET has no body.- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + _, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, false, effectiveSendBackRawResponse)core/providers/openai/openai.go (5)
171-180: Double evaluation of context override in listModelsByKey.The function receives
sendBackRawRequestas a parameter (already computed viaShouldSendBackRawRequestby the caller at lines 96, 108), but then callsShouldSendBackRawRequest(ctx, sendBackRawRequest)again at line 173. This is redundant and could cause confusion.Since callers already pass the effective value, use it directly:
- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + // Set raw request if enabled + if sendBackRawRequest { + response.ExtraFields.RawRequest = rawRequest + }Also note: this is a GET endpoint with
nilrequest body - raw request capture should likely be disabled.
302-310: Same double-evaluation pattern in HandleOpenAITextCompletionRequest.Line 303 calls
ShouldSendBackRawRequest(ctx, sendBackRawRequest)whensendBackRawRequestwas already computed by the caller. Use the parameter directly, or have all callsites pass the raw provider default and compute once inside.- if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { + if sendBackRawRequest {
703-711: HandleOpenAIChatCompletionRequest has double-evaluation pattern.Same issue: line 704 re-evaluates
ShouldSendBackRawRequest(ctx, sendBackRawRequest). Also, the raw response check at line 709 usessendBackRawResponsedirectly (correct), showing inconsistency within the same function.- if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { + if sendBackRawRequest { response.ExtraFields.RawRequest = rawRequest } - // Set raw response if enabled - if sendBackRawResponse { + // Set raw response if enabled (consistent with raw request pattern) + if sendBackRawResponse {
1189-1197: HandleOpenAIResponsesRequest has inconsistent pattern.Double-evaluation at line 1190 (
ShouldSendBackRawRequest(ctx, sendBackRawRequest)) but then direct use at line 1195 (sendBackRawResponse). Should be consistent.
1550-1558: Embedding handler has same double-evaluation inconsistency.Line 1551 re-evaluates while line 1556 uses the parameter directly.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(9 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(7 hunks)core/providers/gemini/gemini.go(12 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(3 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
✅ Files skipped from review due to trivial changes (1)
- core/changelog.md
🚧 Files skipped from review as they are similar to previous changes (15)
- framework/streaming/audio.go
- plugins/logging/main.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- core/providers/cerebras/cerebras.go
- framework/streaming/responses.go
- transports/bifrost-http/lib/config.go
- framework/streaming/types.go
- plugins/logging/operations.go
- framework/configstore/clientconfig.go
- plugins/logging/changelog.md
- core/providers/parasail/parasail.go
- transports/bifrost-http/lib/account.go
- ui/lib/types/schemas.ts
- ui/app/workspace/providers/fragments/performanceFormFragment.tsx
- core/schemas/bifrost.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
framework/configstore/tables/provider.gocore/providers/groq/groq.gocore/providers/ollama/ollama.gocore/providers/mistral/mistral.goframework/streaming/chat.gocore/providers/openrouter/openrouter.gotransports/bifrost-http/handlers/providers.gocore/providers/sgl/sgl.gocore/providers/elevenlabs/elevenlabs.gotransports/changelog.mdcore/providers/bedrock/bedrock.goui/lib/types/logs.tscore/providers/vertex/vertex.gocore/schemas/provider.goframework/logstore/migrations.goframework/streaming/transcription.gocore/providers/anthropic/anthropic.gocore/providers/perplexity/perplexity.goframework/configstore/migrations.gocore/providers/gemini/gemini.goframework/configstore/rdb.gocore/providers/openai/openai.goframework/logstore/tables.goui/lib/types/config.tscore/providers/utils/utils.gocore/providers/azure/azure.gocore/providers/cohere/cohere.goui/app/workspace/providers/page.tsxframework/changelog.md
🧬 Code graph analysis (14)
core/providers/groq/groq.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/ollama/ollama.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/mistral/mistral.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(510-515)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/vertex/vertex.go (2)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ShouldSendBackRawResponse(518-523)core/schemas/models.go (1)
Model(109-129)
framework/logstore/migrations.go (2)
framework/migrator/migrator.go (3)
DefaultOptions(100-106)New(131-149)Migration(62-69)framework/logstore/tables.go (2)
Log(74-137)Log(140-142)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)
framework/configstore/migrations.go (3)
framework/configstore/tables/virtualkey.go (4)
TableVirtualKeyProviderConfig(23-36)TableVirtualKeyProviderConfig(39-41)TableVirtualKeyProviderConfigKey(12-15)TableVirtualKeyProviderConfigKey(18-20)framework/migrator/migrator.go (3)
New(131-149)DefaultOptions(100-106)Migration(62-69)framework/configstore/tables/provider.go (2)
TableProvider(13-38)TableProvider(41-41)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
core/providers/utils/utils.go (3)
core/schemas/bifrost.go (4)
BifrostError(355-364)ErrorField(373-380)BifrostResponseExtraFields(285-295)BifrostContextKeySendBackRawRequest(118-118)ui/lib/types/logs.ts (2)
BifrostError(226-232)ErrorField(217-224)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/azure/azure.go (1)
core/providers/utils/utils.go (2)
HandleProviderResponse(344-407)ShouldSendBackRawRequest(510-515)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-407)ShouldSendBackRawResponse(518-523)ShouldSendBackRawRequest(510-515)ParseAndSetRawRequest(409-416)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (55)
framework/changelog.md (1)
1-2: LGTM!The changelog entries accurately reflect the feature additions in the PR stack: raw response accumulation from the downstack PR and raw request configuration from the current PR.
ui/lib/types/logs.ts (1)
282-283: LGTM!The
raw_requestfield is correctly added as an optional string, matching the backendLog.RawRequestfield type and positioned logically alongsideraw_response.transports/changelog.md (1)
1-2: LGTM!The changelog entries correctly document the transport layer features added in this PR stack.
framework/logstore/migrations.go (2)
43-45: LGTM!The migration is correctly wired into
triggerMigrationsaftermigrationUpdateTimestampFormat, maintaining proper migration ordering.
645-674: LGTM!The migration implementation follows established patterns in this file:
- Uses
UseTransaction = truefor atomicity- Properly propagates context via
tx.WithContext(ctx)- Idempotent migration with
HasColumncheck beforeAddColumn- Includes rollback functionality consistent with other column migrations
core/providers/perplexity/perplexity.go (3)
23-24: LGTM!The new
sendBackRawRequestfield is correctly added alongside the existingsendBackRawResponsefield, maintaining structural consistency.
54-55: LGTM!The field is properly initialized from the provider configuration, consistent with the existing
sendBackRawResponseinitialization pattern.
186-201: LGTM!The streaming path correctly uses
ShouldSendBackRawRequestandShouldSendBackRawResponseto respect context overrides when passing flags to the shared OpenAI-compatible streaming handler.core/providers/ollama/ollama.go (1)
22-23: LGTM! Clean implementation of sendBackRawRequest feature.The
sendBackRawRequestfield is correctly added to the struct, initialized from config, and propagated through all OpenAI-compatible handler calls using theShouldSendBackRawRequesthelper for context-based overrides.Also applies to: 59-60, 82-84
core/providers/elevenlabs/elevenlabs.go (1)
112-120: LGTM! Correct implementation of raw request/response handling in listModelsByKey.The
HandleProviderResponsereturn values are properly captured and conditionally assigned toExtraFieldsusing the helper functions.core/providers/gemini/gemini.go (1)
330-331: Streaming and embedding handlers correctly use context-aware helpers.The
ChatCompletionStreamandEmbeddingmethods properly useShouldSendBackRawRequestwhen passing the flag to the shared handlers.Also applies to: 384-385
core/providers/vertex/vertex.go (1)
605-606: Streaming methods correctly use context-aware helpers.
ChatCompletionStream(both Anthropic and OpenAI paths) andResponsesStreamproperly useShouldSendBackRawRequestwhen passing flags to shared streaming handlers.Also applies to: 681-682, 947-948
core/providers/utils/utils.go (2)
409-416: LGTM! ParseAndSetRawRequest helper is well implemented.The helper correctly parses JSON into an interface and sets it on
ExtraFields.RawRequest, with appropriate warning logging on failure without failing the flow.
510-515: LGTM! ShouldSendBackRawRequest helper follows established pattern.The implementation mirrors
ShouldSendBackRawResponsecorrectly, checking context for override before falling back to the default.core/providers/groq/groq.go (1)
20-21: LGTM! Consistent implementation of raw request feature.The
sendBackRawRequestfield is properly initialized from configuration and consistently threaded through all relevant API calls (ListModels, ChatCompletion, ChatCompletionStream). The implementation mirrors the existingsendBackRawResponsepattern.Also applies to: 56-56, 76-76, 166-166, 190-190
ui/app/workspace/providers/page.tsx (1)
73-73: LGTM! Proper initialization of new field.The
send_back_raw_requestfield is correctly initialized toundefinedin the default provider configuration, matching the pattern of adjacent optional fields.framework/configstore/tables/provider.go (1)
20-20: LGTM! Field addition is straightforward and well-positioned.The
SendBackRawRequestboolean field is correctly added to the database model with appropriate JSON tagging, positioned logically next to the existingSendBackRawResponsefield.framework/logstore/tables.go (1)
107-108: LGTM! Consistent storage field addition.The
RawRequestfield mirrors the existingRawResponsefield structure, using appropriate text storage and JSON tagging for potentially large request payloads.core/schemas/provider.go (1)
31-31: LGTM! Clean additions following established patterns.Both the error constant and configuration field follow existing conventions. The
SendBackRawRequestfield is properly positioned and tagged, enabling the raw request feature across all providers.Also applies to: 241-241
ui/lib/types/config.ts (1)
166-166: LGTM! TypeScript definitions align with backend.The
send_back_raw_requestfield is consistently added as an optional boolean across all provider configuration interfaces, properly aligned with the Go backend structures.Also applies to: 190-190, 201-201
core/providers/sgl/sgl.go (1)
22-22: LGTM! Comprehensive and consistent implementation.The
sendBackRawRequestfield is properly initialized and threaded through all relevant API methods (ListModels, TextCompletion, TextCompletionStream, ChatCompletion, ChatCompletionStream, Embedding). The implementation is consistent with other providers in the stack.Also applies to: 59-59, 79-79, 95-95, 112-112, 130-130, 150-150, 197-197
core/providers/bedrock/bedrock.go (5)
551-554: LGTM!Raw request capture for
TextCompletioncorrectly usesParseAndSetRawRequestwith the actual request body (jsonData).
730-733: LGTM!Raw request capture for
ChatCompletionfollows the same correct pattern.
902-905: LGTM!
ChatCompletionStreamcorrectly sets raw request on the final chunk usingParseAndSetRawRequest.
971-974: LGTM!
Responseshandler correctly captures raw request.
1059-1062: LGTM!
ResponsesStreamcorrectly sets raw request on the final response.core/providers/mistral/mistral.go (3)
161-165: LGTM!
ChatCompletioncorrectly passes the raw request flag to the shared OpenAI handler.
185-193: LGTM!
ChatCompletionStreamcorrectly passes the raw request flag to the shared OpenAI streaming handler.
234-237: LGTM!
Embeddingcorrectly passes the raw request flag.framework/configstore/rdb.go (4)
214-216: LGTM!
SendBackRawRequestis correctly wired into the provider upsert logic, consistent withSendBackRawResponse.
328-330: LGTM!
UpdateProvidercorrectly copies the new field.
439-441: LGTM!
AddProvidercorrectly initializes the new field.
628-630: LGTM!
GetProvidersConfigcorrectly retrieves and populatesSendBackRawRequest.core/providers/openrouter/openrouter.go (6)
146-149: LGTM!
TextCompletioncorrectly passes the raw request flag.
167-173: LGTM!
TextCompletionStreamcorrectly passes the flag.
185-189: LGTM!
ChatCompletioncorrectly passes the flag.
209-217: LGTM!
ChatCompletionStreamcorrectly passes the flag.
229-233: LGTM!
Responsescorrectly passes the flag.
249-256: LGTM!
ResponsesStreamcorrectly passes the flag.framework/configstore/migrations.go (3)
91-93: LGTM!The new migration is correctly added to the migration sequence after
migrationAddPluginVersionColumn.
1143-1154: LGTM!The refactored join table setup logic is cleaner and correctly sets up the join table before checking/creating it.
1284-1311: LGTM!The migration follows established patterns:
- Unique and descriptive ID (
add_send_back_raw_request_columns)- Idempotent forward migration with
HasColumncheck- Proper rollback that drops the column
- Context propagation via
tx.WithContext(ctx)core/providers/cohere/cohere.go (1)
482-485: Streaming raw request handling looks correct.The streaming paths correctly use
ShouldSendBackRawRequestto gate theParseAndSetRawRequestcall at end-of-stream, andjsonBodyis available in scope. This approach is consistent with the streaming pattern.core/providers/anthropic/anthropic.go (2)
405-411: Streaming entry points correctly compute effective flags.
ChatCompletionStreamproperly computesShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)once and passes it to the shared handler. This ensures context overrides are honored throughout the streaming flow.
656-659: Final chunk raw request handling is correct.The streaming handler correctly sets raw request at end-of-stream using
ParseAndSetRawRequestwhensendBackRawRequestis true. SincesendBackRawRequestwas already computed from context at the entry point, this works correctly.core/providers/azure/azure.go (2)
495-501: Streaming delegation correctly passes effective flags.Both Anthropic and OpenAI streaming paths receive
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)ensuring context overrides are honored by the downstream handlers.Also applies to: 523-531
405-421: ChatCompletion branches both pass jsonData correctly.Both the Anthropic model branch (line 411) and the OpenAI branch (line 417) pass
jsonDataas the request body toHandleProviderResponse, enabling proper raw request capture for POST endpoints.transports/bifrost-http/handlers/providers.go (5)
61-65: SendBackRawRequest field properly added to ProviderResponse.The new field follows the same pattern as the existing
SendBackRawResponsefield, with appropriate JSON tag for API serialization.
179-182: Payload field uses pointer for optional semantics.Using
*boolwithomitemptyallows clients to omit the field entirely, distinguishing "not provided" from "explicitly false".
250-252: Safe nil-to-false conversion for config construction.The expression
payload.SendBackRawRequest != nil && *payload.SendBackRawRequestcorrectly handles the nil case, defaulting tofalsewhen not provided. This matches the existingSendBackRawResponsepattern.
431-433: Update path correctly applies payload value.The conditional assignment only updates if payload value is provided, preserving existing behavior when field is omitted.
856-860: Response construction includes new field.
getProviderResponseFromConfigproperly propagatesconfig.SendBackRawRequestto the response, ensuring API consumers can see the configured value.core/providers/openai/openai.go (3)
596-603: Streaming text completion correctly uses effective flag directly.
HandleOpenAITextCompletionStreamingreceivessendBackRawRequest(already effective) and uses it directly without re-evaluating context. This is the correct pattern.
1083-1090: Chat completion streaming final chunk handles raw request correctly.Uses
sendBackRawRequestdirectly (the effective value) to gateParseAndSetRawRequest. This is the correct pattern for streaming paths.
1428-1435: Responses streaming correctly handles raw request at completion.Uses the effective
sendBackRawRequestflag directly without re-evaluation. Pattern is correct for streaming.
Pratham-Mishra04
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
feb7ebe to
899722a
Compare
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
2 times, most recently
from
f40e99c to
927142b
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (3)
core/providers/perplexity/perplexity.go (1)
145-145: Inconsistent usage ofShouldSendBackRawRequestinHandleProviderResponsecall.The raw response parameter uses
providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)but raw request passesprovider.sendBackRawRequestdirectly. This causes a bug when context overrides the config: ifprovider.sendBackRawRequest=falsebut context hasBifrostContextKeySendBackRawRequest=true, the raw request won't be captured (returns nil), yet line 159 will attempt to set it.- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/anthropic/anthropic.go (1)
204-221: Compute effective raw-request/response flags once to honor context overrides.The current implementation passes
provider.sendBackRawRequestdirectly toHandleProviderResponse(line 204) but later evaluatesShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)to gate assignment (line 214). This creates a context-override bug:
- If
BifrostContextKeySendBackRawRequestistruebutprovider.sendBackRawRequestisfalse,HandleProviderResponsereceivesfalseand never capturesrawRequest, yet line 214 evaluates totrueand assignsniltoExtraFields.RawRequest.Compute effective flags once and reuse them consistently:
+ effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &anthropicResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + &anthropicResponse, + nil, // GET: no request body + false, // Skip raw-request capture for GET endpoints + effectiveSendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + // (Optional) omit RawRequest assignment for GET, or guard with: if effectiveSendBackRawRequest && rawRequest != nil @@ - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if effectiveSendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }Apply the same pattern to
TextCompletion(lines 274, 288–290),ChatCompletion(lines 335, 349–351), andResponses(lines 697, 712–714). Streaming paths already follow this pattern correctly (lines 405, 657–659, 765, 992–994).core/providers/cohere/cohere.go (1)
223-243: Consider computing effective flags once for efficiency.The current pattern computes effective flags twice per request:
- Line 223:
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)in the call toHandleProviderResponse- Line 234:
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)again to guard assignmentWhile functionally correct (both see the same context), this is inefficient. Compute once and reuse:
+ effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, &cohereResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + body, + &cohereResponse, + nil, // GET: no request body + false, // Skip raw-request capture for GET endpoints + effectiveSendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + // (Optional) omit RawRequest for GET or guard: if effectiveSendBackRawRequest && rawRequest != nil @@ - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if effectiveSendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }Apply the same pattern to
ChatCompletion(lines 305, 319–321),Responses(lines 531, 547–549), andEmbedding(lines 774, 788–790).
🧹 Nitpick comments (4)
framework/logstore/tables.go (1)
106-108: Consider addingomitemptyto avoid serializing empty strings.Both
RawRequestandRawResponsewill serialize as empty strings when not populated. Addingomitemptywould reduce response payload size when these features are disabled.- ContentSummary string `gorm:"type:text" json:"-"` - RawRequest string `gorm:"type:text" json:"raw_request"` // Populated when `send-back-raw-request` is on - RawResponse string `gorm:"type:text" json:"raw_response"` // Populated when `send-back-raw-response` is on + ContentSummary string `gorm:"type:text" json:"-"` + RawRequest string `gorm:"type:text" json:"raw_request,omitempty"` // Populated when `send-back-raw-request` is on + RawResponse string `gorm:"type:text" json:"raw_response,omitempty"` // Populated when `send-back-raw-response` is oncore/providers/gemini/gemini.go (2)
164-164: Inconsistent parameter usage inHandleProviderResponse.Same issue as other providers:
provider.sendBackRawRequestis passed directly while raw response uses the context-aware helper. If context enables raw request when config disables it, the capturedrawRequestwill be nil.- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))
270-270: Same inconsistency inChatCompletion.Apply the same fix to use the context-aware helper for the raw request parameter.
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, response, jsonData, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, response, jsonData, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/azure/azure.go (1)
197-197: InconsistentHandleProviderResponsecalls across multiple methods.Multiple methods pass
provider.sendBackRawRequestandprovider.sendBackRawResponsedirectly toHandleProviderResponseinstead of using the context-aware helpers. This affectslistModelsByKey,TextCompletion,ChatCompletion,Responses, andEmbedding.While the ExtraFields assignment correctly uses helpers, the underlying data capture won't respect context overrides. Consider applying the context-aware helpers at the
HandleProviderResponsecall sites for consistency with the streaming handlers.Example fix for line 197:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))Apply the same pattern to lines 274, 411, 417, 599, 605, and 757.
Also applies to: 274-274, 411-411, 417-417, 599-599, 605-605, 757-757
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(9 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(7 hunks)core/providers/gemini/gemini.go(12 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(3 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (22)
- plugins/logging/main.go
- framework/streaming/audio.go
- core/schemas/provider.go
- transports/bifrost-http/lib/config.go
- framework/configstore/rdb.go
- core/providers/openrouter/openrouter.go
- ui/app/workspace/providers/page.tsx
- framework/streaming/chat.go
- ui/app/workspace/providers/fragments/performanceFormFragment.tsx
- framework/configstore/clientconfig.go
- ui/lib/types/config.ts
- framework/configstore/tables/provider.go
- framework/streaming/transcription.go
- core/providers/vertex/vertex.go
- framework/streaming/types.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- core/providers/sgl/sgl.go
- transports/bifrost-http/lib/account.go
- core/providers/ollama/ollama.go
- core/schemas/bifrost.go
- core/providers/mistral/mistral.go
- ui/lib/types/logs.ts
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
core/providers/parasail/parasail.gotransports/changelog.mdplugins/logging/operations.gocore/providers/elevenlabs/elevenlabs.gocore/providers/groq/groq.gocore/providers/gemini/gemini.goplugins/logging/changelog.mdcore/providers/perplexity/perplexity.gocore/providers/utils/utils.gocore/providers/bedrock/bedrock.goframework/streaming/responses.gocore/providers/azure/azure.gocore/providers/anthropic/anthropic.gocore/providers/cohere/cohere.gocore/changelog.mdframework/logstore/migrations.gotransports/bifrost-http/handlers/providers.gocore/providers/openai/openai.goframework/configstore/migrations.gocore/providers/cerebras/cerebras.goui/lib/types/schemas.tsframework/logstore/tables.goframework/changelog.md
🧬 Code graph analysis (14)
core/providers/parasail/parasail.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(524-529)
core/providers/elevenlabs/elevenlabs.go (2)
core/providers/utils/utils.go (5)
HandleProviderResponse(344-421)ShouldSendBackRawResponse(532-537)ShouldSendBackRawRequest(524-529)ParseAndSetRawRequest(423-430)NewBifrostOperationError(492-503)core/schemas/provider.go (1)
ErrProviderRawResponseUnmarshal(32-32)
core/providers/groq/groq.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(524-529)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-421)ShouldSendBackRawResponse(532-537)ShouldSendBackRawRequest(524-529)ParseAndSetRawRequest(423-430)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-421)ShouldSendBackRawResponse(532-537)ShouldSendBackRawRequest(524-529)
core/providers/utils/utils.go (3)
core/schemas/bifrost.go (4)
BifrostError(355-364)ErrorField(373-380)BifrostResponseExtraFields(285-295)BifrostContextKeySendBackRawRequest(118-118)ui/lib/types/logs.ts (2)
BifrostError(226-232)ErrorField(217-224)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-421)ShouldSendBackRawResponse(532-537)ShouldSendBackRawRequest(524-529)ParseAndSetRawRequest(423-430)
framework/streaming/responses.go (1)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeResponses(17-17)
core/providers/azure/azure.go (2)
core/providers/utils/utils.go (2)
HandleProviderResponse(344-421)ShouldSendBackRawRequest(524-529)core/schemas/utils.go (1)
IsAnthropicModel(1043-1045)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-421)ShouldSendBackRawRequest(524-529)ParseAndSetRawRequest(423-430)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-421)ShouldSendBackRawRequest(524-529)ShouldSendBackRawResponse(532-537)ParseAndSetRawRequest(423-430)
framework/logstore/migrations.go (2)
framework/migrator/migrator.go (3)
DefaultOptions(100-106)New(131-149)Migration(62-69)framework/logstore/tables.go (2)
Log(74-137)Log(140-142)
framework/configstore/migrations.go (2)
framework/migrator/migrator.go (3)
New(131-149)DefaultOptions(100-106)Migration(62-69)framework/configstore/tables/provider.go (2)
TableProvider(13-38)TableProvider(41-41)
core/providers/cerebras/cerebras.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(524-529)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (23)
transports/changelog.md (1)
1-5: Clarify changelog entries and their scope.The changelog includes entries for "reasoning in chat completions" (lines 3–4) and "provider conversions for integrations" (line 5) that are not mentioned in the PR objectives, which focus specifically on raw request logging and exposure. Additionally, line 1 appears to relate to the target branch rather than this PR.
Please clarify:
- Are lines 3–5 actually part of this PR, or do they belong to other PRs in the stack?
- Should this changelog aggregate all stack changes, or only changes from this specific PR?
- If this is for the entire stack, consider adding structure (e.g., per-PR or per-feature sections) to improve clarity.
framework/logstore/migrations.go (1)
43-45: LGTM! Migration follows established patterns.The new
migrationAddRawRequestColumnfunction correctly mirrors existing migration patterns in the codebase (e.g.,migrationAddParentRequestIDColumn). It properly usesHasColumncheck before adding, wraps in a transaction, and includes a rollback function.Also applies to: 644-674
core/changelog.md (1)
1-4: LGTM! Changelog entries are clear and concise.The entries appropriately document the new raw request feature and related enhancements added in this PR stack.
framework/streaming/responses.go (1)
866-879: LGTM! RawRequest propagation for non-OpenAI providers is correct.The nil checks are properly applied and the
rawRequestis correctly propagated to the finalProcessedStreamResponse.ui/lib/types/schemas.ts (2)
372-375: LGTM! Schema additions are consistent and well-structured.The
send_back_raw_requestfield is correctly added across all relevant schemas with appropriate optionality, mirroring the existingsend_back_raw_responsepattern.
486-488: LGTM! Form schema correctly requires explicit boolean value.Using non-optional
z.boolean()inperformanceFormSchemaensures the form always captures an explicit user choice for this setting, which is appropriate for a configuration toggle.core/providers/elevenlabs/elevenlabs.go (2)
527-533: LGTM: Raw response bug fixed!The past review comments correctly identified that
rawResponsewas unmarshaled but never assigned toresponse.ExtraFields.RawResponse. This has been fixed—line 532 now correctly assigns the value.
27-27: LGTM: Consistent raw request propagation!The raw request feature is implemented consistently across all operation types:
- Field added to provider struct (line 27)
- Initialized from config (line 60)
- Conditionally exposed via
ShouldSendBackRawRequesthelper- Populated in both streaming and non-streaming paths
Also applies to: 60-60, 112-116, 256-258, 437-439
framework/configstore/migrations.go (1)
1284-1311: LGTM: Standard migration pattern!The migration follows the established pattern:
- Checks for column existence before adding
- Provides a rollback function
- Uses descriptive migration ID
- Aligns with the
TableProviderschema changescore/providers/bedrock/bedrock.go (1)
34-34: LGTM: Comprehensive raw request support across all Bedrock operations!The implementation consistently propagates raw request data through all request types (ListModels, TextCompletion, ChatCompletion, Responses) and both streaming and non-streaming paths. The streaming finalization correctly includes raw request data in the final chunk for accumulated metrics.
Also applies to: 77-77, 465-469, 551-554, 730-733, 902-905, 971-974, 1059-1062
transports/bifrost-http/handlers/providers.go (1)
61-61: LGTM: Complete HTTP API integration!The
SendBackRawRequestfield is properly wired through all provider management endpoints:
- Added to response struct (line 61)
- Captured from create payload (line 179)
- Captured from update payload (line 320)
- Included in responses via
getProviderResponseFromConfig(line 856)The nil checks (lines 250, 431-433) correctly preserve existing values when the field is not provided in partial updates.
Also applies to: 179-180, 250-250, 320-321, 431-433, 856-856
plugins/logging/changelog.md (1)
1-1: LGTM: Clear changelog entry!The changelog accurately describes the new raw request logging capability.
framework/changelog.md (1)
1-3: LGTM: Changelog reflects framework enhancements!The changelog entries document the related features being added: raw response accumulation, raw request logging, and reasoning accumulation.
plugins/logging/operations.go (1)
126-134: LGTM: Consistent content logging guards!The past review comment correctly identified inconsistent handling of
disableContentLoggingbetween streaming and non-streaming paths. This has been fixed—both raw request handling blocks are now properly guarded inside theif p.disableContentLogging == nil || !*p.disableContentLoggingconditional, ensuring raw request data is only logged when content logging is enabled.Error handling appropriately logs marshal failures without breaking the update operation.
Also applies to: 305-313
core/providers/parasail/parasail.go (1)
21-21: LGTM: Clean integration with OpenAI-compatible handlers!The raw request flag is properly propagated through Parasail's OpenAI-compatible implementation:
- Field added to provider (line 21)
- Initialized from config (line 52)
- Passed to all shared OpenAI handlers (lines 72, 99, 123)
This aligns with the pattern used by other OpenAI-compatible providers in the codebase.
Also applies to: 52-52, 72-72, 99-99, 123-123
core/providers/groq/groq.go (1)
20-21: LGTM! Consistent use of context-aware helpers.The
sendBackRawRequestfield is correctly initialized and consistently used withproviderUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)across all API methods that delegate to OpenAI handlers. This ensures context overrides are properly respected.Also applies to: 56-57, 76-77, 166-167, 190-191
core/providers/perplexity/perplexity.go (1)
23-24: LGTM for the remaining raw request integration.The field declaration, initialization, conditional setting of
ExtraFields.RawRequest, and streaming handler call all follow the correct pattern.Also applies to: 54-55, 158-166, 193-194
core/providers/cerebras/cerebras.go (1)
20-21: LGTM! Consistent implementation across all endpoints.The
sendBackRawRequestfeature is correctly integrated with context-aware helper usage in all API methods (ListModels,TextCompletion,TextCompletionStream,ChatCompletion,ChatCompletionStream).Also applies to: 51-52, 71-72, 89-90, 111-112, 129-130, 153-154
core/providers/gemini/gemini.go (1)
27-28: LGTM for the remaining raw request integration.The field declaration, initialization, and all usages in streaming handlers, embedding, speech, and transcription endpoints correctly apply the
ShouldSendBackRawRequesthelper. TheParseAndSetRawRequestutility is properly guarded.Also applies to: 60-61, 173-181, 294-302, 330-331, 384-385, 423-426, 638-641, 681-683, 909-912
core/providers/azure/azure.go (1)
26-27: LGTM for the remaining raw request integration.The field declaration, initialization, all streaming handler calls, and ExtraFields conditional assignments are correctly implemented with context-aware helpers.
Also applies to: 51-52, 210-218, 285-293, 340-341, 429-437, 495-496, 523-524, 617-625, 678-679, 707-708, 768-776
core/providers/anthropic/anthropic.go (1)
405-411: Streaming paths correctly honor context overrides.The streaming implementations properly compute effective raw-request/response flags once (line 405:
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)) and pass them through the call chain, avoiding the context-override issue present in non-streaming paths. This is the recommended pattern.Also applies to: 656-659
core/providers/utils/utils.go (1)
344-421: Core raw-request handling implementation is correct.The updated
HandleProviderResponseproperly:
- Guards raw-request capture when
requestBodyisnil(line 352), preventing unmarshal errors for GET requests- Unmarshals raw requests concurrently (lines 369–374) for efficiency
- Returns dedicated error messages (
ErrProviderRawRequestUnmarshal) for raw-request failures (lines 399–402)- Provides
ParseAndSetRawRequesthelper (lines 423–430) with graceful error logging for streaming paths- Implements
ShouldSendBackRawRequest(lines 524–529) to honor context overrides with proper fallbackThis implementation aligns with the PR's goal of exposing raw requests for debugging and advanced use cases.
Also applies to: 423-430, 524-529
core/providers/openai/openai.go (1)
597-599: Streaming paths use parameters directly without re-checking context.The streaming implementations (lines 597–599, 1084–1086, 1428–1431) directly use
sendBackRawRequestto guardParseAndSetRawRequestwithout callingShouldSendBackRawRequestagain. This suggests the parameter is already the effective value. Non-streaming paths should follow the same pattern for consistency (see previous comment).Also applies to: 1084-1086, 1428-1431
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
core/providers/perplexity/perplexity.go (1)
145-166: Inconsistent usage of context-based raw request flag.At line 145,
provider.sendBackRawRequestis passed directly toHandleProviderResponse, but at line 159,ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)is used to check whether to set the raw request.This creates a potential bug: if the context override enables raw request (
BifrostContextKeySendBackRawRequest = true) butprovider.sendBackRawRequestisfalse, then:
HandleProviderResponsewon't capturerawRequest(it will benil)- Line 159's condition will be
truenilwill be assigned toExtraFields.RawRequestFor consistency and correctness:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))core/providers/bedrock/bedrock.go (1)
450-473: Honor context raw‑request overrides when capturing inlistModelsByKey.Here
HandleProviderResponseusesprovider.sendBackRawRequest, but exposure is gated viaShouldSendBackRawRequest(ctx, ...). If the provider default is false and a caller enables raw requests via context,rawRequestis never captured andExtraFields.RawRequestis set to nil.Compute effective flags once and reuse them for both capture and exposure, e.g.:
- bedrockResponse := &BedrockListModelsResponse{} - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, bedrockResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + bedrockResponse := &BedrockListModelsResponse{} + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, + bedrockResponse, + nil, // GET: no request body + sendBackRawRequest, + sendBackRawResponse, + ) @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + // Set raw request/response if effectively enabled and present + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }core/providers/mistral/mistral.go (1)
104-123: Use effective raw‑payload flags for both capture and exposure inlistModelsByKey.
HandleProviderResponseseesprovider.sendBackRawRequest, while ExtraFields usesShouldSendBackRawRequest(ctx, ...). A context override enabling raw requests won’t actually capturerawRequest.Recommend computing effective flags once:
- var mistralResponse MistralListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + var mistralResponse MistralListModelsResponse + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, + &mistralResponse, + nil, // GET: no request body + sendBackRawRequest, + sendBackRawResponse, + ) @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }core/providers/gemini/gemini.go (1)
23-30: Fix raw‑request capture in GeminilistModelsByKeyandChatCompletionto respect context overrides.Here
HandleProviderResponseusesprovider.sendBackRawRequest, butExtraFields.RawRequestis gated byShouldSendBackRawRequest(ctx, ...). If the default is false and a caller enables raw requests via context,rawRequestremains nil and the response incorrectly looks as if a raw request was returned.Suggested adjustments:
func (provider *GeminiProvider) listModelsByKey(...) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { @@ - var geminiResponse GeminiListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &geminiResponse, nil, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + var geminiResponse GeminiListModelsResponse + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + &geminiResponse, + nil, // GET: no request body + sendBackRawRequest, + sendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }and in
ChatCompletion:func (provider *GeminiProvider) ChatCompletion(...) (*schemas.BifrostChatResponse, *schemas.BifrostError) { @@ - response := &schemas.BifrostChatResponse{} - - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, response, jsonData, provider.sendBackRawRequest, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + response := &schemas.BifrostChatResponse{} + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + body, + response, + jsonData, + sendBackRawRequest, + sendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse { + response.ExtraFields.RawResponse = rawResponse + }Speech/Transcription (including streaming) already use
ParseAndSetRawRequestwithShouldSendBackRawRequest, which is fine.Also applies to: 55-62, 162-181, 173-181, 216-304, 270-302
♻️ Duplicate comments (5)
core/providers/openrouter/openrouter.go (1)
18-24: OpenRouter raw‑payload handling is now aligned with helpers and GET semantics.You pass
ShouldSendBackRawRequest/ResponseintoHandleProviderResponsefor list‑models and gate ExtraFields with the same helpers; streaming/non‑streaming OpenAI‑compatible paths consistently forward these effective flags. This addresses the earlier capture vs exposure mismatch.Also applies to: 49-55, 99-120, 112-120, 147-151, 161-175, 179-191, 203-219, 223-235, 243-257
core/providers/cohere/cohere.go (1)
71-73: Cohere raw‑request handling is consistent; you might optionally skip setting nil for GET list‑models.
- Non‑stream paths now pass
ShouldSendBackRawRequest/ResponseintoHandleProviderResponse, so context overrides correctly control capture and ExtraFields.- Streaming chat/responses correctly attach
RawRequestonly in the final chunk viaParseAndSetRawRequest.If you want slightly cleaner
ListModelsbehavior, you could avoid settingExtraFields.RawRequestwhen the GET has no body:- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }Also applies to: 105-112, 221-241, 233-241, 305-326, 318-326, 531-553, 546-553, 774-795, 787-795, 481-485, 721-724
core/providers/azure/azure.go (1)
22-28: Ensure Azure non‑stream handlers use context‑effective raw flags for both capture and ExtraFields.In
listModelsByKey,TextCompletion,ChatCompletion(Anthropic and non‑Anthropic),Responses, andEmbeddingyou:
- Call
HandleProviderResponsewithprovider.sendBackRawRequest/provider.sendBackRawResponse.- Later gate
ExtraFields.RawRequest/RawResponsewithShouldSendBackRawRequest/Response(ctx, ...).If a caller enables raw payloads via context while the provider defaults are false,
HandleProviderResponsenever capturesrawRequest/rawResponse, but ExtraFields is still populated (with nil). Conversely, context cannot disable capture when defaults are true.Pattern to fix (example for
listModelsByKey):- azureResponse := &AzureListModelsResponse{} - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + azureResponse := &AzureListModelsResponse{} + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, + azureResponse, + nil, // GET: no request body + sendBackRawRequest, + sendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }Apply the same idea in:
TextCompletion:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, response, jsonData, provider.sendBackRawRequest, provider.sendBackRawResponse) + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, + response, + jsonData, + sendBackRawRequest, + sendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse { + response.ExtraFields.RawResponse = rawResponse + }
ChatCompletionandResponsesAnthropic/non‑Anthropic branches (use the samesendBackRawRequest/sendBackRawResponsevariables for both branches).Embedding, where you currently callHandleProviderResponse(responseBody, response, jsonData, provider.sendBackRawRequest, provider.sendBackRawResponse)and then gate ExtraFields.Streaming methods already pass
ShouldSendBackRawRequest/Responseinto the Anthropic/OpenAI helpers and don’t need changes.Also applies to: 47-53, 195-218, 210-218, 272-293, 285-293, 404-421, 408-421, 592-609, 617-625, 754-776, 768-776
core/providers/vertex/vertex.go (1)
147-152: Align Vertex raw‑payload capture with context overrides and avoid aggregating nil entries.Across
listModelsByKey,ChatCompletion, and AnthropicResponses,HandleProviderResponseis called withprovider.sendBackRawRequest/sendBackRawResponse, while ExtraFields and list‑models aggregation useShouldSendBackRawRequest/Response. This means:
- Per‑request context overrides enabling raw payloads don’t actually trigger capture.
listModelsByKeycan append nilrawRequest/rawResponseentries and later expose slices of nils.Compute effective flags once per call and reuse them for both capture and exposure, and guard aggregation on non‑nil values, e.g.:
func (provider *VertexProvider) listModelsByKey(...) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { @@ - var rawRequests []interface{} - var rawResponses []interface{} + var rawRequests []interface{} + var rawResponses []interface{} pageToken := "" + + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) @@ - var vertexResponse VertexListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &vertexResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + var vertexResponse VertexListModelsResponse + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + &vertexResponse, + nil, // GET: no request body + sendBackRawRequest, + sendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - rawRequests = append(rawRequests, rawRequest) - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - rawResponses = append(rawResponses, rawResponse) - } + if sendBackRawRequest && rawRequest != nil { + rawRequests = append(rawRequests, rawRequest) + } + if sendBackRawResponse && rawResponse != nil { + rawResponses = append(rawResponses, rawResponse) + } @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequests - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponses - } + if sendBackRawRequest && len(rawRequests) > 0 { + response.ExtraFields.RawRequest = rawRequests + } + if sendBackRawResponse && len(rawResponses) > 0 { + response.ExtraFields.RawResponse = rawResponses + }Apply the same pattern in
ChatCompletionand AnthropicResponses:func (provider *VertexProvider) ChatCompletion(...) { @@ - deployment := provider.getModelDeployment(key, request.Model) + deployment := provider.getModelDeployment(key, request.Model) + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) @@ anthropic path - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), anthropicResponse, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + anthropicResponse, + jsonBody, + sendBackRawRequest, + sendBackRawResponse, + ) @@ non‑Anthropic path - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), response, jsonBody, provider.sendBackRawRequest, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + response, + jsonBody, + sendBackRawRequest, + sendBackRawResponse, + ) @@ ExtraFields - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse { + response.ExtraFields.RawResponse = rawResponse + }and similarly in the Anthropic branch of
Responses. This makes context overrides authoritative for both capture and exposure and prevents confusing nil aggregates.Also applies to: 149-152, 201-213, 224-236, 271-283, 438-502, 693-703, 795-825
core/providers/anthropic/anthropic.go (1)
170-221: Non‑streaming Anthropic paths still ignore context overrides when capturing raw request/response
listModelsByKey,TextCompletion,ChatCompletion, andResponsespass the provider defaultsprovider.sendBackRawRequest/provider.sendBackRawResponseintoHandleProviderResponse, but later gateExtraFields.RawRequest/RawResponsewithShouldSendBackRawRequest/Response(ctx, provider.sendBackRaw*). When a caller enables raw capture via context (while provider defaults are false),HandleProviderResponsenever actually captures raw payloads, soExtraFields.RawRequest/RawResponseend up nil even thoughShould*returns true. For list‑models (GET), you can also end up withRawRequestexplicitly set to nil even though there is no request body to capture.Suggest computing effective flags once per call using the context helpers, passing those into
HandleProviderResponse, and only setting ExtraFields when the flag is true and the captured value is non‑nil. For example:@@ func (provider *AnthropicProvider) listModelsByKey( ... ) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { - var anthropicResponse AnthropicListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &anthropicResponse, nil, provider.sendBackRawRequest, provider.sendBackRawResponse) + var anthropicResponse AnthropicListModelsResponse + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + _, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + &anthropicResponse, + nil, // GET: no JSON body + false, // don't attempt raw-request capture for list-models + effectiveSendBackRawResponse, + ) @@ - response.ExtraFields.Latency = latency.Milliseconds() - - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + response.ExtraFields.Latency = latency.Milliseconds() + if effectiveSendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }And similarly for the non‑streaming completion endpoints:
@@ func (provider *AnthropicProvider) TextCompletion(...) (*schemas.BifrostTextCompletionResponse, *schemas.BifrostError) { - response := acquireAnthropicTextResponse() - defer releaseAnthropicTextResponse(response) - - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, response, jsonData, provider.sendBackRawRequest, provider.sendBackRawResponse) + response := acquireAnthropicTextResponse() + defer releaseAnthropicTextResponse(response) + + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, + response, + jsonData, + effectiveSendBackRawRequest, + effectiveSendBackRawResponse, + ) @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - bifrostResponse.ExtraFields.RawRequest = rawRequest - } - // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - bifrostResponse.ExtraFields.RawResponse = rawResponse - } + // Set raw fields if enabled and captured + if effectiveSendBackRawRequest && rawRequest != nil { + bifrostResponse.ExtraFields.RawRequest = rawRequest + } + if effectiveSendBackRawResponse && rawResponse != nil { + bifrostResponse.ExtraFields.RawResponse = rawResponse + }Apply the same pattern to
ChatCompletionandResponsesso that context overrides reliably control both capture and exposure of raw payloads.Also applies to: 246-298, 307-359, 669-721
🧹 Nitpick comments (10)
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
128-156: Consider calling out sensitive-data implications in the “Include Raw Request” help textThe new toggle and wiring look correct and mirror
send_back_raw_response. Given that raw requests can contain API keys and user/PII data and are logged/returned when this is enabled, consider slightly stronger copy (or a tooltip) to warn users that this is for debugging, may expose sensitive data in logs/responses, and should be enabled only in trusted environments.transports/bifrost-http/handlers/providers.go (1)
315-323: updateProvider preserves prior SendBackRawRequest unless explicitly changed; consider updating the doc commentThe pointer-based handling for
SendBackRawRequest(and the sanitized fallback response) matches the existingSendBackRawResponsepattern and safely preserves the previous value when the field is omitted, which is desirable. The function‑level comment still says “ALL fields” must be provided and “Partial updates are not supported,” which is now slightly misleading for these optional flags; you might want to tweak that wording to call out these pointer fields as exceptions.Also applies to: 431-436, 458-465
core/providers/elevenlabs/elevenlabs.go (1)
519-533: Consider adding raw request support for Transcription.The
Transcriptionmethod handlesRawResponsebut doesn't captureRawRequest. WhileSpeechandSpeechStreammethods include raw request handling, this method doesn't.This may be intentional since transcription uses multipart form data (built via
writeTranscriptionMultipart) rather than JSON, making raw request capture more complex. However, for feature parity, you might want to either:
- Capture a JSON representation of the request parameters, or
- Document that raw request is not supported for multipart transcription requests
core/providers/utils/utils.go (2)
340-421:HandleProviderResponseconcurrency and API look good; consider checkingrawResponseErrwhen both flags are enabledThe new helper correctly:
- Skips raw‑request capture when
requestBody == nil(fixing GET cases).- Unmarshals structured + raw payloads concurrently.
- Distinguishes structured vs raw unmarshal failures via different error messages.
One subtlety: when both
sendBackRawRequestandsendBackRawResponseare true, you only validaterawRequestErrand return immediately, so a non‑nilrawResponseErris silently ignored. It’s unlikely in practice (same JSON, more permissive target type), but if you want strict per‑path error reporting you can tighten this without changing call sites:if structuredErr != nil { return nil, nil, &schemas.BifrostError{ /* ... */ } } - if shouldCaptureRawRequest { - if rawRequestErr != nil { - return nil, nil, &schemas.BifrostError{ - IsBifrostError: true, - Error: &schemas.ErrorField{ - Message: schemas.ErrProviderRawRequestUnmarshal, - Error: rawRequestErr, - }, - } - } - return rawRequest, rawResponse, nil - } - - if sendBackRawResponse { - if rawResponseErr != nil { - return nil, nil, &schemas.BifrostError{ - IsBifrostError: true, - Error: &schemas.ErrorField{ - Message: schemas.ErrProviderRawResponseUnmarshal, - Error: rawResponseErr, - }, - } - } - return rawRequest, rawResponse, nil - } + if shouldCaptureRawRequest && sendBackRawResponse { + if rawRequestErr != nil { + return nil, nil, &schemas.BifrostError{Error: &schemas.ErrorField{Message: schemas.ErrProviderRawRequestUnmarshal, Error: rawRequestErr}, IsBifrostError: true} + } + if rawResponseErr != nil { + return nil, nil, &schemas.BifrostError{Error: &schemas.ErrorField{Message: schemas.ErrProviderRawResponseUnmarshal, Error: rawResponseErr}, IsBifrostError: true} + } + return rawRequest, rawResponse, nil + } + if shouldCaptureRawRequest { + if rawRequestErr != nil { + return nil, nil, &schemas.BifrostError{Error: &schemas.ErrorField{Message: schemas.ErrProviderRawRequestUnmarshal, Error: rawRequestErr}, IsBifrostError: true} + } + return rawRequest, rawResponse, nil + } + if sendBackRawResponse { + if rawResponseErr != nil { + return nil, nil, &schemas.BifrostError{Error: &schemas.ErrorField{Message: schemas.ErrProviderRawResponseUnmarshal, Error: rawResponseErr}, IsBifrostError: true} + } + return rawRequest, rawResponse, nil + } --- `423-430`: **Raw‑request helpers are straightforward and consistent with existing patterns** - `ParseAndSetRawRequest` safely attempts to unmarshal the JSON body into `interface{}` and only logs on failure, avoiding new error paths in streaming finalization. - `ShouldSendBackRawRequest` mirrors `ShouldSendBackRawResponse`: context can force the flag on, but not off (i.e., a `false` value in context is treated as “no override”). That’s consistent with the current design; if you ever need per‑request opt‑out when the provider default is true, this is the place to extend. No changes required. Also applies to: 524-529 </blockquote></details> <details> <summary>core/providers/openai/openai.go (5)</summary><blockquote> `81-112`: **ListModels raw‑response capture is solid; raw‑request exposure on GETs is effectively a no‑op** - `ListModels` computes effective `sendBackRawRequest/Response` via `ShouldSendBackRaw*` and threads them consistently through `HandleOpenAIListModelsRequest` and `listModelsByKey`. - `listModelsByKey` correctly uses `HandleProviderResponse(responseBody, openaiResponse, nil, sendBackRawRequest, sendBackRawResponse)`, so raw responses are captured while raw‑request capture is skipped (nil `requestBody`). Because this is a GET, `rawRequest` will always be nil; yet `ExtraFields.RawRequest` is conditionally set based on `ShouldSendBackRawRequest(ctx, sendBackRawRequest)`. If you want to avoid ever exposing a meaningless `null` raw_request for ListModels, you could tighten this to only set it when non‑nil or drop it entirely for this endpoint, but current behavior is functionally safe. Also applies to: 116-183, 186-211 --- `213-231`: **Text completion raw‑request plumbing (sync + streaming) is correct; only minor cleanup opportunities** - Non‑streaming: `TextCompletion` computes effective `sendBackRawRequest/Response` via `ShouldSendBackRaw*` and passes them into `HandleOpenAITextCompletionRequest`, which forwards them to `HandleProviderResponse`. ExtraFields are then populated based on those flags, so context overrides are honored end‑to‑end. - Streaming: `HandleOpenAITextCompletionStreaming` receives `sendBackRawRequest` and calls `ParseAndSetRawRequest` on the synthesized final chunk when enabled, using the same `jsonBody` that was sent to OpenAI. If you want to simplify slightly, you can avoid re‑calling `ShouldSendBackRawRequest` in the handler and also skip assigning nil raw requests: ```diff - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + // Set raw request if enabled and captured + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }Same pattern applies to the chat/responses/embedding handlers.
Also applies to: 234-245, 263-313, 319-340, 343-357, 368-385, 387-407, 453-603
609-630: Chat completion raw‑request wiring (including Responses fallback) is solid
- Non‑streaming chat completions correctly use effective
sendBackRawRequest/Responsefor both capture (HandleProviderResponse) and exposure (ExtraFields), so context overrides work as intended.- Streaming chat completions pass
sendBackRawRequestintoHandleOpenAIChatCompletionStreaming, which:
- Adds
RawResponseper chunk when enabled.- Calls
ParseAndSetRawRequeston the final synthetic chat chunk, and on the finalResponsesStreamResponseTypeCompletedchunk in the responses‑fallback branch, using the originaljsonBody.As with text completions, you can optionally simplify ExtraFields gating in
HandleOpenAIChatCompletionRequestto:- if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - if sendBackRawResponse { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }Functionally, current behavior is already correct.
Also applies to: 633-644, 662-714, 719-745, 748-763, 788-809, 977-980, 1079-1090
1096-1115: Responses API now exposes raw requests consistently across sync and streaming paths
- Non‑streaming
Responsescorrectly feeds effectivesendBackRawRequest/ResponseintoHandleOpenAIResponsesRequest, which usesHandleProviderResponseto capture both raw payloads and then sets ExtraFields according to those flags.- Streaming
HandleOpenAIResponsesStreamingreceivessendBackRawRequestand:
- Tags each chunk with
RawResponsewhen enabled.- On terminal
Completed/Incompleteevents, callsParseAndSetRawRequestwith the originaljsonBodyand emits a final chunk with total latency.Optional consistency tweak (non‑blocking): gate ExtraFields on the effective booleans and non‑nil values, similar to:
- if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - if sendBackRawResponse { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }Also applies to: 1118-1129, 1147-1199, 1202-1227, 1233-1245, 1259-1275, 1391-1393, 1427-1435
1453-1475: Embedding endpoint correctly participates in raw‑request/response capture
EmbeddingandHandleOpenAIEmbeddingRequestfollow the same pattern as other non‑streaming endpoints:
- Effective
sendBackRawRequest/Responseare computed once per call.HandleProviderResponsereceives those flags along with the JSON request body.- ExtraFields are populated from the returned
rawRequest/rawResponse.You might optionally align the gating with other handlers:
- if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } - if sendBackRawResponse { - response.ExtraFields.RawResponse = rawResponse - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + } + if sendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }Functionally, the current code is already correct.
Also applies to: 1478-1490, 1508-1560
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(9 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(7 hunks)core/providers/gemini/gemini.go(12 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(3 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
✅ Files skipped from review due to trivial changes (2)
- core/changelog.md
- plugins/logging/changelog.md
🚧 Files skipped from review as they are similar to previous changes (21)
- ui/app/workspace/providers/page.tsx
- ui/lib/types/config.ts
- core/schemas/bifrost.go
- framework/logstore/migrations.go
- core/schemas/provider.go
- framework/streaming/types.go
- framework/logstore/tables.go
- framework/streaming/audio.go
- framework/streaming/transcription.go
- framework/configstore/tables/provider.go
- transports/bifrost-http/lib/config.go
- ui/lib/types/schemas.ts
- framework/configstore/rdb.go
- core/providers/parasail/parasail.go
- transports/bifrost-http/lib/account.go
- framework/streaming/chat.go
- transports/changelog.md
- framework/changelog.md
- ui/lib/types/logs.ts
- framework/configstore/clientconfig.go
- core/providers/groq/groq.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/logging/operations.gocore/providers/cerebras/cerebras.goplugins/logging/main.gocore/providers/mistral/mistral.gocore/providers/ollama/ollama.gocore/providers/anthropic/anthropic.gocore/providers/sgl/sgl.gocore/providers/cohere/cohere.goui/app/workspace/logs/views/logDetailsSheet.tsxframework/streaming/responses.gocore/providers/gemini/gemini.gocore/providers/perplexity/perplexity.gocore/providers/azure/azure.goframework/configstore/migrations.goui/app/workspace/providers/fragments/performanceFormFragment.tsxcore/providers/openrouter/openrouter.gocore/providers/utils/utils.gocore/providers/bedrock/bedrock.gocore/providers/vertex/vertex.gocore/providers/openai/openai.gocore/providers/elevenlabs/elevenlabs.gotransports/bifrost-http/handlers/providers.go
🧬 Code graph analysis (16)
core/providers/cerebras/cerebras.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(524-529)
core/providers/mistral/mistral.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-421)ShouldSendBackRawResponse(532-537)ShouldSendBackRawRequest(524-529)
core/providers/ollama/ollama.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(524-529)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-421)ShouldSendBackRawRequest(524-529)ParseAndSetRawRequest(423-430)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(524-529)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-421)ShouldSendBackRawRequest(524-529)ShouldSendBackRawResponse(532-537)ParseAndSetRawRequest(423-430)
ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
ui/app/workspace/logs/views/codeEditor.tsx (1)
CodeEditor(82-245)
framework/streaming/responses.go (1)
framework/streaming/types.go (2)
ProcessedStreamResponse(116-124)StreamType(10-10)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-421)ShouldSendBackRawResponse(532-537)ShouldSendBackRawRequest(524-529)ParseAndSetRawRequest(423-430)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-421)ShouldSendBackRawResponse(532-537)ShouldSendBackRawRequest(524-529)
core/providers/azure/azure.go (1)
core/providers/utils/utils.go (2)
HandleProviderResponse(344-421)ShouldSendBackRawRequest(524-529)
framework/configstore/migrations.go (3)
framework/configstore/tables/virtualkey.go (4)
TableVirtualKeyProviderConfig(23-36)TableVirtualKeyProviderConfig(39-41)TableVirtualKeyProviderConfigKey(12-15)TableVirtualKeyProviderConfigKey(18-20)framework/migrator/migrator.go (3)
New(131-149)DefaultOptions(100-106)Migration(62-69)framework/configstore/tables/provider.go (2)
TableProvider(13-38)TableProvider(41-41)
core/providers/openrouter/openrouter.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-421)ShouldSendBackRawRequest(524-529)ShouldSendBackRawResponse(532-537)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-421)ShouldSendBackRawResponse(532-537)ShouldSendBackRawRequest(524-529)ParseAndSetRawRequest(423-430)
core/providers/vertex/vertex.go (2)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-421)ShouldSendBackRawRequest(524-529)ShouldSendBackRawResponse(532-537)core/schemas/models.go (1)
Model(109-129)
core/providers/openai/openai.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(524-529)HandleProviderResponse(344-421)ParseAndSetRawRequest(423-430)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (15)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (39)
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
35-37: Consistent wiring ofsend_back_raw_requestlooks goodDefaulting, resetting, and submitting
send_back_raw_requestall use the sameprovider.send_back_raw_request ?? falseconvention and pass the form value back intoModelProvider, which keeps behavior predictable and opt‑in.Also applies to: 51-53, 65-67
plugins/logging/main.go (2)
45-46: LGTM! RawRequest field added to UpdateLogData.The new
RawRequestfield mirrors the existingRawResponsefield structure, maintaining consistency in the logging data model.
500-502: LGTM! Raw request extraction respects content logging settings.The code correctly extracts
RawRequestfromextraFieldswithin the content logging guard (line 499), mirroring theRawResponsehandling at lines 503-505. This ensures raw requests are only logged when content logging is enabled.ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
364-388: LGTM! Raw Request viewer implemented correctly.The new Raw Request block mirrors the structure of the Raw Response viewer and correctly handles JSON parsing. The fallback at line 379 now properly returns
log.raw_request(previously flagged issue has been addressed).plugins/logging/operations.go (2)
127-134: LGTM! Raw request marshaling properly guarded for non-streaming.The raw request marshaling is now correctly placed inside the
disableContentLoggingcheck (starting at line 89), ensuring consistency with raw response handling and respecting the content logging flag. The previous review concern has been addressed.
305-313: LGTM! Raw request marshaling properly guarded for streaming.The streaming path correctly places raw request marshaling inside the
disableContentLoggingcheck (starting at line 257). The double-pointer check at line 306 is appropriate for the*interface{}type, and error handling is consistent with the non-streaming path.transports/bifrost-http/handlers/providers.go (2)
55-65: SendBackRawRequest is cleanly added to ProviderResponse and mapped from configThe new
SendBackRawRequestfield is wired consistently fromconfigstore.ProviderConfigintoProviderResponse, mirroring the existingSendBackRawResponsebehavior and keeping the external API shape coherent. No functional issues here.Also applies to: 842-860
170-182: addProvider correctly initializes and returns SendBackRawRequest with safe defaultsUsing a
*boolin the payload andpayload.SendBackRawRequest != nil && *payload.SendBackRawRequestforProviderConfigensures the flag defaults tofalseunless explicitly enabled, and the fallback response path also propagates the same value. This keeps the new capability opt‑in and consistent across success and error paths.Also applies to: 244-253, 270-282
framework/configstore/migrations.go (2)
1284-1311: LGTM! Migration follows established patterns.The migration correctly:
- Checks for column existence before adding
- Provides a rollback function
- Uses the same pattern as other column migrations in this file
Minor note: The migration ID says "columns" (plural) but only adds one column (
send_back_raw_request). This is fine if you anticipate adding more columns in the future, otherwise consider renaming toadd_send_back_raw_request_columnfor consistency with other single-column migrations likeadd_plugin_version_column.
91-93: Migration ordering looks correct.The new migration is properly sequenced after
migrationAddPluginVersionColumnand follows the established error handling pattern.core/providers/elevenlabs/elevenlabs.go (5)
23-30: Struct field addition follows the established pattern.The
sendBackRawRequestfield is correctly added alongsidesendBackRawResponse, maintaining consistency with the existing design.
55-62: Provider initialization correctly wires the new config field.
103-120: Raw request and response handling in listModelsByKey looks correct.The implementation properly:
- Captures
rawRequestfromHandleProviderResponse- Conditionally sets
ExtraFields.RawRequestwhen enabled- Maintains symmetry with raw response handling
256-258: Raw request handling in Speech is correctly implemented.Uses
ParseAndSetRawRequesthelper consistently with the pattern established in other providers.
436-439: Raw request handling in SpeechStream final response is correct.The raw request is properly attached to the final "done" response in the streaming flow.
core/providers/perplexity/perplexity.go (3)
18-25: Struct field addition is correct.
50-56: Provider initialization correctly wires the config field.
186-201: ChatCompletionStream correctly uses context-aware helper.The streaming path properly uses
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)for consistency.core/providers/ollama/ollama.go (8)
17-24: Struct field addition follows established pattern.
55-61: Provider initialization correctly wires the config field.
70-86: ListModels correctly uses context-aware raw request helper.The
ShouldSendBackRawRequesthelper is properly used to respect context overrides.
88-102: TextCompletion implementation is consistent.
104-122: TextCompletionStream implementation is consistent.
124-138: ChatCompletion implementation is consistent.
140-162: ChatCompletionStream implementation is consistent.
190-204: Embedding implementation is consistent.All OpenAI-delegated methods consistently use
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), ensuring context overrides are properly respected.core/providers/sgl/sgl.go (8)
17-24: Struct field addition follows established pattern.
55-61: Provider initialization correctly wires the config field.
69-83: ListModels correctly uses context-aware raw request helper.
85-99: TextCompletion implementation is consistent.
101-119: TextCompletionStream implementation is consistent.
121-135: ChatCompletion implementation is consistent.
137-159: ChatCompletionStream implementation is consistent.
187-201: Embedding implementation is consistent.All methods consistently use
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), maintaining symmetry with the Ollama provider and ensuring context overrides are respected.core/providers/bedrock/bedrock.go (1)
551-555: Raw‑request plumbing in non‑list flows looks consistent.In
TextCompletion,ChatCompletion,ChatCompletionStream, andResponses/ResponsesStreamyou correctly gateParseAndSetRawRequestbehindShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)and use the originaljsonDatabody; this aligns capture with exposure and avoids extra unmarshalling paths.Also applies to: 730-734, 902-906, 971-975, 1058-1063
core/providers/cerebras/cerebras.go (1)
20-22: Cerebras raw‑request flag wiring looks correct and consistent.
sendBackRawRequestis initialized from config and always passed throughShouldSendBackRawRequestinto the shared OpenAI handlers across list‑models, text, chat, and streaming; this keeps behavior aligned with other OpenAI‑compatible providers.Also applies to: 47-53, 61-75, 80-93, 98-118, 121-134, 140-162
core/providers/anthropic/anthropic.go (2)
20-29: Anthropic provider wiring forsendBackRawRequestlooks correctField + constructor wiring from
config.SendBackRawRequestis consistent withsendBackRawResponseand other providers; no functional issues here.Also applies to: 102-110
365-411: Anthropic streaming raw‑request support is consistent and context‑awareFor both chat and responses streaming, you compute effective
sendBackRawRequest/Responseat the provider entry, thread them into the shared handlers, and only callParseAndSetRawRequeston the final chunk when enabled. This correctly captures the JSON body actually sent and aligns streaming behavior with the non‑streaming helpers.Also applies to: 414-429, 398-407, 656-659, 725-772, 775-789, 991-994
core/providers/openai/openai.go (1)
23-31: OpenAI provider wiring forsendBackRawRequestis correctThe new
sendBackRawRequestfield and its initialization fromconfig.SendBackRawRequestare symmetric withsendBackRawResponseand match the pattern used in other providers.Also applies to: 36-69
Pratham-Mishra04
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
899722a to
98bee25
Compare
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
927142b to
49c8a7d
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
plugins/logging/main.go (1)
34-47: Ensure pooledUpdateLogDataclearsRawRequestandRawResponseon reuse (avoid cross-request leakage).The
putUpdateLogDatafunction inpool.goresets 9 fields but omits the newly addedRawRequestandRawResponsefields. Since these are conditionally assigned (only when present inextraFields), stale values from a previous request could persist in the pooled object when reused. Add:data.RawRequest = nil data.RawResponse = nilto the reset logic in
putUpdateLogData(around line 44 in pool.go).
♻️ Duplicate comments (3)
core/changelog.md (1)
4-4: Hyphenate “inter-provider” for readability.This matches the earlier review note; please apply the hyphenation.
-enhancement: improved internal inter provider conversions for integrations +enhancement: improved internal inter-provider conversions for integrationsframework/streaming/chat.go (1)
304-316: Don’t always emit a non-nilRawRequestpointer; also support text-stream final chunks.
RawRequest: &rawRequestmakesRawRequestalways non-nil (even when no raw request exists).- Final-chunk extraction only checks
result.ChatResponse...; forTextCompletionStreamRequest, the raw request would live underresult.TextCompletionResponse.ExtraFields.RawRequest.- var rawRequest interface{} - if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ChatResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil { + if result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { + v := result.ChatResponse.ExtraFields.RawRequest + rawRequest = &v + } else if result.TextCompletionResponse != nil && result.TextCompletionResponse.ExtraFields.RawRequest != nil { + v := result.TextCompletionResponse.ExtraFields.RawRequest + rawRequest = &v + } + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: streamType, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, niltransports/changelog.md (1)
5-5: Hyphenate compound adjective: “inter-provider”.-enhancement: improved internal inter provider conversions for integrations +enhancement: improved internal inter-provider conversions for integrations
🧹 Nitpick comments (7)
core/schemas/provider.go (1)
21-34: Minor: consider rewording ErrProviderRawRequestUnmarshal message (request is “to” provider).E.g. “failed to unmarshal raw request for provider API”.
core/providers/perplexity/perplexity.go (1)
19-25: Minor: compute raw-request toggle once to avoid duplicate context lookups.- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + sendBackRawReq := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResp := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &response, jsonBody, sendBackRawReq, sendBackRawResp) ... - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if sendBackRawReq { bifrostResponse.ExtraFields.RawRequest = rawRequest } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if sendBackRawResp { bifrostResponse.ExtraFields.RawResponse = rawResponse }Also applies to: 50-56, 145-166, 185-201
core/providers/mistral/mistral.go (1)
17-23: Minor: only setExtraFields.RawRequestwhen a raw request was actually captured.- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { response.ExtraFields.RawRequest = rawRequest }Also applies to: 53-59, 105-123, 154-165, 172-193, 226-237
core/providers/openrouter/openrouter.go (1)
99-120: Avoid re-evaluating flags and avoid setting RawRequest/RawResponse when the captured value is nil (GET path).
This endpoint passesrequestBody=nil, sorawRequestwill always be nil (by design inHandleProviderResponse). Consider computing effective flags once and only assigning when non-nil to keepextra_fieldsclean and avoid repeated context reads.var openrouterResponse schemas.BifrostListModelsResponse -// Pass nil requestBody for GET requests - HandleProviderResponse will skip raw request capture -rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &openrouterResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) +sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) +sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) +// GET: requestBody=nil, so raw-request capture is skipped in HandleProviderResponse +rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, + &openrouterResponse, + nil, + sendBackRawRequest, + sendBackRawResponse, +) ... -// Set raw request if enabled -if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { +if sendBackRawRequest && rawRequest != nil { openrouterResponse.ExtraFields.RawRequest = rawRequest } ... -// Set raw response if enabled -if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { +if sendBackRawResponse && rawResponse != nil { openrouterResponse.ExtraFields.RawResponse = rawResponse }core/providers/gemini/gemini.go (1)
162-181: Minor: for GET list-models, RawRequest will always be nil—consider skipping the assignment.Since
requestBody=nil,HandleProviderResponsewon’t populaterawRequest. Consider:-if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { +if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { response.ExtraFields.RawRequest = rawRequest }core/providers/cohere/cohere.go (1)
221-241: Minor: GET list-models sets RawRequest but capture is skipped (nil requestBody).Same cleanup as elsewhere—only set when
rawRequest != nil(or explicitly disable raw-request capture for this GET path to reduce confusion).core/providers/bedrock/bedrock.go (1)
450-473: Minor: GET list-models RawRequest will always be nil—avoid assigning it.-if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { +if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { response.ExtraFields.RawRequest = rawRequest }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(9 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(7 hunks)core/providers/gemini/gemini.go(12 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(3 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(6 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (19)
- framework/logstore/migrations.go
- framework/configstore/tables/provider.go
- ui/lib/types/schemas.ts
- framework/streaming/audio.go
- ui/app/workspace/providers/page.tsx
- ui/app/workspace/providers/fragments/performanceFormFragment.tsx
- ui/lib/types/config.ts
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- framework/streaming/transcription.go
- framework/logstore/tables.go
- plugins/logging/changelog.md
- ui/lib/types/logs.ts
- framework/configstore/rdb.go
- core/providers/anthropic/anthropic.go
- core/providers/elevenlabs/elevenlabs.go
- core/providers/groq/groq.go
- plugins/logging/operations.go
- framework/changelog.md
- framework/streaming/responses.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/lib/account.gocore/schemas/provider.goframework/configstore/clientconfig.gocore/providers/perplexity/perplexity.gocore/providers/sgl/sgl.gocore/providers/ollama/ollama.goframework/streaming/types.gocore/providers/bedrock/bedrock.gocore/providers/azure/azure.gocore/schemas/bifrost.gocore/providers/openrouter/openrouter.gotransports/bifrost-http/handlers/providers.gocore/providers/cerebras/cerebras.gocore/providers/utils/utils.gotransports/bifrost-http/lib/config.gocore/providers/gemini/gemini.goframework/streaming/chat.gocore/providers/cohere/cohere.gocore/providers/parasail/parasail.gocore/providers/vertex/vertex.gocore/providers/mistral/mistral.goplugins/logging/main.gocore/providers/openai/openai.goframework/configstore/migrations.gocore/changelog.mdtransports/changelog.md
🧠 Learnings (2)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/lib/account.gocore/schemas/provider.goframework/configstore/clientconfig.gocore/providers/perplexity/perplexity.gocore/providers/sgl/sgl.gocore/providers/ollama/ollama.goframework/streaming/types.gocore/providers/bedrock/bedrock.gocore/providers/azure/azure.gocore/schemas/bifrost.gocore/providers/openrouter/openrouter.gotransports/bifrost-http/handlers/providers.gocore/providers/cerebras/cerebras.gocore/providers/utils/utils.gotransports/bifrost-http/lib/config.gocore/providers/gemini/gemini.goframework/streaming/chat.gocore/providers/cohere/cohere.gocore/providers/parasail/parasail.gocore/providers/vertex/vertex.gocore/providers/mistral/mistral.goplugins/logging/main.gocore/providers/openai/openai.goframework/configstore/migrations.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/openai.go
🧬 Code graph analysis (15)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/ollama/ollama.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/azure/azure.go (2)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)core/schemas/utils.go (1)
IsAnthropicModel(1043-1045)
core/providers/cerebras/cerebras.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/utils/utils.go (3)
core/schemas/bifrost.go (4)
BifrostError(356-365)ErrorField(374-381)BifrostResponseExtraFields(286-296)BifrostContextKeySendBackRawRequest(118-118)ui/lib/types/logs.ts (2)
BifrostError(226-232)ErrorField(217-224)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
framework/streaming/chat.go (1)
framework/streaming/types.go (2)
ProcessedStreamResponse(116-124)StreamType(10-10)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/parasail/parasail.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/vertex/vertex.go (2)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)core/schemas/models.go (1)
Model(109-129)
core/providers/mistral/mistral.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
core/providers/openai/openai.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(536-541)HandleProviderResponse(344-430)ParseAndSetRawRequest(432-439)
framework/configstore/migrations.go (2)
framework/migrator/migrator.go (3)
New(131-149)DefaultOptions(100-106)Migration(62-69)framework/configstore/tables/provider.go (2)
TableProvider(15-44)TableProvider(47-47)
🪛 LanguageTool
core/changelog.md
[grammar] ~4-~4: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
transports/changelog.md
[grammar] ~5-~5: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (28)
core/providers/cerebras/cerebras.go (1)
16-22: Clean, consistent SendBackRawRequest wiring (constructor + all OpenAI-compatible entrypoints).Please sanity-check (in the stacked PRs) that the OpenAI handler(s) used here persist/attach
raw_requestfor both non-streaming and streaming Cerebras calls whenShouldSendBackRawRequest(...)is true.Also applies to: 47-53, 61-74, 77-92, 95-117, 120-133, 136-162
plugins/logging/main.go (1)
497-506: Good: RawRequest is only persisted when content logging is enabled.transports/bifrost-http/lib/account.go (1)
102-110: Good: propagateSendBackRawRequestinGetConfigForProvider.framework/configstore/clientconfig.go (1)
146-157: LGTM:SendBackRawRequestadded + included in provider config hash.Also applies to: 159-215
core/schemas/provider.go (1)
232-244: Good:SendBackRawRequestis now part of coreProviderConfig.transports/bifrost-http/lib/config.go (2)
1689-1697: Good: DB → in-memory propagation forSendBackRawRequest.
2098-2106: Good: redacted provider config now exposesSendBackRawRequest(safe boolean).framework/streaming/types.go (1)
115-125: LGTM: streamed final responses now propagateRawRequestviaExtraFields.Also applies to: 126-265
core/providers/ollama/ollama.go (1)
18-24: ConsistentSendBackRawRequestpropagation through handler calls.Also applies to: 55-61, 74-85, 90-101, 108-121, 126-137, 146-161, 192-203
core/schemas/bifrost.go (1)
105-122: Schema + context plumbing for raw request looks consistent.Also applies to: 285-296
core/providers/sgl/sgl.go (1)
18-24: ConsistentSendBackRawRequestwiring across SGL endpoints.Also applies to: 55-61, 71-82, 87-98, 105-118, 123-134, 143-158, 189-200
framework/configstore/migrations.go (1)
1440-1440: No concerns.core/providers/openrouter/openrouter.go (1)
18-24: Good: provider-level SendBackRawRequest wiring is straightforward and consistent with SendBackRawResponse.Also applies to: 49-55
core/providers/vertex/vertex.go (1)
56-63: Good: SendBackRawRequest is wired consistently in the provider struct/constructor.Also applies to: 78-85
core/providers/gemini/gemini.go (2)
23-30: LGTM: SendBackRawRequest is wired consistently in GeminiProvider.Also applies to: 55-63
270-303: Good: raw-request capture is correctly sourced from the JSON body for POST + stream finalization.Also applies to: 424-426, 638-641, 681-683, 909-912
core/providers/cohere/cohere.go (2)
66-74: LGTM: Cohere provider raw-request flag is wired and used consistently (capture + exposure).Also applies to: 105-113
305-326: Good: POST paths correctly capture raw request fromjsonBodyand stream finalization fills RawRequest once.Also applies to: 531-554, 774-795, 481-486, 721-725
core/providers/bedrock/bedrock.go (3)
29-36: LGTM: BedrockProvider SendBackRawRequest wiring is clean and consistent.Also applies to: 72-80
551-555: Good: raw-request is attached at the right “finalization” points for Bedrock non-GET + streaming paths.Also applies to: 730-734, 902-905, 971-975, 1059-1063
1169-1262: Verify Bedrock embedding RawRequest behavior (looks missing).
I don’t seeParseAndSetRawRequest(or equivalent) infunc (provider *BedrockProvider) Embedding(...), so enabling SendBackRawRequest may not surface the embedding request payload. If that’s intentional, all good; if not, please add it for parity with other operations.core/providers/parasail/parasail.go (1)
21-22: LGTM! Consistent implementation of raw request capture.The Parasail provider correctly integrates the new
sendBackRawRequestfunctionality following the established pattern:
- Field properly declared and initialized from config
- Threaded through all relevant endpoints (ListModels, ChatCompletion, ChatCompletionStream)
- Uses the
ShouldSendBackRawRequesthelper consistentlyThe implementation aligns with the broader raw-request capture feature across all providers.
Also applies to: 52-53, 72-75, 99-103, 123-131
transports/bifrost-http/handlers/providers.go (1)
61-61: LGTM! Complete HTTP API integration.The provider handler properly exposes and manages the
SendBackRawRequestconfiguration through the HTTP API:
- Added to
ProviderResponsefor client visibility- Handled in both add and update operations with proper optional field semantics
- Conditional update logic (lines 431-433) correctly preserves existing config when not provided
- Fallback paths maintain consistency
getProviderResponseFromConfigpropagates the field uniformlyThe implementation provides complete CRUD support for the feature.
Also applies to: 179-180, 250-250, 279-280, 320-321, 431-433, 462-463, 856-856
core/providers/azure/azure.go (1)
26-27: LGTM! Comprehensive integration across all Azure endpoints.The Azure provider correctly implements raw request capture across its diverse endpoint portfolio:
- Properly initialized from config
- Non-streaming paths use
HandleProviderResponsewithjsonDataparameter- Streaming paths propagate flags and finalize with
ParseAndSetRawRequest- Both OpenAI-compatible and Anthropic-specific code paths covered
- GET requests (ListModels) correctly pass
nilforrequestBodyThe implementation handles Azure's unique multi-backend architecture (OpenAI and Anthropic) consistently.
Also applies to: 51-52, 197-213, 274-288, 405-432, 760-775
core/providers/utils/utils.go (3)
344-430: LGTM! Core utility properly implements dual raw data capture.
HandleProviderResponsehas been well-refactored to support both raw request and raw response capture:
- Signature correctly expanded to accept
requestBodyand return both raw data types- Concurrency handled properly with goroutines and WaitGroup
shouldCaptureRawRequestlogic correctly skips whenrequestBodyis nil (e.g., GET requests)- Error handling checks both
rawRequestErrandrawResponseErrindependently- Past review concerns about error handling have been addressed (lines 404-411)
The implementation enables the feature across all providers while maintaining correct error semantics.
432-439: LGTM! Streaming helper uses appropriate non-fatal error handling.
ParseAndSetRawRequestis designed for streaming finalization where raw request parsing errors should not abort the stream. The warn-only error handling is appropriate for this use case, allowing the stream to complete successfully even if raw request capture fails.
533-541: LGTM! Documentation added per past review.The asymmetric override behavior is now properly documented (lines 534-535), clarifying that context values can only promote
false→trueand cannot override atrueconfig tofalse. This addresses the past review comment and makes the design intent clear.core/providers/openai/openai.go (1)
28-29: LGTM! Comprehensive integration across all OpenAI endpoints.The OpenAI provider thoroughly implements raw request capture across its extensive endpoint portfolio:
Non-streaming paths:
- ListModels, TextCompletion, ChatCompletion, Responses, Embedding all use
HandleProviderResponsewithjsonData- Conditional
RawRequestassignment based on effective flag valueStreaming paths:
- TextCompletionStream, ChatCompletionStream, ResponsesStream all propagate flags
- Finalization uses
ParseAndSetRawRequestat stream end (e.g., lines 597-599, 1083-1086, 1428-1431)- Responses-to-ChatCompletion fallback correctly handled (lines 977-980)
Shared handlers:
- All handler signatures consistently updated:
HandleOpenAIListModelsRequest,HandleOpenAITextCompletionRequest,HandleOpenAIChatCompletionRequest,HandleOpenAIResponsesRequest,HandleOpenAIEmbeddingRequest, etc.- These handlers are reused by other OpenAI-compatible providers, extending the feature across the ecosystem
The implementation is thorough, consistent, and integrates seamlessly with OpenAI's diverse API surface.
Also applies to: 65-66, 96-111, 161-175, 227-230, 242-313, 333-607, 625-714, 736-1094, 1110-1199, 1220-1451, 1471-1561
Pratham-Mishra04
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
8adce3d to
f47ea6b
Compare
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
abdcda6 to
1a511ff
Compare
TejasGhatte
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
f47ea6b to
fc91124
Compare
TejasGhatte
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
1a511ff to
6951a77
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (7)
core/providers/mistral/mistral.go (1)
100-123: Avoid emittingraw_request: nullfor GET endpoints
HandleProviderResponse(..., requestBody=nil, ...)won’t capture raw request, sorawRequestis expected to be nil; assigning it can still surface asnull.- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }plugins/logging/operations.go (2)
173-180: Fix operator precedence indisableContentLoggingguard for raw response
Current condition can enter the block whenp.disableContentLogging == nileven ifdata.RawResponse == nil.- if p.disableContentLogging == nil || !*p.disableContentLogging && data.RawResponse != nil { + if (p.disableContentLogging == nil || !*p.disableContentLogging) && data.RawResponse != nil { rawResponseBytes, err := sonic.Marshal(data.RawResponse) ... }
305-318: Fix raw_response storage in streaming path to match schema and non-streaming consistencyIn the streaming path (line 316),
raw_responseis assigned*streamResponse.Data.RawResponsewithout dereferencing. SinceRawResponseis*stringin the struct, this stores a pointer value instead of the dereferenced string. The logstore schema expectsraw_responseto be astringcolumn. This is inconsistent with:
- The non-streaming path (line 178) which converts to string via
string(rawResponseBytes)- The streaming raw_request handling (line 311) which does
string(rawRequestBytes)Change line 316 to dereference the pointer:
updates["raw_response"] = *streamResponse.Data.RawResponsecore/providers/openrouter/openrouter.go (1)
99-120: Avoid emittingraw_request: nullon list-models (GET)
Even though capture is correctly skipped, the unconditional assignment can still serialize asnull.- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - openrouterResponse.ExtraFields.RawRequest = rawRequest - } + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + openrouterResponse.ExtraFields.RawRequest = rawRequest + }core/providers/vertex/vertex.go (1)
147-236: Fix Vertex list-models raw-request accumulation + avoid defers in pagination loop
requestBodyis nil for GET, soHandleProviderResponsewon’t capture raw request; appendingrawRequeststill builds[null, null, ...](Lines 203-209, 230-232).defer fasthttp.ReleaseRequest/Response(...)inside theforloop delays releases until function return, not per page (Lines 172-176).@@ - var rawRequests []interface{} + var rawRequests []interface{} var rawResponses []interface{} pageToken := "" + + sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) @@ - req := fasthttp.AcquireRequest() - resp := fasthttp.AcquireResponse() - defer fasthttp.ReleaseRequest(req) - defer fasthttp.ReleaseResponse(resp) + req := fasthttp.AcquireRequest() + resp := fasthttp.AcquireResponse() @@ - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &vertexResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + &vertexResponse, + nil, // GET: no request body => raw-request capture skipped + sendBackRawRequest, + sendBackRawResponse, + ) if bifrostErr != nil { return nil, bifrostErr } - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - rawRequests = append(rawRequests, rawRequest) - } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if sendBackRawRequest && rawRequest != nil { + rawRequests = append(rawRequests, rawRequest) + } + if sendBackRawResponse && rawResponse != nil { rawResponses = append(rawResponses, rawResponse) } + + fasthttp.ReleaseRequest(req) + fasthttp.ReleaseResponse(resp) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequests - } + if sendBackRawRequest && len(rawRequests) > 0 { + response.ExtraFields.RawRequest = rawRequests + }(If you explicitly don’t want raw request for list-models, an even cleaner option is passing
sendBackRawRequest=falsefor this GET path.)plugins/logging/main.go (1)
35-47: Add reset logic toUpdateLogDatapool lifecycle to prevent PII leakageThe
UpdateLogDatastruct usessync.Poolfor memory reuse (lines 102, 130-131), butgetUpdateLogData()(pool.go) does not reset the struct on checkout, andputUpdateLogData()(pool.go) does not reset it on return. SinceRawRequestandRawResponseare only conditionally set during PostHook whenextraFieldsexist and content logging is enabled (lines 499-503), stale request/response data from previous log entries can persist into subsequent logs—creating a PII and security risk.Reset the struct in
getUpdateLogData():func (p *LoggerPlugin) getUpdateLogData() *UpdateLogData { d := p.updateDataPool.Get().(*UpdateLogData) *d = UpdateLogData{} // Clear all fields to prevent data leakage return d }framework/logstore/migrations.go (1)
12-47: Make rollback idempotent forraw_requestcolumn.MigratechecksHasColumn, butRollbackunconditionally drops the column; consider guarding withHasColumnto avoid rollback errors in edge cases.Rollback: func(tx *gorm.DB) error { tx = tx.WithContext(ctx) migrator := tx.Migrator() - if err := migrator.DropColumn(&Log{}, "raw_request"); err != nil { - return err + if migrator.HasColumn(&Log{}, "raw_request") { + if err := migrator.DropColumn(&Log{}, "raw_request"); err != nil { + return err + } } return nil },Also applies to: 645-674
♻️ Duplicate comments (6)
core/changelog.md (1)
4-4: Hyphenate compound adjective “inter-provider”.
Change “inter provider” → “inter-provider” for readability/grammar.-enhancement: improved internal inter provider conversions for integrations +enhancement: improved internal inter-provider conversions for integrationstransports/changelog.md (1)
5-5: Hyphenate compound adjective “inter-provider”.
Change “inter provider” → “inter-provider” for readability/grammar.-enhancement: improved internal inter provider conversions for integrations +enhancement: improved internal inter-provider conversions for integrationsui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
26-38: Add an explicit security warning to the “Include Raw Request” description. Enabling this can store/return sensitive payloads (API keys, user data) per PR objectives.<FormLabel>Include Raw Request</FormLabel> <p className="text-muted-foreground text-xs"> - Include the raw provider request alongside the parsed request for debugging and advanced use cases + Include the raw provider request alongside the parsed request for debugging and advanced use cases.{" "} + <span className="font-medium text-yellow-600"> + Warning: this may expose sensitive data (API keys, user information) in responses and logs. + </span> </p>Also applies to: 44-56, 57-67, 128-156
framework/streaming/responses.go (1)
723-726: KeepProcessedStreamResponse.RawRequestnil when the raw request is absent (avoidraw_request: null). Right nowRawRequest: &rawRequestis always non-nil, even ifExtraFields.RawRequestwas never set.if isFinalChunk { - var rawRequest interface{} - if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequest interface{} + var rawRequestPtr *interface{} + if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest + rawRequestPtr = &rawRequest + } ... return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeResponses, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequestPtr, }, nil ... - var rawRequest interface{} - if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequest interface{} + var rawRequestPtr *interface{} + if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest + rawRequestPtr = &rawRequest + } ... return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeResponses, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequestPtr, }, nilAlso applies to: 782-790, 866-879
core/providers/cohere/cohere.go (1)
223-241: Double-evaluation pattern persists despite past review feedback.The past review suggested computing effective flags once per request and reusing them to avoid redundant
ShouldSendBackRawRequestcalls. The current implementation still re-evaluates at each usage:
- Line 223: computes and passes to
HandleProviderResponse- Line 234: re-evaluates when setting
ExtraFields.RawRequestSame pattern in
ChatCompletion(305, 319),Responses(531, 547), andEmbedding(774, 788).While functionally correct (same context throughout), consider the originally-suggested refactor:
func (provider *CohereProvider) ChatCompletion(...) { + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(..., providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(..., effectiveSendBackRawRequest, effectiveSendBackRawResponse) - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if effectiveSendBackRawRequest && rawRequest != nil { bifrostResponse.ExtraFields.RawRequest = rawRequest }Apply consistently in
listModelsByKey,ChatCompletion,Responses, andEmbedding.Also applies to: 305-326, 531-569, 774-797
core/providers/openai/openai.go (1)
96-99: Parameter ambiguity: issendBackRawRequesta raw config value or already-effective flag?The calling code computes:
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)and passes the result as parameter
sendBackRawRequesttolistModelsByKey.Inside
listModelsByKey(line 173), the code re-evaluates:if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) {Since the parameter is already the effective value (context-resolved), re-calling
ShouldSendBackRawRequestis redundant. The same pattern appears inHandleOpenAITextCompletionRequest(242-243, 303-305),HandleOpenAIChatCompletionRequest(640-641, 704-706),HandleOpenAIResponsesRequest(1125-1126, 1190-1192), andHandleOpenAIEmbeddingRequest(1487-1488, 1551-1553).Consider treating parameters as already-effective values:
- if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }Apply consistently across all handlers.
Also applies to: 123-124, 173-175
🧹 Nitpick comments (2)
framework/logstore/tables.go (1)
74-109: RawRequest field addition is consistent with the new logging feature. Optional: if you want to avoid emitting empty strings in log payloads when disabled, considerjson:"raw_request,omitempty".core/providers/azure/azure.go (1)
197-218: Redundant re-evaluation of context-resolved flags.Multiple methods compute effective flags via
ShouldSendBackRawRequestwhen callingHandleProviderResponse, then re-evaluate the same helper when settingExtraFields:
listModelsByKey: lines 197, 211TextCompletion: lines 274, 286ChatCompletion: lines 411, 417, 430Responses: lines 601, 607, 620Embedding: lines 761, 773While functionally correct, consider computing once and reusing:
func (provider *AzureProvider) Embedding(...) { + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(..., providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(..., effectiveSendBackRawRequest, effectiveSendBackRawResponse) - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if effectiveSendBackRawRequest && rawRequest != nil { response.ExtraFields.RawRequest = rawRequest }Also applies to: 274-293, 411-437, 601-627, 761-780
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(9 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(7 hunks)core/providers/gemini/gemini.go(12 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(4 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(6 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (20)
- ui/lib/types/config.ts
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- core/providers/ollama/ollama.go
- core/schemas/bifrost.go
- ui/app/workspace/providers/page.tsx
- core/schemas/provider.go
- ui/lib/types/schemas.ts
- core/providers/cerebras/cerebras.go
- ui/lib/types/logs.ts
- transports/bifrost-http/handlers/providers.go
- framework/streaming/chat.go
- framework/changelog.md
- framework/streaming/transcription.go
- transports/bifrost-http/lib/config.go
- framework/configstore/tables/provider.go
- core/providers/parasail/parasail.go
- framework/streaming/audio.go
- core/providers/perplexity/perplexity.go
- plugins/logging/changelog.md
- transports/bifrost-http/lib/account.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
framework/logstore/migrations.goframework/configstore/rdb.goplugins/logging/operations.goplugins/logging/main.gocore/providers/mistral/mistral.gocore/providers/sgl/sgl.gocore/providers/groq/groq.goframework/streaming/types.gocore/changelog.mdcore/providers/openrouter/openrouter.gocore/providers/elevenlabs/elevenlabs.gocore/providers/vertex/vertex.gocore/providers/openai/openai.gotransports/changelog.mdcore/providers/gemini/gemini.gocore/providers/cohere/cohere.goframework/configstore/clientconfig.goframework/logstore/tables.gocore/providers/utils/utils.gocore/providers/anthropic/anthropic.gocore/providers/bedrock/bedrock.gocore/providers/azure/azure.goframework/streaming/responses.goframework/configstore/migrations.goui/app/workspace/providers/fragments/performanceFormFragment.tsx
🧠 Learnings (2)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
framework/logstore/migrations.goframework/configstore/rdb.goplugins/logging/operations.goplugins/logging/main.gocore/providers/mistral/mistral.gocore/providers/sgl/sgl.gocore/providers/groq/groq.goframework/streaming/types.gocore/providers/openrouter/openrouter.gocore/providers/elevenlabs/elevenlabs.gocore/providers/vertex/vertex.gocore/providers/openai/openai.gocore/providers/gemini/gemini.gocore/providers/cohere/cohere.goframework/configstore/clientconfig.goframework/logstore/tables.gocore/providers/utils/utils.gocore/providers/anthropic/anthropic.gocore/providers/bedrock/bedrock.gocore/providers/azure/azure.goframework/streaming/responses.goframework/configstore/migrations.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/openai.go
🧬 Code graph analysis (14)
framework/logstore/migrations.go (2)
framework/migrator/migrator.go (3)
DefaultOptions(100-106)New(131-149)Migration(62-69)framework/logstore/tables.go (2)
Log(74-137)Log(140-142)
core/providers/mistral/mistral.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/groq/groq.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/openrouter/openrouter.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
core/providers/vertex/vertex.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
core/providers/openai/openai.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(536-541)HandleProviderResponse(344-430)ParseAndSetRawRequest(432-439)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/azure/azure.go (2)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)core/schemas/utils.go (1)
IsAnthropicModel(1043-1045)
framework/streaming/responses.go (1)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeResponses(17-17)
framework/configstore/migrations.go (2)
framework/migrator/migrator.go (3)
New(131-149)DefaultOptions(100-106)Migration(62-69)framework/configstore/tables/provider.go (2)
TableProvider(15-44)TableProvider(47-47)
🪛 LanguageTool
core/changelog.md
[grammar] ~4-~4: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
transports/changelog.md
[grammar] ~5-~5: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (48)
plugins/logging/main.go (1)
499-506: Good:RawRequestcapture is gated bydisableContentLogging
This keeps raw request exposure consistent with other content logging paths.core/providers/mistral/mistral.go (1)
17-23: LGTM: provider-levelsendBackRawRequestwiringAlso applies to: 53-60
core/providers/elevenlabs/elevenlabs.go (3)
23-30: LGTM: raw-request plumbing + conditional exposureAlso applies to: 55-63, 102-120
256-259: Good: usingParseAndSetRawRequestavoids duplicating JSON-unmarshal logicAlso applies to: 436-442
527-533: Good: raw response is now actually assigned (and gated)core/providers/utils/utils.go (2)
344-430: LGTM:HandleProviderResponsecorrectly skips raw-request capture for nil request bodies
Also good that when both raw captures are enabled, raw-response errors aren’t silently dropped.
533-541: Nice: documented asymmetric override semantics for raw-requestplugins/logging/operations.go (1)
125-135: LGTM: raw request logging now respectsdisableContentLoggingcore/providers/sgl/sgl.go (1)
18-24: LGTM: consistent threading ofShouldSendBackRawRequestthrough OpenAI-compatible pathsAlso applies to: 55-62, 69-83, 85-99, 101-119, 121-135, 137-159, 187-201
core/providers/openrouter/openrouter.go (1)
17-24: LGTM: config wiring forsendBackRawRequestAlso applies to: 49-56
core/providers/vertex/vertex.go (1)
56-63: LGTM: provider-levelsendBackRawRequestwiringAlso applies to: 78-85
framework/configstore/clientconfig.go (1)
148-157: Provider config + hash propagation forSendBackRawRequestlooks correct. The new field is present and incorporated intoGenerateConfigHash, so reconciliation should detect toggles reliably.Also applies to: 204-212
framework/streaming/types.go (1)
115-124: Good propagation pattern inToBifrostResponse(only set when non-nil). This matches the intended “optional raw request” semantics and keeps extra fields clean when the feature is off.Also applies to: 152-263
framework/configstore/rdb.go (1)
200-218: DB persistence wiring forSendBackRawRequestis consistent across create/update/read.Also applies to: 331-339, 447-457, 644-653
framework/configstore/migrations.go (3)
92-94: Migration ordering is correct.The new migration
migrationAddSendBackRawRequestColumnsis correctly placed beforemigrationAddConfigHashColumn, ensuring the column exists before hash generation attempts to read it.
1294-1321: Migration implementation follows established patterns.The migration correctly:
- Uses a unique ID (
add_send_back_raw_request_columns)- Checks for column existence before adding (idempotent)
- Includes a proper rollback function
- Follows the same structure as other column migrations in this file
1347-1348: Config hash generation now includes SendBackRawRequest.This ensures changes to the
SendBackRawRequestflag are reflected in the config hash, enabling proper change detection during config synchronization.core/providers/groq/groq.go (3)
20-21: Field addition follows the established pattern.The
sendBackRawRequestfield is correctly positioned alongsidesendBackRawResponse, maintaining consistency in the struct layout.
56-57: Correct initialization from config.The field is properly initialized from
config.SendBackRawRequestduring provider construction.
76-77: Context-aware flag handling is consistent.All paths (
ListModels,ChatCompletion,ChatCompletionStream) correctly useproviderUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)to honor context overrides while falling back to the provider default. This aligns with the pattern used forsendBackRawResponse.Also applies to: 166-167, 190-191
core/providers/anthropic/anthropic.go (10)
26-28: Field addition follows established pattern.The
sendBackRawRequestfield is correctly positioned alongsidesendBackRawResponsein the struct.
107-109: Correct initialization from config.The field is properly initialized from
config.SendBackRawRequestduring provider construction.
204-221: Non-streaming list models handles raw request correctly.The implementation correctly uses context-aware helpers for both flags in
HandleProviderResponseand conditionally setsRawRequestin ExtraFields. Since this is a GET request,nilis passed for requestBody, andHandleProviderResponsewill skip raw request capture appropriately.
274-295: TextCompletion raw request handling is correct.Uses context-aware flags consistently in both
HandleProviderResponseand when setting ExtraFields.
335-356: ChatCompletion raw request handling is correct.Pattern matches TextCompletion with consistent context-aware flag usage.
405-411: ChatCompletionStream correctly passes raw request flag.The context-aware flag is computed once at the call site and passed to the streaming handler, which then uses it in final chunk handling.
Also applies to: 423-424
656-659: Streaming final chunk correctly captures raw request.
ParseAndSetRawRequestis called only whensendBackRawRequestis true, attaching the original request body to the final response's ExtraFields.
697-721: Responses endpoint handles raw request correctly.Consistent pattern with other non-streaming endpoints.
765-771: ResponsesStream correctly passes raw request flag.The context-aware flag is computed and passed to the streaming handler.
Also applies to: 783-784
991-994: ResponsesStream final chunk correctly captures raw request.
ParseAndSetRawRequestis called in the final chunk when enabled, consistent with ChatCompletionStream.core/providers/bedrock/bedrock.go (8)
34-35: Field addition follows established pattern.The
sendBackRawRequestfield is correctly positioned alongsidesendBackRawResponsein the struct.
77-78: Correct initialization from config.The field is properly initialized from
config.SendBackRawRequestduring provider construction.
452-473: List models handles raw request correctly for GET requests.Uses context-aware helpers and correctly passes
nilfor requestBody since this is a GET request without a body. TheHandleProviderResponseimplementation skips raw request capture when requestBody is nil.
551-554: TextCompletion uses ParseAndSetRawRequest appropriately.Since Bedrock has model-specific response parsing (Anthropic vs Mistral), it uses
ParseAndSetRawRequestdirectly rather thanHandleProviderResponse. The context-aware check ensures the flag is honored correctly.
730-733: ChatCompletion raw request handling is correct.Consistent pattern with TextCompletion.
902-905: ChatCompletionStream final chunk correctly captures raw request.Raw request is captured only in the final response chunk, avoiding redundant data in intermediate chunks.
971-974: Responses endpoint handles raw request correctly.Consistent pattern with other non-streaming endpoints.
1059-1062: ResponsesStream final chunk correctly captures raw request.Captured in the last finalization response, consistent with other streaming endpoints.
core/providers/gemini/gemini.go (10)
27-29: Field addition follows established pattern.The
sendBackRawRequestfield is correctly positioned alongsidesendBackRawResponsein the struct.
60-61: Correct initialization from config.The field is properly initialized from
config.SendBackRawRequestduring provider construction.
164-181: List models handles raw request correctly.Uses context-aware helpers consistently in both
HandleProviderResponseand when setting ExtraFields. Previous inconsistency (past review) has been addressed.
270-302: ChatCompletion raw request handling is correct.Previous inconsistency (past review) has been addressed - now uses
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)consistently for bothHandleProviderResponseand ExtraFields assignment.
330-331: ChatCompletionStream correctly passes context-aware flag.The flag is computed using the context-aware helper before being passed to the OpenAI-compatible streaming handler.
384-385: Embedding correctly passes context-aware flag.Consistent with other delegated handlers.
423-426: Speech endpoint handles raw request correctly.Uses
ParseAndSetRawRequestwith context-aware flag check, consistent with the Gemini-specific response handling pattern.
638-641: SpeechStream final chunk correctly captures raw request.Raw request is captured only in the final "done" response, avoiding redundant data in delta chunks.
681-683: Transcription endpoint handles raw request correctly.Consistent pattern with Speech endpoint.
909-912: TranscriptionStream final chunk correctly captures raw request.Consistent with SpeechStream - raw request captured only in the final "done" response.
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
plugins/logging/operations.go (1)
173-180: Fix operator precedence inraw_responseguard to avoid storing"null"whendisableContentLoggingis nil.
As written,p.disableContentLogging == nilbypasses thedata.RawResponse != nilcheck due to&&binding tighter than||, so you can end up writing"raw_response": "null"for entries without a raw response (whendisableContentLoggingis unset). Suggest parenthesizing and always requiring non-nil raw response.- if p.disableContentLogging == nil || !*p.disableContentLogging && data.RawResponse != nil { + if (p.disableContentLogging == nil || !*p.disableContentLogging) && data.RawResponse != nil { rawResponseBytes, err := sonic.Marshal(data.RawResponse) if err != nil { p.logger.Error("failed to marshal raw response: %v", err) } else { updates["raw_response"] = string(rawResponseBytes) } }core/providers/bedrock/bedrock.go (1)
1247-1261: Add raw request support to the Embedding method.The
Embeddingmethod handles raw response (lines 1254-1260) but lacks raw request support, unlike other methods in this provider and other providers that implement embeddings (OpenAI, Azure, Cohere, etc.).To implement this consistently, refactor the method to declare
jsonDataat method scope rather than inside the switch cases, making it accessible for raw request handling:// Convert request and execute based on model type var rawResponse []byte var bifrostError *schemas.BifrostError var latency time.Duration var path string var deployment string + var jsonData interface{} switch modelType { case "titan": - jsonData, bifrostErr := providerUtils.CheckContextAndGetRequestBody( + var bifrostErr *schemas.BifrostError + jsonData, bifrostErr = providerUtils.CheckContextAndGetRequestBody(Then add raw request handling alongside the existing raw response handling:
// Set ExtraFields bifrostResponse.ExtraFields.Provider = providerName bifrostResponse.ExtraFields.ModelRequested = request.Model bifrostResponse.ExtraFields.ModelDeployment = deployment bifrostResponse.ExtraFields.RequestType = schemas.EmbeddingRequest bifrostResponse.ExtraFields.Latency = latency.Milliseconds() + // Set raw request if enabled + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + providerUtils.ParseAndSetRawRequest(&bifrostResponse.ExtraFields, jsonData) + } + // Set raw response if enabledcore/providers/gemini/gemini.go (1)
119-124: Use the correct error message for raw-response unmarshal failures.
This branch is specifically raw-response decoding, so the error should beschemas.ErrProviderRawResponseUnmarshal:- return nil, nil, latency, providerUtils.NewBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, providerName) + return nil, nil, latency, providerUtils.NewBifrostOperationError(schemas.ErrProviderRawResponseUnmarshal, err, providerName)
♻️ Duplicate comments (7)
core/changelog.md (1)
1-4: Hyphenate “inter-provider” in changelog entry (Line 4).
Changeinter provider→inter-providerfor readability/grammar.-enhancement: improved internal inter provider conversions for integrations +enhancement: improved internal inter-provider conversions for integrationsui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
128-156: Add security warning to the UI description.Per the PR objectives: "Raw requests may contain sensitive data (API keys, user data); enabling this feature will store and return this information in logs and responses." The current description lacks this warning.
Apply this diff to add a security warning:
<div className="space-y-0.5"> <FormLabel>Include Raw Request</FormLabel> <p className="text-muted-foreground text-xs"> - Include the raw provider request alongside the parsed request for debugging and advanced use cases + Include the raw provider request alongside the parsed request for debugging and advanced use cases. + <span className="text-yellow-600 font-medium"> Warning: Raw requests may contain sensitive data including API keys and user information.</span> </p> </div>framework/streaming/chat.go (1)
304-316: FixRawRequestpointer semantics: don’t always return a non-nil pointer.
Right nowRawRequest: &rawRequestwill always be non-nil (even whenrawRequestis nil). Prefer only setting the pointer when present:- var rawRequest interface{} - if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ChatResponse.ExtraFields.RawRequest - } + var rawRequestPtr *interface{} + if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.ChatResponse.ExtraFields.RawRequest) + rawRequestPtr = &v + } return &ProcessedStreamResponse{ ... - RawRequest: &rawRequest, + RawRequest: rawRequestPtr, }, niltransports/changelog.md (1)
1-5: Hyphenate “inter-provider”.-enhancement: improved internal inter provider conversions for integrations +enhancement: improved internal inter-provider conversions for integrationsframework/streaming/audio.go (1)
170-182: FixRawRequestpointer semantics in final audio response.- var rawRequest interface{} - if result != nil && result.SpeechStreamResponse != nil && result.SpeechStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.SpeechStreamResponse.ExtraFields.RawRequest - } + var rawRequestPtr *interface{} + if result != nil && result.SpeechStreamResponse != nil && result.SpeechStreamResponse.ExtraFields.RawRequest != nil { + v := interface{}(result.SpeechStreamResponse.ExtraFields.RawRequest) + rawRequestPtr = &v + } return &ProcessedStreamResponse{ ... - RawRequest: &rawRequest, + RawRequest: rawRequestPtr, }, nilframework/streaming/responses.go (1)
723-726: FixRawRequestoptional semantics in final responses (currently always non-nil).
Apply the samerawRequestPtr *interface{}pattern in both final-chunk return sites (OpenAI-compatible and non-OpenAI), and setProcessedStreamResponse.RawRequestto nil when absent.Also applies to: 782-790, 866-879
core/providers/openai/openai.go (1)
173-175: Duplicate: redundant context evaluation in non-streaming handlers persists.As noted in the past review comment, this double-evaluation pattern remains unfixed across all non-streaming handlers. The provider evaluates
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)at lines 96, 227, 625, 1110, and 1471, passes the effective value to handlers, then handlers re-evaluate the same check at lines 173, 303, 704, 1190, and 1551.The streaming handlers (lines 597, 978-980, 1084-1086, 1428-1431) correctly use the parameter directly, showing this pattern was intentionally avoided there.
Apply the fix from the past review comment:
- if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }Update all affected handlers:
listModelsByKey,HandleOpenAITextCompletionRequest,HandleOpenAIChatCompletionRequest,HandleOpenAIResponsesRequest, andHandleOpenAIEmbeddingRequest.Also applies to: 303-305, 704-706, 1190-1192, 1551-1553
🧹 Nitpick comments (12)
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
145-148: Consider removing redundantform.trigger()call.The
form.trigger("send_back_raw_request")call is unnecessary here. This field is a simple boolean with no dependent validation rules—callingfield.onChange(checked)already updates form state and triggers validation through themode: "onChange"configuration.The same applies to line 177 for
send_back_raw_response.<Switch size="md" checked={field.value} - onCheckedChange={(checked) => { - field.onChange(checked); - form.trigger("send_back_raw_request"); - }} + onCheckedChange={field.onChange} />plugins/logging/operations.go (1)
89-135: Raw request logging is correctly gated bydisableContentLogging, but consider DRY + payload controls.
This looks consistent with the intent (only persist raw payloads when content logging is enabled). One suggestion: the raw-request/raw-response marshaling blocks are now duplicated across streaming and non-streaming; consider extracting a small helper (e.g.,marshalExtraField(updates, key, any)) to keep behavior identical (and make it easier to add size limits / redaction later).core/providers/openrouter/openrouter.go (1)
100-120: Consider adding nil check for rawRequest in GET endpoint.For
listModelsByKey,HandleProviderResponseis called withnilrequestBody, which correctly skips raw request capture. However, lines 113-115 still unconditionally assignrawRequesttoExtraFields.RawRequest. While this is not harmful (assigning nil), adding a nil check would be more explicit:// Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { openrouterResponse.ExtraFields.RawRequest = rawRequest }This aligns with the pattern used in some other providers and makes the intent clearer for GET endpoints where raw request capture is not applicable.
core/providers/bedrock/bedrock.go (1)
450-473: List-models raw-request assignment is always nil for GET endpoint.Similar to the Vertex provider,
listModelsByKeyis a GET request withrequestBody == nil, soHandleProviderResponsereturnsnilforrawRequest. The assignment at lines 466-468 will setExtraFields.RawRequest = nil, which is not harmful but unnecessary.Consider skipping raw request handling for this GET endpoint:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, bedrockResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + _, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, bedrockResponse, nil, false, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))And remove the raw request assignment block (lines 465-468).
core/providers/cohere/cohere.go (1)
221-241: List-models raw-request assignment is always nil for GET endpoint.Similar to other providers,
listModelsByKeyis a GET request withrequestBody == nil.HandleProviderResponsereturnsnilforrawRequest, so lines 234-236 will setExtraFields.RawRequest = nil.Consider skipping raw request handling for this GET endpoint:
- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, &cohereResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + _, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, &cohereResponse, nil, false, providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse))And remove the raw request assignment block (lines 233-236).
core/providers/utils/utils.go (1)
340-430: Update the doc comment + consider how to handle empty request bodies when raw-request capture is enabled.
The function-level comment still describes only raw response behavior, andrequestBody != nilwill treat empty bodies as “capture” and can turn on a hard error path.Suggested diff (doc + safer gate — only if empty bodies shouldn’t break requests):
-// If sendBackRawResponse is true, it returns the raw response interface, otherwise nil. +// If sendBackRawRequest / sendBackRawResponse are true, it attempts to also unmarshal and return +// the raw request / raw response (as interface{}), otherwise they are nil. func HandleProviderResponse[T any](responseBody []byte, response *T, requestBody []byte, sendBackRawRequest bool, sendBackRawResponse bool) (interface{}, interface{}, *schemas.BifrostError) { @@ - // Skip raw request capture if requestBody is nil (e.g., for GET requests) - shouldCaptureRawRequest := sendBackRawRequest && requestBody != nil + // Skip raw request capture if requestBody is nil (e.g., for GET requests) or empty. + shouldCaptureRawRequest := sendBackRawRequest && len(requestBody) > 0core/providers/elevenlabs/elevenlabs.go (2)
102-120: Avoid emittingraw_request: nullfor GET list-models; compute effective flags once.
BecauseHandleProviderResponse(..., requestBody=nil, ...)can’t capture a raw request,rawRequestwill be nil even when enabled; currentlyExtraFields.RawRequestcan still be set to nil (Line 112-115). Consider:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &elevenlabsResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + resp.Body(), + &elevenlabsResponse, + nil, // GET => no request body to capture + effectiveSendBackRawRequest, + effectiveSendBackRawResponse, + ) ... - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if effectiveSendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - response.ExtraFields.RawResponse = rawResponse - } + if effectiveSendBackRawResponse && rawResponse != nil { + response.ExtraFields.RawResponse = rawResponse + }
447-535: Define expected behavior for multipart Transcription raw-request capture (currently absent).
Transcriptionbuilds a multipart body (Line 469-495) but never attaches anyRawRequest. If the product expectation is “raw_request for every provider call”, you’ll need an explicit strategy here (e.g., capture a structured/sanitized representation of multipart fields and omit file bytes), or explicitly document that multipart raw-request capture is not supported to avoid huge payloads/PII leaks.core/providers/perplexity/perplexity.go (1)
144-166: Use “effective flags” once; only set RawRequest/RawResponse when non-nil.
This avoids repeated context lookups and prevents accidentalraw_request: null/raw_response: nulloutputs.core/providers/gemini/gemini.go (1)
162-181: Avoid settingRawRequestfor list-models when it cannot be captured (GET body is nil).
Same pattern as other GET list-models: compute effective flags once and only assign whenrawRequest != nil.core/providers/anthropic/anthropic.go (2)
203-221: Avoid emittingraw_request: nullin list-models (GET has no request body).
Consider skippingRawRequestassignment entirely for list-models, or guard withrawRequest != nil(and compute effective flags once).
274-295: Non-streaming raw-request/raw-response capture is consistently plumbed.
Small improvement: reuse computed effective flags and guard onrawRequest/rawResponse != nilbefore assigning to avoidnullfields.Also applies to: 335-356, 697-719
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(9 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(7 hunks)core/providers/gemini/gemini.go(12 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(4 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(6 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (18)
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- transports/bifrost-http/lib/account.go
- plugins/logging/main.go
- ui/app/workspace/providers/page.tsx
- framework/changelog.md
- framework/logstore/tables.go
- plugins/logging/changelog.md
- framework/configstore/rdb.go
- transports/bifrost-http/lib/config.go
- core/schemas/bifrost.go
- framework/configstore/clientconfig.go
- framework/streaming/types.go
- core/providers/mistral/mistral.go
- framework/configstore/tables/provider.go
- core/providers/groq/groq.go
- framework/streaming/transcription.go
- ui/lib/types/schemas.ts
- framework/configstore/migrations.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
core/providers/perplexity/perplexity.goframework/logstore/migrations.goui/app/workspace/providers/fragments/performanceFormFragment.tsxframework/streaming/responses.goframework/streaming/audio.gocore/providers/openrouter/openrouter.gocore/providers/elevenlabs/elevenlabs.gocore/providers/utils/utils.gocore/schemas/provider.gocore/providers/gemini/gemini.gotransports/bifrost-http/handlers/providers.gocore/providers/anthropic/anthropic.gocore/providers/parasail/parasail.gocore/providers/ollama/ollama.gocore/providers/vertex/vertex.goplugins/logging/operations.gocore/providers/sgl/sgl.gocore/providers/cohere/cohere.goui/lib/types/logs.tscore/providers/bedrock/bedrock.gocore/providers/azure/azure.goui/lib/types/config.tscore/providers/cerebras/cerebras.gocore/providers/openai/openai.goframework/streaming/chat.gocore/changelog.mdtransports/changelog.md
🧠 Learnings (3)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
core/providers/perplexity/perplexity.goframework/logstore/migrations.goframework/streaming/responses.goframework/streaming/audio.gocore/providers/openrouter/openrouter.gocore/providers/elevenlabs/elevenlabs.gocore/providers/utils/utils.gocore/schemas/provider.gocore/providers/gemini/gemini.gotransports/bifrost-http/handlers/providers.gocore/providers/anthropic/anthropic.gocore/providers/parasail/parasail.gocore/providers/ollama/ollama.gocore/providers/vertex/vertex.goplugins/logging/operations.gocore/providers/sgl/sgl.gocore/providers/cohere/cohere.gocore/providers/bedrock/bedrock.gocore/providers/azure/azure.gocore/providers/cerebras/cerebras.gocore/providers/openai/openai.goframework/streaming/chat.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/providers.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/openai.go
🧬 Code graph analysis (19)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
framework/logstore/migrations.go (2)
framework/migrator/migrator.go (3)
DefaultOptions(100-106)New(131-149)Migration(62-69)framework/logstore/tables.go (2)
Log(74-137)Log(140-142)
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (2)
ui/components/ui/form.tsx (4)
FormItem(161-161)FormLabel(162-162)FormControl(163-163)FormMessage(165-165)ui/components/ui/switch.tsx (1)
Switch(36-36)
framework/streaming/responses.go (2)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeResponses(17-17)core/schemas/provider.go (1)
Provider(282-309)
framework/streaming/audio.go (2)
framework/streaming/types.go (2)
ProcessedStreamResponse(116-124)StreamType(10-10)core/providers/gemini/types.go (1)
Type(778-778)
core/providers/openrouter/openrouter.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
core/providers/elevenlabs/elevenlabs.go (2)
core/providers/utils/utils.go (5)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)NewBifrostOperationError(501-512)core/schemas/provider.go (1)
ErrProviderRawResponseUnmarshal(32-32)
core/providers/utils/utils.go (2)
core/schemas/bifrost.go (4)
BifrostError(356-365)ErrorField(374-381)BifrostResponseExtraFields(286-296)BifrostContextKeySendBackRawRequest(118-118)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/parasail/parasail.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/ollama/ollama.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/vertex/vertex.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/azure/azure.go (3)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)core/schemas/utils.go (1)
IsAnthropicModel(1043-1045)core/providers/anthropic/anthropic.go (2)
AcquireAnthropicMessageResponse(46-50)ReleaseAnthropicMessageResponse(53-57)
core/providers/cerebras/cerebras.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/openai/openai.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(536-541)HandleProviderResponse(344-430)ParseAndSetRawRequest(432-439)
framework/streaming/chat.go (1)
framework/streaming/types.go (2)
ProcessedStreamResponse(116-124)StreamType(10-10)
🪛 LanguageTool
core/changelog.md
[grammar] ~4-~4: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
transports/changelog.md
[grammar] ~5-~5: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (30)
ui/lib/types/logs.ts (1)
250-284: Confirmraw_requestwire format matchesstring(and decide ifunknown/structured type is needed).
If the backend sometimes emits raw request as JSON (object) or binary/base64, typing this asstringwill be misleading and can cause downstream UI assumptions (e.g., pretty-print, truncation) to break.Also: since raw requests can contain secrets/PII, double-check any UI rendering path for
LogEntry.raw_requestapplies the same redaction/truncation rules asraw_response.ui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
35-36: LGTM!The
send_back_raw_requestfield is correctly wired through:
- Default values initialization (line 35)
- Form reset on provider change (line 51)
- Provider update payload (line 65)
The implementation follows the established pattern for
send_back_raw_response.Also applies to: 51-52, 65-66
ui/lib/types/config.ts (2)
166-167: LGTM!The
send_back_raw_requestfield is correctly added as an optional boolean adjacent tosend_back_raw_response, maintaining consistency in the type definitions.
190-191: LGTM!The field is consistently added to both
AddProviderRequestandUpdateProviderRequestinterfaces, ensuring the API contract supports the new feature for both creating and updating providers.Also applies to: 201-202
plugins/logging/operations.go (1)
305-313: Double-pointer nil check pattern is correct; no JSON serialization issue.The
streamResponse.RawRequest != nil && *streamResponse.RawRequest != nilpattern is appropriate here.RawRequestis typed as*interface{}inProcessedStreamResponse, and the dual check correctly guards against both a nil pointer and a nil interface value before dereferencing and marshaling. The value being marshaled is always aninterface{}containing decoded JSON data (never[]byte), sosonic.Marshal()produces proper JSON encoding, not base64. The end-to-end flow—from unmarshaling inParseAndSetRawRequest()through to storage as a text field in the logstore—is correctly implemented.framework/logstore/migrations.go (1)
43-45: Good wiring, but keep migration ordering in mind (stack awareness).
CallingmigrationAddRawRequestColumn()aftermigrationUpdateTimestampFormat()is fine, but given this PR is part of a Graphite stack, please double-check no downstack PR assumeslogs.raw_requestexists earlier in bootstrap/initialization flows (e.g., tests that run only a subset of migrations).core/schemas/provider.go (1)
31-31: LGTM!The new error constant
ErrProviderRawRequestUnmarshalfollows the existing naming pattern, and theSendBackRawRequestfield is properly placed alongsideSendBackRawResponsewith consistent JSON tagging and documentation.Also applies to: 241-241
core/providers/openrouter/openrouter.go (1)
137-258: LGTM!All request methods (TextCompletion, TextCompletionStream, ChatCompletion, ChatCompletionStream, Responses, ResponsesStream) correctly pass the effective
ShouldSendBackRawRequestflag to the underlying OpenAI-compatible handlers.core/providers/vertex/vertex.go (1)
438-501: LGTM!The ChatCompletion method correctly captures raw requests for both Anthropic and non-Anthropic model paths. The streaming methods (ChatCompletionStream, ResponsesStream) properly pass the effective
ShouldSendBackRawRequestflag to the underlying streaming handlers. The Responses method also correctly handles raw request capture for Anthropic models.Also applies to: 605-606, 681-682
core/providers/bedrock/bedrock.go (1)
551-554: LGTM!The raw request handling is correctly implemented across non-streaming methods (TextCompletion, ChatCompletion, Responses) using
ParseAndSetRawRequest, and in streaming methods (ChatCompletionStream, ResponsesStream) where the raw request is appropriately set only in the final response chunk.Also applies to: 730-733, 902-905, 971-974, 1059-1062
core/providers/cohere/cohere.go (1)
305-326: LGTM!The raw request handling is correctly implemented across all methods:
- Non-streaming methods (ChatCompletion, Responses, Embedding) properly capture raw request via
HandleProviderResponseand set it inExtraFieldswhen enabled.- Streaming methods (ChatCompletionStream, ResponsesStream) correctly use
ParseAndSetRawRequestto set the raw request only in the final chunk.Also applies to: 482-485, 531-554, 721-724, 774-795
core/providers/parasail/parasail.go (2)
17-23: Provider config wiring forsendBackRawRequestlooks consistent with the flag’s intent.
Good: unexported field, initialized fromconfig.SendBackRawRequest, no behavior change unless enabled.Also applies to: 48-54
62-75: PropagatingShouldSendBackRawRequest(...)into OpenAI-compatible handlers is clean/consistent.
This matches the pattern used across providers and keeps the context override behavior centralized inproviderUtils.Also applies to: 90-104, 110-132
core/providers/utils/utils.go (3)
432-439:ParseAndSetRawRequestis a good non-fatal helper; consider using it as the default policy everywhere raw-request parsing is “best-effort”.
Right nowHandleProviderResponsetreats raw-request unmarshal failures as fatal; that’s fine if you want “if you enable it, it must work”, but it’s stricter thanParseAndSetRawRequest. Worth double-checking intended UX.
533-541: Asymmetric override comment forShouldSendBackRawRequestis clear and matches existing raw-response semantics.
340-430: All call sites correctly use the newHandleProviderResponsesignature with proper error handling. When raw request body contains invalid JSON, the error is caught at line 372 and returned asBifrostErrorwithErrProviderRawRequestUnmarshal. All 30+ call sites across providers properly check the error return value and propagate failures. No issues found.core/providers/ollama/ollama.go (2)
18-24: Ollama provider config wiring forsendBackRawRequestis minimal and consistent.Also applies to: 55-61
69-86: Handler call updates look correct; nice that all relevant endpoints are covered (incl. streaming + embeddings).Also applies to: 88-102, 104-122, 124-138, 140-162, 190-204
core/providers/sgl/sgl.go (2)
18-24: SGL provider:sendBackRawRequestplumbing is clean and consistent with other providers.Also applies to: 55-61
69-83: Good coverage across sync + streaming paths for the new raw-request flag.Also applies to: 85-99, 101-119, 121-135, 137-159, 187-201
core/providers/cerebras/cerebras.go (2)
16-22: Cerebras provider: config-to-field wiring for raw request capture looks good.Also applies to: 47-54
61-75: Call-site propagation ofShouldSendBackRawRequest(...)is consistent (incl. streaming paths).Also applies to: 77-93, 95-118, 120-134, 136-162
core/providers/elevenlabs/elevenlabs.go (2)
23-30: Provider flag wiring looks correct.sendBackRawRequestis plumbed from config consistently withsendBackRawResponse.Also applies to: 55-62
256-258: Raw-request capture for Speech/SpeechStream finalization is consistent and safe (no hard-fail on parse).Also applies to: 436-440
core/providers/perplexity/perplexity.go (2)
19-25: Provider flag wiring looks correct.Also applies to: 50-56
193-201: Streaming plumbing for raw-request flag is consistent.core/providers/gemini/gemini.go (2)
23-30: Provider flag wiring looks correct.Also applies to: 55-62
270-303: Raw-request capture viaParseAndSetRawRequestis consistently wired across operations/streams.Also applies to: 424-431, 638-642, 681-687, 909-912
core/providers/anthropic/anthropic.go (2)
21-29: Provider flag wiring looks correct.Also applies to: 102-110
416-429: Streaming finalization correctly attaches raw request only when enabled.Also applies to: 656-659, 776-789, 991-995
Pratham-Mishra04
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
fc91124 to
f59299a
Compare
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
6951a77 to
2d213a4
Compare
TejasGhatte
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
2d213a4 to
a22dc43
Compare
TejasGhatte
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
f59299a to
86357e6
Compare
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
a22dc43 to
37e01b9
Compare
Pratham-Mishra04
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
86357e6 to
5f77971
Compare
TejasGhatte
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
37e01b9 to
9551b34
Compare
TejasGhatte
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
5f77971 to
921fdd8
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (8)
plugins/logging/main.go (1)
34-47: ResetRawRequestandRawResponseinputUpdateLogData()to prevent cross-request leakage from the sync.Pool.
UpdateLogDatais reused viasync.Pool, butputUpdateLogData()(pool.go, lines 34–45) fails to reset the newRawRequestandRawResponseinterface{} fields. This means a later request will inherit the previous request's raw payload, potentially retaining large HTTP bodies indefinitely and exposing sensitive data across requests. Adddata.RawRequest = nilanddata.RawResponse = nilto the reset function.core/providers/elevenlabs/elevenlabs.go (1)
70-123: Avoid settingRawRequestfor GET/list-models when there’s no request body.
HandleProviderResponse(..., requestBody=nil, ...)won’t capture raw request, so this can surfaceraw_request: null. Consider guarding onrawRequest != nil(or just skip entirely for GET).- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + // Set raw request if enabled (only if captured) + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }transports/bifrost-http/handlers/providers.go (1)
302-488: Bug risk: update flow can unintentionally reset raw flags to false when payload omits them.
Initializeconfig.SendBackRawRequest/config.SendBackRawResponsefromoldConfigRawbefore applyingpayload.*overrides.config := configstore.ProviderConfig{ Keys: oldConfigRaw.Keys, NetworkConfig: oldConfigRaw.NetworkConfig, ConcurrencyAndBufferSize: oldConfigRaw.ConcurrencyAndBufferSize, ProxyConfig: oldConfigRaw.ProxyConfig, CustomProviderConfig: oldConfigRaw.CustomProviderConfig, + SendBackRawRequest: oldConfigRaw.SendBackRawRequest, + SendBackRawResponse: oldConfigRaw.SendBackRawResponse, }framework/streaming/types.go (1)
115-124:ProcessedStreamResponse.RawRequesttype mismatch:*interface{}cannot be assigned tointerface{}.Line 123 declares
RawRequestas*interface{}, butBifrostResponseExtraFields.RawRequest(core/schemas/bifrost.go:293) is typed asinterface{}. The assignments at lines 160, 212, 230, 246, and 261 directly assignp.RawRequest(*interface{}) toExtraFields.RawRequest(interface{}) without dereferencing, which violates Go's type system. Either changeProcessedStreamResponse.RawRequesttointerface{}(remove the pointer) or dereference the value in assignments (*p.RawRequest).core/providers/openrouter/openrouter.go (1)
99-115: Don’t setExtraFields.RawRequestfor GET list-models when it wasn’t captured.
HandleProviderResponse(..., requestBody=nil, ...)will skip raw-request capture by design; assigningRawRequest = rawRequestunder the flag likely producesraw_request: null.- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - openrouterResponse.ExtraFields.RawRequest = rawRequest - } + // Set raw request only if it was actually captured + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + openrouterResponse.ExtraFields.RawRequest = rawRequest + }core/providers/gemini/gemini.go (1)
163-181: Avoid emitting/storingraw_request: nullon GET list-models.
requestBody=nilmeans raw request won’t be captured; only assign when non-nil.- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }core/providers/bedrock/bedrock.go (1)
450-473: Avoidraw_request: nullin Bedrock list-models (GET, no body).- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }core/providers/anthropic/anthropic.go (1)
203-221: Anthropic list-models: don’t set RawRequest when it can’t be captured (GET).- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }
♻️ Duplicate comments (15)
transports/changelog.md (1)
5-5: Hyphenate compound adjective (“inter-provider”).Line 5 should read “inter-provider” to match standard usage.
-enhancement: improved internal inter provider conversions for integrations +enhancement: improved internal inter-provider conversions for integrationscore/providers/cohere/cohere.go (4)
223-241: GETlistModelsByKey: raw-request capture withrequestBody=nilis unsafe / misleading when enabled.
IfSendBackRawRequest(or ctx override) is true,HandleProviderResponse(..., requestBody=nil, sendBackRawRequest=true)can’t faithfully capture a “raw request payload” for a GET and may error (or always producenullraw_request). Consider explicitly disabling raw-request capture for this endpoint (or capture method+URL+headers via a different mechanism) and also guard against settingRawRequestwhen it’s nil.@@ // Parse Cohere list models response var cohereResponse CohereListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, &cohereResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + // GET: no JSON request body; disable raw-request capture here unless you implement a GET-specific capture format. + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + body, + &cohereResponse, + nil, + false, + effectiveSendBackRawResponse, + ) if bifrostErr != nil { return nil, bifrostErr } @@ - // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + // Set raw request if enabled (and available) + if effectiveSendBackRawRequest && rawRequest != nil { response.ExtraFields.RawRequest = rawRequest } @@ - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if effectiveSendBackRawResponse && rawResponse != nil { response.ExtraFields.RawResponse = rawResponse }
305-326: Non-stream ChatCompletion: consider computing effective flags once + nil-guarding extra fields.
Current behavior is likely fine, but a singleeffectiveSendBackRawRequest/Responseimproves consistency (esp. if ctx is decorated later) and avoids emittingnullraw fields.
531-554: Responses: same “effective flags once + nil-guard” cleanup as ChatCompletion.
774-795: Embedding: same “effective flags once + nil-guard” cleanup as ChatCompletion.framework/logstore/migrations.go (1)
645-674: Make rollback idempotent + wrap migration errors with%w.
Rollback currently drops the column unconditionally (can fail if partially applied) and the migrate error uses%sinstead of%w. This was already called out in prior review comments—please apply that fix here.core/providers/azure/azure.go (1)
197-197: Redundant context evaluation persists across non-streaming handlers.The double-evaluation pattern flagged in the previous review remains unresolved. Each non-streaming endpoint computes the effective value at the call site (e.g., line 197:
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)), passes it toHandleProviderResponse, then re-evaluates the identical expression when settingExtraFields(e.g., line 211). Since context and configuration are unchanged, this adds unnecessary overhead.Recommended fix: Compute the effective values once and reuse them:
+ sendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + sendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, sendBackRawRequest, sendBackRawResponse) if bifrostErr != nil { return nil, bifrostErr } // ... set other fields ... - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if sendBackRawRequest && rawRequest != nil { response.ExtraFields.RawRequest = rawRequest } - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if sendBackRawResponse && rawResponse != nil { response.ExtraFields.RawResponse = rawResponse }Apply this pattern to all non-streaming endpoints:
listModelsByKey,TextCompletion,ChatCompletion,Responses, andEmbedding.Also applies to: 211-213, 274-274, 286-288, 411-411, 417-417, 430-432, 601-601, 607-607, 620-622, 760-760, 773-775
core/changelog.md (1)
1-4: Hyphenate “inter-provider” in changelog entry.
Line 4 reads “inter provider”; prior review already requested “inter-provider” for readability.-enhancement: improved internal inter provider conversions for integrations +enhancement: improved internal inter-provider conversions for integrationsui/app/workspace/providers/fragments/performanceFormFragment.tsx (1)
128-156: Add security warning to the UI description.Per PR objectives: "Raw requests may contain sensitive information (API keys, user data); enabling this feature will store and return this information in logs and responses." Users should be informed of the security implications before enabling this feature.
Apply this diff to add a security warning:
<div className="space-y-0.5"> <FormLabel>Include Raw Request</FormLabel> <p className="text-muted-foreground text-xs"> - Include the raw provider request alongside the parsed request for debugging and advanced use cases + Include the raw provider request alongside the parsed request for debugging and advanced use cases. + <span className="text-yellow-600 dark:text-yellow-500 font-medium"> Warning: Raw requests may contain sensitive data including API keys and user information.</span> </p> </div>framework/streaming/chat.go (1)
304-316:RawRequestpointer is always non-nil even when no raw request exists.The nil guard at line 305 correctly prevents panics. However, line 315 always assigns
&rawRequest, which meansProcessedStreamResponse.RawRequestwill be a non-nil pointer to a nilinterface{}when no raw request is present, rather than a truly nil pointer.This creates inconsistent semantics: consumers checking
if response.RawRequest != nilwill gettrueeven when there's no actual raw request data.- var rawRequest interface{} - if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ChatResponse.ExtraFields.RawRequest - } + var rawRequest *interface{} + if result != nil && result.ChatResponse != nil && result.ChatResponse.ExtraFields.RawRequest != nil { + v := result.ChatResponse.ExtraFields.RawRequest + rawRequest = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: streamType, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequest, }, nilframework/streaming/transcription.go (1)
189-201: KeepProcessedStreamResponse.RawRequestnil when absent (avoid pointer-to-nil-interface).
Current code setsRawRequest: &rawRequestunconditionally, soRawRequestis always non-nil on final chunks even whenExtraFields.RawRequestis missing. Prefer an optional pointer that’s only set when present.- var rawRequest interface{} - if result != nil && result.TranscriptionStreamResponse != nil && result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.TranscriptionStreamResponse.ExtraFields.RawRequest - } + var rawRequestPtr *interface{} + if result != nil && + result.TranscriptionStreamResponse != nil && + result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { + v := result.TranscriptionStreamResponse.ExtraFields.RawRequest + rawRequestPtr = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeTranscription, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequestPtr, }, nilframework/streaming/responses.go (2)
723-726: Fix RawRequest optional pointer semantics in OpenAI final-chunk path.- var rawRequest interface{} - if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequestPtr *interface{} + if result != nil && + result.ResponsesStreamResponse != nil && + result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + v := result.ResponsesStreamResponse.ExtraFields.RawRequest + rawRequestPtr = &v + } @@ return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeResponses, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequestPtr, }, nilAlso applies to: 782-790
866-879: Fix RawRequest optional pointer semantics in non-OpenAI final-chunk path.- var rawRequest interface{} - if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequestPtr *interface{} + if result != nil && + result.ResponsesStreamResponse != nil && + result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + v := result.ResponsesStreamResponse.ExtraFields.RawRequest + rawRequestPtr = &v + } @@ return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeResponses, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequestPtr, }, nilcore/providers/openai/openai.go (2)
96-99: Double evaluation pattern persists from past review.This exact issue was flagged in a previous review but remains unaddressed. The code evaluates the context-resolved boolean twice:
- Line 96: Computes effective value →
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)- Line 173: Re-evaluates inside
listModelsByKey→ShouldSendBackRawRequest(ctx, sendBackRawRequest)The parameter
sendBackRawRequestat line 123 is already the effective context-resolved value, but line 173 treats it as a raw config needing resolution. While functionally correct (context check happens first), this creates ambiguity about parameter semantics.Streaming handlers show the correct pattern (e.g., lines 597-599):
if sendBackRawRequest { providerUtils.ParseAndSetRawRequest(&response.ExtraFields, jsonBody) }Apply this fix to treat the parameter as already-effective:
- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + // Set raw request if enabled + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }Also applies to: 123-124, 173-175
227-228: Inconsistent pattern: non-streaming handlers re-evaluate while streaming handlers use parameters directly.All non-streaming handlers (TextCompletion, ChatCompletion, Responses, Embedding) follow the same double evaluation pattern flagged in the previous review, while streaming handlers correctly use parameters directly.
Non-streaming (lines 303-305, 704-706, 1190-1192, 1551-1553):
// Parameter is already effective, but re-evaluated: if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { response.ExtraFields.RawRequest = rawRequest }Streaming (lines 597-599, 1084-1086, 1429-1431) — correct approach:
// Uses parameter directly: if sendBackRawRequest { providerUtils.ParseAndSetRawRequest(&response.ExtraFields, jsonBody) }This inconsistency makes the codebase harder to maintain and creates ambiguity about whether parameters represent raw config or effective values.
Unify the pattern across all handlers by using parameters directly:
- if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }Apply this change at lines 303-305, 704-706, 1190-1192, and 1551-1553 to match the streaming code pattern.
Also applies to: 242-243, 303-305, 625-626, 640-641, 704-706, 1110-1111, 1125-1126, 1190-1192, 1471-1472, 1487-1488, 1551-1553
🧹 Nitpick comments (3)
framework/logstore/tables.go (1)
104-109: Consideromitempty(and redaction/size limits) forraw_request/raw_responseJSON fields.
Right now these will always appear in API responses for logs; if the intent is “only when enabled/present”, switching tojson:"raw_request,omitempty"/json:"raw_response,omitempty"reduces accidental surfacing. Separately, ensure the stack enforces redaction/limits since these fields can contain secrets/PII.ui/lib/types/schemas.ts (1)
367-420: Consider defaultingperformanceFormSchema.send_back_raw_requesttofalse(or making it optional).
Right now it’s required (Line 486) while other schemas treat it as optional, which can cause validation failures when the field is absent.export const performanceFormSchema = z.object({ concurrency_and_buffer_size: z.object({ concurrency: z.coerce .number("Concurrency must be a number") .min(1, "Concurrency must be greater than 0") .max(100000, "Concurrency must be less than 100000"), buffer_size: z.coerce .number("Buffer size must be a number") .min(1, "Buffer size must be greater than 0") .max(100000, "Buffer size must be less than 100000"), }), - send_back_raw_request: z.boolean(), + send_back_raw_request: z.boolean().default(false), send_back_raw_response: z.boolean(), });Also applies to: 474-488
core/providers/mistral/mistral.go (1)
105-123: Consider caching the helper result to avoid redundant evaluation.
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)is called twice: once at line 105 forHandleProviderResponseand again at line 116 for the conditional assignment. While functionally correct, this duplicates the context lookup.// Parse Mistral's response var mistralResponse MistralListModelsResponse - rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + shouldSendRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + shouldSendRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &mistralResponse, nil, shouldSendRawRequest, shouldSendRawResponse) if bifrostErr != nil { return nil, bifrostErr } // Create final response response := mistralResponse.ToBifrostListModelsResponse(key.Models) response.ExtraFields.Latency = latency.Milliseconds() // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if shouldSendRawRequest { response.ExtraFields.RawRequest = rawRequest } // Set raw response if enabled - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { + if shouldSendRawResponse { response.ExtraFields.RawResponse = rawResponse }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(9 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(7 hunks)core/providers/gemini/gemini.go(12 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(4 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(6 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
✅ Files skipped from review due to trivial changes (1)
- plugins/logging/changelog.md
🚧 Files skipped from review as they are similar to previous changes (16)
- core/providers/cerebras/cerebras.go
- ui/lib/types/logs.ts
- core/schemas/bifrost.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- framework/configstore/rdb.go
- transports/bifrost-http/lib/account.go
- framework/streaming/audio.go
- core/providers/groq/groq.go
- core/providers/ollama/ollama.go
- plugins/logging/operations.go
- core/providers/vertex/vertex.go
- framework/changelog.md
- core/schemas/provider.go
- framework/configstore/tables/provider.go
- transports/bifrost-http/lib/config.go
- ui/lib/types/config.ts
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
framework/configstore/clientconfig.gocore/providers/perplexity/perplexity.goframework/streaming/types.goframework/logstore/tables.goframework/logstore/migrations.goplugins/logging/main.goui/lib/types/schemas.tscore/providers/mistral/mistral.goframework/streaming/transcription.gocore/providers/elevenlabs/elevenlabs.goframework/streaming/responses.goframework/streaming/chat.gocore/providers/anthropic/anthropic.goui/app/workspace/providers/page.tsxframework/configstore/migrations.gocore/providers/azure/azure.gocore/providers/openrouter/openrouter.gocore/providers/sgl/sgl.gocore/providers/parasail/parasail.gocore/providers/utils/utils.gotransports/bifrost-http/handlers/providers.gocore/providers/cohere/cohere.gocore/providers/gemini/gemini.goui/app/workspace/providers/fragments/performanceFormFragment.tsxcore/providers/openai/openai.gocore/providers/bedrock/bedrock.gocore/changelog.mdtransports/changelog.md
🧠 Learnings (3)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
framework/configstore/clientconfig.gocore/providers/perplexity/perplexity.goframework/streaming/types.goframework/logstore/tables.goframework/logstore/migrations.goplugins/logging/main.gocore/providers/mistral/mistral.goframework/streaming/transcription.gocore/providers/elevenlabs/elevenlabs.goframework/streaming/responses.goframework/streaming/chat.gocore/providers/anthropic/anthropic.goframework/configstore/migrations.gocore/providers/azure/azure.gocore/providers/openrouter/openrouter.gocore/providers/sgl/sgl.gocore/providers/parasail/parasail.gocore/providers/utils/utils.gotransports/bifrost-http/handlers/providers.gocore/providers/cohere/cohere.gocore/providers/gemini/gemini.gocore/providers/openai/openai.gocore/providers/bedrock/bedrock.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/providers.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/openai.go
🧬 Code graph analysis (18)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
framework/logstore/migrations.go (2)
framework/migrator/migrator.go (3)
DefaultOptions(100-106)New(131-149)Migration(62-69)framework/logstore/tables.go (2)
Log(74-137)Log(140-142)
core/providers/mistral/mistral.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
core/providers/elevenlabs/elevenlabs.go (2)
core/providers/utils/utils.go (5)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)NewBifrostOperationError(501-512)core/schemas/provider.go (1)
ErrProviderRawResponseUnmarshal(32-32)
framework/streaming/responses.go (1)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeResponses(17-17)
framework/streaming/chat.go (1)
framework/streaming/types.go (3)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)
core/providers/anthropic/anthropic.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
framework/configstore/migrations.go (2)
framework/migrator/migrator.go (3)
New(131-149)DefaultOptions(100-106)Migration(62-69)framework/configstore/tables/provider.go (2)
TableProvider(15-44)TableProvider(47-47)
core/providers/azure/azure.go (2)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)core/schemas/utils.go (1)
IsAnthropicModel(1043-1045)
core/providers/openrouter/openrouter.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/parasail/parasail.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/utils/utils.go (2)
core/schemas/bifrost.go (4)
BifrostError(356-365)ErrorField(374-381)BifrostResponseExtraFields(286-296)BifrostContextKeySendBackRawRequest(118-118)core/schemas/provider.go (2)
ErrProviderRawRequestUnmarshal(31-31)ErrProviderRawResponseUnmarshal(32-32)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
ui/app/workspace/providers/fragments/performanceFormFragment.tsx (2)
ui/components/ui/form.tsx (4)
FormItem(161-161)FormLabel(162-162)FormControl(163-163)FormMessage(165-165)ui/components/ui/switch.tsx (1)
Switch(36-36)
core/providers/openai/openai.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(536-541)HandleProviderResponse(344-430)ParseAndSetRawRequest(432-439)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
🪛 LanguageTool
core/changelog.md
[grammar] ~4-~4: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
transports/changelog.md
[grammar] ~5-~5: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (47)
core/providers/cohere/cohere.go (4)
67-74: CohereProvider: addedsendBackRawRequestwiring is straightforward.
105-112: Constructor correctly initializessendBackRawRequestfrom config.
482-485: Streaming end-of-stream raw-request capture viaParseAndSetRawRequestis the right approach.
721-724: ResponsesStream: end-of-stream raw-request capture viaParseAndSetRawRequestlooks good.plugins/logging/main.go (1)
498-505: Raw request capture is correctly gated behinddisableContentLogging.
PopulatingupdateData.RawRequestfromextraFields.RawRequestalongsideRawResponsematches the feature intent and avoids logging when content logging is disabled.framework/logstore/migrations.go (1)
43-45: Migration wiring order looks fine.
AddingmigrationAddRawRequestColumnafter the timestamp migration is reasonable and keeps schema evolution linear.core/providers/azure/azure.go (1)
26-26: LGTM: Clean addition of sendBackRawRequest support.The new field follows the established pattern for
sendBackRawResponseand is correctly wired from the provider configuration.Also applies to: 51-51
framework/configstore/clientconfig.go (1)
146-157: Config hash now correctly accounts forSendBackRawRequest.
This closes the drift hole and keeps parity withSendBackRawResponse.Also applies to: 159-215
core/providers/perplexity/perplexity.go (1)
18-57: Perplexity raw-request wiring looks consistent (ctx override + ExtraFields propagation).Also applies to: 127-169, 171-202
framework/configstore/migrations.go (1)
16-105: Migration ordering + hash prepopulation are correctly updated forsend_back_raw_request.
Please ensure this migration’s add/drop behavior is exercised on all supported DBs.Also applies to: 1294-1321, 1341-1351
core/providers/elevenlabs/elevenlabs.go (1)
175-289: Speech/SpeechStream raw-request finalization + Transcription raw-response assignment look correct.
Good use ofParseAndSetRawRequestfor JSON request bodies and guarded raw-response unmarshalling for transcription.Also applies to: 291-445, 447-536
transports/bifrost-http/handlers/providers.go (1)
54-65: Add/get responses correctly surfacesend_back_raw_requestfrom stored config.
Propagation throughgetProviderResponseFromConfigis consistent.Also applies to: 170-300, 842-861
core/providers/utils/utils.go (3)
344-430: LGTM! Well-structured concurrent parsing with proper error handling.The
HandleProviderResponsefunction correctly:
- Skips raw request capture when
requestBodyis nil (e.g., GET requests)- Uses concurrent goroutines efficiently with dynamic count
- Handles error paths correctly for both raw request and raw response when both flags are enabled (addressed from past review)
432-439: LGTM! Simple helper for streaming raw request capture.The
ParseAndSetRawRequesthelper provides a non-fatal path for setting raw request data, appropriately logging failures without blocking execution.
533-541: LGTM! Asymmetric override behavior is now documented.The comment on lines 534-535 correctly documents the intentional asymmetric behavior where context can only promote
false→true, matching the pattern inShouldSendBackRawResponse. This addresses the past review comment.core/providers/sgl/sgl.go (8)
22-23: LGTM! Field addition follows established pattern.
55-61: LGTM! Proper initialization from config.
79-82: LGTM! Consistent use of context-aware helper.
95-98: LGTM!
112-118: LGTM!
130-134: LGTM!
150-158: LGTM!
197-200: LGTM!core/providers/mistral/mistral.go (5)
21-23: LGTM! Field addition follows established pattern.
53-59: LGTM! Proper initialization from config.
161-165: LGTM! Consistent use of context-aware helper.
185-193: LGTM!
234-237: LGTM!core/providers/parasail/parasail.go (5)
21-23: LGTM! Field addition follows established pattern.
48-54: LGTM! Proper initialization from config.
72-75: LGTM! Consistent use of context-aware helper.
99-103: LGTM!
123-131: LGTM!ui/app/workspace/providers/page.tsx (2)
74-78: LGTM! Initial state correctly includes new field.The
send_back_raw_request: undefinedinitialization aligns with the new provider configuration capability introduced across the PR stack.
225-225: LGTM! Minor formatting cleanup.ui/app/workspace/providers/fragments/performanceFormFragment.tsx (3)
35-37: LGTM! Default values correctly initialized.
51-53: LGTM! Form reset includes new field.
65-67: LGTM! Submit handler includes new field.core/providers/openrouter/openrouter.go (1)
22-24: Provider-level flag wiring looks consistent.Also applies to: 49-55
core/providers/gemini/gemini.go (2)
23-62: Provider flag wiring is straightforward.
270-303: Raw request capture for non-GET + streaming finalization looks consistent with the stack’s intent.Also applies to: 424-426, 638-642, 681-687, 909-915
core/providers/bedrock/bedrock.go (2)
28-80: Flag wiring via ProviderConfig looks consistent.
551-555: Raw request attachment viaParseAndSetRawRequestin non-GET + streaming finalization looks good.Also applies to: 730-734, 902-906, 971-975, 1058-1064
core/providers/anthropic/anthropic.go (2)
20-29: Provider flag wiring looks consistent across providers in the stack.Also applies to: 102-110
397-411: Streaming helpers: newsendBackRawRequestparam + final-chunk raw-request attachment is coherent.Also applies to: 416-429, 656-663, 758-771, 774-789, 991-999
core/providers/openai/openai.go (2)
28-28: LGTM: Field declaration and initialization follow existing pattern.The
sendBackRawRequestfield is correctly declared and initialized from the config, mirroring thesendBackRawResponsepattern.Also applies to: 65-65
333-334: LGTM: Streaming handlers implement the correct pattern.All streaming handlers (TextCompletionStream, ChatCompletionStream, ResponsesStream) correctly:
- Resolve the effective value at call sites using
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)- Pass the effective boolean to streaming functions
- Use the parameter directly without re-evaluation:
if sendBackRawRequest { ... }- Use the
ParseAndSetRawRequesthelper for end-of-stream raw request captureThis pattern is cleaner and should be adopted by non-streaming handlers as well.
Also applies to: 351-352, 597-599, 736-737, 756-757, 978-980, 1084-1086, 1220-1221, 1239-1240, 1429-1431
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
framework/streaming/types.go (1)
115-264: FixRawRequesttype propagation: don’t store*interface{}insideinterface{}.
Right now,ExtraFields.RawRequest = p.RawRequestmakes the dynamic type*interface{}; downstream code expectingmap[string]any/[]anyvia type assertion will fail. Prefer assigning*p.RawRequest.- if p.RawRequest != nil { - resp.ChatResponse.ExtraFields.RawRequest = p.RawRequest - } + if p.RawRequest != nil { + resp.ChatResponse.ExtraFields.RawRequest = *p.RawRequest + }Apply the same change to all stream-type branches in this method.
plugins/logging/main.go (1)
34-47: PooledUpdateLogDatais missing reset forRawRequestandRawResponsefields.The
putUpdateLogData()function inplugins/logging/pool.godoes not resetRawRequestandRawResponsebefore returning the struct to the pool. Since these fields can hold large request/response bodies, they will be retained in memory across requests, causing a memory leak. Add the following resets:func (p *LoggerPlugin) putUpdateLogData(data *UpdateLogData) { // Reset all fields to avoid memory leaks data.Status = "" data.TokenUsage = nil data.ChatOutput = nil data.ResponsesOutput = nil data.ErrorDetails = nil data.SpeechOutput = nil data.TranscriptionOutput = nil data.EmbeddingOutput = nil data.Cost = nil + data.RawRequest = nil + data.RawResponse = nil p.updateDataPool.Put(data) }core/providers/elevenlabs/elevenlabs.go (1)
102-120: Avoid settingExtraFields.RawRequestto nil on GET/v1/models.
HandleProviderResponse(..., requestBody=nil, ...)will never populaterawRequest, so the current assignment can add a misleadingraw_request: nullwhen enabled. Consider guarding withrawRequest != nil(or passfalsefor raw-request capture on GET endpoints).- // Set raw request if enabled - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + // Set raw request if enabled (GET has no JSON body, so rawRequest will be nil) + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }core/providers/bedrock/bedrock.go (1)
451-473: ListModels is a GET; don’t populateRawRequestfrom a nil body.
SincerequestBodyisnil,HandleProviderResponsewon’t capture a raw request. Guard the assignment (or passfalsefor raw-request capture here) to avoidraw_request: null.- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }
♻️ Duplicate comments (12)
core/changelog.md (1)
1-4: LGTM - changelog entries document the new features appropriately.The entries accurately reflect the changes in this PR stack. The hyphenation issue on line 4 ("inter provider" → "inter-provider") was already flagged in a previous review.
transports/changelog.md (1)
1-5: LGTM - changelog entries document the transport layer changes.The entries accurately reflect the raw request/response streaming support and reasoning enhancements. The hyphenation issue on line 5 was already flagged in a previous review.
framework/logstore/migrations.go (1)
660-671: Make rollback idempotent and preserve error chain.The rollback function unconditionally calls
DropColumn, which will fail if the column doesn't exist (e.g., on partial migration failure). This was flagged in a previous review. Additionally, the error wrapping uses%sinstead of%w, which loses the error chain.Apply this diff to fix both issues:
Rollback: func(tx *gorm.DB) error { tx = tx.WithContext(ctx) migrator := tx.Migrator() - if err := migrator.DropColumn(&Log{}, "raw_request"); err != nil { - return err + if migrator.HasColumn(&Log{}, "raw_request") { + if err := migrator.DropColumn(&Log{}, "raw_request"); err != nil { + return err + } } return nil }, }}) err := m.Migrate() if err != nil { - return fmt.Errorf("error while adding raw request column: %s", err.Error()) + return fmt.Errorf("error while adding raw request column: %w", err) }core/providers/vertex/vertex.go (1)
207-212: List models raw request accumulation will produce nil entries for GET requests.Since
listModelsByKeyis a GET request with no request body,HandleProviderResponse(line 203) is called withnilas the requestBody parameter, which returnsnilforrawRequest. The code still appends this nil value torawRequestson each page iteration, resulting in[null, null, ...]in the response.This was flagged in a previous review. Consider adding a nil check before appending:
- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && rawRequest != nil { rawRequests = append(rawRequests, rawRequest) }And for the final assignment (lines 230-232):
- if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { + if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) && len(rawRequests) > 0 { response.ExtraFields.RawRequest = rawRequests }ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
364-388: Raw Request viewer fallback is correct (falls back tolog.raw_request).
This avoids the previous “Raw Response shown under Raw Request” confusion.framework/streaming/responses.go (1)
723-726: KeepProcessedStreamResponse.RawRequestnil when absent (avoid&rawRequestalways).
This is the same optional-pointer issue previously raised: taking&rawRequestmakes the pointer non-nil even when the value is nil.- var rawRequest interface{} - if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.ResponsesStreamResponse.ExtraFields.RawRequest - } + var rawRequestPtr *interface{} + if result != nil && result.ResponsesStreamResponse != nil && result.ResponsesStreamResponse.ExtraFields.RawRequest != nil { + v := result.ResponsesStreamResponse.ExtraFields.RawRequest + rawRequestPtr = &v + } @@ - RawRequest: &rawRequest, + RawRequest: rawRequestPtr,Apply the same pattern to the non-OpenAI final path (around Lines 866-879).
Also applies to: 789-790, 866-869, 878-879
framework/streaming/transcription.go (1)
189-201: Don’t always setProcessedStreamResponse.RawRequest; keep itnilwhen absent (regresses optional semantics).
Right nowRawRequest: &rawRequestis always non-nil even whenExtraFields.RawRequestis nil, so callers may seeraw_request: nulland need double-nil checks.- var rawRequest interface{} - if result != nil && result.TranscriptionStreamResponse != nil && result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { - rawRequest = result.TranscriptionStreamResponse.ExtraFields.RawRequest - } + var rawRequestPtr *interface{} + if result != nil && result.TranscriptionStreamResponse != nil && result.TranscriptionStreamResponse.ExtraFields.RawRequest != nil { + v := result.TranscriptionStreamResponse.ExtraFields.RawRequest + rawRequestPtr = &v + } return &ProcessedStreamResponse{ Type: StreamResponseTypeFinal, RequestID: requestID, StreamType: StreamTypeTranscription, Provider: provider, Model: model, Data: data, - RawRequest: &rawRequest, + RawRequest: rawRequestPtr, }, nilcore/providers/elevenlabs/elevenlabs.go (1)
527-533: Raw response now correctly assigned in Transcription path.core/providers/cohere/cohere.go (1)
221-241: GETListModels: current raw-request wiring can only produceraw_request: null.
BecauserequestBodyisnil,HandleProviderResponseskips raw-request capture, butExtraFields.RawRequestis still set when enabled. GuardrawRequest != nil(or explicitly disable raw-request capture for this GET).- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(body, &cohereResponse, nil, - providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), - providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), - ) + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + body, &cohereResponse, + nil, // GET: no JSON request body + false, // or: effectiveSendBackRawRequest, if you later decide to capture something non-body for GET + effectiveSendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if effectiveSendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }core/providers/azure/azure.go (2)
195-218: GETlistModelsByKey: don’t assignRawRequestfrom a nil request body + avoid doubleShould*evaluation.
This is the same pattern previously flagged:HandleProviderResponse(..., requestBody=nil, sendBackRawRequest=ShouldSendBackRawRequest(...))can’t produce a raw request, and re-callingShouldSendBackRawRequestagain before assignment is redundant.- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, - providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), - providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), - ) + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, azureResponse, nil, false, effectiveSendBackRawResponse) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if effectiveSendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }
272-294: Non-stream handlers: consider treating passed booleans as already-effective (skip re-callingShould*).
This is functionally correct today, but the repetition adds noise and was previously raised.Also applies to: 404-437, 594-627, 758-781
core/providers/openai/openai.go (1)
96-96: Double evaluation pattern persists—treat parameters as already-effective values.A past review comment flagged this exact issue, but the pattern remains:
- Line 96 computes
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)and passes it tolistModelsByKey- Line 173 re-evaluates
ShouldSendBackRawRequest(ctx, sendBackRawRequest)using the parameterThis creates ambiguity: if
sendBackRawRequestis a parameter, it should represent an already-effective value. Re-evaluating it withShouldSendBackRawRequestis redundant and confusing.The same pattern appears throughout this file:
- TextCompletion: lines 227/303-305
- ChatCompletion: lines 625/704-706
- Responses: lines 1110/1190-1192
- Embedding: lines 1471/1551-1553
Recommendation (from past review): Compute the effective value once, pass it as a parameter, and use it directly without re-evaluation:
- if providerUtils.ShouldSendBackRawRequest(ctx, sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if sendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }This pattern should be applied consistently across all non-streaming handlers in both OpenAI and Anthropic providers to eliminate ambiguity and improve code clarity.
Also applies to: lines 227/303-305, 625/704-706, 1110/1190-1192, 1471/1551-1553
Based on past review comments: A previous review flagged this exact pattern at lines 96-99, 123-124, 173-175.
Also applies to: 173-175
🧹 Nitpick comments (6)
ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
364-388: Consider memoizing JSON pretty-print to avoid re-parsing on every render.
Not required, butJSON.parseinside render can be a small perf hit for large payloads (especially when opening/closing the sheet).framework/logstore/tables.go (1)
104-109: Consideromitemptyonraw_request/raw_responseJSON fields to reduce payload noise.
Right now empty strings will serialize as""whenever the column exists but wasn’t populated.- RawRequest string `gorm:"type:text" json:"raw_request"` - RawResponse string `gorm:"type:text" json:"raw_response"` + RawRequest string `gorm:"type:text" json:"raw_request,omitempty"` + RawResponse string `gorm:"type:text" json:"raw_response,omitempty"`core/providers/perplexity/perplexity.go (1)
144-166: Avoid double-evaluatingShouldSendBackRawRequest/Response(minor cleanup).
Compute the booleans once and reuse forHandleProviderResponseandExtraFieldsassignment.core/providers/openrouter/openrouter.go (1)
99-120: Prefer computing effective flags once (and avoid settingRawRequestwhen nil).
Keeps capture/exposure consistent and avoids emittingraw_request: nullfor GET paths.- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(responseBody, &openrouterResponse, nil, - providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), - providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse), - ) + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse( + responseBody, &openrouterResponse, nil, effectiveSendBackRawRequest, effectiveSendBackRawResponse, + ) @@ - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - openrouterResponse.ExtraFields.RawRequest = rawRequest - } + if effectiveSendBackRawRequest && rawRequest != nil { + openrouterResponse.ExtraFields.RawRequest = rawRequest + } @@ - if providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) { - openrouterResponse.ExtraFields.RawResponse = rawResponse - } + if effectiveSendBackRawResponse && rawResponse != nil { + openrouterResponse.ExtraFields.RawResponse = rawResponse + }core/providers/gemini/gemini.go (1)
162-181: Consider: reuse effective flags once; gateRawRequestonrawRequest != nil.
Particularly for list-models (GET) where request body is nil and rawRequest will remain nil.Also applies to: 268-302
core/providers/anthropic/anthropic.go (1)
204-204: Prefer treating parameters as already-effective values to avoid redundant evaluation.Lines 204 and 214 both call
ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)with identical arguments:
- Line 204 computes the effective value and passes it to
HandleProviderResponse- Line 214 re-evaluates the same context and flag to decide whether to set
ExtraFields.RawRequestThis double evaluation creates ambiguity about whether parameters represent the raw configuration value or the context-resolved effective value. The same pattern appears in other handlers (TextCompletion lines 274/288, ChatCompletion lines 335/349, Responses lines 697/712).
Recommendation: Compute the effective value once, pass it as a parameter, and treat that parameter as already effective. Remove internal
ShouldSendBackRawRequestcalls and use the parameter directly with a nil check:- rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &anthropicResponse, nil, providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest), providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)) + effectiveSendBackRawRequest := providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) + effectiveSendBackRawResponse := providerUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse) + rawRequest, rawResponse, bifrostErr := providerUtils.HandleProviderResponse(resp.Body(), &anthropicResponse, nil, effectiveSendBackRawRequest, effectiveSendBackRawResponse) if bifrostErr != nil { return nil, bifrostErr } // ... - if providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest) { - response.ExtraFields.RawRequest = rawRequest - } + if effectiveSendBackRawRequest && rawRequest != nil { + response.ExtraFields.RawRequest = rawRequest + }Apply this pattern consistently across all non-streaming handlers (TextCompletion, ChatCompletion, Responses, listModelsByKey) to improve code clarity.
Also applies to: lines 274/287-290, 335/348-351, 697/711-714
Also applies to: 213-216
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (45)
core/changelog.md(1 hunks)core/providers/anthropic/anthropic.go(16 hunks)core/providers/azure/azure.go(17 hunks)core/providers/bedrock/bedrock.go(9 hunks)core/providers/cerebras/cerebras.go(7 hunks)core/providers/cohere/cohere.go(12 hunks)core/providers/elevenlabs/elevenlabs.go(7 hunks)core/providers/gemini/gemini.go(12 hunks)core/providers/groq/groq.go(5 hunks)core/providers/mistral/mistral.go(7 hunks)core/providers/ollama/ollama.go(8 hunks)core/providers/openai/openai.go(32 hunks)core/providers/openrouter/openrouter.go(10 hunks)core/providers/parasail/parasail.go(5 hunks)core/providers/perplexity/perplexity.go(5 hunks)core/providers/sgl/sgl.go(8 hunks)core/providers/utils/utils.go(3 hunks)core/providers/vertex/vertex.go(14 hunks)core/schemas/bifrost.go(2 hunks)core/schemas/provider.go(2 hunks)framework/changelog.md(1 hunks)framework/configstore/clientconfig.go(2 hunks)framework/configstore/migrations.go(4 hunks)framework/configstore/rdb.go(4 hunks)framework/configstore/tables/provider.go(1 hunks)framework/logstore/migrations.go(2 hunks)framework/logstore/tables.go(1 hunks)framework/streaming/audio.go(1 hunks)framework/streaming/chat.go(1 hunks)framework/streaming/responses.go(3 hunks)framework/streaming/transcription.go(1 hunks)framework/streaming/types.go(6 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/main.go(2 hunks)plugins/logging/operations.go(2 hunks)transports/bifrost-http/handlers/providers.go(8 hunks)transports/bifrost-http/lib/account.go(1 hunks)transports/bifrost-http/lib/config.go(2 hunks)transports/changelog.md(1 hunks)ui/app/workspace/logs/views/logDetailsSheet.tsx(1 hunks)ui/app/workspace/providers/fragments/performanceFormFragment.tsx(4 hunks)ui/app/workspace/providers/page.tsx(2 hunks)ui/lib/types/config.ts(3 hunks)ui/lib/types/logs.ts(1 hunks)ui/lib/types/schemas.ts(5 hunks)
✅ Files skipped from review due to trivial changes (1)
- plugins/logging/changelog.md
🚧 Files skipped from review as they are similar to previous changes (13)
- core/providers/cerebras/cerebras.go
- ui/app/workspace/providers/page.tsx
- ui/app/workspace/providers/fragments/performanceFormFragment.tsx
- plugins/logging/operations.go
- framework/streaming/audio.go
- core/providers/ollama/ollama.go
- core/providers/mistral/mistral.go
- core/providers/utils/utils.go
- ui/lib/types/logs.ts
- transports/bifrost-http/lib/account.go
- framework/changelog.md
- framework/streaming/chat.go
- ui/lib/types/config.ts
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/logging/main.gocore/providers/elevenlabs/elevenlabs.gocore/schemas/provider.goframework/streaming/types.gocore/providers/perplexity/perplexity.goframework/configstore/rdb.goframework/logstore/migrations.gocore/providers/cohere/cohere.goframework/configstore/tables/provider.gocore/providers/vertex/vertex.gocore/providers/sgl/sgl.gocore/schemas/bifrost.gotransports/bifrost-http/handlers/providers.goui/app/workspace/logs/views/logDetailsSheet.tsxcore/providers/gemini/gemini.gocore/providers/bedrock/bedrock.goframework/logstore/tables.gotransports/bifrost-http/lib/config.gocore/providers/azure/azure.goframework/configstore/migrations.goframework/streaming/transcription.gocore/providers/anthropic/anthropic.goframework/configstore/clientconfig.goui/lib/types/schemas.tscore/providers/parasail/parasail.gocore/providers/openai/openai.goframework/streaming/responses.gocore/providers/groq/groq.gocore/providers/openrouter/openrouter.gocore/changelog.mdtransports/changelog.md
🧠 Learnings (3)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
plugins/logging/main.gocore/providers/elevenlabs/elevenlabs.gocore/schemas/provider.goframework/streaming/types.gocore/providers/perplexity/perplexity.goframework/configstore/rdb.goframework/logstore/migrations.gocore/providers/cohere/cohere.goframework/configstore/tables/provider.gocore/providers/vertex/vertex.gocore/providers/sgl/sgl.gocore/schemas/bifrost.gotransports/bifrost-http/handlers/providers.gocore/providers/gemini/gemini.gocore/providers/bedrock/bedrock.goframework/logstore/tables.gotransports/bifrost-http/lib/config.gocore/providers/azure/azure.goframework/configstore/migrations.goframework/streaming/transcription.gocore/providers/anthropic/anthropic.goframework/configstore/clientconfig.gocore/providers/parasail/parasail.gocore/providers/openai/openai.goframework/streaming/responses.gocore/providers/groq/groq.gocore/providers/openrouter/openrouter.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/providers.gotransports/bifrost-http/lib/config.go
📚 Learning: 2025-12-11T11:58:25.307Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: core/providers/openai/responses.go:42-84
Timestamp: 2025-12-11T11:58:25.307Z
Learning: In core/providers/openai/responses.go (and related OpenAI response handling), document and enforce the API format constraint: if ResponsesReasoning != nil and the response contains content blocks, all content blocks should be treated as reasoning blocks by default. Implement type guards or parsing logic accordingly, and add unit tests to verify that when ResponsesReasoning is non-nil, content blocks are labeled as reasoning blocks. Include clear comments in the code explaining the rationale and ensure downstream consumers rely on this behavior.
Applied to files:
core/providers/openai/openai.go
🧬 Code graph analysis (16)
core/providers/elevenlabs/elevenlabs.go (2)
core/providers/utils/utils.go (5)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)NewBifrostOperationError(501-512)core/schemas/provider.go (1)
ErrProviderRawResponseUnmarshal(32-32)
core/providers/perplexity/perplexity.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
framework/logstore/migrations.go (2)
framework/migrator/migrator.go (3)
DefaultOptions(100-106)New(131-149)Migration(62-69)framework/logstore/tables.go (2)
Log(74-137)Log(140-142)
core/providers/cohere/cohere.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/sgl/sgl.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
ui/app/workspace/logs/views/codeEditor.tsx (1)
CodeEditor(82-245)
core/providers/gemini/gemini.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/bedrock/bedrock.go (1)
core/providers/utils/utils.go (4)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)ParseAndSetRawRequest(432-439)
core/providers/azure/azure.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
framework/configstore/migrations.go (1)
framework/configstore/tables/provider.go (2)
TableProvider(15-44)TableProvider(47-47)
framework/streaming/transcription.go (1)
framework/streaming/types.go (2)
ProcessedStreamResponse(116-124)StreamType(10-10)
core/providers/parasail/parasail.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/openai/openai.go (1)
core/providers/utils/utils.go (3)
ShouldSendBackRawRequest(536-541)HandleProviderResponse(344-430)ParseAndSetRawRequest(432-439)
framework/streaming/responses.go (1)
framework/streaming/types.go (4)
ProcessedStreamResponse(116-124)StreamResponseTypeFinal(24-24)StreamType(10-10)StreamTypeResponses(17-17)
core/providers/groq/groq.go (1)
core/providers/utils/utils.go (1)
ShouldSendBackRawRequest(536-541)
core/providers/openrouter/openrouter.go (1)
core/providers/utils/utils.go (3)
HandleProviderResponse(344-430)ShouldSendBackRawRequest(536-541)ShouldSendBackRawResponse(544-549)
🪛 LanguageTool
core/changelog.md
[grammar] ~4-~4: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
transports/changelog.md
[grammar] ~5-~5: Use a hyphen to join words.
Context: ...api enhancement: improved internal inter provider conversions for integrations
(QB_NEW_EN_HYPHEN)
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (50)
core/schemas/provider.go (2)
31-31: LGTM - Error constant follows established naming conventions.The new
ErrProviderRawRequestUnmarshalconstant is consistent with the existingErrProviderRawResponseUnmarshalpattern.
241-241: LGTM - Field addition is consistent with existing patterns.The
SendBackRawRequestfield mirrors the existingSendBackRawResponsefield with consistent naming, JSON tagging, and placement.framework/configstore/clientconfig.go (2)
153-153: LGTM - Field addition mirrors existing pattern.The
SendBackRawRequestfield is correctly added with appropriate JSON tag and comment, consistent withSendBackRawResponse.
204-207: LGTM - Hash implementation is consistent.The
SendBackRawRequesthashing follows the same pattern asSendBackRawResponse(lines 209-212), ensuring config drift detection works correctly for this new field. This addresses the concern from the previous review.core/providers/sgl/sgl.go (2)
22-23: LGTM - Raw request flag added consistently.The
sendBackRawRequestfield mirrors the existingsendBackRawResponsefield with proper initialization and usage across all provider methods.
79-80: LGTM - Context-aware raw request handling.The use of
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)correctly allows runtime context overrides while respecting the provider-level default. This pattern is consistently applied across all methods (ListModels, TextCompletion, TextCompletionStream, ChatCompletion, ChatCompletionStream, Embedding).ui/lib/types/schemas.ts (2)
372-373: LGTM - Schema additions are consistent.The
send_back_raw_requestfield is correctly added as optional boolean across config schemas (modelProviderConfigSchema,formModelProviderConfigSchema,addProviderRequestSchema,updateProviderRequestSchema), mirroring the existingsend_back_raw_responsefield.
486-487: LGTM - Form schema uses required boolean appropriately.The
performanceFormSchemacorrectly usesz.boolean()(required) rather thanz.boolean().optional()since form state needs an explicit value for the checkbox control.core/providers/vertex/vertex.go (3)
61-62: LGTM - Raw request flag added to Vertex provider.The
sendBackRawRequestfield is correctly added and initialized from configuration, consistent with the existingsendBackRawResponsepattern.
443-471: LGTM - Chat completion raw request handling is correct.For POST requests with a body, the raw request handling correctly captures and conditionally includes the request payload in the response. The pattern is consistently applied for both Anthropic and non-Anthropic model paths.
605-606: LGTM - Streaming handlers correctly pass raw request flag.The context-aware
ShouldSendBackRawRequestis properly passed to the Anthropic and OpenAI streaming handlers.plugins/logging/main.go (1)
497-505: RawRequest capture behinddisableContentLogginglooks consistent.
CopyingextraFields.RawRequestonly when content logging is enabled matches the stated security posture for this feature.core/schemas/bifrost.go (2)
103-122: Context key + schema field additions look consistent with existing RawResponse flow.
285-296:RawRequest interface{} json:",omitempty"is the right shape for “raw JSON payload”.core/providers/perplexity/perplexity.go (2)
18-25: Per-providersendBackRawRequestflag wiring looks correct.
186-201: Streaming path now respects raw-request toggle via the shared helper—good.framework/configstore/rdb.go (5)
208-219: Provider upsert includesSendBackRawRequest—LGTM.
331-339: Single-provider update includesSendBackRawRequest—LGTM.
447-458: Provider create includesSendBackRawRequest—LGTM.
644-654: Provider config round-trip (DB → runtime) includesSendBackRawRequest—LGTM.
200-304: Stack check: ensure migrations land before runtime reads/writes this field.
Given the Graphite stack, please confirm the PR that introduces the DB column forconfig_providers.send_back_raw_requestis in an earlier (downstack) PR that merges first, or this PR can break at runtime.Also applies to: 306-429, 431-515, 545-657
framework/configstore/tables/provider.go (1)
12-44: BothSendBackRawRequestandSendBackRawResponseare properly included in theGenerateConfigHashfunction (framework/configstore/clientconfig.go, lines 204–212). The hash logic correctly incorporates these fields, and migration support exists. Extensive test coverage confirms hash generation works correctly across all scenarios.transports/bifrost-http/lib/config.go (3)
1689-1697: Good:SendBackRawRequestis loaded from DB into in-memory provider config.
2098-2106: Good: redacted provider config includesSendBackRawRequest(safe boolean exposure).
639-648: No action needed.SendBackRawRequestis already properly integrated into the config hash generation and database schema.The
GenerateConfigHash()method inframework/configstore/clientconfig.go(lines 204–205) explicitly includesSendBackRawRequestin the hash computation. Additionally, the migrationmigrationAddSendBackRawRequestColumns()inframework/configstore/migrations.go(lines 1294–1332) properly adds thesend_back_raw_requestcolumn to the database and pre-populates config hashes for existing providers using the same hash logic that includesSendBackRawRequest.core/providers/groq/groq.go (2)
16-22: LGTM: provider storessendBackRawRequestfrom config.Also applies to: 52-58
66-80: LGTM: correctly passes effectiveShouldSendBackRawRequestandShouldSendBackRawResponsethrough to OpenAI-compatible handlers.All three methods (ListModels at 66-80, ChatCompletion at 157-171, ChatCompletionStream at 173-199) consistently use
providerUtils.ShouldSendBackRawRequest(ctx, provider.sendBackRawRequest)andproviderUtils.ShouldSendBackRawResponse(ctx, provider.sendBackRawResponse)in the correct parameter positions for their respective handler calls, ensuring context-aware configuration is properly propagated.core/providers/openrouter/openrouter.go (1)
18-24: LGTM: OpenRouterProvider wiressendBackRawRequestfrom config.Also applies to: 49-55
core/providers/gemini/gemini.go (2)
23-30: LGTM: GeminiProvider storessendBackRawRequestfrom config.Also applies to: 55-62
638-642: LGTM: streaming “done” events attach parsed raw request when enabled.Also applies to: 909-913
framework/configstore/migrations.go (3)
92-96: Good migration ordering (add column before config_hash backfill).
This preventsmigrationAddConfigHashColumnfrom potentially selecting a non-existentsend_back_raw_requestcolumn on older DBs.
1294-1321: Newsend_back_raw_requestcolumn migration looks correct and idempotent.
HasColumn/AddColumn + rollback DropColumn is consistent with the rest of this file.
1343-1350: Config hash now correctly reflectsSendBackRawRequest.core/providers/elevenlabs/elevenlabs.go (2)
23-30:sendBackRawRequestwiring is straightforward.Also applies to: 55-62
256-258: Raw request capture viaParseAndSetRawRequeston JSON bodies looks good.Also applies to: 436-439
core/providers/bedrock/bedrock.go (2)
29-36:sendBackRawRequestflag is correctly wired into the provider.Also applies to: 72-79
551-555: JSON request-body capture points look consistent (including stream finalization).Also applies to: 730-734, 902-906, 971-975, 1059-1063
core/providers/cohere/cohere.go (2)
66-74: Provider flag wiring is fine.Also applies to: 105-112
305-326: POST paths correctly passjsonBodyintoHandleProviderResponsefor raw-request capture.Also applies to: 531-554, 774-795
core/providers/parasail/parasail.go (2)
16-23: Clean plumbing ofsendBackRawRequestinto Parasail provider.Also applies to: 48-54
64-75: Passing effective raw-request/raw-response flags into shared OpenAI handlers looks correct.Also applies to: 92-104, 116-131
core/providers/azure/azure.go (1)
21-28: Provider flag wiring is fine.Also applies to: 47-53
transports/bifrost-http/handlers/providers.go (4)
61-61: LGTM – Field addition is consistent.The
SendBackRawRequestfield follows the established pattern forSendBackRawResponseand is properly positioned and documented.
179-179: LGTM – Payload field and initialization follow established patterns.The optional
SendBackRawRequestfield in the add-provider payload and its initialization with a nil check are consistent with the existingSendBackRawResponsehandling.Also applies to: 250-250
320-320: LGTM – Update logic correctly handles partial updates.The optional
SendBackRawRequestfield in the update payload and the conditional update logic (lines 431-433) correctly preserve existing values when the field is nil, allowing partial updates as documented in the function comment.Also applies to: 431-433
279-279: LGTM – Response construction consistently surfaces the new field.The fallback response construction (lines 279, 462) and the
getProviderResponseFromConfighelper (line 856) correctly propagateSendBackRawRequestthrough all provider response paths, ensuring the field is always included in API responses.Also applies to: 462-462, 856-856
core/providers/anthropic/anthropic.go (2)
26-26: LGTM – Provider field addition and initialization.The
sendBackRawRequestfield is correctly added to the provider struct and initialized from the config, mirroring the establishedsendBackRawResponsepattern.Also applies to: 107-107
423-423: LGTM – Streaming paths follow a cleaner pattern.The streaming handlers correctly compute the effective
sendBackRawRequestvalue once (line 405), pass it as a parameter, and use it directly inside the handler (lines 656-659) without re-evaluation. This is the preferred pattern and is clearer than the double-evaluation approach used in non-streaming handlers.Also applies to: lines 783-783, 991-994
Also applies to: 656-659
core/providers/openai/openai.go (2)
28-28: LGTM – Provider field addition and initialization.The
sendBackRawRequestfield is correctly added and initialized from the config, following the established pattern.Also applies to: 65-65
596-599: LGTM – Streaming paths follow the preferred pattern.The streaming handlers correctly compute the effective
sendBackRawRequestvalue once, pass it as a parameter, and use it directly in stream finalization (lines 597, 1084, 1429) without re-evaluation. This is the cleaner pattern that non-streaming handlers should adopt.Also applies to: 1083-1086, 1428-1431
Pratham-Mishra04
force-pushed
the
12-04-feat_raw_response_accumulation_for_streaming
branch
from
921fdd8 to
5f77971
Compare
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
9551b34 to
26864dc
Compare
Merge activity
|
Pratham-Mishra04
changed the base branch from
12-04-feat_raw_response_accumulation_for_streaming
to
graphite-base/1010
Pratham-Mishra04
force-pushed
the
12-05-feat_send_back_raw_request_support
branch
from
26864dc to
1f53185
Compare
Pratham-Mishra04
deleted the
12-05-feat_send_back_raw_request_support
branch
Summary
This PR adds the ability to send back raw requests in API responses, complementing the existing raw response functionality. This feature helps with debugging and advanced use cases by providing visibility into the exact request payload sent to provider APIs.
Changes
SendBackRawRequestfield to provider configurationsType of change
Affected areas
How to test
raw_requestfield inextra_fieldsScreenshots/Recordings
N/A
Breaking changes
Related issues
N/A
Security considerations
Raw requests may contain sensitive information like API keys or user data. Users should be aware that enabling this feature will store this information in logs and return it in responses.
Checklist