plugins v2 architecture #888
Conversation
📝 WalkthroughSummary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings. WalkthroughThis PR replaces the Plugin TransportInterceptor hook with an HTTPTransportMiddleware (fasthttp middleware), adds a tracing/observability subsystem (Trace/Span models, Tracer, TraceStore, propagation, helpers), threads tracing through core request and streaming flows (TTFT), updates many handlers/plugins to the new middleware API, and adds a dev pprof profiler and UI integration. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant HTTPHandler
participant TracingMW as TracingMiddleware
participant Tracer
participant PluginPipeline
participant PluginMW as HTTPTransportMiddleware
participant Provider
Client->>HTTPHandler: HTTP request
HTTPHandler->>TracingMW: pass RequestCtx
TracingMW->>Tracer: CreateTrace() / StartSpan
TracingMW->>PluginPipeline: execute with traced ctx
PluginPipeline->>PluginMW: middleware(next)
PluginMW->>PluginMW: modify req/ctx, call next(ctx)
PluginMW->>Provider: forward to provider
Provider-->>PluginMW: response/chunk(s)
PluginMW-->>PluginPipeline: response
PluginPipeline-->>TracingMW: response
TracingMW->>Tracer: EndSpan / EndTrace (finalize)
TracingMW->>OtelPlugin: Inject(ctx, trace) %% Observability plugins receive completed trace
TracingMW-->>HTTPHandler: response sent (stream finalizer may run after flush)
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 inconclusive)
✅ Passed checks (2 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: Organization UI Review profile: CHILL Plan: Pro ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (2)
🧰 Additional context used📓 Path-based instructions (1)**⚙️ CodeRabbit configuration file
Files:
🔇 Additional comments (3)
Comment |
🧪 Test Suite AvailableThis PR can be tested by a repository admin. |
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
2 times, most recently
from
ec9703b to
de6897f
Compare
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
de6897f to
e32d6a2
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/schemas/plugin.go (1)
48-48: Update comment reference to use HTTPTransportMiddleware.Line 48 still references "TransportInterceptor" in the execution order documentation. This should be updated to "HTTPTransportMiddleware" for consistency.
Apply this diff:
-// 1. TransportInterceptor (HTTP transport only, modifies raw headers/body before entering Bifrost core) +// 1. HTTPTransportMiddleware (HTTP transport only, modifies raw headers/body before entering Bifrost core)transports/bifrost-http/handlers/middlewares.go (1)
48-81: Critical: Middleware execution instead of composition.Line 78 immediately invokes the chained handler with
(ctx)instead of returning it. This breaks the middleware pattern—the function should return a handler, not execute it.Apply this diff:
- lib.ChainMiddlewares(next, pluginsMiddlewareChain...)(ctx) + lib.ChainMiddlewares(next, pluginsMiddlewareChain...)(ctx) } } }Wait, let me reconsider. Looking at the structure:
func TransportInterceptorMiddleware(config *lib.Config) schemas.BifrostHTTPMiddleware { return func(next fasthttp.RequestHandler) fasthttp.RequestHandler { return func(ctx *fasthttp.RequestCtx) { // ... logic ... lib.ChainMiddlewares(next, pluginsMiddlewareChain...)(ctx) } } }Actually, the outer function returns
func(next) handler, and the inner function returnsfunc(ctx). At line 78, we're inside the innermost function that takesctx, so calling(ctx)makes sense here. But the issue is thatlib.ChainMiddlewaresreturns a handler, and we need to call it with ctx.Actually, on closer inspection, this looks correct. The pattern is:
- TransportInterceptorMiddleware returns a middleware (a function that takes a handler and returns a handler)
- That middleware's returned handler is a function that takes ctx
- Inside that function, we chain middlewares and invoke the result with ctx
So the code is correct. Let me reconsider...
Actually, I need to look more carefully. Line 78:
lib.ChainMiddlewares(next, pluginsMiddlewareChain...)(ctx)This chains the middlewares starting with
nextas the base handler, wrapping it with the plugin middlewares, and then calls the resulting handler withctx. This seems correct for the pattern.But wait - there's a potential issue. If any of the earlier paths call
next(ctx), and then we also call the chained middlewares, we'd be calling the handler twice. Let me check the early returns...Lines 54-56: If no plugins, call
next(ctx)and return ✓
Lines 66-68: If no governance, callnext(ctx)and return ✓
Line 78: Otherwise, call chained middlewaresThis looks correct actually. My initial concern was wrong.
However, I notice another issue: if a plugin returns nil for HTTPTransportMiddleware(), we skip it (lines 73-75), but what if ALL plugins return nil? Then pluginsMiddlewareChain would be empty, and we'd call ChainMiddlewares with just
nextand empty variadic args, which should work fine according to lines 12-15 of lib/middleware.go.Actually, I think the code is correct. Let me reconsider if there are any other issues...
One thing I notice: the governance skip logic (lines 57-68) only executes the middleware chain if the governance plugin is loaded. This seems like a significant behavioral change - previously, TransportInterceptor would be called on all plugins, but now the HTTP middleware is only chained if governance is present. This might be intentional but should be verified.
[scratchpad_end] -->
♻️ Duplicate comments (2)
plugins/telemetry/main.go (1)
278-281: Same nil-middleware semantics concern as logging pluginThis
HTTPTransportMiddlewarestub mirrors logging’s pattern (returningnilto signal “no transport middleware”). That’s fine as long as the framework checks fornilbefore chaining plugin middlewares; otherwise a nil call will panic.Please ensure the shared plugin wiring handles
nilconsistently; no change needed here beyond that.plugins/mocker/main.go (1)
483-486: Mocker plugin’s HTTPTransportMiddleware stub follows the same patternReturning
nilhere is consistent with the other plugins and is a reasonable “no HTTP middleware” signal, provided the caller treatsnilas “don’t apply this middleware” rather than invoking it.Given the repeated pattern, it’s worth double‑checking the central wiring once to ensure nils are filtered out before chaining.
🧹 Nitpick comments (3)
plugins/governance/go.mod (1)
10-11: Verify version appropriateness and assess architectural necessity of direct fasthttp/sonic dependencies.The governance plugin now has direct dependencies on
bytedance/sonicandvalyala/fasthttp, which aligns with the new HTTPTransportMiddleware pattern. However, clarify:
- Whether these versions are secure and stable (especially given the "chill" review context, but still important).
- Whether the governance plugin itself directly uses these libraries, or if they're being exposed as transitive dependencies from the core/framework modules.
If the governance plugin doesn't directly instantiate fasthttp handlers or use sonic for serialization, consider whether these dependencies should be imported only in the transport layer (core or framework) rather than propagating to plugins.
transports/bifrost-http/handlers/health.go (1)
9-32: HealthHandler middleware type updated; small comment mismatchThe change to
...schemas.BifrostHTTPMiddlewareplus the newschemasimport correctly aligns the health route with the shared middleware type and keeps/healthbehavior intact.The doc comment for
getHealthstill saysGET /api/healthwhile the actual route is/health; consider updating the comment to avoid confusion.plugins/logging/main.go (1)
186-189: Document and validate the semantics of a nil HTTPTransportMiddleware
HTTPTransportMiddlewarereturningnilis a reasonable “no-op” implementation, but only if the transport wiring checks fornilbefore composing middleware chains. Otherwise, a nil call could panic at runtime.I’d recommend:
- Treating
nilexplicitly as “no middleware” in the code that collects and applies plugin middlewares, and- Leaving this method as-is here, given the comment and to avoid extra fasthttp dependencies.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
examples/plugins/hello-world/go.sumis excluded by!**/*.sum
📒 Files selected for processing (32)
core/schemas/plugin.go(3 hunks)examples/plugins/hello-world/go.mod(1 hunks)examples/plugins/hello-world/main.go(2 hunks)framework/plugins/dynamicplugin.go(3 hunks)framework/plugins/dynamicplugin_test.go(2 hunks)plugins/governance/go.mod(1 hunks)plugins/governance/main.go(2 hunks)plugins/jsonparser/main.go(1 hunks)plugins/logging/main.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/mocker/main.go(1 hunks)plugins/otel/main.go(1 hunks)plugins/semanticcache/main.go(1 hunks)plugins/telemetry/main.go(1 hunks)transports/bifrost-http/handlers/cache.go(1 hunks)transports/bifrost-http/handlers/config.go(2 hunks)transports/bifrost-http/handlers/governance.go(2 hunks)transports/bifrost-http/handlers/health.go(2 hunks)transports/bifrost-http/handlers/inference.go(1 hunks)transports/bifrost-http/handlers/integrations.go(2 hunks)transports/bifrost-http/handlers/logging.go(1 hunks)transports/bifrost-http/handlers/mcp.go(1 hunks)transports/bifrost-http/handlers/middlewares.go(5 hunks)transports/bifrost-http/handlers/middlewares_test.go(8 hunks)transports/bifrost-http/handlers/plugins.go(1 hunks)transports/bifrost-http/handlers/providers.go(1 hunks)transports/bifrost-http/handlers/session.go(2 hunks)transports/bifrost-http/handlers/ui.go(2 hunks)transports/bifrost-http/handlers/websocket.go(2 hunks)transports/bifrost-http/integrations/router.go(2 hunks)transports/bifrost-http/lib/middleware.go(1 hunks)transports/bifrost-http/server/server.go(4 hunks)
🧰 Additional context used
🧬 Code graph analysis (29)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/integrations/router.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/websocket.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/cache.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/logging/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)
plugins/maxim/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)
transports/bifrost-http/handlers/governance.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/middlewares.go (4)
transports/bifrost-http/lib/config.go (1)
Config(139-169)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/mcp.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/plugins.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)
transports/bifrost-http/handlers/inference.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/providers.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/telemetry/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/health.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/governance/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/server/server.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/plugin.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-25)
🔇 Additional comments (38)
examples/plugins/hello-world/go.mod (2)
10-22: Indirect dependencies properly declared.The transitive dependencies from fasthttp are correctly listed with
// indirectmarkers. This formatting aligns with Go module best practices and makes the dependency graph transparent.
5-8: ---Dependencies are appropriately versioned for the middleware architecture migration.
The addition of
github.com/valyala/fasthttp v1.67.0andgithub.com/maximhq/bifrost/core v1.2.27correctly supports the HTTPTransportMiddleware pattern. Both versions are valid and stable, with fasthttp v1.67.0 including security fixes for URL validation and host parsing.transports/bifrost-http/handlers/session.go (1)
11-39: Align SessionHandler middleware type with core schemasRegisterRoutes now uses
...schemas.BifrostHTTPMiddlewareand the newschemasimport; this cleanly aligns session routes with the centralized middleware type while preserving the existing routing behavior vialib.ChainMiddlewares.transports/bifrost-http/handlers/cache.go (1)
26-29: CacheHandler RegisterRoutes migrated to schemas middleware typeUsing
...schemas.BifrostHTTPMiddlewarefor cache routes is consistent with the shared middleware abstraction and works seamlessly withlib.ChainMiddlewares.transports/bifrost-http/handlers/logging.go (1)
41-47: LoggingHandler now uses schemas.BifrostHTTPMiddlewareSwitching RegisterRoutes to
...schemas.BifrostHTTPMiddlewarestandardizes logging routes on the new middleware type without altering any logging behavior.transports/bifrost-http/handlers/providers.go (1)
78-86: ProviderHandler RegisterRoutes aligned with schemas middlewareThe provider management routes now take
...schemas.BifrostHTTPMiddleware, matching the shared middleware type across the transport layer while preserving all existing provider CRUD semantics.transports/bifrost-http/handlers/mcp.go (1)
42-50: MCPHandler routes migrated to schemas.BifrostHTTPMiddlewareUsing
...schemas.BifrostHTTPMiddlewarehere brings MCP endpoints in line with the new middleware contract while keeping all MCP behavior and routing identical.transports/bifrost-http/handlers/integrations.go (1)
5-40: IntegrationHandler standardized on schemas.BifrostHTTPMiddlewareAdding the
schemasimport and updating RegisterRoutes to...schemas.BifrostHTTPMiddlewarecleanly propagates the new middleware type into all integration extensions viaextension.RegisterRoutes(r, middlewares...)without changing routing semantics.transports/bifrost-http/handlers/config.go (1)
14-55: ConfigHandler routes moved to schemas.BifrostHTTPMiddlewareThe added
schemasimport and updated RegisterRoutes signature (...schemas.BifrostHTTPMiddleware) bring config and version endpoints in line with the unified middleware abstraction while leaving all config behavior as-is.transports/bifrost-http/handlers/plugins.go (1)
53-59: Aligning PluginsHandler with shared middleware type looks correctSwitching
RegisterRoutesto...schemas.BifrostHTTPMiddlewarekeeps handler wiring consistent with the new central middleware contract while preserving existing route behavior vialib.ChainMiddlewares. No issues from this change alone.transports/bifrost-http/handlers/middlewares_test.go (1)
6-6: Tests correctly updated to use the shared BifrostHTTPMiddleware typeSwitching the middleware constructions to
schemas.BifrostHTTPMiddleware(with the new import) keeps the test behavior identical while aligning with the central middleware type definition. The ordering, short‑circuit, and context‑mutation scenarios are all still clearly exercised, and the explicit conversions are type‑correct.Also applies to: 295-301, 322-342, 368-373, 395-402, 405-418, 459-465, 467-475, 477-483
transports/bifrost-http/handlers/governance.go (1)
15-22: Governance routes now correctly use the shared middleware contractUpdating
RegisterRoutesto accept...schemas.BifrostHTTPMiddlewareand leaving thelib.ChainMiddlewarescalls intact cleanly aligns governance handlers with the new middleware model without changing route behavior.Also applies to: 154-175
transports/bifrost-http/handlers/inference.go (1)
274-285: Inference handler middleware signature migration is consistent
CompletionHandler.RegisterRoutesnow taking...schemas.BifrostHTTPMiddlewarematches the repository‑wide middleware type migration and should remain compatible withlib.ChainMiddlewaresas updated in this PR.examples/plugins/hello-world/main.go (1)
8-9: Example HTTPTransportMiddleware implementation looks correctThe hello‑world plugin’s
HTTPTransportMiddlewarehas the expectedfunc(next fasthttp.RequestHandler) fasthttp.RequestHandlersignature, logs once per request, and correctly forwards tonext(ctx), making it a clear, minimal example for plugin authors.Also applies to: 20-25
transports/bifrost-http/integrations/router.go (2)
70-70: LGTM! Consistent middleware type migration.The migration from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddlewareis clean and consistent across both the interface definition and implementation. This centralizes the middleware type in the schemas package, improving code organization.Also applies to: 232-232
11-11: LGTM! Clean middleware type migration.The handler correctly adopts the new
schemas.BifrostHTTPMiddlewaretype with the necessary import addition. Consistent with the broader architectural refactor.Also applies to: 29-29
transports/bifrost-http/handlers/websocket.go (1)
14-14: LGTM! Consistent with the middleware migration.The WebSocket handler correctly adopts the new middleware type from the schemas package.
Also applies to: 51-51
framework/plugins/dynamicplugin_test.go (1)
53-83: LGTM! Well-structured test for the new middleware API.The test correctly verifies the middleware pattern:
- Creates a mock next handler
- Wraps it with the plugin's middleware
- Verifies the wrapped handler calls the next handler
This properly validates the middleware contract established by the new
HTTPTransportMiddlewareAPI.plugins/jsonparser/main.go (1)
87-90: LGTM! Appropriate no-op middleware.The plugin correctly implements the new
HTTPTransportMiddlewaremethod, returningnilto indicate no middleware is needed. This is the expected pattern for plugins that don't require HTTP transport interception.core/schemas/plugin.go (1)
36-38: LGTM! Well-defined middleware type.The
BifrostHTTPMiddlewaretype follows the standard fasthttp middleware pattern:func(next fasthttp.RequestHandler) fasthttp.RequestHandler. This is a clean and idiomatic approach for HTTP middleware.plugins/semanticcache/main.go (1)
338-341: LGTM! Consistent no-op middleware implementation.The semantic cache plugin correctly implements the new middleware API, returning
nilas it doesn't require HTTP transport interception.plugins/otel/main.go (1)
146-149: LGTM! Consistent with the plugins v2 architecture.The OTEL plugin correctly implements the new
HTTPTransportMiddlewaremethod, returningnilas it doesn't need HTTP-level middleware.transports/bifrost-http/lib/middleware.go (1)
3-23: LGTM! Clean type migration.The refactor correctly centralizes the middleware type definition in
core/schemaswhile preserving the chaining logic.transports/bifrost-http/handlers/middlewares.go (3)
20-45: LGTM! Type signature updated correctly.The CORS middleware logic remains unchanged; only the return type was updated to use the centralized type definition.
99-200: LGTM! Type signature updated correctly.The authentication middleware logic is preserved; only the return type signature was updated.
57-68: Governance gate is functionally correct but adds hidden behavioral coupling.The governance gate (lines 57-68) is safe—only the governance plugin provides non-nil middleware, so gating execution when governance is absent doesn't skip any plugins. However, it introduces a brittle assumption: the code assumes governance is the exclusive middleware provider. If another plugin adds middleware in the future, it would silently bypass the gate and never execute.
Rather than gating on a specific plugin name, the code should filter collected middlewares and check if any are non-nil, making intent explicit and future-proof.
pluginsMiddlewareChain := []schemas.BifrostHTTPMiddleware{} for _, plugin := range plugins { middleware := plugin.HTTPTransportMiddleware() if middleware == nil { continue } pluginsMiddlewareChain = append(pluginsMiddlewareChain, middleware) } if len(pluginsMiddlewareChain) > 0 { lib.ChainMiddlewares(next, pluginsMiddlewareChain...)(ctx) } else { next(ctx) }plugins/maxim/main.go (1)
125-128: LGTM! Correct no-op middleware implementation.The maxim plugin correctly returns
nilfor HTTP transport middleware since it operates at the PreHook/PostHook level rather than the HTTP transport layer.transports/bifrost-http/server/server.go (5)
844-850: LGTM! Method signature updated correctly.The route registration methods now properly accept the centralized middleware type.
853-930: LGTM! Consistent middleware type usage.All handler registrations correctly use the updated middleware type throughout the method.
933-936: LGTM! UI route registration updated correctly.The method signature properly reflects the middleware type change.
965-976: LGTM! Common middleware preparation updated correctly.The method correctly builds and returns a slice of the centralized middleware type.
1077-1082: LGTM! Server middleware chain configured correctly.The middleware composition properly chains CORS → TransportInterceptor → Router, ensuring the correct execution order.
plugins/governance/main.go (3)
158-174: LGTM! Middleware structure and early validation correct.The middleware factory pattern is properly implemented with appropriate early returns for missing or invalid virtual keys.
195-209: LGTM! Load balancing integration correct.The middleware properly modifies the request body with provider load balancing before forwarding to the next handler.
213-305: LGTM! Load balancing logic is sound.The weighted random selection with budget/rate limit awareness and fallback handling is well-implemented with appropriate defensive checks.
framework/plugins/dynamicplugin.go (2)
26-40: LGTM! Dynamic plugin API updated correctly.The field and method changes properly align with the new middleware-based plugin interface.
141-148: LGTM! Symbol lookup updated correctly.The dynamic plugin loader properly looks up and casts the new
HTTPTransportMiddlewaresymbol with clear error messages.transports/bifrost-http/handlers/ui.go (1)
11-31: Middleware type migration tocore/schemaslooks consistentSwitching
RegisterRoutesto useschemas.BifrostHTTPMiddlewareand importingcore/schemasaligns this handler with the shared middleware contract used elsewhere. The variadic signature and continued use oflib.ChainMiddlewareskeep the API surface unchanged for callers while centralizing the type definition in core.
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
2 times, most recently
from
9be42ea to
c08c82f
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/schemas/plugin.go (1)
71-80: Update execution-order docs to reference HTTPTransportMiddleware instead of TransportInterceptorThe
Plugininterface now exposesHTTPTransportMiddleware()but the execution-order comments above still refer toTransportInterceptor; updating those comments to describeHTTPTransportMiddlewareand its place in the pipeline will avoid confusion for plugin authors.
♻️ Duplicate comments (1)
plugins/governance/main.go (1)
214-227: Fix misleading log message on JSON unmarshal errorThe log message at the
sonic.Unmarshalfailure path still says “failed to marshal”, which is inaccurate and repeats a previously flagged issue.Apply this diff to clarify the error:
- err = sonic.Unmarshal(ctx.Request.Body(), &payload) - if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + err = sonic.Unmarshal(ctx.Request.Body(), &payload) + if err != nil { + p.logger.Error("failed to unmarshal request body to check for virtual key: %v", err) next(ctx) return }
🧹 Nitpick comments (3)
transports/bifrost-http/lib/middleware.go (1)
3-22: Middleware chaining now correctly standardizes on schemas.BifrostHTTPMiddlewareSwitching
ChainMiddlewaresto useschemas.BifrostHTTPMiddlewarekeeps behavior identical while centralizing the type inschemas; just ensure all callers continue to filter outnilmiddlewares, or consider skippingnilinside the loop as a small safety net in the future.plugins/governance/main.go (2)
13-20: HTTP middleware and virtual key parsing look sound; consider minor robustness tweaksThe new
parseVirtualKey+HTTPTransportMiddlewareflow is well-structured: it cleanly short-circuits when there’s no/invalid VK, validates viap.store.GetVirtualKey, enriches headers viaaddMCPIncludeTools, and rewrites the JSON body withloadBalanceProviderwhile always falling back tonext(ctx)on errors. That gives safe behavior even with malformed or non-JSON bodies.Two small optional improvements:
- Body presence check
ctx.Request.Body()can be non‑nil but empty; in that casesonic.Unmarshalwill just fail and you fall back, which is safe but noisy. You could avoid unnecessary log noise with a length check:- if ctx.Request.Body() == nil { + if len(ctx.Request.Body()) == 0 { next(ctx) return }
- Unused error from addMCPIncludeTools
addMCPIncludeToolsnever returns a non‑nil error, so theerrhandling here is effectively dead. Either drop the error from the helper’s signature (if you control all call sites) or add a brief comment noting that it currently can’t fail to avoid confusion for future readers.Both are non‑blocking polish items; functionally this middleware looks correct.
Also applies to: 158-181, 183-235
70-79: Update Init doc comment to reference HTTPTransportMiddleware instead of TransportInterceptorThe Init comment still mentions
TransportInterceptorfor howinMemoryStoreis used, but the implementation has moved that responsibility intoHTTPTransportMiddleware. Updating the comment will prevent confusion for future maintainers.For example:
-// - `inMemoryStore` is used by TransportInterceptor to validate configured providers +// - `inMemoryStore` is used by HTTPTransportMiddleware to validate configured providers
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
examples/plugins/hello-world/go.sumis excluded by!**/*.sum
📒 Files selected for processing (32)
core/schemas/plugin.go(3 hunks)examples/plugins/hello-world/go.mod(1 hunks)examples/plugins/hello-world/main.go(2 hunks)framework/plugins/dynamicplugin.go(3 hunks)framework/plugins/dynamicplugin_test.go(2 hunks)plugins/governance/go.mod(1 hunks)plugins/governance/main.go(3 hunks)plugins/jsonparser/main.go(1 hunks)plugins/logging/main.go(1 hunks)plugins/maxim/main.go(1 hunks)plugins/mocker/main.go(1 hunks)plugins/otel/main.go(1 hunks)plugins/semanticcache/main.go(1 hunks)plugins/telemetry/main.go(1 hunks)transports/bifrost-http/handlers/cache.go(1 hunks)transports/bifrost-http/handlers/config.go(2 hunks)transports/bifrost-http/handlers/governance.go(2 hunks)transports/bifrost-http/handlers/health.go(2 hunks)transports/bifrost-http/handlers/inference.go(1 hunks)transports/bifrost-http/handlers/integrations.go(2 hunks)transports/bifrost-http/handlers/logging.go(1 hunks)transports/bifrost-http/handlers/mcp.go(1 hunks)transports/bifrost-http/handlers/middlewares.go(3 hunks)transports/bifrost-http/handlers/middlewares_test.go(8 hunks)transports/bifrost-http/handlers/plugins.go(1 hunks)transports/bifrost-http/handlers/providers.go(1 hunks)transports/bifrost-http/handlers/session.go(2 hunks)transports/bifrost-http/handlers/ui.go(2 hunks)transports/bifrost-http/handlers/websocket.go(2 hunks)transports/bifrost-http/integrations/router.go(2 hunks)transports/bifrost-http/lib/middleware.go(1 hunks)transports/bifrost-http/server/server.go(4 hunks)
🚧 Files skipped from review as they are similar to previous changes (13)
- transports/bifrost-http/handlers/inference.go
- transports/bifrost-http/integrations/router.go
- transports/bifrost-http/handlers/governance.go
- transports/bifrost-http/handlers/config.go
- plugins/telemetry/main.go
- transports/bifrost-http/handlers/session.go
- framework/plugins/dynamicplugin.go
- transports/bifrost-http/server/server.go
- transports/bifrost-http/handlers/cache.go
- transports/bifrost-http/handlers/health.go
- plugins/semanticcache/main.go
- plugins/otel/main.go
- plugins/governance/go.mod
🧰 Additional context used
🧬 Code graph analysis (18)
transports/bifrost-http/handlers/websocket.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(51-53)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/providers.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/maxim/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-27)
transports/bifrost-http/handlers/plugins.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/plugin.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-27)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-27)
transports/bifrost-http/handlers/mcp.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/logging/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/middlewares.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-27)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(20-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
🔇 Additional comments (20)
examples/plugins/hello-world/go.mod (2)
7-7: fasthttp addition aligns with middleware pattern transition.The addition of
github.com/valyala/fasthttp v1.67.0as a direct dependency is an expected change that aligns with the PR's transition to an HTTP middleware pattern. The transitive indirect dependencies (brotli, compress, bytebufferpool) are standard fasthttp dependencies and are appropriate.
6-6: Unable to access repository files for verification.The review comment references
examples/plugins/hello-world/go.mod, but I cannot access the codebase files in the current sandbox environment to independently verify the core dependency downgrade claim or check version consistency across other modules in the repository.The original analysis flagged a downgrade of
github.com/maximhq/bifrost/corefrom v1.2.29 to v1.2.27 as potentially problematic. Without access to the actual files, I cannot confirm:
- Whether this downgrade is intentional
- If version consistency exists across other modules
- What functionality or security implications might be affected
transports/bifrost-http/handlers/providers.go (1)
78-78: LGTM! Middleware type migration is consistent.The update from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddlewarealigns with the architectural shift to centralize the middleware type definition in the core schemas package.transports/bifrost-http/handlers/ui.go (1)
11-11: LGTM! Clean type migration with proper import.The import addition and middleware type update are consistent with the broader migration to
schemas.BifrostHTTPMiddleware.Also applies to: 29-29
transports/bifrost-http/handlers/logging.go (1)
42-42: LGTM! Middleware type updated consistently.The RegisterRoutes signature update to
schemas.BifrostHTTPMiddlewareis consistent with the architectural refactor across the codebase.transports/bifrost-http/handlers/plugins.go (1)
53-53: LGTM! Type migration applied consistently.The middleware type update maintains consistency with the broader architectural shift to centralize middleware types in the schemas package.
plugins/jsonparser/main.go (1)
87-90: LGTM! Plugin correctly implements new middleware interface.The implementation of
HTTPTransportMiddleware()returningnilis appropriate for this plugin, as indicated by the comment. This aligns with the migration fromTransportInterceptorto the new middleware-based approach.transports/bifrost-http/handlers/mcp.go (1)
42-42: LGTM! Middleware type migration is consistent.The RegisterRoutes signature update follows the standard pattern for migrating to
schemas.BifrostHTTPMiddleware.transports/bifrost-http/handlers/middlewares_test.go (1)
6-6: LGTM! Tests updated consistently with type migration.All test middleware references have been properly updated from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddleware. The test logic remains unchanged, ensuring the migration is purely structural.Also applies to: 295-295, 322-322, 329-329, 336-336, 368-368, 395-395, 405-405, 413-413, 458-458, 467-467, 477-477
transports/bifrost-http/handlers/integrations.go (1)
8-8: LGTM! Integration handler properly migrated.The import addition and RegisterRoutes signature update are consistent with the architectural refactor. The variadic middleware delegation to extension routers remains functionally identical.
Also applies to: 35-35
framework/plugins/dynamicplugin_test.go (1)
53-83: Dynamic plugin middleware test correctly validates chaining behaviorThe new
HTTPTransportMiddlewaresubtest exercises the middleware constructor, wrapped handler, and ensuresnextis invoked with a realisticfasthttp.RequestCtxsetup; this is sufficient to validate the new dynamic-plugin hook wiring.plugins/logging/main.go (1)
186-189: No-op HTTPTransportMiddleware is a valid opt-out for this pluginReturning
nilhere is a clean way for the logging plugin to signal “no HTTP-level middleware”, and the transport layer already guards againstnilmiddlewares before chaining.examples/plugins/hello-world/main.go (1)
8-27: Example plugin’s HTTPTransportMiddleware correctly implements the new contractThe exported
HTTPTransportMiddlewarereturns aschemas.BifrostHTTPMiddlewarethat logs then callsnext, which is exactly what the dynamic plugin loader expects for the new HTTP transport hook.transports/bifrost-http/handlers/websocket.go (1)
14-53: WebSocket route registration now aligned with shared middleware typeUsing
...schemas.BifrostHTTPMiddlewareinRegisterRoutesand feeding them throughlib.ChainMiddlewaresaligns WebSocket routes with the unified middleware abstraction without changing behavior.core/schemas/plugin.go (1)
4-39: BifrostHTTPMiddleware type cleanly defines the shared HTTP middleware contractDefining
BifrostHTTPMiddlewareinschemasgives a single, clear function type for all HTTP middleware, and usingfasthttp.RequestHandlerdirectly matches the transport layer’s expectations.plugins/mocker/main.go (1)
479-486: Mocker plugin cleanly opts out of HTTP transport middlewareReturning
nilfromMockerPlugin.HTTPTransportMiddlewareis consistent with the interface and works with the interceptor middleware’s nil filtering to indicate this plugin has no HTTP-layer behavior.transports/bifrost-http/handlers/middlewares.go (2)
18-44: CORS middleware correctly updated to use schemas.BifrostHTTPMiddlewareThe CORS logic is unchanged and wrapping it in
schemas.BifrostHTTPMiddlewarematches the new shared middleware abstraction with no behavioral impact.
86-187: Auth middleware’s type change cleanly adopts the shared middleware abstractionChanging
AuthMiddlewareto returnschemas.BifrostHTTPMiddlewarewhile keeping the behavior (including the identity middleware when auth is disabled or store is nil) preserves existing auth semantics and fits naturally into the new middleware chain.plugins/maxim/main.go (1)
125-128: Nil HTTPTransportMiddleware is an acceptable no-op implementationReturning
nilhere is a clear way to indicate this plugin does not participate in HTTP middleware and cleanly satisfies the updatedPlugininterface. As long as the transport layer skipsnilmiddlewares (as it does for other plugins in this PR), this is fine.plugins/governance/main.go (1)
370-377: Updated PreHook error message better matches behaviorChanging the PreHook error message to “virtual key is missing or invalid” aligns with the actual semantics (empty/missing or rejected VK) and improves clarity for clients without altering logic.
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
c08c82f to
13f8738
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/schemas/plugin.go (1)
46-46: Update documentation to reflect new method name.Line 46 still refers to "TransportInterceptor" in the execution order documentation, but the method has been renamed to "HTTPTransportMiddleware". Update the comment for consistency.
🔎 Proposed fix
-// 1. TransportInterceptor (HTTP transport only, modifies raw headers/body before entering Bifrost core) +// 1. HTTPTransportMiddleware (HTTP transport only, modifies raw headers/body before entering Bifrost core)
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
core/schemas/plugin.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
core/schemas/plugin.go
🧠 Learnings (1)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
core/schemas/plugin.go
🧬 Code graph analysis (1)
core/schemas/plugin.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
🔇 Additional comments (3)
core/schemas/plugin.go (3)
4-6: LGTM! Import correctly added for middleware pattern.The fasthttp import is necessary for the
BifrostHTTPMiddlewaretype definition and follows the standard middleware pattern.
34-36: LGTM! Clean middleware type definition.The
BifrostHTTPMiddlewaretype correctly follows the standard middleware constructor pattern and is well-documented.
73-77: LGTM! Clean migration to middleware pattern.The
HTTPTransportMiddleware()method properly replaces the oldTransportInterceptorapproach with a standard middleware constructor pattern. The documentation clearly explains the new behavior.Note: The hello-world example issue (missing method receiver) has already been flagged in previous review comments.
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
13f8738 to
fa4b8d4
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/schemas/plugin.go (1)
46-46: Update comment to reference HTTPTransportMiddleware.Line 46 still references the old method name "TransportInterceptor" in the execution order documentation.
🔎 Proposed fix
-// 1. TransportInterceptor (HTTP transport only, modifies raw headers/body before entering Bifrost core) +// 1. HTTPTransportMiddleware (HTTP transport only, modifies raw headers/body before entering Bifrost core) // 2. PreHook (executed in registration order) // 3. Provider call // 4. PostHook (executed in reverse order of PreHooks)
♻️ Duplicate comments (3)
examples/plugins/hello-world/main.go (1)
19-26: Fix the hello-world example: HTTPTransportMiddleware must be a method with a receiver.The
HTTPTransportMiddlewarefunction is defined as a free function without a receiver, which does not properly implement thePlugininterface. To correctly implement the interface, it should be a method on a plugin struct type (e.g.,func (p *HelloWorldPlugin) HTTPTransportMiddleware() schemas.BifrostHTTPMiddleware).🔎 Proposed fix
First, define a plugin struct type near the top of the file:
+type HelloWorldPlugin struct{} + func Init(config any) error { fmt.Println("Init called") - return nil + return nil }Then update the function to be a method on that struct:
-func HTTPTransportMiddleware() schemas.BifrostHTTPMiddleware { +func (p *HelloWorldPlugin) HTTPTransportMiddleware() schemas.BifrostHTTPMiddleware { return func(next fasthttp.RequestHandler) fasthttp.RequestHandler { return func(ctx *fasthttp.RequestCtx) { fmt.Println("HTTPTransportMiddleware called") next(ctx) } } }Similarly, update all other exported functions to be methods on the plugin struct (PreHook, PostHook, GetName, Cleanup).
transports/bifrost-http/handlers/middlewares.go (1)
47-68: Update comments to reflect the new middleware terminology.The function and inline comments still reference "interceptors" (lines 47, 60) despite the code now using
HTTPTransportMiddleware(). Update the comments to reference "HTTP transport middleware" for consistency with the new API.plugins/governance/main.go (1)
212-217: Fix error message accuracy.Line 214: The error message says "failed to marshal" but the operation is
sonic.Unmarshal. This was flagged in a previous review.🔎 Proposed fix
- p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body: %v", err)
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
examples/plugins/hello-world/go.sumis excluded by!**/*.sum
📒 Files selected for processing (36)
core/mcp.gocore/schemas/plugin.goexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goplugins/governance/go.modplugins/governance/main.goplugins/jsonparser/main.goplugins/logging/main.goplugins/maxim/main.goplugins/mocker/main.goplugins/otel/main.goplugins/semanticcache/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/go.mod
💤 Files with no reviewable changes (1)
- framework/configstore/tables/mcp.go
🚧 Files skipped from review as they are similar to previous changes (12)
- transports/bifrost-http/handlers/logging.go
- transports/bifrost-http/handlers/middlewares_test.go
- transports/bifrost-http/handlers/session.go
- plugins/telemetry/main.go
- transports/bifrost-http/lib/middleware.go
- framework/plugins/dynamicplugin_test.go
- transports/bifrost-http/handlers/governance.go
- transports/bifrost-http/handlers/cache.go
- transports/bifrost-http/integrations/router.go
- transports/bifrost-http/handlers/ui.go
- plugins/semanticcache/main.go
- plugins/governance/go.mod
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/inference.gocore/mcp.gotransports/bifrost-http/handlers/websocket.goplugins/otel/main.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/providers.gocore/schemas/plugin.goexamples/plugins/hello-world/go.modtransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/health.goplugins/logging/main.gotransports/go.modplugins/mocker/main.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/server/server.goplugins/maxim/main.goexamples/plugins/hello-world/main.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/mcp.goplugins/jsonparser/main.goplugins/governance/main.go
🧠 Learnings (2)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/inference.gocore/mcp.gotransports/bifrost-http/handlers/websocket.goplugins/otel/main.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/providers.gocore/schemas/plugin.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/health.goplugins/logging/main.goplugins/mocker/main.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/server/server.goplugins/maxim/main.goexamples/plugins/hello-world/main.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/mcp.goplugins/jsonparser/main.goplugins/governance/main.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/mcp.go
🧬 Code graph analysis (20)
transports/bifrost-http/handlers/plugins.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/inference.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/websocket.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/otel/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
transports/bifrost-http/handlers/middlewares.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)
transports/bifrost-http/handlers/providers.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
core/schemas/plugin.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/health.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/logging/main.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)transports/bifrost-http/handlers/inference.go (1)
TranscriptionRequest(285-289)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/mcpServer.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/server/server.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)transports/bifrost-http/handlers/middlewares.go (1)
TransportInterceptorMiddleware(48-69)
plugins/maxim/main.go (1)
core/schemas/plugin.go (2)
Plugin(69-95)BifrostHTTPMiddleware(36-36)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/mcp.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
🔇 Additional comments (29)
core/mcp.go (1)
1135-1150: In-process client now uses strongly typedInProcessServerPassing
config.InProcessServerdirectly intoclient.NewInProcessClientis fine and removes redundant runtime type assertions, assumingschemas.MCPClientConfig.InProcessServeris already declared as*server.MCPServeracross the codebase.Please confirm that
InProcessServeris declared as*server.MCPServer(or a compatible concrete type) inschemas.MCPClientConfigso that this remains type-safe at compile time.transports/bifrost-http/handlers/providers.go (1)
79-89: Provider routes now consumeschemas.BifrostHTTPMiddlewareUpdating
RegisterRoutesto take...schemas.BifrostHTTPMiddlewarewhile still chaining vialib.ChainMiddlewaresmatches the new global middleware type and keeps the handler behavior unchanged.transports/bifrost-http/handlers/config.go (1)
15-62: Config routes aligned toschemas.BifrostHTTPMiddlewareImporting
core/schemasand updatingRegisterRoutesto accept...schemas.BifrostHTTPMiddleware(still passed throughlib.ChainMiddlewares) keeps the runtime behavior but unifies the middleware type with the rest of the HTTP stack.transports/bifrost-http/handlers/plugins.go (1)
52-59: Plugins handler now uses schema-level middleware typeSwitching
PluginsHandler.RegisterRoutesto...schemas.BifrostHTTPMiddlewarebrings this handler in line with the new plugin middleware contract and with other HTTP handlers using the shared type.transports/bifrost-http/handlers/inference.go (1)
364-390: Inference routes switched toschemas.BifrostHTTPMiddlewareUpdating
CompletionHandler.RegisterRoutesto take...schemas.BifrostHTTPMiddlewarewhile still delegating tolib.ChainMiddlewarescleanly adopts the new shared middleware type for all inference endpoints without changing routing behavior.transports/bifrost-http/handlers/mcpServer.go (1)
71-76: MCP server routes migrated toschemas.BifrostHTTPMiddleware
MCPServerHandler.RegisterRoutesnow accepts...schemas.BifrostHTTPMiddleware, which is consistent with the rest of the HTTP stack and the centralized Bifrost middleware type; the POST/GET/mcpbehavior remains the same.examples/plugins/hello-world/go.mod (1)
6-23: go.mod: fasthttp dependency aligns with existing modulesAdding
github.com/valyala/fasthttp v1.67.0to the hello-world example is correct and already consistent across all modules in the codebase (core, framework, and other plugins). No version misalignment issues.transports/go.mod (1)
5-25: mcp-go v0.41.1 dependency is consistent across all modulesThe
github.com/mark3labs/mcp-go v0.41.1dependency in transports/go.mod matches all other modules in the repository. Verified that core, framework, all plugins, and test modules consistently use the same version.transports/bifrost-http/handlers/websocket.go (1)
14-14: LGTM! Consistent middleware type migration.The import addition and signature update align with the repository-wide migration to
schemas.BifrostHTTPMiddleware. No functional changes to routing logic.Also applies to: 51-51
transports/bifrost-http/handlers/health.go (1)
9-9: LGTM! Middleware type migration is consistent.The import and signature changes align with the new
schemas.BifrostHTTPMiddlewaretype used across the codebase.Also applies to: 27-27
transports/bifrost-http/handlers/mcp.go (1)
43-51: LGTM! MCP handler aligned with new middleware pattern.The signature update and route registrations properly use the new
schemas.BifrostHTTPMiddlewaretype, maintaining existing MCP functionality.transports/bifrost-http/handlers/integrations.go (1)
8-8: LGTM! Integration handler properly delegates new middleware type.The import and signature changes align with the migration, and the method correctly passes the new middleware type to extension routers.
Also applies to: 37-41
plugins/logging/main.go (1)
196-199: LGTM! Logging plugin correctly implements new middleware interface.The method properly replaces
TransportInterceptorwithHTTPTransportMiddlewareusing the correct receiver signature and returnsnil(appropriate for plugins that don't need transport-layer interception).core/schemas/plugin.go (1)
34-36: LGTM! Core middleware type and interface properly defined.The
BifrostHTTPMiddlewaretype follows standard middleware patterns, and thePlugin.HTTPTransportMiddleware()method signature is correctly defined.Also applies to: 73-77
plugins/otel/main.go (1)
167-170: LGTM! OTEL plugin correctly migrated to new middleware interface.The method signature with receiver properly implements the
Plugininterface, and returningnilis appropriate for plugins that don't require transport-layer middleware.plugins/mocker/main.go (1)
481-484: LGTM!The
HTTPTransportMiddleware()correctly returnsnilsince this plugin usesPreHookfor its mocking logic rather than HTTP transport-level interception. This aligns with the new middleware pattern introduced in this PR.transports/bifrost-http/handlers/middlewares.go (2)
19-19: LGTM!Return type correctly updated to
schemas.BifrostHTTPMiddleware, aligning with the new centralized middleware type definition.
87-87: LGTM!Return type correctly updated to
schemas.BifrostHTTPMiddleware.plugins/maxim/main.go (1)
124-127: LGTM!The
HTTPTransportMiddleware()correctly returnsnilsince this plugin usesPreHook/PostHookfor Maxim tracing integration rather than HTTP transport-level interception.plugins/jsonparser/main.go (1)
86-89: LGTM!The
HTTPTransportMiddleware()correctly returnsnilsince this plugin usesPostHookfor streaming JSON parsing rather than HTTP transport-level interception.plugins/governance/main.go (3)
13-13: LGTM!Using
sonicfor JSON marshaling in the hot path is a good choice for performance.
158-181: LGTM!The
parseVirtualKeyhelper correctly extracts virtual keys from multiple header sources (x-bf-vk,Authorization: Bearer,x-api-key) with proper prefix validation and priority ordering.
183-234: HTTPTransportMiddleware implementation follows the new middleware pattern correctly.The middleware properly:
- Extracts and validates virtual keys
- Adds MCP include tools headers
- Performs provider load balancing
- Gracefully continues on errors (avoiding request failures for governance issues)
The implementation aligns with the PR's architecture shift from
TransportInterceptortoHTTPTransportMiddleware.framework/plugins/dynamicplugin.go (1)
24-39: Struct and method look good, pending the loader fix.The field and accessor method are correctly structured. Once the symbol lookup is fixed to call the factory function, this will work as intended.
transports/bifrost-http/server/server.go (5)
958-965: LGTM!Signature correctly updated to use the new
schemas.BifrostHTTPMiddlewaretype. The route registration logic is unchanged.
968-968: LGTM!Consistent type migration for the API routes registration.
1054-1057: LGTM!Type migration applied consistently.
1086-1097: LGTM!Return type and slice initialization correctly updated to use the new middleware type.
1230-1231: LGTM!The
TransportInterceptorMiddlewareis correctly prepended to the inference middleware chain, ensuring plugin middlewares wrap the entire request lifecycle.
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
fa4b8d4 to
ddfbaf4
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
transports/bifrost-http/handlers/middlewares.go (1)
47-68: Comments still reference old "interceptor" terminology.As noted in a previous review, the function name and inline comments still use "TransportInterceptor" terminology even though the code now uses
plugin.HTTPTransportMiddleware(). Update the comment on line 47 to reference "HTTP transport middleware" and line 59's comment to say "Collect plugin HTTP transport middleware" instead of "Call TransportInterceptor on all plugins".🔎 Suggested comment updates
-// TransportInterceptorMiddleware collects all plugin interceptors and calls them one by one +// TransportInterceptorMiddleware collects all plugin HTTP transport middleware and chains them func TransportInterceptorMiddleware(config *lib.Config) schemas.BifrostHTTPMiddleware { return func(next fasthttp.RequestHandler) fasthttp.RequestHandler { return func(ctx *fasthttp.RequestCtx) { plugins := config.GetLoadedPlugins() if len(plugins) == 0 { next(ctx) return } pluginsMiddlewareChain := []schemas.BifrostHTTPMiddleware{} for _, plugin := range plugins { middleware := plugin.HTTPTransportMiddleware() - // Call TransportInterceptor on all plugins + // Collect plugin HTTP transport middleware if middleware == nil { continue }plugins/governance/main.go (1)
212-217: Error message still incorrect after previous fix.Line 214: The error message says "failed to marshal" but the operation on line 212 is
sonic.Unmarshal. This was flagged in a previous review and marked as addressed in commit fa4b8d4, but the incorrect message remains in the current code.🔎 Proposed fix
- p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body: %v", err)framework/plugins/dynamicplugin.go (1)
140-147: Critical: Type mismatch will cause dynamic plugin loading to fail.The example plugin at
examples/plugins/hello-world/main.go:18-25exportsHTTPTransportMiddlewareas a factory function:func HTTPTransportMiddleware() schemas.BifrostHTTPMiddleware { ... }However, line 145 attempts to cast the symbol directly to
schemas.BifrostHTTPMiddleware:if dp.httpTransportMiddleware, ok = httpTransportMiddlewareSym.(schemas.BifrostHTTPMiddleware); !ok {This type mismatch will cause the assertion to always fail, preventing any dynamic plugins from loading successfully. The symbol is a function that returns the middleware, not the middleware itself.
Compare with the
GetNamepattern on line 137, which correctly handles factory functions.🔎 Proposed fix
// Looking up for HTTPTransportMiddleware method httpTransportMiddlewareSym, err := plugin.Lookup("HTTPTransportMiddleware") if err != nil { return nil, err } - if dp.httpTransportMiddleware, ok = httpTransportMiddlewareSym.(schemas.BifrostHTTPMiddleware); !ok { - return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func(next fasthttp.RequestHandler) fasthttp.RequestHandler") + httpTransportMiddlewareFunc, ok := httpTransportMiddlewareSym.(func() schemas.BifrostHTTPMiddleware) + if !ok { + return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func() schemas.BifrostHTTPMiddleware") } + dp.httpTransportMiddleware = httpTransportMiddlewareFunc()
🧹 Nitpick comments (1)
core/schemas/plugin.go (1)
45-46: Update outdated comment terminology.Line 46 still references "TransportInterceptor" in the execution order documentation, but the interface now uses
HTTPTransportMiddleware. This should be updated for consistency.🔎 Proposed fix
// Execution order: -// 1. TransportInterceptor (HTTP transport only, modifies raw headers/body before entering Bifrost core) +// 1. HTTPTransportMiddleware (HTTP transport only, wraps request handling before entering Bifrost core) // 2. PreHook (executed in registration order)
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
examples/plugins/hello-world/go.sumis excluded by!**/*.sum
📒 Files selected for processing (37)
core/mcp.gocore/schemas/plugin.goexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goplugins/governance/go.modplugins/governance/main.goplugins/jsonparser/main.goplugins/logging/main.goplugins/maxim/main.goplugins/mocker/main.goplugins/otel/main.goplugins/semanticcache/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/go.modui/components/sidebar.tsx
💤 Files with no reviewable changes (1)
- framework/configstore/tables/mcp.go
✅ Files skipped from review due to trivial changes (1)
- ui/components/sidebar.tsx
🚧 Files skipped from review as they are similar to previous changes (17)
- transports/bifrost-http/handlers/session.go
- transports/bifrost-http/handlers/governance.go
- transports/bifrost-http/handlers/cache.go
- transports/go.mod
- transports/bifrost-http/handlers/inference.go
- transports/bifrost-http/integrations/router.go
- transports/bifrost-http/handlers/providers.go
- plugins/governance/go.mod
- core/mcp.go
- plugins/maxim/main.go
- examples/plugins/hello-world/go.mod
- plugins/otel/main.go
- plugins/semanticcache/main.go
- examples/plugins/hello-world/main.go
- transports/bifrost-http/handlers/websocket.go
- transports/bifrost-http/handlers/plugins.go
- transports/bifrost-http/handlers/config.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
framework/plugins/dynamicplugin_test.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/mcpServer.gocore/schemas/plugin.goplugins/mocker/main.goplugins/telemetry/main.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/server/server.goplugins/governance/main.gotransports/bifrost-http/handlers/health.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/mcp.goplugins/logging/main.gotransports/bifrost-http/handlers/middlewares.goplugins/jsonparser/main.go
🧠 Learnings (2)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
framework/plugins/dynamicplugin_test.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/mcpServer.gocore/schemas/plugin.goplugins/mocker/main.goplugins/telemetry/main.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/server/server.goplugins/governance/main.gotransports/bifrost-http/handlers/health.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/mcp.goplugins/logging/main.gotransports/bifrost-http/handlers/middlewares.goplugins/jsonparser/main.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/middlewares.go
🧬 Code graph analysis (18)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/mcpServer.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
core/schemas/plugin.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/telemetry/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/server/server.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)transports/bifrost-http/handlers/middlewares.go (1)
TransportInterceptorMiddleware(48-68)transports/bifrost-http/lib/config.go (1)
Config(184-215)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/health.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)examples/plugins/hello-world/main.go (2)
GetName(15-17)HTTPTransportMiddleware(19-26)
transports/bifrost-http/handlers/mcp.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/logging/main.go (4)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)ui/lib/types/logs.ts (1)
TranscriptionInput(15-17)transports/bifrost-http/handlers/inference.go (1)
TranscriptionRequest(285-289)
transports/bifrost-http/handlers/middlewares.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
🔇 Additional comments (19)
transports/bifrost-http/handlers/middlewares_test.go (1)
308-308: LGTM! Consistent middleware type migration.All test middleware instantiations have been correctly updated from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddleware. The test logic remains unchanged and continues to validate middleware chaining, execution order, context modification, and short-circuiting behavior.Also applies to: 335-335, 342-342, 349-349, 381-381, 408-408, 418-418, 426-426, 471-471, 480-480, 490-490
transports/bifrost-http/handlers/logging.go (1)
42-42: LGTM! RegisterRoutes signature updated consistently.The method signature has been updated to accept
schemas.BifrostHTTPMiddlewareinstead oflib.BifrostHTTPMiddleware, aligning with the broader middleware type migration. The route registration logic remains unchanged.transports/bifrost-http/handlers/ui.go (1)
11-11: LGTM! Middleware type migration completed.The import and RegisterRoutes signature have been updated to use
schemas.BifrostHTTPMiddleware, consistent with the codebase-wide migration. The UI serving logic remains unchanged.Also applies to: 29-29
transports/bifrost-http/handlers/mcp.go (1)
43-43: LGTM! Signature updated for consistency.The RegisterRoutes method now accepts
schemas.BifrostHTTPMiddleware, aligning with the middleware type migration. All MCP route registrations continue to apply middlewares via ChainMiddlewares.transports/bifrost-http/handlers/mcpServer.go (1)
72-72: LGTM! MCP server routes updated consistently.The RegisterRoutes signature has been updated to accept
schemas.BifrostHTTPMiddleware, maintaining consistency with the broader middleware type migration. The MCP server route handling remains unchanged.transports/bifrost-http/handlers/integrations.go (1)
8-8: LGTM! Integration handler updated consistently.The import and RegisterRoutes signature have been updated to use
schemas.BifrostHTTPMiddleware. The method continues to delegate route registration to each integration extension with the provided middlewares.Also applies to: 37-37
plugins/telemetry/main.go (1)
279-282: LGTM! Correct no-op middleware implementation.The
HTTPTransportMiddlewaremethod correctly returnsnilsince this plugin doesn't require HTTP transport middleware. The telemetry plugin collects metrics throughPreHookandPostHookmethods instead, which is the appropriate pattern for request/response lifecycle tracking.framework/plugins/dynamicplugin_test.go (1)
16-16: LGTM! Test properly validates new middleware API.The test has been correctly updated to validate
HTTPTransportMiddlewareinstead of the deprecatedTransportInterceptor. The test:
- Creates a mock next handler to track invocation
- Obtains the middleware function from the plugin
- Wraps the next handler with the middleware
- Sets up a realistic request context with URI, method, and headers
- Verifies the next handler is called through the middleware chain
This properly tests the new middleware-based invocation pattern.
Also applies to: 54-84
transports/bifrost-http/lib/middleware.go (1)
3-22: LGTM! Clean migration to schemas-based middleware type.The transition from a local
BifrostHTTPMiddlewaretype toschemas.BifrostHTTPMiddlewareis straightforward and maintains the existing chaining logic correctly.plugins/logging/main.go (1)
196-199: LGTM! Correct no-op middleware implementation.The logging plugin appropriately returns
nilfor HTTP transport middleware since it operates via PreHook/PostHook rather than at the transport layer.core/schemas/plugin.go (1)
73-77: LGTM! Clean middleware-based API design.The new
HTTPTransportMiddleware()method returning aBifrostHTTPMiddlewareconstructor follows standard middleware patterns and provides better composability than the previous interception approach.plugins/jsonparser/main.go (1)
86-89: LGTM! Appropriate no-op middleware for this plugin.The JSON parser plugin correctly returns
nilsince it operates on responses in PostHook rather than at the HTTP transport layer.plugins/mocker/main.go (1)
481-484: LGTM! Correct no-op implementation.The mocker plugin appropriately returns
nilsince it operates via PreHook to short-circuit requests, not at the HTTP transport layer.transports/bifrost-http/handlers/health.go (1)
27-29: LGTM! Clean signature migration.The health handler correctly adopts the new
schemas.BifrostHTTPMiddlewaretype without any functional changes.transports/bifrost-http/handlers/middlewares.go (1)
56-65: LGTM! Correct middleware collection and chaining.The logic properly collects non-nil middleware from each plugin and chains them using
lib.ChainMiddlewares, maintaining plugin ordering and allowing short-circuiting.transports/bifrost-http/server/server.go (2)
958-958: LGTM! Consistent signature updates across server API.All route registration and middleware preparation methods have been correctly updated to use
schemas.BifrostHTTPMiddleware, maintaining consistency throughout the server initialization flow.Also applies to: 968-968, 1054-1054, 1086-1087
1230-1230: LGTM! Correct middleware prepending.The inference middleware chain correctly prepends
TransportInterceptorMiddlewareusing the newschemas.BifrostHTTPMiddlewaretype.plugins/governance/main.go (2)
409-409: LGTM!The error message accurately reflects the validation failure condition.
183-234: HTTPTransportMiddleware operates at HTTP transport layer for request transformation, not governance enforcement.The middleware correctly fails open when virtual keys are missing or errors occur in transformations (MCP tools, JSON parsing, load balancing). Governance enforcement is handled at a separate layer via PreHook, which respects the
IsVkMandatoryconfiguration setting:
- When
IsVkMandatory=false(default): missing virtual keys bypass HTTPTransportMiddleware transformations intentionally- When
IsVkMandatory=true: PreHook rejects missing/invalid virtual keys with a 400 error before request processingErrors in HTTPTransportMiddleware operations (JSON unmarshaling, load balancing) should not block requests since this layer focuses on request transformation for availability. However, to improve observability, add metrics/alerts for these error paths to ensure visibility when governance transformations fail silently.
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
ddfbaf4 to
d632297
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
plugins/governance/main.go (1)
158-181: Missing prefix validation for x-bf-vk header (previously flagged).The
x-bf-vkheader on line 162 is returned without validating that it starts withVirtualKeyPrefix(sk-bf-), while theAuthorizationBearer (line 168) andx-api-key(line 178) headers correctly validate the prefix. This inconsistency could allow invalid virtual keys to pass when provided via thex-bf-vkheader.🔎 Proposed fix
vkHeader := ctx.Request.Header.Peek("x-bf-vk") if string(vkHeader) != "" { - return bifrost.Ptr(string(vkHeader)) + vkValue := string(vkHeader) + if strings.HasPrefix(strings.ToLower(vkValue), VirtualKeyPrefix) { + return bifrost.Ptr(vkValue) + } }Based on past review comments from commit history; this issue remains unaddressed.
framework/plugins/dynamicplugin.go (1)
140-147: Critical: Type mismatch in symbol lookup - this issue appears unresolved.This is the same critical issue flagged in the previous review comment (marked as "✅ Addressed in commit ddfbaf4"), but the code still shows the incorrect pattern:
Line 145 attempts to cast
httpTransportMiddlewareSymdirectly toschemas.BifrostHTTPMiddleware, but the example plugin atexamples/plugins/hello-world/main.goexportsHTTPTransportMiddlewareas a factory function with signaturefunc() schemas.BifrostHTTPMiddleware.This cast will always fail because the symbol is a function that returns the middleware, not the middleware itself. Compare with the correct pattern used for
GetNameon line 137.🔎 Required fix (same as previous review)
// Looking up for HTTPTransportMiddleware method httpTransportMiddlewareSym, err := plugin.Lookup("HTTPTransportMiddleware") if err != nil { return nil, err } - if dp.httpTransportMiddleware, ok = httpTransportMiddlewareSym.(schemas.BifrostHTTPMiddleware); !ok { - return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func(next fasthttp.RequestHandler) fasthttp.RequestHandler") + httpTransportMiddlewareFunc, ok := httpTransportMiddlewareSym.(func() schemas.BifrostHTTPMiddleware) + if !ok { + return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func() schemas.BifrostHTTPMiddleware") } + dp.httpTransportMiddleware = httpTransportMiddlewareFunc()Note: The previous review indicated this was addressed in commit ddfbaf4, but the issue persists in the current code. Please verify whether the fix was reverted or if this PR is based on an older commit.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
examples/plugins/hello-world/go.sumis excluded by!**/*.sum
📒 Files selected for processing (41)
core/mcp.gocore/schemas/plugin.godocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goplugins/governance/go.modplugins/governance/main.goplugins/jsonparser/main.goplugins/logging/main.goplugins/maxim/main.goplugins/mocker/main.goplugins/otel/main.goplugins/semanticcache/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/go.modui/components/sidebar.tsx
💤 Files with no reviewable changes (1)
- framework/configstore/tables/mcp.go
✅ Files skipped from review due to trivial changes (1)
- docs/plugins/getting-started.mdx
🚧 Files skipped from review as they are similar to previous changes (14)
- transports/bifrost-http/handlers/providers.go
- core/mcp.go
- transports/bifrost-http/handlers/integrations.go
- transports/bifrost-http/handlers/logging.go
- examples/plugins/hello-world/go.mod
- transports/bifrost-http/handlers/session.go
- plugins/otel/main.go
- transports/bifrost-http/handlers/governance.go
- transports/bifrost-http/handlers/middlewares_test.go
- core/schemas/plugin.go
- transports/go.mod
- transports/bifrost-http/handlers/cache.go
- plugins/semanticcache/main.go
- plugins/maxim/main.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
docs/docs.jsontransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/plugins.goframework/plugins/dynamicplugin_test.gotransports/bifrost-http/handlers/inference.godocs/plugins/writing-plugin.mdxplugins/logging/main.goui/components/sidebar.tsxplugins/telemetry/main.goplugins/governance/go.moddocs/plugins/migration-guide.mdxplugins/governance/main.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.goplugins/mocker/main.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/server/server.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/config.goplugins/jsonparser/main.gotransports/bifrost-http/handlers/health.goexamples/plugins/hello-world/main.go
🧠 Learnings (2)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/plugins.goframework/plugins/dynamicplugin_test.gotransports/bifrost-http/handlers/inference.goplugins/logging/main.goplugins/telemetry/main.goplugins/governance/main.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.goplugins/mocker/main.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/server/server.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/config.goplugins/jsonparser/main.gotransports/bifrost-http/handlers/health.goexamples/plugins/hello-world/main.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/health.go
🧬 Code graph analysis (19)
transports/bifrost-http/handlers/mcpServer.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/websocket.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/plugins.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
transports/bifrost-http/handlers/inference.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/telemetry/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/mcp.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/integrations/router.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/middlewares.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)
transports/bifrost-http/server/server.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)transports/bifrost-http/handlers/middlewares.go (1)
TransportInterceptorMiddleware(48-68)transports/bifrost-http/lib/config.go (1)
Config(184-215)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)examples/plugins/hello-world/main.go (2)
GetName(15-17)HTTPTransportMiddleware(19-26)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
transports/bifrost-http/handlers/health.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(36-36)
🔇 Additional comments (25)
ui/components/sidebar.tsx (2)
208-208: Verify whether this UI change belongs in this PR.This styling change (reducing border radius from
rounded-mdtorounded-sm) appears unrelated to the PR's stated objectives, which focus on plugins v2 architecture and replacingTransportInterceptorwithHTTPTransportMiddlewarein the Go backend.Consider whether this visual adjustment should be:
- Moved to a separate UI-focused PR for clearer change tracking
- Documented in the PR description if intentionally bundled
The change itself is technically correct.
Based on coding guidelines, which emphasize reviewing changes in light of the whole PR stack context.
243-243: Same styling adjustment as line 208.This change maintains visual consistency with the main menu button adjustment, but the same concern about PR scope applies.
plugins/governance/go.mod (1)
8-11: Pinned dependency versions are current and secure.The promotion of
bytedance/sonic v1.14.1andvalyala/fasthttp v1.67.0from indirect to direct dependencies aligns with the middleware refactor. v1.67.0 includes security fixes from multiple security researchers, with improvements to URL validation, IPv6 validation, scheme validation, and host validation. bytedance/sonic shows 0 unresolved vulnerabilities. These versions are consistent across all plugins in the PR stack.docs/plugins/migration-guide.mdx (2)
1-307: Comprehensive and well-structured migration guide.The documentation thoroughly covers the migration from
TransportInterceptortoHTTPTransportMiddleware, including:
- Clear comparison table of differences
- Step-by-step migration instructions
- Common migration patterns with before/after examples
- Troubleshooting section for common issues
The code examples are consistent with the actual implementation in the codebase (e.g.,
examples/plugins/hello-world/main.goshows the same middleware pattern).
259-264: The documented execution order in the migration guide is accurate. No changes needed.Likely an incorrect or invalid review comment.
docs/docs.json (1)
135-136: LGTM!The migration guide is appropriately placed after the "writing-plugin" documentation in the navigation structure, providing a logical flow for plugin developers.
transports/bifrost-http/handlers/inference.go (1)
365-390: LGTM!The
RegisterRoutessignature update to acceptschemas.BifrostHTTPMiddlewarealigns with the project-wide migration to the new middleware pattern. The change is consistent with the type definition incore/schemas/plugin.goand other handler files in this PR.transports/bifrost-http/handlers/plugins.go (1)
53-59: LGTM!The
RegisterRoutessignature update is consistent with the middleware type migration across all handler files in this PR.plugins/telemetry/main.go (1)
279-282: LGTM - No-op implementation is appropriate.The telemetry plugin correctly implements
HTTPTransportMiddlewarereturningnilsince it doesn't need to intercept HTTP transport requests. The framework safely filters out nil middlewares before chaining them. The plugin's metrics collection is handled viaPreHook/PostHookfor Bifrost-level metrics and the separateHTTPMiddlewaremethod for HTTP request metrics.transports/bifrost-http/handlers/config.go (1)
15-15: LGTM: Consistent type migration to schemas.BifrostHTTPMiddleware.The import addition and signature change align with the broader architectural migration from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddleware. The routing logic and middleware chaining remain unchanged.Also applies to: 55-62
transports/bifrost-http/handlers/websocket.go (1)
14-14: LGTM: Consistent type migration.The import and signature change align with the project-wide migration to
schemas.BifrostHTTPMiddleware. WebSocket route registration logic remains unchanged.Also applies to: 51-53
transports/bifrost-http/handlers/ui.go (1)
11-11: LGTM: Consistent type migration.The import and signature change align with the architectural migration to
schemas.BifrostHTTPMiddleware. UI route registration logic remains unchanged.Also applies to: 29-32
transports/bifrost-http/handlers/mcpServer.go (1)
72-76: LGTM: Consistent type migration.The signature change aligns with the project-wide migration to
schemas.BifrostHTTPMiddleware. MCP server route registration logic remains unchanged.transports/bifrost-http/handlers/mcp.go (1)
43-51: LGTM: Consistent type migration.The signature change aligns with the project-wide migration to
schemas.BifrostHTTPMiddleware. MCP tool route registration logic remains unchanged.plugins/logging/main.go (1)
196-199: LGTM: Appropriate no-op middleware implementation.The logging plugin correctly returns
nilfromHTTPTransportMiddleware()since it performs its functionality throughPreHook/PostHookrather than HTTP transport interception. This aligns with the new plugin architecture where middleware is optional.framework/plugins/dynamicplugin_test.go (1)
54-84: LGTM: Comprehensive test coverage for new middleware API.The test properly exercises the new
HTTPTransportMiddleware()API by:
- Obtaining the middleware factory from the plugin
- Wrapping a mock next handler
- Invoking the wrapped handler
- Verifying the next handler is called
This validates the middleware chain pattern introduced by the plugins v2 architecture.
transports/bifrost-http/integrations/router.go (1)
72-73: LGTM: Consistent interface and implementation updates.The
ExtensionRouterinterface andGenericRouterimplementation both correctly update theirRegisterRoutessignatures to acceptschemas.BifrostHTTPMiddleware. The route registration logic and middleware chaining remain unchanged.Also applies to: 320-368
transports/bifrost-http/handlers/health.go (1)
27-29: LGTM: Clean type migration.The middleware parameter type correctly migrated from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddleware, aligning with the PR's centralization goal.plugins/mocker/main.go (1)
481-484: LGTM: Appropriate no-op implementation.Returning
nilis correct for the mocker plugin since it operates at the PreHook/PostHook level rather than HTTP transport layer. The middleware chain correctly handlesnilvalues (as seen in handlers/middlewares.go line 60).transports/bifrost-http/lib/middleware.go (1)
11-23: LGTM: Type migration maintains correct chaining logic.The signature update to use
schemas.BifrostHTTPMiddlewareis clean and maintains the correct right-to-left construction order for middleware chaining.docs/plugins/writing-plugin.mdx (1)
65-241: LGTM: Comprehensive documentation for breaking change.The versioned tabs clearly distinguish v1.3.x from v1.4.x+ patterns, and the HTTPTransportMiddleware section provides appropriate warnings about runtime scope and invocation context. The migration guide is thorough and will help plugin developers adapt to the new middleware pattern.
transports/bifrost-http/handlers/middlewares.go (1)
47-68: LGTM: Correct plugin middleware collection and chaining.The middleware now properly collects non-nil
HTTPTransportMiddleware()results from all plugins and chains them vialib.ChainMiddlewares. The logic correctly handlesnilreturns (line 60-62) and maintains proper execution order.plugins/governance/main.go (1)
184-234: LGTM: HTTPTransportMiddleware correctly implements governance logic.The middleware properly:
- Parses virtual keys from headers
- Validates virtual key existence and active status
- Adds MCP include-tools headers
- Performs load balancing and provider selection
- Marshals/unmarshals request body using Sonic
The error handling appropriately continues with
next(ctx)on errors, allowing requests to proceed without governance when issues occur.transports/bifrost-http/server/server.go (1)
958-965: LGTM: Consistent middleware type migration across route registration.All route registration methods (
RegisterInferenceRoutes,RegisterAPIRoutes,RegisterUIRoutes) andPrepareCommonMiddlewarescorrectly migrated fromlib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddleware. The middleware ordering at line 1230 appropriately placesTransportInterceptorMiddlewarefirst to ensure plugin HTTP transport middleware executes before other inference middlewares.Also applies to: 968-1051, 1054-1057, 1086-1097
plugins/jsonparser/main.go (1)
86-89: The nil return value is safely handled—nil middleware is filtered out before invocation.Looking at the middleware chain construction in
transports/bifrost-http/handlers/middlewares.go, the code checks each plugin's middleware before using it:middleware := plugin.HTTPTransportMiddleware() if middleware == nil { continue // Nil values are skipped, not added to the chain } pluginsMiddlewareChain = append(pluginsMiddlewareChain, middleware)Only non-nil middlewares are added to
pluginsMiddlewareChain, so nil function pointers never reach invocation. This is a well-established pattern used by 7 other plugins (telemetry, logging, maxim, mocker, otel, semanticcache, governance) with the same "is not used for this plugin" comment. The implementation is safe and requires no changes.Likely an incorrect or invalid review comment.
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
d632297 to
bfb02bb
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (4)
examples/plugins/hello-world/main.go (1)
19-26: Note: Existing issue flagged in previous review.The middleware doesn't set the context value that PreHook/PostHook expect to read (line 29, 38-39). This has already been flagged in a previous review.
plugins/governance/main.go (2)
158-181: Missing prefix validation for x-bf-vk header.The
x-bf-vkheader (lines 161-163) is returned without validating theVirtualKeyPrefix, while theAuthorization(line 168) andx-api-key(line 177) headers have this validation. This inconsistency was flagged in a previous review.🔎 Proposed fix
vkHeader := ctx.Request.Header.Peek("x-bf-vk") if string(vkHeader) != "" { - return bifrost.Ptr(string(vkHeader)) + vkValue := string(vkHeader) + if strings.HasPrefix(strings.ToLower(vkValue), VirtualKeyPrefix) { + return bifrost.Ptr(vkValue) + } }
212-217: Fix error messages to reflect unmarshal operation.Lines 214 and 226 both log "failed to marshal" but the operations are
sonic.Unmarshal(line 212) andsonic.Marshal(line 224) respectively. Line 214's message should say "unmarshal".🔎 Proposed fix
- p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body: %v", err)framework/plugins/dynamicplugin.go (1)
140-147: Type mismatch between dynamic plugin loading and example plugin export still present.The example plugin at
examples/plugins/hello-world/main.go:18-25exportsHTTPTransportMiddlewareas a factory functionfunc() schemas.BifrostHTTPMiddleware, but line 145 casts directly toschemas.BifrostHTTPMiddleware. This type assertion will fail at runtime.Either:
- Update this code to cast to
func() schemas.BifrostHTTPMiddlewareand invoke it (consistent with theGetNamepattern on line 137), or- Update the example plugin to export the middleware directly instead of a factory function.
🔎 Proposed fix (Option 1: align with factory pattern)
// Looking up for HTTPTransportMiddleware method httpTransportMiddlewareSym, err := plugin.Lookup("HTTPTransportMiddleware") if err != nil { return nil, err } - if dp.httpTransportMiddleware, ok = httpTransportMiddlewareSym.(schemas.BifrostHTTPMiddleware); !ok { - return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func(next fasthttp.RequestHandler) fasthttp.RequestHandler") + httpTransportMiddlewareFunc, ok := httpTransportMiddlewareSym.(func() schemas.BifrostHTTPMiddleware) + if !ok { + return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func() schemas.BifrostHTTPMiddleware") } + dp.httpTransportMiddleware = httpTransportMiddlewareFunc()
🧹 Nitpick comments (6)
core/bifrost.go (2)
70-71: Tracer initialization and retry span wiring are soundThe new
tracerfield onBifrostand initialization inInit(usingconfig.Tracerorschemas.DefaultTracer()) guarantee a non‑nil tracer for downstream use, and passing it intoexecuteRequestWithRetriescentralizes span creation for all provider calls. The generic retry helper correctly:
- Records retry count in context and attributes,
- Wraps each attempt in its own span (
llm.callfor first,retry.attempt.Nfor retries),- Attaches provider/model/requestType metadata and error/status details.
One behavioral nuance: because
*ctxis updated with the span context each time, later attempts become children of the previous attempt’s span (nested retries). If you’d prefer all attempts as siblings under the same parent span, you could preserve the original parent span ID and avoid chaining through the previous attempt’s span, but the current behavior is still trace‑valid.Also applies to: 108-127, 2729-2837
2283-2311: Fallback, key-selection, and plugin hook tracing are coherent; only minor hierarchy nitsThe added spans around:
- Fallback attempts (
fallback.<provider>.<model>in both unary and streaming paths),- Key selection (
key.selectioninrequestWorker), and- Per‑plugin
PreHook/PostHookexecutions,are all well-scoped and tagged (provider, model, fallback index, plugin name, errors). They also correctly propagate the active span ID via
BifrostContextKeySpanIDso downstream operations become children of the right span.Two minor, optional refinements you could consider:
For key selection, you currently end the
key.selectionspan before updatingreq.ContexttokeySpanCtx, so subsequent LLM spans become children of a finished span. That’s valid for tracing, but if you want a cleaner hierarchy you could:
- Set
req.Context = keySpanCtxbefore callingselectKeyFromProviderForModel, and- Keep the parent span as the outer request or root span instead of making the LLM call a child of the key-selection span.
In plugin hooks, you set a new span ID in the plugin context but only sync back to the parent context after all hooks via
GetParentCtxWithUserValues(). If you ever need per-plugin child spans to be visible immediately to nested operations, you might want to push span IDs into the parent context earlier, though the current approach is consistent and safe.No functional bugs here; these are purely trace-structure considerations.
Also applies to: 2381-2408, 2896-2924, 3157-3212, 3221-3247, 3267-3271
core/schemas/plugin.go (1)
4-8: Update Plugin execution-order docs to matchHTTPTransportMiddlewareThe interface and
BifrostHTTPMiddlewaretype correctly move transport interception to a middleware model, andObservabilityPluginis well-documented. The comment block abovePluginstill describes step 1 asTransportInterceptorthough the method is nowHTTPTransportMiddleware()returningBifrostHTTPMiddleware. To avoid confusion for plugin authors, consider updating that execution-order description to referenceHTTPTransportMiddlewareand the HTTP middleware chain rather than the oldTransportInterceptorterminology. No runtime issues here.Also applies to: 36-39, 47-79, 109-137
framework/tracing/propagation.go (1)
10-22: W3C trace-context helpers look correct; consider clarifyingExtractParentIDnamingThe W3C parsing/validation helpers (
ParseTraceparent,FormatTraceparent,InjectTraceContext) are defensively implemented and match the standardversion-traceid-parentid-traceflagsformat.One minor naming nit:
ExtractParentIDcurrently returnsctx.TraceID(the upstream trace ID) rather thanctx.ParentID(the parent span ID). The comment says “parent trace ID”, and you pass this intoCreateTrace(parentID)inTracingMiddleware, so the behavior is consistent, but the function name and struct field names can be confusing. Consider either:
- Renaming it to something like
ExtractUpstreamTraceID, or- Returning both trace ID and parent span ID via
ExtractTraceContextwhere you already expose both.Functionally this is fine; this is just about avoiding future misunderstandings.
Also applies to: 24-37, 38-95, 97-117, 119-127
plugins/otel/converter.go (2)
165-170: Potential overflow when converting largeuint64values toint64.Lines 169-170 convert
uint64toint64which will overflow for values greater thanmath.MaxInt64(~9.2 quintillion). While this is unlikely for typical use cases, it could silently produce incorrect negative values for very large unsigned integers stored in span attributes.🔎 Suggested handling for large uint64
case uint64: + if v > uint64(^uint(0)>>1) { // v > MaxInt64 + return kvStr(key, fmt.Sprintf("%d", v)) + } return kvInt(key, int64(v))
68-69: Silently ignored error fromhex.DecodeString.While the input is sanitized and padded, explicitly handling the error would be more defensive. The error is discarded with
bytes, _ := hex.DecodeString(cleaned).🔎 Optional: log or return empty on error
- bytes, _ := hex.DecodeString(cleaned) - return bytes + bytes, err := hex.DecodeString(cleaned) + if err != nil { + return make([]byte, length) + } + return bytes
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
core/go.sumis excluded by!**/*.sumexamples/plugins/hello-world/go.sumis excluded by!**/*.sum
📒 Files selected for processing (55)
core/bifrost.gocore/bifrost_test.gocore/go.modcore/mcp.gocore/schemas/bifrost.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.godocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goplugins/governance/go.modplugins/governance/main.goplugins/jsonparser/main.goplugins/logging/main.goplugins/maxim/main.goplugins/mocker/main.goplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/semanticcache/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/go.modui/components/sidebar.tsxui/components/ui/switch.tsx
💤 Files with no reviewable changes (2)
- framework/configstore/tables/mcp.go
- plugins/otel/ttlsyncmap.go
🚧 Files skipped from review as they are similar to previous changes (16)
- core/mcp.go
- transports/bifrost-http/handlers/websocket.go
- transports/bifrost-http/handlers/inference.go
- transports/bifrost-http/handlers/plugins.go
- transports/bifrost-http/handlers/ui.go
- docs/plugins/migration-guide.mdx
- transports/bifrost-http/integrations/router.go
- transports/bifrost-http/handlers/integrations.go
- docs/plugins/writing-plugin.mdx
- transports/go.mod
- plugins/governance/go.mod
- transports/bifrost-http/handlers/health.go
- ui/components/sidebar.tsx
- transports/bifrost-http/handlers/mcp.go
- plugins/mocker/main.go
- plugins/logging/main.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/semanticcache/main.goplugins/telemetry/main.gocore/go.modtransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/session.gocore/schemas/bifrost.goplugins/jsonparser/main.goframework/tracing/propagation.goplugins/maxim/main.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/middlewares.goframework/plugins/dynamicplugin_test.gotransports/bifrost-http/lib/middleware.goframework/tracing/store.gotransports/bifrost-http/handlers/cache.goframework/tracing/helpers.goexamples/plugins/hello-world/go.modframework/tracing/tracer.gocore/schemas/plugin.gocore/schemas/tracer.goplugins/otel/main.gotransports/bifrost-http/handlers/config.goui/components/ui/switch.tsxexamples/plugins/hello-world/main.gocore/bifrost_test.gotransports/bifrost-http/handlers/middlewares_test.gocore/schemas/trace.goplugins/otel/converter.godocs/docs.jsonframework/tracing/llmspan.goplugins/governance/main.gocore/bifrost.godocs/plugins/getting-started.mdx
🧠 Learnings (3)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
plugins/semanticcache/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/session.gocore/schemas/bifrost.goplugins/jsonparser/main.goframework/tracing/propagation.goplugins/maxim/main.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/middlewares.goframework/plugins/dynamicplugin_test.gotransports/bifrost-http/lib/middleware.goframework/tracing/store.gotransports/bifrost-http/handlers/cache.goframework/tracing/helpers.goframework/tracing/tracer.gocore/schemas/plugin.gocore/schemas/tracer.goplugins/otel/main.gotransports/bifrost-http/handlers/config.goexamples/plugins/hello-world/main.gocore/bifrost_test.gotransports/bifrost-http/handlers/middlewares_test.gocore/schemas/trace.goplugins/otel/converter.goframework/tracing/llmspan.goplugins/governance/main.gocore/bifrost.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/middlewares_test.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
🧬 Code graph analysis (27)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
plugins/telemetry/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/governance.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/mcpServer.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/bifrost.go (1)
core/schemas/tracer.go (1)
Tracer(13-30)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/maxim/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (2)
GetName(15-17)HTTPTransportMiddleware(19-26)
transports/bifrost-http/handlers/providers.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/server/server.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)transports/bifrost-http/handlers/middlewares.go (1)
TransportInterceptorMiddleware(49-69)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/store.go (2)
core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
transports/bifrost-http/handlers/cache.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/tracer.go (5)
framework/tracing/store.go (1)
TraceStore(13-23)core/schemas/trace.go (4)
SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)core/schemas/tracer.go (2)
SpanHandle(8-8)Tracer(13-30)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (1)
BifrostContextKeySpanID(144-144)
core/schemas/plugin.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
core/schemas/tracer.go (2)
core/schemas/trace.go (2)
SpanKind(115-115)SpanStatus(143-143)framework/tracing/helpers.go (1)
EndSpan(50-56)
plugins/otel/main.go (3)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)core/schemas/context.go (1)
BifrostContext(31-41)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/bifrost_test.go (1)
core/schemas/tracer.go (1)
DefaultTracer(51-53)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/trace.go (2)
framework/tracing/helpers.go (1)
AddSpan(32-38)ui/lib/constants/logs.ts (1)
Status(163-163)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (16)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
core/bifrost.go (1)
framework/tracing/helpers.go (1)
EndSpan(50-56)
🔇 Additional comments (39)
ui/components/ui/switch.tsx (1)
16-16: Confirm the switch design change from pill-shaped to slightly rounded corners.Standard implementations like Radix UI use
rounded-fullfor both the Root and Thumb of switch components, creating the characteristic pill-shaped appearance. Changing torounded-sm(2px border-radius) introduces a visual breaking change that deviates from established UI patterns. Ensure this design direction aligns with the broader UI system redesign and document this change in migration notes so users understand the visual impact.core/go.mod (1)
27-27: LGTM! Routine dependency updates.The addition of
semver/v3and update topprofare indirect dependencies, likely pulled in transitively by the new plugin architecture changes. These updates appear routine and appropriate for the PR's scope.Also applies to: 50-50
docs/plugins/getting-started.mdx (1)
51-103: LGTM! Excellent version-specific documentation.The use of tabs to document both v1.4.x+ (with
HTTPTransportMiddleware) and v1.3.x (withTransportInterceptor) provides a clear migration path for users. The execution order is properly documented for each version, making the breaking change easy to understand.docs/docs.json (1)
135-136: LGTM! Essential migration guide added.Adding the migration guide to the documentation navigation is appropriate for a breaking change. The placement after "writing-plugin" makes logical sense for users exploring plugin development.
transports/bifrost-http/handlers/config.go (1)
15-15: LGTM! Consistent middleware type migration.The change from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddlewarealigns with the PR's goal of consolidating middleware types in the core schemas package. This is part of the broader migration to the new plugin architecture.Also applies to: 55-55
transports/bifrost-http/handlers/providers.go (1)
80-80: LGTM! Consistent middleware type migration.The middleware type change matches the pattern applied across all handler files, maintaining consistency with the new plugin architecture.
transports/bifrost-http/handlers/logging.go (1)
42-42: LGTM! Consistent middleware type migration.The change maintains consistency with the middleware type migration applied across all handler files in this PR.
transports/bifrost-http/handlers/middlewares_test.go (1)
308-308: LGTM! Comprehensive test coverage maintained.All test cases have been consistently updated to use
schemas.BifrostHTTPMiddlewareinstead oflib.BifrostHTTPMiddleware. The test logic remains unchanged, ensuring that middleware chaining, execution order, context modification, and short-circuit behavior continue to be properly verified.Also applies to: 335-335, 342-342, 349-349, 381-381, 408-408, 418-418, 426-426, 471-471, 480-480, 490-490
core/bifrost_test.go (1)
71-71: LGTM! Tests updated for tracer parameter.All test cases have been consistently updated to pass
schemas.DefaultTracer()toexecuteRequestWithRetries, aligning with the new signature that supports tracing. The use ofDefaultTracer()(a no-op tracer) is appropriate for unit tests, maintaining test isolation while exercising the new tracing parameter.Also applies to: 105-105, 139-139, 202-202, 275-275, 504-504
transports/bifrost-http/handlers/cache.go (1)
26-29: LGTM! Middleware type migration is correct.The RegisterRoutes signature has been properly updated to use
schemas.BifrostHTTPMiddlewareinstead oflib.BifrostHTTPMiddleware, aligning with the new centralized middleware type definition in core/schemas.transports/bifrost-http/handlers/mcpServer.go (1)
72-76: LGTM! Middleware type migration is consistent.The RegisterRoutes signature correctly adopts
schemas.BifrostHTTPMiddleware, maintaining consistency with the framework-wide middleware migration.transports/bifrost-http/handlers/session.go (1)
32-36: LGTM! Middleware type update is correct.RegisterRoutes properly migrated to use
schemas.BifrostHTTPMiddlewarewith no functional changes.core/schemas/bifrost.go (2)
25-25: LGTM! Tracer integration is well-designed.The optional Tracer field with documented nil-as-NoOp behavior allows distributed tracing to be integrated without breaking existing code. This follows good design patterns for optional observability features.
143-144: LGTM! Trace context keys follow established conventions.The new
BifrostContextKeyTraceIDandBifrostContextKeySpanIDconstants are properly typed and follow the existingBifrostContextKeynaming convention, enabling trace propagation through the Bifrost context.framework/plugins/dynamicplugin_test.go (1)
54-84: LGTM! Test correctly validates new middleware pattern.The updated test properly exercises the
HTTPTransportMiddleware()method by:
- Creating a mock next handler with a flag to track invocation
- Obtaining and wrapping the middleware function
- Verifying the wrapped handler correctly calls the next handler
This test implementation correctly validates the new middleware contract.
transports/bifrost-http/handlers/governance.go (1)
154-175: LGTM! Governance routes properly migrated to new middleware type.RegisterRoutes correctly updated to use
schemas.BifrostHTTPMiddleware, maintaining consistency with the repository-wide middleware migration.examples/plugins/hello-world/go.mod (1)
6-23: fasthttp v1.67.0 is secure for use.The version includes security improvements for URL validation and IPv6 address validation, and no known vulnerabilities were identified for this release. The addition is appropriate for the new HTTP middleware pattern.
plugins/telemetry/main.go (1)
279-282: No action needed. The telemetry plugin correctly returnsnilfromHTTPTransportMiddleware()since this plugin does not require HTTP transport-level interception. The middleware chain collection logic intransports/bifrost-http/handlers/middlewares.goexplicitly filters out nil middleware functions before passing them toChainMiddlewares, preventing any nil pointer dereferences. This is a safe and intentional pattern.Likely an incorrect or invalid review comment.
plugins/semanticcache/main.go (1)
338-341: LGTM!The no-op implementation is correct for this plugin, which doesn't require HTTP transport middleware.
transports/bifrost-http/lib/middleware.go (1)
3-6: LGTM!The migration to the centralized
schemas.BifrostHTTPMiddlewaretype is correct and maintains backward compatibility.Also applies to: 11-11
plugins/maxim/main.go (1)
124-127: LGTM!The no-op implementation is appropriate for this plugin, which doesn't require HTTP transport middleware.
core/schemas/tracer.go (1)
1-57: LGTM!The Tracer interface is well-designed with a minimal surface area. The NoOpTracer provides a safe default, and the compile-time interface check ensures correctness.
plugins/jsonparser/main.go (1)
86-89: LGTM!The no-op implementation is correct for this plugin, which processes responses in PostHook rather than at the transport layer.
framework/tracing/store.go (1)
1-302: LGTM!The TraceStore implementation is well-designed with proper:
- Object pooling to reduce allocations
- Thread-safe concurrent access via sync.Map
- TTL-based cleanup with graceful shutdown
- Pre-allocated capacities for slices to minimize reallocations
transports/bifrost-http/server/server.go (1)
958-965: Middleware type migration toschemas.BifrostHTTPMiddlewarelooks consistentAll server route registration and middleware preparation now consistently use
schemas.BifrostHTTPMiddleware, and inference middlewares correctly prependTransportInterceptorMiddleware(s.Config)while preserving existing behavior. No issues spotted here.Also applies to: 968-1057, 1086-1097, 1220-1234
transports/bifrost-http/handlers/middlewares.go (1)
20-46: HTTP middleware type changes and new tracing middleware integrate cleanly
CorsMiddleware,AuthMiddleware, andTransportInterceptorMiddlewarenow correctly exposeschemas.BifrostHTTPMiddleware, matching the newPlugincontract.TransportInterceptorMiddlewaresafely skips plugins that returnnilmiddleware and chains the rest vialib.ChainMiddlewareswithout altering ordering semantics.TracingMiddlewarecleanly wires request‑scoped traces:
- Derives a parent ID from W3C trace headers and calls
store.CreateTrace(parentID).- Stores the resulting trace ID and root span ID in
RequestCtxuser values for downstream components.- Ends the root span with an OK or error status based on the HTTP response code.
- Asynchronously completes the trace, forwards it to all non‑nil observability plugins via
Inject, and then releases it back to the store.This is a solid foundation for HTTP‑level tracing; any further enhancements (e.g., sampling or additional HTTP attributes) can be layered on later without changing the API.
Also applies to: 48-69, 87-188, 190-264, 266-281
framework/tracing/tracer.go (1)
1-21: StoreTracer correctly bridgesschemas.TracertoTraceStore
StoreTracercleanly implementsschemas.Tracer: it:
- Reads the current trace ID and optional parent span ID from context,
- Starts either root or child spans and propagates the new span ID back into context,
- Safely no‑ops when there’s no trace ID or when the handle is nil/invalid,
- Updates attributes and events via
TraceStoreandSpanmethods.As long as callers only construct
StoreTracerwith a non‑nil*TraceStore, this provides a solid bridge between the core tracer interface and the in‑memory trace store.Also applies to: 23-29, 30-57, 59-67, 69-82, 84-102, 104-106
framework/tracing/helpers.go (1)
1-21: Tracing helper functions are straightforward and defensiveThe helpers (
GetTraceID,GetTrace,AddSpan,AddChildSpan,EndSpan,SetSpanAttribute,AddSpanEvent) provide a clean, minimal API overTraceStore, with appropriate nil/empty checks for context, trace, and span. They’re safe to use across the codebase and complement the higher-levelStoreTracernicely.Also applies to: 22-29, 31-38, 40-47, 49-56, 58-69, 71-82
plugins/otel/main.go (1)
55-75: No action required. The code correctly usesstrings.SplitSeq, a standard library function added in Go 1.24 that returns an iterator over substrings. This pattern is valid and preferred for iterating over split results without allocating a slice.Likely an incorrect or invalid review comment.
framework/plugins/dynamicplugin.go (1)
36-39: LGTM for the middleware accessor method.The
HTTPTransportMiddleware()method correctly returns the stored middleware. SinceloadDynamicPluginrequires all symbols to be present and properly cast before returning, the field will never be nil when accessed.plugins/otel/converter.go (2)
72-88: Well-structured trace-to-OTEL conversion.The
convertTraceToResourceSpanfunction cleanly iterates through spans and assembles a properly structuredResourceSpanwith resource attributes and instrumentation scope. The implementation follows OTEL conventions.
231-269: Span kind and status mappings are comprehensive.Both
convertSpanKindandconvertSpanStatuscover all definedSpanKindandSpanStatusconstants fromcore/schemas/trace.go, with appropriate defaults for unrecognized values.core/schemas/trace.go (3)
21-38: Thread-safe span operations look correct.
AddSpanandGetSpanproperly acquire the mutex before accessing the sharedSpansslice. The linear search inGetSpanis acceptable for typical trace sizes.
67-82: Thread-safe attribute and event operations look correct.
SetAttributeandAddEventproperly acquire the mutex and handle nil map initialization. These follow the expected patterns for concurrent access.
154-325: Comprehensive attribute key definitions following OpenTelemetry conventions.The extensive set of
gen_ai.*namespace attribute keys is well-organized by category and follows OpenTelemetry semantic conventions. Good documentation comments explain the purpose.framework/tracing/llmspan.go (4)
1118-1132: Message extraction helpers are well-implemented.
extractChatMessagesproperly handles nil content by skipping such messages. The helper uses a pre-allocated slice with appropriate capacity.
1207-1228: Efficient content extraction withstrings.Builder.
extractMessageContentcorrectly usesstrings.Builderfor efficient concatenation when processing multiple content blocks.
12-57: Clean dispatcher pattern for request attribute population.
PopulateRequestAttributesprovides a well-organized entry point that dispatches to type-specific handlers based onRequestType. The nil check at the start prevents panics.
59-100: Response attribute dispatcher uses switch on pointer fields.
PopulateResponseAttributesusesswitch { case resp.X != nil: }pattern which works but differs from the request dispatcher'sswitch req.RequestTypepattern. This is acceptable sinceBifrostResponsemay not have an explicit type field.
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
bfb02bb to
b8f95ce
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
transports/bifrost-http/integrations/router.go (1)
513-518: Context parameter change breaks downstream context value propagation and requires fixing.The code sets Bifrost-specific context values in
bifrostCtxusing custom-typed keys (e.g.,schemas.BifrostContextKeySendBackRawResponse,schemas.BifrostContextKeyDirectKey) viacontext.WithValue. However, client methods now receivectx(fasthttp.RequestCtx) instead ofbifrostCtx. Since fasthttp.RequestCtx.Value() only supports string keys, these custom-typed context values set inbifrostCtxwill not be accessible to the client methods, breaking features that depend on them (such as direct key selection, raw response handling, and integration-type routing). PassbifrostCtxto client methods instead ofctxto preserve context value propagation.core/bifrost.go (1)
2731-2743: ClarifyPopulateLLMResponseAttributes’s contract whenrespis nilIn
executeRequestWithRetries, you always end the span and, for non-streaming calls, invoke:if resp, ok := any(result).(*schemas.BifrostResponse); ok { tracer.PopulateLLMResponseAttributes(handle, resp, bifrostError) }When the handler returns an error,
resultcan be a typed(*schemas.BifrostResponse)(nil), sookis true andrespis nil. Tracer implementations that assumerespis non-nil could panic here.Either:
- Guard the call with
resp != nil, or- Explicitly document in
Tracer.PopulateLLMResponseAttributesthatrespmay be nil whileerris non-nil and must be handled defensively.This keeps tracing backends robust under error paths.
Also applies to: 2789-2817
♻️ Duplicate comments (5)
examples/plugins/hello-world/main.go (1)
19-26: HTTPTransportMiddleware implementation is correct for dynamic plugins.The free function signature is appropriate for Go's plugin system, where exported symbols are looked up by name and wrapped by the
DynamicPluginadapter.However, note that
PreHook(line 29) andPostHook(lines 38-39) still attempt to read the context value"hello-world-plugin-transport-interceptor", which is no longer set by this middleware. This results invalue1always beingnil, breaking the example's context propagation demonstration. This issue was flagged in previous reviews.plugins/governance/main.go (2)
158-181: Missing prefix validation for x-bf-vk header.The
parseVirtualKeyfunction validates that values from theAuthorization(Bearer) andx-api-keyheaders start withVirtualKeyPrefix(lines 168, 177), but thex-bf-vkheader on lines 160-163 is returned without prefix validation. This inconsistency could allow invalid virtual keys to pass through when provided via thex-bf-vkheader.🔎 Proposed fix
vkHeader := ctx.Request.Header.Peek("x-bf-vk") if string(vkHeader) != "" { + vkValue := string(vkHeader) + if strings.HasPrefix(strings.ToLower(vkValue), VirtualKeyPrefix) { - return bifrost.Ptr(string(vkHeader)) + return bifrost.Ptr(vkValue) + } }
211-217: Fix error message accuracy.Line 214: The error message says "failed to marshal" but the operation is unmarshaling (
sonic.Unmarshal).🔎 Proposed fix
- p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body: %v", err)core/schemas/trace.go (1)
40-49: GuardTrace.Reset,Span.End, andSpan.Resetwith the existing mutex to avoid races
TraceandSpanboth embed amu sync.Mutex, and mutating methods likeAddSpan,GetSpan,SetAttribute, andAddEventuse it. However:
Trace.Resetclears IDs, spans, and attributes without locking.Span.EndupdatesEndTime,Status, andStatusMsgwithout locking.Span.Resetclears all fields and slices without locking.If these methods are ever called while other goroutines still hold references obtained via
GetTrace/GetSpan(or via the store), this can lead to data races despite the presence ofmu.Given you already rely on the mutex for other mutations, it would be safer and more consistent to wrap these methods with the same lock, e.g.:
func (t *Trace) Reset() { - t.TraceID = "" - t.ParentID = "" - t.RootSpan = nil - t.Spans = t.Spans[:0] - t.StartTime = time.Time{} - t.EndTime = time.Time{} - t.Attributes = nil + t.mu.Lock() + defer t.mu.Unlock() + t.TraceID = "" + t.ParentID = "" + t.RootSpan = nil + t.Spans = t.Spans[:0] + t.StartTime = time.Time{} + t.EndTime = time.Time{} + t.Attributes = nil }and similarly for
Span.EndandSpan.Reset.This keeps span/trace lifecycle mutations consistent with the rest of the API and avoids subtle races when pooling is combined with concurrent readers.
Also applies to: 84-104
framework/tracing/llmspan.go (1)
190-193: ExtraParams written directly to attrs map may cause attribute key collisions.This issue was previously flagged: user-provided keys from
ExtraParamsare written directly to the attributes map without namespacing. If a user sets a key like"gen_ai.request.model"in ExtraParams, it would overwrite the standard attribute set on line 22.This pattern repeats throughout the file at lines 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 768-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029.
🔎 Suggested approach: namespace ExtraParams
// ExtraParams for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) + attrs["gen_ai.extra."+k] = fmt.Sprintf("%v", v) }Apply this pattern consistently at all ExtraParams locations throughout the file.
🧹 Nitpick comments (3)
ui/components/ui/sheet.tsx (1)
103-103: Simplify redundantmb-6application.The base className already includes
mb-6, and the conditional also appliesmb-6whenexpandableis false, resulting in duplicate class application. While Tailwind handles this gracefully, the code can be simplified for clarity.🔎 Suggested simplification
-<div data-slot="sheet-header" className={cn("mb-6 flex items-center", sheetContext?.expandable ? "p-0" : "mb-6")} {...props}> +<div data-slot="sheet-header" className={cn("flex items-center", sheetContext?.expandable ? "p-0" : "mb-6")} {...props}>ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
181-201: Add aria-label to the action menu trigger for better accessibility.The dropdown menu implementation follows best practices by consolidating actions into an overflow menu. However, the icon-only trigger button should include an
aria-labelto improve screen reader accessibility.🔎 Suggested improvement
<DropdownMenuTrigger asChild> - <Button variant="ghost" size="icon"> + <Button variant="ghost" size="icon" aria-label="Log actions"> <MoreVertical className="h-3 w-3" /> </Button> </DropdownMenuTrigger>framework/tracing/helpers.go (1)
10-29: Helpers are sound; optionally guard against nilTraceStoreThe helper layer around
TraceStore(GetTraceID,GetTrace,AddSpan*,EndSpan,SetSpanAttribute,AddSpanEvent) is clean and includes good nil/empty-traceID checks, so they safely no-op when there is no active trace.If you ever expect these to be called before a tracer is fully wired, you might want an extra defensive
if store == nil { return }in the helpers that dereferencestoreto avoid panics from accidental nil stores. Right now, they assume the tracer always passes a non-nil store, which is OK but a bit brittle.Also applies to: 49-82
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
core/go.sumis excluded by!**/*.sumexamples/plugins/hello-world/go.sumis excluded by!**/*.sum
📒 Files selected for processing (57)
core/bifrost.gocore/bifrost_test.gocore/go.modcore/mcp.gocore/schemas/bifrost.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.godocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goplugins/governance/go.modplugins/governance/main.goplugins/jsonparser/main.goplugins/logging/main.goplugins/maxim/main.goplugins/mocker/main.goplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/semanticcache/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/go.modui/app/workspace/logs/views/logDetailsSheet.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsx
💤 Files with no reviewable changes (2)
- framework/configstore/tables/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (1)
- docs/plugins/migration-guide.mdx
🚧 Files skipped from review as they are similar to previous changes (17)
- core/mcp.go
- transports/bifrost-http/handlers/inference.go
- transports/bifrost-http/handlers/mcpServer.go
- plugins/mocker/main.go
- plugins/telemetry/main.go
- plugins/governance/go.mod
- plugins/maxim/main.go
- transports/bifrost-http/handlers/cache.go
- transports/bifrost-http/handlers/logging.go
- transports/bifrost-http/handlers/mcp.go
- transports/go.mod
- core/bifrost_test.go
- plugins/semanticcache/main.go
- ui/components/sidebar.tsx
- transports/bifrost-http/handlers/middlewares_test.go
- transports/bifrost-http/handlers/websocket.go
- ui/components/ui/switch.tsx
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/plugins.goframework/plugins/dynamicplugin_test.gotransports/bifrost-http/handlers/config.gocore/go.modtransports/bifrost-http/handlers/session.goui/app/workspace/logs/views/logDetailsSheet.tsxcore/schemas/bifrost.godocs/plugins/getting-started.mdxcore/schemas/plugin.goui/components/ui/sheet.tsxplugins/governance/main.gotransports/bifrost-http/handlers/providers.godocs/plugins/writing-plugin.mdxcore/schemas/tracer.goframework/tracing/store.gotransports/bifrost-http/server/server.goplugins/otel/main.gotransports/bifrost-http/handlers/health.goplugins/logging/main.gotransports/bifrost-http/handlers/middlewares.goframework/plugins/dynamicplugin.gocore/bifrost.goplugins/otel/converter.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/lib/middleware.goexamples/plugins/hello-world/main.goplugins/jsonparser/main.goframework/tracing/tracer.gocore/schemas/trace.goframework/tracing/propagation.goframework/tracing/helpers.goexamples/plugins/hello-world/go.modtransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/integrations.godocs/docs.jsonframework/tracing/llmspan.go
🧠 Learnings (3)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/plugins.goframework/plugins/dynamicplugin_test.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/session.gocore/schemas/bifrost.gocore/schemas/plugin.goplugins/governance/main.gotransports/bifrost-http/handlers/providers.gocore/schemas/tracer.goframework/tracing/store.gotransports/bifrost-http/server/server.goplugins/otel/main.gotransports/bifrost-http/handlers/health.goplugins/logging/main.gotransports/bifrost-http/handlers/middlewares.goframework/plugins/dynamicplugin.gocore/bifrost.goplugins/otel/converter.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/lib/middleware.goexamples/plugins/hello-world/main.goplugins/jsonparser/main.goframework/tracing/tracer.gocore/schemas/trace.goframework/tracing/propagation.goframework/tracing/helpers.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/integrations.goframework/tracing/llmspan.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/integrations.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
🧬 Code graph analysis (27)
transports/bifrost-http/handlers/governance.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/plugins.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
ui/app/workspace/logs/views/logDetailsSheet.tsx (3)
ui/components/ui/sheet.tsx (1)
SheetContent(137-137)ui/components/ui/alertDialog.tsx (9)
AlertDialog(83-83)AlertDialogTrigger(93-93)AlertDialogContent(86-86)AlertDialogHeader(89-89)AlertDialogTitle(92-92)AlertDialogDescription(87-87)AlertDialogFooter(88-88)AlertDialogCancel(85-85)AlertDialogAction(84-84)ui/components/ui/dropdownMenu.tsx (5)
DropdownMenu(193-193)DropdownMenuTrigger(207-207)DropdownMenuContent(195-195)DropdownMenuItem(197-197)DropdownMenuSeparator(202-202)
core/schemas/bifrost.go (2)
core/schemas/tracer.go (1)
Tracer(13-47)framework/tracing/tracer.go (1)
Tracer(14-16)
core/schemas/plugin.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/handlers/providers.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/tracer.go (4)
core/schemas/trace.go (3)
Trace(10-19)SpanKind(115-115)SpanStatus(143-143)framework/tracing/helpers.go (1)
EndSpan(50-56)core/schemas/bifrost.go (3)
BifrostRequest(168-188)BifrostResponse(330-350)BifrostError(469-478)framework/tracing/tracer.go (1)
Tracer(14-16)
transports/bifrost-http/server/server.go (4)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)transports/bifrost-http/handlers/middlewares.go (2)
TransportInterceptorMiddleware(50-70)NewTracingMiddleware(207-214)framework/tracing/store.go (1)
NewTraceStore(26-53)framework/tracing/tracer.go (1)
NewTracer(19-21)
plugins/otel/main.go (3)
core/schemas/plugin.go (1)
ObservabilityPlugin(123-137)core/schemas/context.go (1)
BifrostContext(31-41)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/handlers/health.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/logging/main.go (5)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)ui/lib/types/logs.ts (1)
TranscriptionInput(15-17)core/schemas/bifrost.go (1)
TranscriptionRequest(102-102)transports/bifrost-http/handlers/inference.go (1)
TranscriptionRequest(285-289)
transports/bifrost-http/handlers/middlewares.go (6)
core/schemas/plugin.go (3)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)Plugin(71-97)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)core/schemas/tracer.go (1)
Tracer(13-47)framework/tracing/propagation.go (1)
ExtractParentID(26-36)core/schemas/trace.go (3)
SpanKindHTTPRequest(131-131)SpanStatusError(151-151)SpanStatusOk(149-149)framework/tracing/helpers.go (1)
EndSpan(50-56)
core/bifrost.go (4)
core/schemas/tracer.go (2)
Tracer(13-47)DefaultTracer(80-82)framework/tracing/tracer.go (1)
Tracer(14-16)core/schemas/trace.go (3)
AttrProviderName(159-159)AttrRequestModel(160-160)SpanKind(115-115)framework/tracing/helpers.go (1)
EndSpan(50-56)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (17)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindHTTPRequest(131-131)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/tracer.go (6)
framework/tracing/store.go (1)
TraceStore(13-23)core/schemas/tracer.go (2)
Tracer(13-47)SpanHandle(8-8)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (4)
BifrostContextKeySpanID(144-144)BifrostRequest(168-188)BifrostResponse(330-350)BifrostError(469-478)framework/tracing/llmspan.go (3)
PopulateRequestAttributes(14-60)PopulateResponseAttributes(64-106)PopulateErrorAttributes(109-124)
core/schemas/trace.go (2)
framework/tracing/helpers.go (1)
AddSpan(32-38)ui/lib/constants/logs.ts (1)
Status(163-163)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(13-23)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
transports/bifrost-http/integrations/router.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (7)
ListModelsRequest(92-92)TextCompletionRequest(93-93)ChatCompletionRequest(95-95)ResponsesRequest(97-97)EmbeddingRequest(99-99)SpeechRequest(100-100)TranscriptionRequest(102-102)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/llmspan.go (6)
core/schemas/bifrost.go (18)
BifrostRequest(168-188)RequestType(89-89)TextCompletionRequest(93-93)EmbeddingRequest(99-99)TranscriptionRequest(102-102)SpeechRequest(100-100)ResponsesRequest(97-97)BatchCreateRequest(104-104)BatchListRequest(105-105)BatchRetrieveRequest(106-106)BatchCancelRequest(107-107)BatchResultsRequest(108-108)FileUploadRequest(109-109)FileListRequest(110-110)FileRetrieveRequest(111-111)FileDeleteRequest(112-112)FileContentRequest(113-113)BifrostResponse(330-350)core/schemas/trace.go (24)
AttrProviderName(159-159)AttrRequestModel(160-160)AttrError(208-208)AttrErrorType(209-209)AttrErrorCode(210-210)AttrVirtualKeyID(220-220)AttrVirtualKeyName(221-221)AttrSelectedKeyID(222-222)AttrSelectedKeyName(223-223)AttrTeamID(224-224)AttrTeamName(225-225)AttrCustomerID(226-226)AttrCustomerName(227-227)AttrNumberOfRetries(228-228)AttrFallbackIndex(229-229)AttrMaxTokens(163-163)AttrTemperature(164-164)AttrTopP(165-165)AttrStopSequences(166-166)AttrPresencePenalty(167-167)AttrFrequencyPenalty(168-168)AttrParallelToolCall(169-169)AttrRequestUser(170-170)AttrN(175-175)core/schemas/chatcompletions.go (1)
BifrostChatRequest(12-19)core/schemas/responses.go (1)
BifrostResponsesRequest(32-39)core/schemas/files.go (6)
BifrostFileUploadRequest(53-67)BifrostFileListRequest(109-128)BifrostFileRetrieveRequest(146-159)BifrostFileDeleteRequest(186-198)BifrostFileContentRequest(215-227)BifrostFileUploadResponse(90-106)framework/modelcatalog/main.go (1)
ModelCatalog(23-44)
🔇 Additional comments (53)
examples/plugins/hello-world/go.mod (2)
7-7: Verify core dependency version reflects breaking changes.The example plugin depends on
github.com/maximhq/bifrost/core v1.2.40. Given the breaking change (replacingTransportInterceptorwithHTTPTransportMiddleware), confirm this core version includes the new interface and that the hello-world plugin's implementation aligns with it.Also verify that the reorganization of direct and indirect dependencies is intentional and consistent with go mod tidy output (rather than manually maintained).
Also applies to: 11-23
8-8: No changes needed. The fasthttp v1.67.0 version in hello-world/go.mod is correct and consistent with the core module v1.2.40. The HTTPTransportMiddleware implementation in main.go properly uses the fasthttp.RequestHandler pattern, and all transitive dependencies (brotli, compress, bytebufferpool) are expected from fasthttp. Versions are aligned across the plugin stack.ui/app/workspace/logs/views/logDetailsSheet.tsx (4)
16-16: LGTM!The new imports for dropdown menu components and the MoreVertical icon are appropriate for the refactored action menu pattern.
Also applies to: 22-22
202-218: LGTM!The AlertDialog implementation for delete confirmation is well-structured. The dialog properly handles the destructive action with clear messaging, cancel option, and correctly closes the sheet after deletion.
171-171: Remove the expandable prop verification concern.The
expandableprop is properly implemented as a custom feature in the SheetContent component. It has a default value offalse, is integrated into the component's context, and controls responsive width and styling behavior. No changes needed.Likely an incorrect or invalid review comment.
195-195: Thedestructivevariant onDropdownMenuItemis already properly implemented in the wrapper component with full type safety and appropriate styling for light/dark modes and focus states. No changes needed.core/go.mod (1)
27-27: LGTM: Routine indirect dependency updates.The addition of
semver/v3and the update togoogle/pprofappear to be indirect dependency changes, likely pulled in by the new tracing infrastructure or other updated dependencies in this PR.Also applies to: 50-50
core/schemas/bifrost.go (2)
25-25: LGTM: Well-documented tracing support.The addition of the
Tracerfield toBifrostConfigwith clear documentation about the nil = NoOpTracer behavior is excellent. This provides a clean opt-in mechanism for distributed tracing.
143-144: LGTM: Tracing context keys follow established patterns.The new context keys for trace ID and span ID follow the existing
BifrostContextKeynaming convention and enable proper trace propagation through the request lifecycle. The comments clearly indicate these are set by the tracing middleware and tracer respectively.docs/docs.json (1)
135-136: LGTM: Essential documentation for breaking changes.Adding the migration guide to the Custom plugins section is crucial for this breaking change. The placement after "writing-plugin" makes logical sense for developers who need to update their plugins from v1.3.x to v1.4.x.
transports/bifrost-http/handlers/providers.go (1)
80-80: LGTM: Middleware type migration aligns with plugin v2 architecture.The migration from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddlewarecentralizes the middleware type definition in the core schemas package. This change is consistent with the broader plugin v2 architecture migration and maintains the same functional signature.transports/bifrost-http/handlers/plugins.go (1)
53-53: LGTM: Consistent middleware type migration.This change mirrors the migration pattern seen across all handler files, updating the middleware type to use the centralized
schemas.BifrostHTTPMiddlewaredefinition.transports/bifrost-http/handlers/integrations.go (1)
8-8: LGTM: Integration handler updated for middleware type migration.The addition of the schemas import and the update to use
schemas.BifrostHTTPMiddlewaremaintains consistency with the rest of the HTTP transport layer. The middleware forwarding logic to extensions remains unchanged.Also applies to: 37-37
transports/bifrost-http/handlers/ui.go (1)
11-11: LGTM: UI handler completes the middleware type migration.This change completes the systematic migration across all handler files. The UI serving logic remains unchanged while the middleware type is updated to the centralized
schemas.BifrostHTTPMiddleware.Also applies to: 29-29
plugins/logging/main.go (1)
196-199: ReturningnilforHTTPTransportMiddleware()is correct since the logging plugin only usesPreHookandPostHookfor request/response logging.The framework properly handles nil middleware returns. The plugin loader explicitly checks for nil middleware at line 62 in
transports/bifrost-http/handlers/middlewares.goand skips them, ensuring only active middlewares are passed to the chaining logic.transports/bifrost-http/handlers/session.go (1)
11-11: LGTM! Clean migration to schemas.BifrostHTTPMiddleware.The import and signature change correctly align with the new middleware type defined in
core/schemas/plugin.go. The routing logic remains unchanged, maintaining backward compatibility at the handler level.Also applies to: 32-32
transports/bifrost-http/handlers/config.go (1)
15-15: LGTM! Consistent middleware type migration.The signature update follows the same pattern as other handlers in this PR, correctly using the new
schemas.BifrostHTTPMiddlewaretype.Also applies to: 55-55
transports/bifrost-http/handlers/health.go (1)
9-9: LGTM! Handler signature properly updated.The middleware type migration is consistent with the broader refactor across all HTTP handlers.
Also applies to: 27-27
framework/plugins/dynamicplugin_test.go (1)
16-16: LGTM! Test properly validates the new middleware pattern.The test correctly:
- Creates a mock next handler to track invocation
- Obtains the middleware function from the plugin
- Wraps the mock handler with the middleware
- Verifies the middleware calls the next handler in the chain
This validates the middleware chaining semantics introduced by the new architecture.
Also applies to: 54-84
transports/bifrost-http/handlers/governance.go (1)
15-15: LGTM! Governance handler signature updated consistently.The middleware type migration aligns with the unified
schemas.BifrostHTTPMiddlewarepattern used across all HTTP handlers.Also applies to: 154-154
docs/plugins/getting-started.mdx (1)
51-68: LGTM! Excellent documentation of the breaking change.The versioned tabs clearly distinguish the v1.4.x+ API (with
HTTPTransportMiddleware) from the v1.3.x API (withTransportInterceptor), helping plugin developers understand the migration path. The execution order is properly documented for both versions.Also applies to: 90-103
core/schemas/plugin.go (3)
36-38: LGTM! Clean middleware type definition.The
BifrostHTTPMiddlewaretype follows the standard HTTP middleware pattern: a function that takes the next handler and returns a new handler. This is more idiomatic than the previousTransportInterceptorapproach.
75-79: LGTM! Plugin interface properly migrated to middleware pattern.The replacement of
TransportInterceptorwithHTTPTransportMiddleware() BifrostHTTPMiddlewareprovides a cleaner, more composable API for HTTP transport interception. The method signature is consistent with standard middleware patterns used in Go HTTP frameworks.
109-137: LGTM! ObservabilityPlugin interface enables async trace forwarding.The new interface properly extends
Pluginand adds theInjectmethod for receiving completed traces. The documentation clearly explains:
- Async invocation after response is written (no client latency impact)
- Usage via Go type assertion
- Expected backend implementations (OTEL, Datadog, Jaeger, etc.)
This is a well-designed extension point for observability integrations.
transports/bifrost-http/integrations/router.go (1)
72-72: Middleware type migration looks correct.The interface and method signatures are updated to use
schemas.BifrostHTTPMiddlewareinstead oflib.BifrostHTTPMiddleware, aligning with the centralized type definition incore/schemas/plugin.go. This is consistent across the codebase migration in this PR.Also applies to: 320-320
docs/plugins/writing-plugin.mdx (1)
65-126: Documentation for v1.4.x+ HTTPTransportMiddleware is clear and comprehensive.The new tab structure with side-by-side v1.3.x and v1.4.x+ examples provides excellent migration guidance. The
HTTPTransportMiddlewareexample correctly demonstrates the middleware pattern withnext(ctx)invocation.One minor observation: The example shows request modification before
next(ctx)and response modification after, which is good practice to document.plugins/jsonparser/main.go (1)
86-89: HTTPTransportMiddleware returning nil is appropriate for this plugin.Since the JSON parser plugin only processes responses (in
PostHook), returningnilfromHTTPTransportMiddlewareis correct. The middleware chain inlib.ChainMiddlewaresproperly skips nil middlewares.plugins/governance/main.go (1)
183-234: HTTPTransportMiddleware implementation looks solid.The middleware correctly:
- Parses virtual keys from multiple header sources
- Gracefully degrades on errors by calling
next(ctx)rather than blocking requests- Modifies headers and body appropriately using fasthttp APIs
- Follows the middleware pattern with
next(ctx)invocationtransports/bifrost-http/lib/middleware.go (1)
3-23: Clean migration to centralized middleware type.The
ChainMiddlewaresfunction now usesschemas.BifrostHTTPMiddlewarefrom the core schemas package instead of a local type definition. The chaining logic (right-to-left composition for left-to-right execution) remains unchanged and correct.framework/tracing/store.go (2)
56-82: Object pooling implementation looks correct.The
CreateTracemethod properly resets pooled objects before reuse:
- Generates new TraceID
- Resets time fields
- Clears RootSpan
- Resets slices while preserving capacity (efficient)
- Clears attributes map
This prevents data leakage between pooled objects.
120-165: Span lifecycle management is well implemented.The
StartSpanmethod properly:
- Gets span from pool
- Resets all fields (including events and attributes)
- Sets parent relationship based on root span
- Adds span to trace
The parent tracking logic (lines 154-159) correctly handles both root spans and child spans.
plugins/otel/main.go (2)
201-224: Inject method implementation is clean and correct.The
Injectmethod properly:
- Handles nil trace gracefully
- Checks client initialization
- Converts trace to OTEL ResourceSpan format
- Emits to collector with error handling
- Logs errors but returns them for upstream handling
This aligns well with the
ObservabilityPlugininterface contract described incore/schemas/plugin.go.
237-238: Good use of compile-time interface check.The
var _ schemas.ObservabilityPlugin = (*OtelPlugin)(nil)pattern ensures the plugin correctly implements the interface at compile time, catching signature mismatches early.transports/bifrost-http/handlers/middlewares.go (4)
49-70: TransportInterceptorMiddleware correctly collects and chains plugin middlewares.The implementation:
- Collects non-nil middlewares from all plugins
- Uses
lib.ChainMiddlewaresto compose them- Gracefully handles empty plugin lists
This is a clean migration from the old per-request interception pattern to middleware composition.
201-214: TracingMiddleware design with atomic plugin updates is well thought out.Using
atomic.Pointer[[]schemas.ObservabilityPlugin]allows thread-safe updates to observability plugins at runtime (e.g., during hot-reload) without requiring locks on the hot path. The constructor properly initializes and stores the initial plugin list.
284-299: GetObservabilityPlugins helper is clean and correct.The type assertion pattern correctly filters plugins implementing
ObservabilityPlugininterface. Early return for empty input is a good optimization.
247-277: Async trace forwarding pattern is sound, but lacks graceful shutdown coordination.The
deferwith goroutine pattern appropriately ensures trace completion and async execution, usingcontext.Background()since request context is done. However, the trace forwarding goroutines spawned here are untracked and will not be coordinated during server shutdown—traces may be lost if the server terminates while these goroutines are in flight.For production deployments, consider tracking in-flight traces with
sync.WaitGroupif guaranteed delivery is required.transports/bifrost-http/server/server.go (1)
958-1037: HTTP middleware type migration and tracing bootstrap look consistentThe migration to
schemas.BifrostHTTPMiddlewareacrossRegisterInferenceRoutes,RegisterAPIRoutes,RegisterUIRoutes, andPrepareCommonMiddlewares, plus the new bootstrap logic that:
- Prepends
handlers.TransportInterceptorMiddlewareto inference middlewares, and- When any
schemas.ObservabilityPluginis loaded, creates aTraceStore/Tracer, injects it intos.ClientviaSetTracer, and prepends the tracing middlewareis internally consistent and preserves the intended middleware ordering for inference traffic.
This neatly centralizes tracing/bootstrap concerns in the server without leaking HTTP details into core.
Also applies to: 1087-1098, 1205-1246
plugins/otel/converter.go (1)
49-70: OTEL conversion logic for traces/spans/events looks solidThe OTEL converter:
- Normalizes IDs with
hexToBytes,- Maps
SpanKind/SpanStatusappropriately,- Translates span and event attributes via
convertAttributesToKeyValues/anyToKeyValue, and- Builds
ResourceSpanwith reasonable resource and instrumentation scope metadata,all in a nil-safe way for attributes and events. As long as callers never pass a nil
*schemas.Trace, this should behave correctly and produce well-formed OTLP payloads.Also applies to: 72-87, 90-107, 112-147, 148-229, 231-257, 259-269, 271-285
core/schemas/tracer.go (1)
6-47: Tracer abstraction andNoOpTracerdefault are well-structuredThe
Tracerinterface cleanly captures the operations Bifrost needs (trace lifecycle, span lifecycle, attributes, LLM request/response enrichment), andNoOpTracerprovides a safe default that keeps all call sites simple (no nil checks) while avoiding overhead when tracing is disabled.The compile-time assertion and
DefaultTracer()helper round out the contract cleanly.Also applies to: 49-86
framework/plugins/dynamicplugin.go (1)
24-29: Dynamic plugin HTTP middleware wiring matches the new plugin contractThe
DynamicPluginnow stores anhttpTransportMiddlewareof typeschemas.BifrostHTTPMiddleware, exposes it viaHTTPTransportMiddleware(), andloadDynamicPluginlooks up theHTTPTransportMiddlewaresymbol and asserts it to that function type. This matches the updated plugin interface (exporting afunc HTTPTransportMiddleware(next fasthttp.RequestHandler) fasthttp.RequestHandler) and integrates cleanly with the HTTP transport middleware chain.Dynamic plugins that don’t need HTTP interception can simply implement a pass-through middleware.
Also applies to: 36-39, 140-147
framework/tracing/tracer.go (6)
23-44: LGTM!The trace lifecycle methods correctly delegate to the underlying store with appropriate nil checks. The comment on line 35 properly documents caller responsibility for releasing traces.
46-75: LGTM!The StartSpan implementation correctly handles parent-child span relationships and maintains context propagation. The defensive nil checks ensure robustness when trace or span creation fails.
77-84: LGTM!EndSpan correctly delegates to the store with defensive type checking. Passing
nilfor attributes is appropriate since attributes should be set viaSetAttributeduring the span's lifetime.
86-100: LGTM!SetAttribute is properly defensive with nil checks at each step. The delegation pattern to the underlying trace and span is clean and correct.
102-120: LGTM!AddEvent follows the same defensive pattern as SetAttribute. Using
time.Now()for the event timestamp is appropriate.
122-162: LGTM!The LLM attribute population methods correctly integrate with the helper functions from
llmspan.go. The pattern of retrieving attribute maps and applying them to spans is clean, and all nil checks are in place.framework/tracing/llmspan.go (6)
12-60: LGTM!PopulateRequestAttributes provides a clean entry point with proper nil checking and comprehensive request type coverage. The delegation pattern to type-specific populators is well-structured.
62-124: LGTM!Both response and error attribute population functions are properly nil-safe and provide comprehensive coverage. The conditional checks for optional error fields are appropriate.
126-153: LGTM!PopulateContextAttributes correctly populates optional context metadata with appropriate conditional checks. Setting retry and fallback counts unconditionally is appropriate since zero is a meaningful value.
206-936: LGTM!The type-specific attribute population functions (Chat, TextCompletion, Embedding, Transcription, Speech, Responses, Batch, and File operations) all follow consistent patterns with proper nil checks and conditional handling of optional fields. The implementation is comprehensive and correct, aside from the ExtraParams collision issue already noted.
1121-1237: LGTM!The message extraction helpers are well-structured with appropriate nil checks throughout. The
MessageSummaryandResponsesMessageSummarytypes provide clean abstractions for tracing, and the content extraction logic correctly handles both string and block-based content formats.
1243-1255: LGTM!PopulateCostAttribute correctly handles optional pricing integration with appropriate nil checks. The integration with
ModelCatalog.CalculateCostWithCacheDebugprovides cost tracking when a pricing manager is available.
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
b8f95ce to
fb945b9
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (5)
ui/components/ui/switch.tsx (1)
13-32: UI changes appear unrelated to the PR's stated objective.This PR is described as "plugins v2 architecture" focusing on Go backend changes (HTTP middleware, plugin interfaces, transport layer). However, this file contains UI component modifications (default size and border radius changes) that seem unrelated to the plugin architecture refactor.
Consider moving these UI changes to a separate PR for better separation of concerns and clearer change tracking.
docs/plugins/writing-plugin.mdx (1)
54-59: Update go.mod example version to supportHTTPTransportMiddlewareThe v1.4.x+ example plugin code uses
HTTPTransportMiddleware()andschemas.BifrostHTTPMiddleware(lines 66–102), but the go.mod snippet pinsgithub.com/maximhq/bifrost/core v1.2.38. The hello-world example in the repository uses v1.2.40 with this API. Update the pinned version to at least v1.2.40 to avoid compilation errors.transports/bifrost-http/integrations/router.go (1)
503-693: Use the bifrost context for all non-streaming client calls
handleNonStreamingRequestpassesctx(a*fasthttp.RequestCtx) to client methods, but it should use acontext.Contextderived frombifrostCtx, matching the pattern inhandleBatchRequest(line 698) andhandleFileRequest.Add
requestCtx := *bifrostCtxafter the comments and userequestCtxfor all client method calls:func (g *GenericRouter) handleNonStreamingRequest(ctx *fasthttp.RequestCtx, config RouteConfig, req interface{}, bifrostReq *schemas.BifrostRequest, bifrostCtx *context.Context) { // Use the cancellable context from ConvertToBifrostContext // While we can't detect client disconnects until we try to write, having a cancellable context // allows providers that check ctx.Done() to cancel early if needed. This is less critical than // streaming requests (where we actively detect write errors), but still provides a mechanism // for providers to respect cancellation. + requestCtx := *bifrostCtx var response interface{} var err error switch { case bifrostReq.ListModelsRequest != nil: - listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) + listModelsResponse, bifrostErr := g.client.ListModelsRequest(requestCtx, bifrostReq.ListModelsRequest)Replace
ctxwithrequestCtxin all client method calls:ListModelsRequest,TextCompletionRequest,ChatCompletionRequest,ResponsesRequest,EmbeddingRequest,SpeechRequest, andTranscriptionRequest.core/bifrost.go (2)
2743-2850: Streaming detection viaBifrostContextKeyStreamStartTimebreaks non-stream plugin spans
executeRequestWithRetriesnow unconditionally sets:streamStartTime := time.Now() *ctx = context.WithValue(*ctx, schemas.BifrostContextKeyStreamStartTime, streamStartTime) *ctx = context.WithValue(*ctx, schemas.BifrostContextKeyTracer, tracer)for all request types (streaming and non-streaming).
PluginPipeline.RunPostHooksdetects “streaming mode” via:isStreaming := (*ctx).Value(schemas.BifrostContextKeyStreamStartTime) != niland, when
isStreamingis true, it:
- Skips per-plugin
posthookspans and- Only accumulates timing into
postHookTimings/chunkCount.Because
BifrostContextKeyStreamStartTimeis now always set in the worker context, every non-stream request is treated as streaming inRunPostHooks, which:
- Silently removes the existing per-plugin post-hook spans for all synchronous calls.
- Accumulates timing that is never flushed for non-stream flows (no finalizer is invoked).
You likely want one of:
- Only setting
BifrostContextKeyStreamStartTimefor stream requests (e.g., pass anisStreamingflag intoexecuteRequestWithRetriesand gate theWithValuecalls), or- Changing
RunPostHooksto key off a dedicated streaming flag (e.g.,BifrostContextKeyDeferTraceCompletionor similar) instead of the presence of a timestamp used for TTFT.As-is, this is a functional regression for plugin observability on non-streaming requests.
Also applies to: 3261-3323
2978-2996: StreamingPluginPipelinelifetime vs pooling can race and drop metricsFor streaming requests,
requestWorkerdoes:pipeline := bifrost.getPluginPipeline() postHookRunner := func(...) { return pipeline.RunPostHooks(...) } // stores postHookSpanFinalizer capturing `pipeline` into req.Context ... stream, bifrostError = executeRequestWithRetries(..., postHookRunner, ...) ... if pipeline != nil { bifrost.releasePluginPipeline(pipeline) // calls reset and returns to pool }Meanwhile, providers receive
postHookRunnerand (by design) call it from their streaming goroutines as chunks arrive. Additionally, HTTP/transport code is expected to call thepostHookSpanFinalizerclosure (which also closes spans onpipeline) at the end of streaming, using the reference stored in the context.This creates two problems:
pipelineis reset and returned to the pool while streaming goroutines and the finalizer may still be using it → data races and incorrectpostHookTimings/chunkCount.- Because
resetPluginPipelineclearspostHookTimingsandpostHookPluginOrder,FinalizeStreamingPostHookSpanswill frequently see an empty map and produce no aggregated spans, even ifRunPostHookswas called many times during streaming.You probably want to:
- Keep the streaming
PluginPipelinealive until the stream truly ends (e.g., release it from the place that closes the stream / calls the finalizer), or- Avoid pooling
PluginPipelinefor streaming at all, and only reuse it for non-streaming flows.As implemented, the combination of pooling + streaming callbacks is unsafe and undermines the new aggregated post-hook span metrics.
Also applies to: 3370-3423
♻️ Duplicate comments (9)
ui/components/ui/switch.tsx (1)
13-13: Counterintuitive naming: "md" is smaller than "default" and is now the default.The past review correctly identified that
"md"renders smaller (h-5 w-9) than"default"(h-6 w-11), which is confusing. Settingsize = "md"as the default parameter exacerbates this—users now get the smaller variant by default, and "medium" being smaller than "default" remains counterintuitive.Consider renaming to make the size relationship clear (e.g.,
"sm"and"md", or"compact"and"default"), and set the default parameter to represent the actual baseline size.examples/plugins/hello-world/main.go (1)
7-7: Hello-world example still reads a context value that HTTPTransportMiddleware never sets.
PreHook/PostHookloghello-world-plugin-transport-interceptor, but the newHTTPTransportMiddlewareonly prints and forwards, so that key is alwaysniland the data‑flow demo is broken.Consider either:
- Updating
HTTPTransportMiddlewareto extract the*schemas.BifrostContextfromfasthttp.RequestCtx(e.g., viactx.UserValue("bifrost-context")) and set the expected key, or- Removing the reads of
"hello-world-plugin-transport-interceptor"fromPreHook/PostHookif you no longer want to demonstrate that flow.Also applies to: 19-43
framework/tracing/store.go (1)
339-367: HardenTraceStore.Stopagainst double calls and clean up deferred spans
Stop()currently stops the ticker and unconditionally closess.stopCleanup:func (s *TraceStore) Stop() { if s.cleanupTicker != nil { s.cleanupTicker.Stop() } close(s.stopCleanup) s.cleanupWg.Wait() }If
Stop()is ever called more than once (or called after a zero-TTL store that never started cleanup), this will panic with “close of closed channel”. A prior review already flagged this pattern.Separately,
cleanupOldTracesonly evicts entries froms.traces. Any matching entries indeferredSpansfor those trace IDs will remain, which is a small but real leak over time for abandoned traces.Proposed fix: guard Stop() with sync.Once and clear deferred spans for evicted traces
type TraceStore struct { @@ - ttl time.Duration - cleanupTicker *time.Ticker - stopCleanup chan struct{} - cleanupWg sync.WaitGroup + ttl time.Duration + cleanupTicker *time.Ticker + stopCleanup chan struct{} + cleanupWg sync.WaitGroup + stopOnce sync.Once } @@ func (s *TraceStore) cleanupOldTraces() { cutoff := time.Now().Add(-s.ttl) count := 0 - s.traces.Range(func(key, value any) bool { - trace := value.(*schemas.Trace) - if trace.StartTime.Before(cutoff) { - if deleted, ok := s.traces.LoadAndDelete(key); ok { - s.ReleaseTrace(deleted.(*schemas.Trace)) - count++ - } - } - return true - }) + s.traces.Range(func(key, value any) bool { + traceID := key.(string) + trace := value.(*schemas.Trace) + if trace.StartTime.Before(cutoff) { + if deleted, ok := s.traces.LoadAndDelete(key); ok { + s.ReleaseTrace(deleted.(*schemas.Trace)) + // also clear any deferred span for this trace + s.deferredSpans.Delete(traceID) + count++ + } + } + return true + }) @@ func (s *TraceStore) Stop() { - if s.cleanupTicker != nil { - s.cleanupTicker.Stop() - } - close(s.stopCleanup) - s.cleanupWg.Wait() + s.stopOnce.Do(func() { + if s.cleanupTicker != nil { + s.cleanupTicker.Stop() + } + close(s.stopCleanup) + s.cleanupWg.Wait() + }) }plugins/governance/main.go (1)
158-181: Tighten virtual key handling and error messages in HTTP transport middleware
Missing prefix validation for
x-bf-vk(duplicate of earlier review)
parseVirtualKeyreturns thex-bf-vkheader as-is:vkHeader := ctx.Request.Header.Peek("x-bf-vk") if string(vkHeader) != "" { return bifrost.Ptr(string(vkHeader)) }but both
Authorization: Bearer ...andx-api-keypaths enforceVirtualKeyPrefix. This inconsistency allows arbitrary values viax-bf-vkthat would be rejected on other headers.Misleading log messages for JSON (un)marshaling (duplicate of earlier review)
InHTTPTransportMiddleware, you log:err = sonic.Unmarshal(ctx.Request.Body(), &payload) if err != nil { p.logger.Error("failed to marshal request body to check for virtual key: %v", err) } ... body, err := sonic.Marshal(payload) if err != nil { p.logger.Error("failed to marshal request body to check for virtual key: %v", err) }The first message should say “unmarshal”, and both messages should describe what the code is actually doing (parsing request JSON / rebuilding body after load-balancing), not “check for virtual key”.
Body presence check is fragile
ctx.Request.Body()returns a[]byte; comparing it tonilmight still treat an empty body as “present” on some paths. That leads to unnecessary JSON parsing attempts and noisy error logs for routes that legitimately have no body.Safer is to check
len(body) == 0and skip parsing in that case.Proposed fixes
func parseVirtualKey(ctx *fasthttp.RequestCtx) *string { var virtualKeyValue string vkHeader := ctx.Request.Header.Peek("x-bf-vk") - if string(vkHeader) != "" { - return bifrost.Ptr(string(vkHeader)) - } + if vk := string(vkHeader); vk != "" && strings.HasPrefix(strings.ToLower(vk), VirtualKeyPrefix) { + return bifrost.Ptr(vk) + } @@ func (p *GovernancePlugin) HTTPTransportMiddleware() schemas.BifrostHTTPMiddleware { return func(next fasthttp.RequestHandler) fasthttp.RequestHandler { return func(ctx *fasthttp.RequestCtx) { @@ - if ctx.Request.Body() == nil { + bodyBytes := ctx.Request.Body() + if len(bodyBytes) == 0 { next(ctx) return } var payload map[string]any - err = sonic.Unmarshal(ctx.Request.Body(), &payload) + err = sonic.Unmarshal(bodyBytes, &payload) if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body for governance HTTP middleware: %v", err) next(ctx) return } @@ payload, err = p.loadBalanceProvider(payload, virtualKey) if err != nil { - p.logger.Error("failed to load balance provider: %v", err) + p.logger.Error("failed to load balance provider in governance HTTP middleware: %v", err) next(ctx) return } body, err := sonic.Marshal(payload) if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to marshal updated request body in governance HTTP middleware: %v", err) next(ctx) return }Also applies to: 183-234
core/bifrost.go (1)
70-75:SetTraceris unsafe for concurrent use and allowsnil(risking panics)
Bifrost.traceris read from hot paths (workers, fallback spans, plugin pipeline) without synchronization, whileSetTracermutates it directly and is public. In addition:
SetTracerallowstracerto be set tonil, but all call sites assume a non-nil tracer (bifrost.tracer.StartSpan(...)etc.), so a runtime call toSetTracer(nil)would lead to panics.Given this is a global cross-cutting dependency:
- Either treat
SetTraceras init-only (document it clearly and possibly guard with a once/“started” flag), or- Make
tracerconcurrency-safe (e.g.,atomic.ValuewithLoad/Store) and normalizeniltoschemas.DefaultTracer()so it’s never nil at use sites.This repeats an earlier concern and is still present in the current diff.
Also applies to: 246-250
framework/tracing/propagation.go (1)
24-36:ExtractParentIDshould returnParentID, notTraceIDThe function is documented to extract the parent ID from the
traceparentheader but currently returnsctx.TraceID. Per W3C format, the parent/span ID is the third field and should map toctx.ParentID.Proposed fix
func ExtractParentID(header *fasthttp.RequestHeader) string { @@ ctx := ParseTraceparent(traceParent) if ctx == nil { return "" } - return ctx.TraceID + return ctx.ParentID }core/schemas/trace.go (1)
40-49:Trace.Reset,Span.End, andSpan.Resetmutate state without locking
Trace.AddSpan/Trace.GetSpanandSpan.SetAttribute/Span.AddEventare mutex-protected, but:
Trace.Resetclears fields and slices with no lock.Span.EndsetsEndTime,Status,StatusMsgwithout locking.Span.Resetclears all fields and theEventsslice without locking.Given these types are shared through
TraceStoreand may be read/updated concurrently, calling these methods while another goroutine accesses the same trace/span can produce data races.Either:
- Guard these methods with the same mutex (
mu.Lock/defer mu.Unlock()), or- Very clearly document and enforce that they may only be called when the trace/span is no longer reachable by any other goroutine (e.g., after removal from maps and before returning to pools).
Example for
Span.End:func (s *Span) End(status SpanStatus, statusMsg string) { - s.EndTime = time.Now() - s.Status = status - s.StatusMsg = statusMsg + s.mu.Lock() + defer s.mu.Unlock() + s.EndTime = time.Now() + s.Status = status + s.StatusMsg = statusMsg }Same pattern applies to
Trace.ResetandSpan.Reset.Also applies to: 84-104
framework/tracing/tracer.go (1)
18-21: Comments still refer to “StoreTracer” instead ofTracerThere are several references to “StoreTracer” in comments:
- Constructor comment:
// NewTracer creates a new StoreTracer...- Type comment for
spanHandle.- Compile-time assertion comment:
// Ensure StoreTracer implements schemas.Tracer...The actual exported type is
Tracer, so these comments should be updated for consistency and to avoid confusion.Example fixes:
-// NewTracer creates a new StoreTracer wrapping the given TraceStore. +// NewTracer creates a new Tracer wrapping the given TraceStore. @@ -// spanHandle is the concrete implementation of schemas.SpanHandle for StoreTracer. +// spanHandle is the concrete implementation of schemas.SpanHandle for Tracer. @@ -// Ensure StoreTracer implements schemas.Tracer at compile time +// Ensure Tracer implements schemas.Tracer at compile timeAlso applies to: 46-51, 447-448
framework/tracing/llmspan.go (1)
190-193: ExtraParams collision issue remains unaddressed.User-provided keys from
ExtraParamsare written directly to the attributes map without namespacing, which can overwrite standard attributes (e.g., if a user sets a key like"gen_ai.request.model").This issue was previously flagged and affects multiple locations in this file (lines 190-193, 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 768-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029).
🔎 Suggested approach: namespace ExtraParams consistently
// ExtraParams for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) + attrs["gen_ai.extra."+k] = fmt.Sprintf("%v", v) }Apply this pattern consistently across all 14 occurrences in the file.
🧹 Nitpick comments (6)
core/providers/utils/utils.go (1)
880-925: Deferred LLM span completion for streaming looks correct, but check error/ordering edge cases.The new tracing hooks in
ProcessAndSendResponse/ProcessAndSendBifrostErrorpluscompleteDeferredSpanare defensively implemented and should safely no-op when tracing isn’t configured. Chunk accumulation +GetAccumulatedChunksgives you a solid basis to populate LLM response attributes, TTFT, and total chunks before ending the deferred span.Two edge cases are worth double‑checking:
Ordering relative to post hooks (final chunk).
completeDeferredSpanis called beforepostHookRunneron the final chunk, while it usesBifrostContextKeyPostHookSpanFinalizerto synthesize aggregated post‑hook spans. If that finalizer is intended to account for post‑hook execution on the last chunk as well, you may want to move thecompleteDeferredSpancall afterpostHookRunnerfor the final chunk (or adjust the finalizer’s expectations).Scanner / transport errors via
ProcessAndSendError.
OnlyProcessAndSendResponseandProcessAndSendBifrostErrortriggercompleteDeferredSpanwhenBifrostContextKeyStreamEndIndicatoris true. For stream failures that go throughProcessAndSendError(e.g., low‑level read/parse errors), there currently isn’t a path here to close the deferred span, which could leave traces open until some external timeout.Also applies to: 932-969, 1412-1501
core/schemas/plugin.go (1)
4-8: Plugin API evolution is coherent, but docs still mention TransportInterceptor.Adding
BifrostHTTPMiddlewareand switching thePlugininterface toHTTPTransportMiddleware() BifrostHTTPMiddleware, plus the newObservabilityPlugininterface, aligns with the rest of the middleware and tracing changes.One minor doc mismatch remains: the “Execution order” comment block still refers to
TransportInterceptoras step 1. It should be updated to describeHTTPTransportMiddlewareas the first stage in the pipeline to avoid confusing plugin authors migrating to the new API.Also applies to: 36-39, 71-80, 109-137
transports/bifrost-http/server/server.go (1)
1231-1246: Consider managing the tracing store’s lifecycle on server shutdownThe Bootstrap logic:
- Prepends
handlers.TransportInterceptorMiddlewaretoinferenceMiddlewares.- When any
ObservabilityPluginis present, allocates aTraceStorewith a 60-minute TTL, builds atracing.Tracer, callss.Client.SetTracer(tracer), and prepends a tracing middleware.This is a good integration of HTTP middleware, client tracing, and observability plugins. One improvement would be to keep a reference to the
TraceStore(or a closer on the tracer) onBifrostHTTPServerand stop it during shutdown, so the cleanup goroutine and its ticker are explicitly torn down rather than relying on process exit.plugins/otel/converter.go (1)
49-70: Consider stricter validation inhexToBytesfor trace/span IDs
hexToBytesstrips non-hex chars, pads/truncates, and ignoreshex.DecodeStringerrors. For trace IDs/span IDs this can silently turn malformed inputs into “valid” OTEL IDs, which hurts debuggability and cross-system correlation.If feasible, consider:
- Failing fast (returning
niland skipping span export) whenDecodeStringerrors, or- Validating length/charset at call sites (e.g., before
convertSpanToOTELSpan) and logging when IDs are bad.This is an optional hardening improvement; current behavior is functional but very permissive.
framework/tracing/helpers.go (1)
22-82: Defensively guard against nil*TraceStorein helpers
GetTrace,AddSpan,AddChildSpan,EndSpan,SetSpanAttribute, andAddSpanEventall dereferencestorewithout checking for nil. If callers ever pass a nil*TraceStore(e.g., during partial initialization or tests), these will panic.Consider early returns like:
if store == nil { return // or nil for the getters }in these helpers to make them safer to use across the codebase.
framework/tracing/llmspan.go (1)
280-280: Consider preserving structured data instead of string conversion.Maps and structured data are being converted to string representations using
fmt.Sprintf("%v", ...)at lines 280 (LogitBias), 613 (Metadata), and 705 (Metadata).This flattens the structure and reduces queryability in tracing backends that support nested attributes. Consider using structured attribute values if the tracing backend supports them, or use JSON serialization for better structure preservation.
💡 Alternative serialization approach
For better structure preservation, consider JSON serialization:
- attrs[schemas.AttrLogitBias] = fmt.Sprintf("%v", req.Params.LogitBias) + if data, err := json.Marshal(req.Params.LogitBias); err == nil { + attrs[schemas.AttrLogitBias] = string(data) + }Note: This requires importing
encoding/json.Also applies to: 613-613, 705-705
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
core/go.sumis excluded by!**/*.sumexamples/plugins/hello-world/go.sumis excluded by!**/*.sum
📒 Files selected for processing (65)
core/bifrost.gocore/bifrost_test.gocore/go.modcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.godocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goplugins/governance/go.modplugins/governance/main.goplugins/jsonparser/main.goplugins/logging/main.goplugins/maxim/main.goplugins/mocker/main.goplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/semanticcache/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/go.modui/app/workspace/logs/views/logDetailsSheet.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsx
💤 Files with no reviewable changes (2)
- framework/configstore/tables/mcp.go
- plugins/otel/ttlsyncmap.go
🚧 Files skipped from review as they are similar to previous changes (22)
- transports/bifrost-http/handlers/websocket.go
- transports/go.mod
- transports/bifrost-http/handlers/plugins.go
- docs/plugins/getting-started.mdx
- transports/bifrost-http/handlers/health.go
- transports/bifrost-http/handlers/middlewares_test.go
- core/bifrost_test.go
- plugins/maxim/main.go
- transports/bifrost-http/handlers/cache.go
- plugins/mocker/main.go
- transports/bifrost-http/handlers/providers.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- plugins/governance/go.mod
- framework/plugins/dynamicplugin_test.go
- ui/components/sidebar.tsx
- core/mcp.go
- framework/plugins/dynamicplugin.go
- ui/components/ui/sheet.tsx
- transports/bifrost-http/handlers/config.go
- examples/plugins/hello-world/go.mod
- plugins/logging/main.go
- docs/plugins/migration-guide.mdx
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
framework/streaming/types.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/logging.goframework/streaming/responses.gocore/schemas/plugin.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/ui.goframework/streaming/transcription.godocs/docs.jsontransports/bifrost-http/server/server.goframework/tracing/store.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.goplugins/telemetry/main.goui/components/ui/switch.tsxdocs/plugins/writing-plugin.mdxplugins/semanticcache/main.goplugins/otel/converter.goplugins/governance/main.goframework/streaming/chat.gocore/schemas/context.goframework/streaming/accumulator.gocore/providers/utils/utils.goframework/tracing/propagation.goplugins/otel/main.gocore/schemas/bifrost.gocore/go.modcore/schemas/trace.gotransports/bifrost-http/handlers/mcpServer.goframework/streaming/audio.gotransports/bifrost-http/handlers/integrations.goexamples/plugins/hello-world/main.gocore/schemas/tracer.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/session.goplugins/jsonparser/main.gocore/bifrost.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/tracer.go
🧠 Learnings (4)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
framework/streaming/types.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/logging.goframework/streaming/responses.gocore/schemas/plugin.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/ui.goframework/streaming/transcription.gotransports/bifrost-http/server/server.goframework/tracing/store.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.goplugins/telemetry/main.goplugins/semanticcache/main.goplugins/otel/converter.goplugins/governance/main.goframework/streaming/chat.gocore/schemas/context.goframework/streaming/accumulator.gocore/providers/utils/utils.goframework/tracing/propagation.goplugins/otel/main.gocore/schemas/bifrost.gocore/schemas/trace.gotransports/bifrost-http/handlers/mcpServer.goframework/streaming/audio.gotransports/bifrost-http/handlers/integrations.goexamples/plugins/hello-world/main.gocore/schemas/tracer.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/session.goplugins/jsonparser/main.gocore/bifrost.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/tracer.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/session.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
🧬 Code graph analysis (32)
transports/bifrost-http/handlers/mcp.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/responses.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
core/schemas/plugin.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/handlers/governance.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/transcription.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/server/server.go (4)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)transports/bifrost-http/handlers/middlewares.go (2)
TransportInterceptorMiddleware(50-70)NewTracingMiddleware(207-214)framework/tracing/store.go (1)
NewTraceStore(38-65)framework/tracing/tracer.go (1)
NewTracer(19-21)
framework/tracing/store.go (5)
core/schemas/tracer.go (1)
Tracer(13-73)framework/tracing/tracer.go (1)
Tracer(14-16)core/schemas/bifrost.go (1)
BifrostResponse(335-355)core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
transports/bifrost-http/handlers/middlewares.go (6)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)Plugin(71-97)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)core/schemas/tracer.go (1)
Tracer(13-73)framework/tracing/propagation.go (1)
ExtractParentID(26-36)core/schemas/bifrost.go (4)
BifrostContextKeyTraceID(143-143)BifrostContextKeyTraceCompleter(148-148)BifrostContextKeySpanID(144-144)BifrostContextKeyDeferTraceCompletion(147-147)core/schemas/trace.go (3)
SpanKindHTTPRequest(131-131)SpanStatusError(151-151)SpanStatusOk(149-149)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/telemetry/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (17)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindHTTPRequest(131-131)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/chat.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
core/schemas/context.go (1)
core/schemas/bifrost.go (1)
BifrostContextKeyDeferTraceCompletion(147-147)
plugins/otel/main.go (3)
core/schemas/plugin.go (3)
BifrostHTTPMiddleware(38-38)PluginShortCircuit(12-16)ObservabilityPlugin(123-137)core/schemas/context.go (1)
BifrostContext(32-42)core/schemas/trace.go (1)
Trace(10-19)
core/schemas/bifrost.go (2)
core/schemas/tracer.go (1)
Tracer(13-73)framework/tracing/tracer.go (1)
Tracer(14-16)
core/schemas/trace.go (2)
framework/tracing/helpers.go (1)
AddSpan(32-38)ui/lib/constants/logs.ts (1)
Status(163-163)
transports/bifrost-http/handlers/mcpServer.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/audio.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/tracer.go (4)
core/schemas/trace.go (3)
Trace(10-19)SpanKind(115-115)SpanStatus(143-143)framework/tracing/helpers.go (1)
EndSpan(50-56)core/schemas/bifrost.go (3)
BifrostRequest(173-193)BifrostResponse(335-355)BifrostError(474-483)framework/tracing/tracer.go (1)
Tracer(14-16)
transports/bifrost-http/integrations/router.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (9)
ListModelsRequest(92-92)TextCompletionRequest(93-93)ChatCompletionRequest(95-95)ResponsesRequest(97-97)EmbeddingRequest(99-99)SpeechRequest(100-100)TranscriptionRequest(102-102)BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)transports/bifrost-http/handlers/inference.go (5)
ChatRequest(172-176)ResponsesRequest(266-270)EmbeddingRequest(273-277)SpeechRequest(279-283)TranscriptionRequest(285-289)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/bifrost.go (3)
core/schemas/tracer.go (3)
Tracer(13-73)DefaultTracer(124-126)SpanHandle(8-8)core/schemas/bifrost.go (5)
BifrostRequest(173-193)BifrostContextKeyStreamStartTime(145-145)BifrostContextKeyTracer(146-146)BifrostResponse(335-355)BifrostError(474-483)framework/tracing/helpers.go (1)
EndSpan(50-56)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(24-35)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
framework/tracing/llmspan.go (5)
core/schemas/bifrost.go (18)
BifrostRequest(173-193)RequestType(89-89)TextCompletionRequest(93-93)EmbeddingRequest(99-99)TranscriptionRequest(102-102)SpeechRequest(100-100)ResponsesRequest(97-97)BatchCreateRequest(104-104)BatchListRequest(105-105)BatchRetrieveRequest(106-106)BatchCancelRequest(107-107)BatchResultsRequest(108-108)FileUploadRequest(109-109)FileListRequest(110-110)FileRetrieveRequest(111-111)FileDeleteRequest(112-112)FileContentRequest(113-113)BifrostResponse(335-355)core/schemas/trace.go (100)
AttrProviderName(159-159)AttrRequestModel(160-160)AttrError(216-216)AttrErrorType(217-217)AttrErrorCode(218-218)AttrVirtualKeyID(228-228)AttrVirtualKeyName(229-229)AttrSelectedKeyID(230-230)AttrSelectedKeyName(231-231)AttrTeamID(232-232)AttrTeamName(233-233)AttrCustomerID(234-234)AttrCustomerName(235-235)AttrNumberOfRetries(236-236)AttrFallbackIndex(237-237)AttrMaxTokens(163-163)AttrTemperature(164-164)AttrTopP(165-165)AttrStopSequences(166-166)AttrPresencePenalty(167-167)AttrFrequencyPenalty(168-168)AttrParallelToolCall(169-169)AttrRequestUser(170-170)AttrMessageCount(188-188)AttrInputMessages(222-222)AttrResponseID(191-191)AttrResponseModel(192-192)AttrObject(197-197)AttrSystemFprint(194-194)AttrCreated(196-196)AttrServiceTier(195-195)AttrOutputMessages(225-225)AttrFinishReason(193-193)AttrTotalTokens(210-210)AttrN(175-175)AttrInputText(221-221)AttrDimensions(178-178)AttrEncodingFormat(179-179)AttrInputEmbedding(224-224)AttrLanguage(180-180)AttrPrompt(181-181)AttrResponseFormat(182-182)AttrFormat(183-183)AttrInputTokens(211-211)AttrOutputTokens(212-212)AttrInputSpeech(223-223)AttrRespInclude(255-255)AttrRespMaxOutputTokens(256-256)AttrRespMaxToolCalls(257-257)AttrRespMetadata(258-258)AttrRespPreviousRespID(259-259)AttrRespPromptCacheKey(260-260)AttrRespReasoningText(261-261)AttrRespReasoningEffort(262-262)AttrRespReasoningGenSum(263-263)AttrRespSafetyIdentifier(264-264)AttrRespStore(265-265)AttrRespTemperature(266-266)AttrRespTextVerbosity(267-267)AttrRespTextFormatType(268-268)AttrRespTopLogProbs(269-269)AttrRespTopP(270-270)AttrRespToolChoiceType(271-271)AttrRespToolChoiceName(272-272)AttrRespTruncation(273-273)AttrRespTools(274-274)AttrBatchInputFileID(281-281)AttrBatchEndpoint(280-280)AttrBatchCompletionWin(284-284)AttrBatchRequestsCount(287-287)AttrBatchMetadata(291-291)AttrBatchLimit(292-292)AttrBatchAfter(293-293)AttrBatchBeforeID(294-294)AttrBatchAfterID(295-295)AttrBatchPageToken(296-296)AttrBatchPageSize(297-297)AttrBatchID(277-277)AttrBatchStatus(278-278)AttrBatchObject(279-279)AttrBatchCreatedAt(285-285)AttrBatchExpiresAt(286-286)AttrBatchOutputFileID(282-282)AttrBatchErrorFileID(283-283)AttrBatchCountTotal(298-298)AttrBatchCountCompleted(299-299)AttrBatchCountFailed(300-300)AttrBatchDataCount(288-288)AttrBatchHasMore(290-290)AttrBatchFirstID(301-301)AttrBatchLastID(302-302)AttrBatchInProgressAt(303-303)AttrBatchFinalizingAt(304-304)AttrBatchCompletedAt(305-305)AttrBatchFailedAt(306-306)AttrBatchExpiredAt(307-307)AttrBatchCancellingAt(308-308)AttrBatchCancelledAt(309-309)AttrBatchNextCursor(310-310)AttrUsageCost(213-213)core/schemas/chatcompletions.go (1)
BifrostChatRequest(12-19)core/schemas/embedding.go (1)
BifrostEmbeddingRequest(9-16)core/schemas/responses.go (1)
BifrostResponsesRequest(32-39)
framework/tracing/tracer.go (5)
framework/tracing/store.go (1)
TraceStore(24-35)core/schemas/tracer.go (2)
Tracer(13-73)SpanHandle(8-8)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (4)
BifrostContextKeySpanID(144-144)BifrostRequest(173-193)BifrostResponse(335-355)BifrostError(474-483)
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
fb945b9 to
1cecccb
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (12)
ui/components/ui/switch.tsx (1)
13-13: Note: Size default change already flagged in previous review.The change from
size = "default"(implicit previous default) tosize = "md"has already been identified as a breaking change in the past review comment. Existing Switch components without an explicit size prop will render smaller (h-6 w-11 → h-5 w-9).examples/plugins/hello-world/main.go (1)
19-26: HTTPTransportMiddleware implementation looks correct but context value propagation is broken.The middleware correctly implements the
schemas.BifrostHTTPMiddlewaresignature with proper delegation tonext(ctx). However,PreHook(line 29) andPostHook(lines 38-39) still read"hello-world-plugin-transport-interceptor"from context, which this middleware never sets—those reads will always returnnil.Either set the context value in the middleware to demonstrate data flow, or remove the stale context reads from the hooks.
plugins/governance/main.go (2)
30-36: Validatex-bf-vkheader consistently with other virtual key sources
parseVirtualKeyreturns the rawx-bf-vkheader without checking it starts withVirtualKeyPrefix("sk-bf-"), while bothAuthorization: Bearerandx-api-keypaths enforce that prefix. This inconsistency can allow malformed or unintended tokens viax-bf-vkto bypass the prefix check.Align
x-bf-vkhandling with the other branches, e.g.:Proposed adjustment
func parseVirtualKey(ctx *fasthttp.RequestCtx) *string { - var virtualKeyValue string - vkHeader := ctx.Request.Header.Peek("x-bf-vk") - if string(vkHeader) != "" { - return bifrost.Ptr(string(vkHeader)) - } + var virtualKeyValue string + + vkHeader := ctx.Request.Header.Peek("x-bf-vk") + if len(vkHeader) > 0 { + vkValue := string(vkHeader) + if strings.HasPrefix(strings.ToLower(vkValue), VirtualKeyPrefix) { + return bifrost.Ptr(vkValue) + } + }This keeps all header sources enforcing the same VK prefix semantics.
Also applies to: 158-181
183-231: Fix HTTP middleware log message and empty-body handlingTwo small correctness/UX issues in
HTTPTransportMiddleware:
- Line 214 logs
"failed to marshal"while callingsonic.Unmarshal. This should say"unmarshal"to reflect the actual operation.- The empty-body check only guards on
ctx.Request.Body() == nil. Infasthttp, an empty body is often a zero-length slice, notnil, so you’ll still try to unmarshal and log an error for valid but empty requests.Consider:
Proposed tweak
- if ctx.Request.Body() == nil { + if body := ctx.Request.Body(); body == nil || len(body) == 0 { next(ctx) return } var payload map[string]any - err = sonic.Unmarshal(ctx.Request.Body(), &payload) + err = sonic.Unmarshal(ctx.Request.Body(), &payload) if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body to check for virtual key: %v", err) next(ctx) return }core/bifrost.go (1)
55-75: ClarifySetTracer’s concurrency model or make it race-free
Bifrost.traceris read on hot paths (handleRequest,handleStreamRequest,requestWorker,executeRequestWithRetries, plugin pipeline) without synchronization, whileSetTracermutates it directly:type Bifrost struct { ... tracer schemas.Tracer } func (bifrost *Bifrost) SetTracer(tracer schemas.Tracer) { bifrost.tracer = tracer }In the current server flow
SetTraceris only called during bootstrap (before requests start), which is safe. But as a public method it invites runtime reconfiguration, which would introduce a data race.Consider either:
- Treating
SetTraceras init-only: document it as such and avoid calling it after workers start, or- Making it concurrency-safe (e.g., store the tracer in an
atomic.Value/atomic.Pointeror a small RW-mutex-protected field, and load via that everywhere).In Go, is writing to a struct field from one goroutine while other goroutines read it without synchronization considered a data race?Also applies to: 120-139, 247-250
framework/tracing/store.go (1)
23-35: GuardTraceStore.Stop()against multiple invocations (double close panic)
Stop()unconditionally closess.stopCleanup. IfStop()is called more than once (e.g., from tests or multiple shutdown paths), closing an already-closed channel will panic. This was previously flagged and the pattern is unchanged.Consider adding a
sync.Oncefield toTraceStoreand wrapping the shutdown logic so it runs only once:Proposed fix
type TraceStore struct { traces sync.Map // map[traceID]*schemas.Trace - thread-safe concurrent access deferredSpans sync.Map // map[traceID]*DeferredSpanInfo - deferred spans for streaming requests tracePool sync.Pool // Reuse Trace objects to reduce allocations spanPool sync.Pool // Reuse Span objects to reduce allocations logger schemas.Logger ttl time.Duration cleanupTicker *time.Ticker stopCleanup chan struct{} cleanupWg sync.WaitGroup + stopOnce sync.Once } // Stop stops the cleanup goroutine and releases resources func (s *TraceStore) Stop() { - if s.cleanupTicker != nil { - s.cleanupTicker.Stop() - } - close(s.stopCleanup) - s.cleanupWg.Wait() + s.stopOnce.Do(func() { + if s.cleanupTicker != nil { + s.cleanupTicker.Stop() + } + close(s.stopCleanup) + s.cleanupWg.Wait() + }) }In Go, what happens if you call close() on a channel more than once?Also applies to: 360-367
framework/tracing/propagation.go (1)
24-36:ExtractParentIDshould returnParentID, notTraceIDThe function is documented as extracting the parent trace/span ID from the
traceparentheader, but after parsing it returnsctx.TraceID:ctx := ParseTraceparent(traceParent) ... return ctx.TraceIDPer the W3C format (
version-traceid-parentid-traceflags), the parent/span ID is the third field and should be surfaced viactx.ParentID. Returning the trace ID here will break proper parent-child linking for incoming traces.Recommend:
func ExtractParentID(header *fasthttp.RequestHeader) string { traceParent := string(header.Peek(TraceParentHeader)) if traceParent == "" { return "" } ctx := ParseTraceparent(traceParent) if ctx == nil { return "" } - return ctx.TraceID + return ctx.ParentID }In the W3C Trace Context `traceparent` header, which field represents the parent/span ID?framework/tracing/llmspan.go (1)
190-193: ExtraParams keys written directly may cause attribute key collisions.User-provided keys from
ExtraParamsare written directly to the attributes map without namespacing. If a user sets a key like"gen_ai.request.model"in ExtraParams, it would overwrite the standard attribute.This pattern repeats throughout the file (lines 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 768-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029).
🔎 Suggested approach: namespace ExtraParams
// ExtraParams for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) + attrs["gen_ai.extra."+k] = fmt.Sprintf("%v", v) }core/schemas/trace.go (3)
40-49:Trace.Reset()lacks mutex protection.This method modifies all fields including
Spanswithout holding the mutex, creating a potential data race ifResetis called while another goroutine is accessing the trace.
84-89:Span.End()lacks mutex protection.This method modifies
EndTime,Status, andStatusMsgwithout holding the mutex, which can cause data races if called concurrently with reads or writes to these fields.
91-104:Span.Reset()lacks mutex protection.Similar to
Trace.Reset(), this method modifies fields without holding the mutex. If reused from a pool while another goroutine still holds a reference, a data race could occur.framework/tracing/tracer.go (1)
393-398: Missing sort for deterministic output inbuildCompleteResponseFromChunks.The
choiceIndices(line 395-398) andtcIndices(line 434-437) slices are built from map iteration but are not sorted before use. This results in non-deterministic ordering of choices and tool calls in the reconstructed response.🔎 Proposed fix
+import "sort" // Build final choices from accumulated data // Sort choice indices for deterministic output choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } +sort.Ints(choiceIndices) for _, idx := range choiceIndices { ... // Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } + sort.Ints(tcIndices)Also applies to: 433-437
🧹 Nitpick comments (9)
transports/bifrost-http/handlers/middlewares.go (1)
49-70: Plugin middleware collection is correct; consider pre-allocating the slice.The implementation correctly:
- Collects
HTTPTransportMiddleware()from all plugins- Skips nil middlewares
- Chains them via
lib.ChainMiddlewaresFor a minor efficiency gain on hot paths, consider pre-allocating the slice:
🔎 Optional optimization
-pluginsMiddlewareChain := []schemas.BifrostHTTPMiddleware{} +pluginsMiddlewareChain := make([]schemas.BifrostHTTPMiddleware, 0, len(plugins))framework/streaming/responses.go (1)
742-756: LGTM! TTFT calculation correctly implemented.The Time to First Token calculation is properly implemented with defensive checks for zero timestamps. The logic correctly computes TTFT in milliseconds and propagates it through AccumulatedData.
Optional: Consider extracting TTFT calculation to a helper function.
The same TTFT calculation logic is duplicated across multiple streaming files (responses.go, chat.go, transcription.go). Consider extracting this into a shared helper function to reduce duplication and improve maintainability.
🔎 Proposed helper function
+// calculateTTFT calculates Time to First Token in milliseconds +func calculateTTFT(startTimestamp, firstChunkTimestamp time.Time) int64 { + if startTimestamp.IsZero() || firstChunkTimestamp.IsZero() { + return 0 + } + return firstChunkTimestamp.Sub(startTimestamp).Nanoseconds() / 1e6 +} // Calculate Time to First Token (TTFT) in milliseconds -var ttft int64 -if !accumulator.StartTimestamp.IsZero() && !accumulator.FirstChunkTimestamp.IsZero() { - ttft = accumulator.FirstChunkTimestamp.Sub(accumulator.StartTimestamp).Nanoseconds() / 1e6 -} +ttft := calculateTTFT(accumulator.StartTimestamp, accumulator.FirstChunkTimestamp)core/bifrost_test.go (1)
71-72: LGTM! Test calls correctly updated for new tracing integration.All test call sites have been consistently updated to include the tracer parameter (
schemas.DefaultTracer()) and request parameter (nil), aligning with the new signature ofexecuteRequestWithRetries.Optional: Consider clarifying the nil parameter.
The trailing
nilparameter passed toexecuteRequestWithRetriescould be clearer with a comment or named constant (e.g.,var nilRequest *schemas.BifrostRequest = nil) to improve test readability.Also applies to: 106-107, 141-142, 205-206, 279-280, 509-510
plugins/semanticcache/main.go (1)
338-341: Semantic cache transport middleware + CacheType gating look sane, with a small edge-caseReturning
nilfromHTTPTransportMiddlewareis fine if the transport chain skips nils. The newCacheTypeKeylogic correctly narrows to direct-only or semantic-only caching when set.Be aware that if a new
CacheTypevalue is ever introduced and written into context, bothperformDirectSearchandperformSemanticSearchwill befalse, effectively disabling cache lookup. If you’d rather fall back to “both enabled” on unknown values, you could keep the defaults and only flip the one that should be disabled.Please confirm that callers only ever set
CacheTypeKeytoCacheTypeDirectorCacheTypeSemanticat this point; otherwise you may want a default/fallback branch.Also applies to: 381-390
core/providers/utils/utils.go (1)
881-900: Streaming tracing integration is coherent; consider covering scanner-error path tooThe new tracing flow makes sense:
- Each streaming chunk is accumulated via
Tracer.AddStreamingChunk.- Final chunks (as marked by
BifrostContextKeyStreamEndIndicator) callcompleteDeferredSpan, which:
- Looks up the deferred LLM span by
traceID.- Uses
GetAccumulatedChunksto drivePopulateLLMResponseAttributes.- Sets TTFT and total chunk attributes.
- Finalizes post-hook spans via
BifrostContextKeyPostHookSpanFinalizerand ends the LLM span with OK/Error.- Clears the deferred span.
ProcessAndSendResponseandProcessAndSendBifrostErrorare both wired into this, so normal and provider-error streaming terminations are handled.One gap you may want to close:
ProcessAndSendError(the low-level stream read/scanner error path) doesn’t currently invokecompleteDeferredSpan, so a streaming request that fails before aBifrostResponse/BifrostErroris produced could leave the deferred LLM span open. If that function is reachable after the LLM span has been created, callingcompleteDeferredSpan(&ctx, nil, bifrostError)there as well would make streaming span completion fully symmetric.Please double-check where
ProcessAndSendErroris used; if it’s part of the same LLM call lifecycle, consider wiring it intocompleteDeferredSpanto avoid dangling spans on scanner failures.Also applies to: 939-945, 1412-1501
docs/plugins/writing-plugin.mdx (1)
53-59: Plugin v2 example is correct; consider aligning the documented core version with v1.4.x+The v1.4.x+ tab’s hello-world plugin:
- Exposes
HTTPTransportMiddleware() schemas.BifrostHTTPMiddleware,- Uses the standard
next fasthttp.RequestHandlerpattern, and- Keeps
Init,GetName,PreHook,PostHook, andCleanupsignatures consistent with the loader,which matches the updated
Plugininterface and the example code inexamples/plugins/hello-world.One documentation tweak: Step 1’s
go.modstill pinsgithub.com/maximhq/bifrost/core v1.2.38, while the v1.4.x+ code relies on the newer HTTP middleware surface. It might be clearer either to:
- Update that snippet to a v1.4.x-compatible core version, or
- Annotate that snippet as v1.3.x-only and add a separate v1.4.x+ go.mod example (or simply say “match the Bifrost version you’re running”).
Also applies to: 65-127, 213-241, 429-444
core/bifrost.go (1)
3270-3436: Streaming post‑hook timing aggregation and spans are well-structuredThe new
PluginPipelinestreaming timing machinery looks coherent:
RunPostHooksswitches to timing accumulation whenBifrostContextKeyStreamStartTimeis present, avoiding per-chunk span spam.accumulatePluginTimingmaintains per-plugin totals and call counts plus error counts, recording order of first occurrence.FinalizeStreamingPostHookSpansbuilds a nested span hierarchy (plugin.<name>.posthook) in that order, sets aggregated metrics (invocations, avg/total duration, error count), and closes spans in reverse order like a call stack.This design gives a clear, low-cardinality representation of streaming plugin cost while keeping allocation and span volume low.
framework/tracing/helpers.go (1)
10-83: Tracing helpers are straightforward; consider optional nil-store guardThe context-based helpers (
GetTraceID,GetTrace,AddSpan,AddChildSpan,EndSpan,SetSpanAttribute,AddSpanEvent) are all defensive w.r.t. missing trace IDs and missing spans, which is good.If you expect tests or optional wiring to sometimes pass a nil
*TraceStore, a small extra guard inGetTrace/AddSpan/EndSpan(early-return on nilstore) would make them fully panic-proof, but that’s optional given current usage.plugins/otel/converter.go (1)
49-70: OTEL conversion path looks good; minor type-coverage nits onlyThe trace/span → OTEL conversion is generally solid:
hexToBytescleans and normalizes IDs to fixed byte lengths before assigningTraceId/SpanId/ParentSpanId.convertTraceToResourceSpanandconvertSpanToOTELSpancorrectly fill IDs, timestamps, attributes, status, events, and parent span linkage.anyToKeyValueandconvertAttributesToKeyValuescover the common scalar/slice/map cases and gracefully skip empty values.If you want to tighten things further, consider:
- Adding explicit handling for
[]boolinanyToKeyValueso boolean slices don’t fall back to string formatting.- Optionally logging or asserting when
hex.DecodeStringinhexToBytesfails (currently the error is ignored, though the input is sanitized first).These are minor polish items; the core conversion logic is sound.
Also applies to: 72-107, 133-229, 271-285
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
core/go.sumis excluded by!**/*.sumexamples/plugins/hello-world/go.sumis excluded by!**/*.sum
📒 Files selected for processing (66)
core/bifrost.gocore/bifrost_test.gocore/go.modcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.godocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goplugins/governance/go.modplugins/governance/main.goplugins/jsonparser/main.goplugins/logging/main.goplugins/logging/utils.goplugins/maxim/main.goplugins/mocker/main.goplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/semanticcache/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/go.modui/app/workspace/logs/views/logDetailsSheet.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsx
💤 Files with no reviewable changes (2)
- framework/configstore/tables/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (1)
- docs/plugins/migration-guide.mdx
🚧 Files skipped from review as they are similar to previous changes (24)
- core/go.mod
- framework/streaming/audio.go
- transports/bifrost-http/handlers/providers.go
- transports/bifrost-http/handlers/logging.go
- framework/plugins/dynamicplugin_test.go
- core/schemas/context.go
- core/mcp.go
- framework/streaming/types.go
- plugins/maxim/main.go
- plugins/governance/go.mod
- transports/bifrost-http/handlers/ui.go
- transports/go.mod
- transports/bifrost-http/handlers/mcp.go
- plugins/mocker/main.go
- transports/bifrost-http/lib/middleware.go
- core/schemas/bifrost.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- transports/bifrost-http/handlers/config.go
- ui/components/sidebar.tsx
- docs/docs.json
- plugins/jsonparser/main.go
- transports/bifrost-http/handlers/inference.go
- transports/bifrost-http/handlers/health.go
- framework/plugins/dynamicplugin.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/logging/utils.gotransports/bifrost-http/handlers/governance.goframework/streaming/responses.goplugins/semanticcache/main.gocore/schemas/plugin.gotransports/bifrost-http/handlers/middlewares_test.gocore/bifrost.gotransports/bifrost-http/handlers/integrations.goexamples/plugins/hello-world/main.gotransports/bifrost-http/handlers/session.goframework/streaming/chat.goframework/streaming/transcription.gocore/bifrost_test.goframework/tracing/store.goplugins/governance/main.godocs/plugins/writing-plugin.mdxcore/providers/utils/utils.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/cache.goframework/tracing/propagation.goplugins/telemetry/main.goplugins/otel/main.goframework/streaming/accumulator.gotransports/bifrost-http/handlers/plugins.goplugins/otel/converter.goui/components/ui/sheet.tsxexamples/plugins/hello-world/go.modframework/tracing/helpers.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/mcpServer.gocore/schemas/trace.goui/components/ui/switch.tsxdocs/plugins/getting-started.mdxcore/schemas/tracer.goframework/tracing/tracer.goplugins/logging/main.gotransports/bifrost-http/handlers/middlewares.goframework/tracing/llmspan.go
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
plugins/logging/utils.gotransports/bifrost-http/handlers/governance.goframework/streaming/responses.goplugins/semanticcache/main.gocore/schemas/plugin.gotransports/bifrost-http/handlers/middlewares_test.gocore/bifrost.gotransports/bifrost-http/handlers/integrations.goexamples/plugins/hello-world/main.gotransports/bifrost-http/handlers/session.goframework/streaming/chat.goframework/streaming/transcription.gocore/bifrost_test.goframework/tracing/store.goplugins/governance/main.gocore/providers/utils/utils.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/cache.goframework/tracing/propagation.goplugins/telemetry/main.goplugins/otel/main.goframework/streaming/accumulator.gotransports/bifrost-http/handlers/plugins.goplugins/otel/converter.goframework/tracing/helpers.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/mcpServer.gocore/schemas/trace.gocore/schemas/tracer.goframework/tracing/tracer.goplugins/logging/main.gotransports/bifrost-http/handlers/middlewares.goframework/tracing/llmspan.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/mcpServer.gotransports/bifrost-http/handlers/middlewares.go
📚 Learning: 2025-12-24T04:36:57.698Z
Learnt from: danpiths
Repo: maximhq/bifrost PR: 1169
File: transports/bifrost-http/handlers/governance.go:1708-1878
Timestamp: 2025-12-24T04:36:57.698Z
Learning: In governance update handlers (e.g., updateModelConfig, updateProviderGovernance), design updates to support clearing individual fields by sending null/empty values (e.g., {"rate_limit": {"token_max_limit": null}} clears token_max_limit). Follow this pattern for partial updates so users can remove specific governance settings without deleting the whole entity. Ensure budget updates follow the same approach using direct field assignment. Review input validation, JSON decoding (e.g., pointers vs values in Go structs), and API documentation to reflect nullable fields and expected behavior.
Applied to files:
transports/bifrost-http/handlers/governance.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
🧬 Code graph analysis (27)
plugins/logging/utils.go (3)
core/schemas/tracer.go (1)
StreamAccumulatorResult(15-33)framework/streaming/types.go (9)
ProcessedStreamResponse(118-126)StreamType(10-10)StreamTypeText(13-13)StreamTypeChat(14-14)StreamTypeResponses(17-17)StreamTypeAudio(15-15)StreamTypeTranscription(16-16)StreamResponseType(20-20)AccumulatedData(28-48)core/schemas/chatcompletions.go (1)
ChatAssistantMessage(649-656)
transports/bifrost-http/handlers/governance.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/responses.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
core/schemas/plugin.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/chat.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
framework/streaming/transcription.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
core/bifrost_test.go (1)
core/schemas/tracer.go (1)
DefaultTracer(174-176)
framework/tracing/store.go (1)
core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-26)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/providers/utils/utils.go (4)
core/schemas/bifrost.go (6)
BifrostResponse(335-355)BifrostContextKeyTracer(146-146)BifrostContextKeyTraceID(143-143)BifrostContextKeyStreamEndIndicator(131-131)BifrostError(474-483)BifrostContextKeyPostHookSpanFinalizer(149-149)core/schemas/tracer.go (1)
Tracer(38-112)framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/trace.go (2)
AttrTimeToFirstToken(198-198)AttrTotalChunks(199-199)
transports/bifrost-http/integrations/router.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (9)
ListModelsRequest(92-92)TextCompletionRequest(93-93)ChatCompletionRequest(95-95)ResponsesRequest(97-97)EmbeddingRequest(99-99)SpeechRequest(100-100)TranscriptionRequest(102-102)BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)transports/bifrost-http/handlers/inference.go (5)
ChatRequest(172-176)ResponsesRequest(266-270)EmbeddingRequest(273-277)SpeechRequest(279-283)TranscriptionRequest(285-289)
transports/bifrost-http/server/server.go (4)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)transports/bifrost-http/handlers/middlewares.go (2)
TransportInterceptorMiddleware(50-70)NewTracingMiddleware(207-214)framework/tracing/store.go (1)
NewTraceStore(38-65)framework/tracing/tracer.go (1)
NewTracer(24-29)
transports/bifrost-http/handlers/cache.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/main.go (2)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)core/schemas/trace.go (1)
Trace(10-19)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
transports/bifrost-http/handlers/plugins.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (15)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(24-35)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
transports/bifrost-http/handlers/websocket.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/mcpServer.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/trace.go (2)
framework/tracing/helpers.go (1)
AddSpan(32-38)ui/lib/constants/logs.ts (1)
Status(163-163)
framework/tracing/tracer.go (5)
framework/tracing/store.go (1)
TraceStore(24-35)core/schemas/tracer.go (3)
Tracer(38-112)SpanHandle(11-11)StreamAccumulatorResult(15-33)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (6)
BifrostContextKeySpanID(144-144)BifrostRequest(173-193)BifrostResponse(335-355)BifrostError(474-483)BifrostContextKeyRequestID(124-124)BifrostContextKeyStreamEndIndicator(131-131)framework/streaming/types.go (1)
StreamResponseTypeFinal(24-24)
framework/tracing/llmspan.go (6)
core/schemas/bifrost.go (24)
BifrostRequest(173-193)RequestType(89-89)ChatCompletionRequest(95-95)ChatCompletionStreamRequest(96-96)TextCompletionRequest(93-93)TextCompletionStreamRequest(94-94)EmbeddingRequest(99-99)TranscriptionRequest(102-102)TranscriptionStreamRequest(103-103)SpeechRequest(100-100)SpeechStreamRequest(101-101)ResponsesRequest(97-97)ResponsesStreamRequest(98-98)BatchCreateRequest(104-104)BatchListRequest(105-105)BatchRetrieveRequest(106-106)BatchCancelRequest(107-107)BatchResultsRequest(108-108)FileUploadRequest(109-109)FileListRequest(110-110)FileRetrieveRequest(111-111)FileDeleteRequest(112-112)FileContentRequest(113-113)BifrostResponse(335-355)core/schemas/trace.go (5)
AttrProviderName(159-159)AttrRequestModel(160-160)AttrError(216-216)AttrErrorType(217-217)AttrErrorCode(218-218)core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)BifrostChatResponse(27-42)core/schemas/responses.go (2)
BifrostResponsesRequest(32-39)BifrostResponsesResponse(45-85)core/schemas/batch.go (10)
BifrostBatchCreateRequest(65-83)BifrostBatchListRequest(118-133)BifrostBatchRetrieveRequest(150-159)BifrostBatchCancelRequest(205-214)BifrostBatchResultsRequest(234-246)BifrostBatchCreateResponse(91-115)BifrostBatchListResponse(136-147)BifrostBatchRetrieveResponse(167-202)BifrostBatchCancelResponse(222-231)BifrostBatchResultsResponse(285-294)core/schemas/files.go (10)
BifrostFileUploadRequest(53-67)BifrostFileListRequest(109-128)BifrostFileRetrieveRequest(146-159)BifrostFileDeleteRequest(186-198)BifrostFileContentRequest(215-227)BifrostFileUploadResponse(90-106)BifrostFileListResponse(136-143)BifrostFileRetrieveResponse(167-183)BifrostFileDeleteResponse(206-212)BifrostFileContentResponse(235-241)
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
35152fe to
18b4142
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (10)
transports/bifrost-http/integrations/router.go (1)
502-693: [Duplicate] Non-streaming requests still using incorrect context.This issue was previously flagged: non-streaming calls at lines 514, 534, 557, 580, 603, 626, and 660 pass
ctx(*fasthttp.RequestCtx) instead of the enriched Bifrost context (*bifrostCtx). This breaks tracing, plugin context propagation, and may cause compilation errors if the client expectscontext.Context.Please refer to the previous review comment for the detailed fix.
framework/changelog.md (1)
1-25: Breaking change well-documented; tracing mention could be expanded.The breaking change from
TransportInterceptortoHTTPTransportMiddlewareis thoroughly documented with code examples. The tracing framework is mentioned on line 1, but per the past review comment, consider adding a separate section with more details about the tracing APIs (Trace/Span/Tracer types, TraceStore, W3C propagation) for plugin authors who want to adopt tracing.docs/plugins/migration-guide.mdx (1)
247-250: Version reference issue persists.The
go get github.com/maximhq/bifrost/[email protected]command references a tag that may not exist yet. This will cause the documented command to fail for users following the guide.Consider one of:
- Create the v1.4.0 tag before merging
- Update to reference an existing version or use
@latest- Use a placeholder like
@vX.Y.Zwith a note to check the latest versioncore/providers/utils/utils.go (1)
1416-1420: Context should be passed by value, not by pointer.The function signature
completeDeferredSpan(ctx *context.Context, ...)violates Go conventions. Thecontext.Contexttype is designed to be passed by value and is safe for concurrent use. Passing it by pointer adds unnecessary indirection and can cause confusion.🔎 Proposed fix
-func completeDeferredSpan(ctx *context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { - if ctx == nil { +func completeDeferredSpan(ctx context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { + if ctx == nil { return } - traceID, ok := (*ctx).Value(schemas.BifrostContextKeyTraceID).(string) + traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string)Update call sites accordingly:
- completeDeferredSpan(&ctx, response, nil) + completeDeferredSpan(ctx, response, nil)transports/changelog.md (1)
16-23: Version reference inconsistency with core changelog.The migration summary states
v1.3.x (removed)andv1.4.x+ (new), but the core changelog indicates the new API is available incore v1.3.0+. This inconsistency could confuse developers trying to understand which versions to target.Please align the version references across all changelogs to ensure consistent migration guidance.
framework/plugins/dynamicplugin.go (1)
140-147: Fix HTTPTransportMiddleware symbol type assertion (factory func vs middleware func)The dynamic plugin loader assumes
HTTPTransportMiddlewareis exported as aschemas.BifrostHTTPMiddleware, but the example plugin exports:func HTTPTransportMiddleware() schemas.BifrostHTTPMiddleware { ... }So
plugin.Lookup("HTTPTransportMiddleware")returns afunc() schemas.BifrostHTTPMiddleware, not the middleware itself; the current type assertion will always fail and dynamic plugins will never load.You should assert to the factory function type, then call it to obtain the middleware. Also update the error message to match the actual expected type.
Proposed fix for HTTPTransportMiddleware lookup
// Looking up for HTTPTransportMiddleware method httpTransportMiddlewareSym, err := plugin.Lookup("HTTPTransportMiddleware") if err != nil { return nil, err } - if dp.httpTransportMiddleware, ok = httpTransportMiddlewareSym.(schemas.BifrostHTTPMiddleware); !ok { - return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func(next fasthttp.RequestHandler) fasthttp.RequestHandler") - } + httpTransportMiddlewareFunc, ok := httpTransportMiddlewareSym.(func() schemas.BifrostHTTPMiddleware) + if !ok { + return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func() schemas.BifrostHTTPMiddleware") + } + + dp.httpTransportMiddleware = httpTransportMiddlewareFunc() + if dp.httpTransportMiddleware == nil { + return nil, fmt.Errorf("HTTPTransportMiddleware factory returned nil middleware") + }transports/bifrost-http/handlers/middlewares.go (1)
191-307: *TracingMiddleware must pass a context.Context to StartSpan, not fasthttp.RequestCtx
TracingMiddleware.Middlewarecurrently does:spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)But
StartSpanexpects acontext.Context, whilectxhere is*fasthttp.RequestCtx. This will not compile and also prevents you from attaching trace ID into the span context properly.You should construct a proper
context.Context(and inject the trace ID into it) before callingStartSpan, then continue to use the returnedspanCtxfor downstream work.Proposed fix for StartSpan usage in TracingMiddleware
func (m *TracingMiddleware) Middleware() schemas.BifrostHTTPMiddleware { return func(next fasthttp.RequestHandler) fasthttp.RequestHandler { return func(ctx *fasthttp.RequestCtx) { @@ - // Extract parent trace ID from W3C headers (if present) - parentID := tracing.ExtractParentID(&ctx.Request.Header) - // Create trace in store - only ID returned (trace data stays in store) - traceID := m.tracer.Load().CreateTrace(parentID) - // Only trace ID goes into context (lightweight, no bloat) - ctx.SetUserValue(schemas.BifrostContextKeyTraceID, traceID) + // Extract parent trace ID from W3C headers (if present) + parentID := tracing.ExtractParentID(&ctx.Request.Header) + // Create trace in store - only ID returned (trace data stays in store) + traceID := m.tracer.Load().CreateTrace(parentID) + // Only trace ID goes into context/user values (lightweight, no bloat) + ctx.SetUserValue(schemas.BifrostContextKeyTraceID, traceID) + + // Build a base context carrying the trace ID for the tracer + baseCtx := context.WithValue(context.Background(), schemas.BifrostContextKeyTraceID, traceID) @@ - // Create root span for the HTTP request - spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) + // Create root span for the HTTP request + spanCtx, rootSpan := m.tracer.Load().StartSpan(baseCtx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) if rootSpan != nil { m.tracer.Load().SetAttribute(rootSpan, "http.method", string(ctx.Method())) m.tracer.Load().SetAttribute(rootSpan, "http.url", string(ctx.RequestURI())) m.tracer.Load().SetAttribute(rootSpan, "http.user_agent", string(ctx.Request.Header.UserAgent())) // Set root span ID in context for child span creation if spanID, ok := spanCtx.Value(schemas.BifrostContextKeySpanID).(string); ok { ctx.SetUserValue(schemas.BifrostContextKeySpanID, spanID) } }This keeps trace ID propagation via
ctx.UserValuewhile satisfying the tracer’scontext.Contextrequirements.core/bifrost.go (1)
70-71: Tracer initialization and accessors mostly solid; guard against SetTracer(nil) panics
Initnow initializes a tracer fromconfig.Traceror falls back toschemas.DefaultTracer(), and stores it inbifrost.tracer(anatomic.Value), so there is always a tracer by default.getPluginPipelinecorrectly injects the current tracer intoPluginPipeline.Shutdowndefensively callsStop()on any stored tracer.However,
SetTracerallows callers to store anyschemas.Tracervalue, includingnil, andgetTracer()unconditionally does:return bifrost.tracer.Load().(schemas.Tracer)If
SetTracer(nil)is ever called, the nextgetTracer()will panic. GivenSetTraceris public, it’s easy to misuse in the future.Consider either:
- Rejecting
nilinSetTracer(log and ignore, or replace withschemas.DefaultTracer()), and/or- Making
getTracer()defensive by checking the loaded value and falling back toschemas.DefaultTracer()when it’s nil or of the wrong type.Example defensive adjustment
func (bifrost *Bifrost) SetTracer(tracer schemas.Tracer) { - bifrost.tracer.Store(tracer) + if tracer == nil { + // Optional: log and fall back to default no-op tracer + tracer = schemas.DefaultTracer() + } + bifrost.tracer.Store(tracer) } func (bifrost *Bifrost) getTracer() schemas.Tracer { - return bifrost.tracer.Load().(schemas.Tracer) + if v := bifrost.tracer.Load(); v != nil { + if t, ok := v.(schemas.Tracer); ok && t != nil { + return t + } + } + return schemas.DefaultTracer() }Also applies to: 78-100, 120-127, 139-140, 247-255
framework/tracing/llmspan.go (1)
191-193: Consider namespacing ExtraParams to prevent attribute key collisions.User-provided keys from
ExtraParamsare written directly to the attributes map without namespacing throughout this file. If a user sets a key that matches a standard attribute name (e.g.,"gen_ai.request.model"), it would overwrite the standard attribute, potentially causing observability issues.🔎 Suggested approach
Namespace all ExtraParams keys with a consistent prefix like
"gen_ai.extra."or"user.extra."to prevent collisions:// ExtraParams for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) + attrs["gen_ai.extra."+k] = fmt.Sprintf("%v", v) }Apply this pattern consistently across all ExtraParams handling locations in the file.
Also applies to: 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 768-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029
framework/tracing/tracer.go (1)
394-441: Add missing sort.Ints() calls for deterministic output.The comments state "Sort choice indices for deterministic output" (line 394) and "Sort tool calls by index" (line 433), but the actual
sort.Ints()calls are missing. Without sorting, map iteration order is nondeterministic in Go, causing inconsistent ordering of choices and tool calls in reconstructed responses.🔎 Proposed fix
// Sort choice indices for deterministic output choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } + sort.Ints(choiceIndices) for _, idx := range choiceIndices { accum := choiceMap[idx] // Build message msg := &schemas.ChatMessage{ Role: accum.role, } // Set content if accum.content != "" { msg.Content = &schemas.ChatMessageContent{ ContentStr: &accum.content, } } // Build assistant message fields if accum.refusal != "" || accum.reasoning != "" || len(accum.reasoningDetails) > 0 || accum.audio != nil || len(accum.toolCalls) > 0 { msg.ChatAssistantMessage = &schemas.ChatAssistantMessage{} if accum.refusal != "" { msg.ChatAssistantMessage.Refusal = &accum.refusal } if accum.reasoning != "" { msg.ChatAssistantMessage.Reasoning = &accum.reasoning } if len(accum.reasoningDetails) > 0 { msg.ChatAssistantMessage.ReasoningDetails = accum.reasoningDetails } if accum.audio != nil { msg.ChatAssistantMessage.Audio = accum.audio } if len(accum.toolCalls) > 0 { // Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } + sort.Ints(tcIndices) toolCalls := make([]schemas.ChatAssistantMessageToolCall, 0, len(accum.toolCalls)) for _, tcIdx := range tcIndices { toolCalls = append(toolCalls, accum.toolCalls[tcIdx]) } msg.ChatAssistantMessage.ToolCalls = toolCalls } }Add the import:
import ( "context" + "sort" "time" "github.com/maximhq/bifrost/core/schemas"Note: Past review comments indicate this was addressed in commit 1cecccb, but the fix appears to be missing in the current code.
🧹 Nitpick comments (5)
plugins/jsonparser/main.go (1)
95-102: Minor: Empty line in doc comment.Line 95 has an empty doc comment line (
//) which is unnecessary.🔎 Proposed fix
// PreHook is not used for this plugin as we only process responses // Parameters: // - ctx: The Bifrost context // - req: The Bifrost request -// // Returns: // - *schemas.BifrostRequest: The processed request // - *schemas.PluginShortCircuit: The plugin short circuit if the request is not allowed // - error: Any error that occurred during processingtransports/changelog.md (1)
16-23: Add language specifier to fenced code block.Static analysis indicates the code block should have a language specified for proper syntax highlighting.
🔎 Proposed fix
**Migration summary:** - ``` + ```go // v1.3.x (removed) TransportInterceptor(ctx *BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) // v1.4.x+ (new) HTTPTransportMiddleware() BifrostHTTPMiddleware // where BifrostHTTPMiddleware = func(next fasthttp.RequestHandler) fasthttp.RequestHandler ```transports/bifrost-http/server/server.go (1)
1040-1043: UI route registration signature matches new middleware type
RegisterUIRoutesnow takes...schemas.BifrostHTTPMiddlewareand forwards them to the UI handler. Currently Bootstrap callss.RegisterUIRoutes()with no middlewares, which is fine; if you later want metrics/tracing on UI endpoints, you can passcommonMiddlewareshere as well.transports/bifrost-http/handlers/middlewares.go (1)
49-68: Plugin HTTP transport middleware chaining is correct; caching chain could be a future optimization
TransportInterceptorMiddlewarecorrectly:
- Pulls loaded plugins from
config.GetLoadedPlugins().- Collects non‑nil
plugin.HTTPTransportMiddleware()values.- Chains them via
lib.ChainMiddlewarespreserving order.This is functionally sound; if performance ever becomes a concern, you could cache the composed chain per
config.Pluginssnapshot instead of rebuilding on every request.framework/tracing/helpers.go (1)
1-83: Context-based tracing helpers are straightforward; consider guarding nil TraceStoreThe helpers (
GetTraceID,GetTrace,AddSpan,AddChildSpan,EndSpan,SetSpanAttribute,AddSpanEvent) correctly:
- Derive
traceIDfromschemas.BifrostContextKeyTraceIDin the context.- No‑op when no trace ID is present.
- Delegate span operations to
TraceStore/schemas.Trace.As long as callers always pass a non‑nil
*TraceStore, this is safe. If there’s any chance of misuse, you could defensively return early whenstore == nilto avoid panics.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sum
📒 Files selected for processing (89)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/versionui/app/workspace/logs/views/logDetailsSheet.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsx
💤 Files with no reviewable changes (3)
- core/mcp.go
- framework/configstore/tables/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (1)
- plugins/logging/version
🚧 Files skipped from review as they are similar to previous changes (35)
- plugins/otel/version
- .github/workflows/scripts/release-bifrost-http.sh
- transports/bifrost-http/handlers/health.go
- .github/workflows/release-pipeline.yml
- plugins/maxim/version
- transports/bifrost-http/handlers/logging.go
- docs/docs.json
- framework/streaming/audio.go
- framework/plugins/dynamicplugin_test.go
- framework/streaming/types.go
- core/version
- .github/workflows/scripts/push-mintlify-changelog.sh
- plugins/telemetry/version
- examples/plugins/hello-world/main.go
- plugins/governance/version
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- plugins/otel/changelog.md
- ui/components/ui/switch.tsx
- plugins/semanticcache/version
- transports/bifrost-http/handlers/middlewares_test.go
- plugins/mocker/main.go
- plugins/maxim/changelog.md
- framework/version
- docs/plugins/getting-started.mdx
- core/schemas/bifrost.go
- plugins/maxim/main.go
- plugins/logging/utils.go
- transports/version
- plugins/semanticcache/changelog.md
- plugins/mocker/version
- ui/components/sidebar.tsx
- core/schemas/trace.go
- transports/bifrost-http/handlers/config.go
- plugins/governance/main.go
- plugins/jsonparser/version
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/mcpserver.goplugins/logging/changelog.mdcore/schemas/plugin.goplugins/jsonparser/changelog.mdtransports/bifrost-http/handlers/governance.goframework/plugins/dynamicplugin.gotransports/bifrost-http/integrations/router.goframework/streaming/chat.gotransports/bifrost-http/handlers/mcp.gocore/bifrost_test.gotransports/bifrost-http/handlers/websocket.goframework/streaming/transcription.goframework/tracing/store.gocore/schemas/context.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/session.gocore/changelog.mdplugins/otel/main.gotransports/changelog.mdframework/streaming/accumulator.goframework/tracing/helpers.gotransports/bifrost-http/server/server.godocs/plugins/writing-plugin.mdxframework/tracing/propagation.goframework/changelog.mdtransports/bifrost-http/handlers/ui.goplugins/governance/go.modtransports/bifrost-http/handlers/middlewares.goui/components/ui/sheet.tsxtransports/bifrost-http/lib/middleware.goframework/streaming/responses.goplugins/telemetry/changelog.mdtransports/bifrost-http/handlers/inference.godocs/plugins/migration-guide.mdxplugins/governance/changelog.mdplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.goexamples/plugins/hello-world/go.modplugins/logging/main.gocore/schemas/tracer.goplugins/otel/converter.gocore/bifrost.goplugins/jsonparser/main.goplugins/semanticcache/main.gocore/providers/utils/utils.goplugins/mocker/changelog.mdframework/tracing/llmspan.goframework/tracing/tracer.gotransports/bifrost-http/handlers/plugins.go
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/mcpserver.gocore/schemas/plugin.gotransports/bifrost-http/handlers/governance.goframework/plugins/dynamicplugin.gotransports/bifrost-http/integrations/router.goframework/streaming/chat.gotransports/bifrost-http/handlers/mcp.gocore/bifrost_test.gotransports/bifrost-http/handlers/websocket.goframework/streaming/transcription.goframework/tracing/store.gocore/schemas/context.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/session.goplugins/otel/main.goframework/streaming/accumulator.goframework/tracing/helpers.gotransports/bifrost-http/server/server.goframework/tracing/propagation.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.goframework/streaming/responses.gotransports/bifrost-http/handlers/inference.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.goplugins/logging/main.gocore/schemas/tracer.goplugins/otel/converter.gocore/bifrost.goplugins/jsonparser/main.goplugins/semanticcache/main.gocore/providers/utils/utils.goframework/tracing/llmspan.goframework/tracing/tracer.gotransports/bifrost-http/handlers/plugins.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/plugins.go
📚 Learning: 2025-12-24T04:36:57.698Z
Learnt from: danpiths
Repo: maximhq/bifrost PR: 1169
File: transports/bifrost-http/handlers/governance.go:1708-1878
Timestamp: 2025-12-24T04:36:57.698Z
Learning: In governance update handlers (e.g., updateModelConfig, updateProviderGovernance), design updates to support clearing individual fields by sending null/empty values (e.g., {"rate_limit": {"token_max_limit": null}} clears token_max_limit). Follow this pattern for partial updates so users can remove specific governance settings without deleting the whole entity. Ensure budget updates follow the same approach using direct field assignment. Review input validation, JSON decoding (e.g., pointers vs values in Go structs), and API documentation to reflect nullable fields and expected behavior.
Applied to files:
transports/bifrost-http/handlers/governance.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
🧬 Code graph analysis (31)
transports/bifrost-http/handlers/providers.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/mcpserver.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/plugin.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/handlers/governance.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (2)
GetName(15-17)HTTPTransportMiddleware(19-27)
transports/bifrost-http/integrations/router.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
framework/streaming/chat.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/handlers/mcp.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/bifrost_test.go (1)
core/schemas/tracer.go (1)
DefaultTracer(181-183)
transports/bifrost-http/handlers/websocket.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/transcription.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
framework/tracing/store.go (4)
framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/bifrost.go (1)
BifrostResponse(335-355)core/schemas/trace.go (5)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
core/schemas/context.go (1)
core/schemas/bifrost.go (1)
BifrostContextKeyDeferTraceCompletion(147-147)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/main.go (2)
core/schemas/plugin.go (1)
ObservabilityPlugin(123-137)core/schemas/trace.go (1)
Trace(10-19)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(25-37)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/middlewares.go (6)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)Plugin(71-97)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)framework/tracing/propagation.go (1)
ExtractParentID(74-84)core/schemas/bifrost.go (3)
BifrostContextKeyTraceID(143-143)BifrostContextKeySpanID(144-144)BifrostContextKeyDeferTraceCompletion(147-147)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/responses.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
plugins/telemetry/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/cache.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/tracer.go (7)
core/schemas/models.go (1)
Model(109-129)core/schemas/bifrost.go (3)
ModelProvider(33-33)BifrostRequest(173-193)BifrostResponse(335-355)core/schemas/chatcompletions.go (1)
BifrostLLMUsage(845-852)core/schemas/speech.go (1)
BifrostSpeechResponse(22-29)core/schemas/transcriptions.go (1)
BifrostTranscriptionResponse(16-26)core/schemas/trace.go (3)
Trace(10-19)SpanKind(115-115)SpanStatus(143-143)core/schemas/context.go (1)
BifrostContext(32-42)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (17)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindHTTPRequest(131-131)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
core/providers/utils/utils.go (2)
core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/helpers.go (1)
EndSpan(50-56)
framework/tracing/tracer.go (6)
framework/tracing/store.go (1)
TraceStore(25-37)framework/streaming/accumulator.go (2)
Accumulator(14-30)NewAccumulator(440-480)core/schemas/tracer.go (3)
Tracer(38-116)SpanHandle(11-11)StreamAccumulatorResult(15-33)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (4)
BifrostContextKeySpanID(144-144)BifrostRequest(173-193)BifrostResponse(335-355)BifrostError(474-483)framework/tracing/llmspan.go (1)
PopulateResponseAttributes(64-106)
transports/bifrost-http/handlers/plugins.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
🪛 markdownlint-cli2 (0.18.1)
transports/changelog.md
16-16: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (12)
core/providers/utils/utils.go (1)
1416-1420: Pass context by value, not by pointer.The function signature
completeDeferredSpan(ctx *context.Context, ...)violates Go conventions.context.Contextis designed to be passed by value and is safe for concurrent use. Passing it by pointer is an anti-pattern that can lead to subtle bugs.The fix requires changing the signature to accept
context.Contextand updating all call sites (lines 897, 942) to passctxinstead of&ctx.docs/plugins/migration-guide.mdx (1)
247-250: Version reference may be invalid at merge time.Line 249 references
@v1.4.0, but this tag may not exist when users follow this guide. Consider using a placeholder like@latestor updating after the release is tagged.transports/changelog.md (1)
16-23: Version reference inconsistency persists.The migration summary shows
v1.3.x (removed)andv1.4.x+ (new), but ensure this aligns with the actual versioning across all changelogs. The core changelog should be checked for consistency.plugins/governance/main.go (2)
286-292: Fix error message: operation is unmarshal, not marshal.Line 289 logs "failed to marshal request body" but the operation on line 287 is
sonic.Unmarshal. The error message should say "unmarshal".🔎 Proposed fix
- p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body to check for virtual key: %v", err)
233-256: x-bf-vk header missing VirtualKeyPrefix validation.The
x-bf-vkheader (lines 236-238) is returned without validating that it starts withVirtualKeyPrefix, while theAuthorizationBearer token (lines 241-246) andx-api-key(lines 251-253) both validate the prefix. This inconsistency could allow invalid virtual keys to pass through when provided via thex-bf-vkheader.🔎 Proposed fix
vkHeader := ctx.Request.Header.Peek("x-bf-vk") if string(vkHeader) != "" { - return bifrost.Ptr(string(vkHeader)) + vkValue := string(vkHeader) + if strings.HasPrefix(strings.ToLower(vkValue), VirtualKeyPrefix) { + return bifrost.Ptr(vkValue) + } }transports/bifrost-http/integrations/router.go (1)
501-507: Non‑streaming Bifrost calls must usebifrostCtx, not*fasthttp.RequestCtx
handleNonStreamingRequeststill passesctx(a*fasthttp.RequestCtx) into Bifrost client methods instead of the cancellable/enriched*bifrostCtxcreated byConvertToBifrostContext. This likely won’t compile against the client signatures and also drops all tracing and plugin context values stored onbifrostCtx.Proposed fix: use the converted context for all non‑streaming Bifrost client calls
func (g *GenericRouter) handleNonStreamingRequest(ctx *fasthttp.RequestCtx, config RouteConfig, req interface{}, bifrostReq *schemas.BifrostRequest, bifrostCtx *context.Context) { - + // Use the cancellable/enriched context created in createHandler + requestCtx := *bifrostCtx @@ switch { case bifrostReq.ListModelsRequest != nil: - listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) + listModelsResponse, bifrostErr := g.client.ListModelsRequest(requestCtx, bifrostReq.ListModelsRequest) @@ case bifrostReq.TextCompletionRequest != nil: - textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, bifrostReq.TextCompletionRequest) + textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(requestCtx, bifrostReq.TextCompletionRequest) @@ case bifrostReq.ChatRequest != nil: - chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, bifrostReq.ChatRequest) + chatResponse, bifrostErr := g.client.ChatCompletionRequest(requestCtx, bifrostReq.ChatRequest) @@ case bifrostReq.ResponsesRequest != nil: - responsesResponse, bifrostErr := g.client.ResponsesRequest(ctx, bifrostReq.ResponsesRequest) + responsesResponse, bifrostErr := g.client.ResponsesRequest(requestCtx, bifrostReq.ResponsesRequest) @@ case bifrostReq.EmbeddingRequest != nil: - embeddingResponse, bifrostErr := g.client.EmbeddingRequest(ctx, bifrostReq.EmbeddingRequest) + embeddingResponse, bifrostErr := g.client.EmbeddingRequest(requestCtx, bifrostReq.EmbeddingRequest) @@ case bifrostReq.SpeechRequest != nil: - speechResponse, bifrostErr := g.client.SpeechRequest(ctx, bifrostReq.SpeechRequest) + speechResponse, bifrostErr := g.client.SpeechRequest(requestCtx, bifrostReq.SpeechRequest) @@ case bifrostReq.TranscriptionRequest != nil: - transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(ctx, bifrostReq.TranscriptionRequest) + transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(requestCtx, bifrostReq.TranscriptionRequest)Also applies to: 512-683
framework/plugins/dynamicplugin.go (1)
24-29: FixHTTPTransportMiddlewaresymbol type: plugin exports a factory function, not a middlewareDynamic plugins export:
func HTTPTransportMiddleware() schemas.BifrostHTTPMiddleware { ... }but
loadDynamicPlugincurrently asserts the symbol directly toschemas.BifrostHTTPMiddleware. That will always fail at runtime; the symbol is afunc() schemas.BifrostHTTPMiddleware.You need to assert to the factory function and then call it to obtain the middleware, similar to how
GetNameis handled.Proposed fix for `HTTPTransportMiddleware` lookup
// Looking up for HTTPTransportMiddleware method httpTransportMiddlewareSym, err := plugin.Lookup("HTTPTransportMiddleware") if err != nil { return nil, err } - if dp.httpTransportMiddleware, ok = httpTransportMiddlewareSym.(schemas.BifrostHTTPMiddleware); !ok { - return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func(next fasthttp.RequestHandler) fasthttp.RequestHandler") - } + httpTransportMiddlewareFunc, ok := httpTransportMiddlewareSym.(func() schemas.BifrostHTTPMiddleware) + if !ok { + return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func() schemas.BifrostHTTPMiddleware") + } + dp.httpTransportMiddleware = httpTransportMiddlewareFunc()Also applies to: 36-39, 140-147
transports/bifrost-http/handlers/middlewares.go (1)
191-279:TracingMiddlewaremust callStartSpanwith acontext.Contextcarrying the trace ID, not*fasthttp.RequestCtx
StartSpanis defined to take acontext.Context, and the tracing helpers expect the trace ID to live in that context (viaBifrostContextKeyTraceID). Currently:spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)where
ctxis*fasthttp.RequestCtx. This both fails the type contract and preventsStartSpanfrom seeing the trace ID you just generated.Proposed fix: build a proper context with the trace ID before calling `StartSpan`
// Create trace in store - only ID returned (trace data stays in store) traceID := m.tracer.Load().CreateTrace(parentID) // Only trace ID goes into context (lightweight, no bloat) ctx.SetUserValue(schemas.BifrostContextKeyTraceID, traceID) @@ - // Create root span for the HTTP request - spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) + // Create a standard context carrying the trace ID for the tracer + spanCtxInput := context.WithValue(context.Background(), schemas.BifrostContextKeyTraceID, traceID) + // Create root span for the HTTP request + spanCtx, rootSpan := m.tracer.Load().StartSpan(spanCtxInput, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)The existing code that extracts
spanIDfromspanCtxand stores it inctxviaSetUserValuecan stay as‑is.framework/tracing/llmspan.go (1)
190-193: NamespaceExtraParamsattributes to avoid collisions with standard keysAcross multiple helpers,
ExtraParamsare written directly into the attrs map:for k, v := range req.Params.ExtraParams { attrs[k] = fmt.Sprintf("%v", v) }The same pattern appears for batch/file
ExtraParams, too. This means user-defined keys can overwrite core attributes likegen_ai.request.modelor other reserved names, which was previously flagged.Consider:
- Prefixing user keys with a dedicated namespace (e.g.
gen_ai.extra.), and- Optionally guarding against overwriting existing keys.
Example of namespacing ExtraParams
- for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) - } + const extraPrefix = "gen_ai.extra." + for k, v := range req.Params.ExtraParams { + key := extraPrefix + k + if _, exists := attrs[key]; !exists { + attrs[key] = fmt.Sprintf("%v", v) + } + }Apply the same pattern to other
ExtraParamsblocks (chat/text/embeddings/responses/batch/file) for consistency.Also applies to: 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 769-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029
core/bifrost.go (1)
70-71: GuardSetTracer(nil)/ invalid values to avoidgetTracerpanics
getTracer()blindly doesbifrost.tracer.Load().(schemas.Tracer), whileSetTraceraccepts any value (includingnil). ASetTracer(nil)(or storing a non‑schemas.Tracerby mistake) will cause a runtime panic the next timegetTracer()runs, even though you default toDefaultTracer()inInit.Since
SetTraceris public and can be called at runtime, this is a real foot‑gun.Consider hardening both the setter and getter:
- In
SetTracer, reject or normalizenil(e.g., log and fall back toschemas.DefaultTracer()).- In
getTracer, handle nil/invalid values defensively and fall back toschemas.DefaultTracer()instead of panicking.Proposed defensive implementation
func (bifrost *Bifrost) SetTracer(tracer schemas.Tracer) { - bifrost.tracer.Store(tracer) + if tracer == nil { + // Optional: log once if you want visibility + if bifrost.logger != nil { + bifrost.logger.Warn("SetTracer called with nil tracer, falling back to DefaultTracer") + } + tracer = schemas.DefaultTracer() + } + bifrost.tracer.Store(tracer) } // getTracer returns the tracer from atomic storage with type assertion. func (bifrost *Bifrost) getTracer() schemas.Tracer { - return bifrost.tracer.Load().(schemas.Tracer) + if v := bifrost.tracer.Load(); v != nil { + if t, ok := v.(schemas.Tracer); ok && t != nil { + return t + } + if bifrost.logger != nil { + bifrost.logger.Warn("invalid or nil tracer in atomic storage, using DefaultTracer") + } + } + return schemas.DefaultTracer() }Also applies to: 122-127, 139-140, 247-255
framework/tracing/tracer.go (2)
54-60: Fix stale comment referencing “StoreTracer”The
spanHandlecomment still refers to “StoreTracer”, but the concrete type is justTracer. Updating the wording will avoid confusion for readers and tools.
233-458: MakebuildCompleteResponseFromChunksdeterministic for choices and tool callsYou build
choiceMapandtoolCallsas maps, then iterate over their keys without sorting, even though the comments mention “Sort … for deterministic output”. Go map iteration order is intentionally random, so:
- Choice ordering across
result.ChatResponse.Choiceswill be nondeterministic.- Tool call ordering within a message will also be nondeterministic.
For trace replay, testing, and observability, this non‑determinism is undesirable.
Recommend:
- Sort
choiceIndicesbefore the loop.- Sort
tcIndicesbefore buildingtoolCalls.You’ll also need to import
sort.Proposed deterministic ordering
import ( "context" "time" "github.com/maximhq/bifrost/core/schemas" "github.com/maximhq/bifrost/framework/modelcatalog" "github.com/maximhq/bifrost/framework/streaming" + "sort" ) @@ // Build final choices from accumulated data // Sort choice indices for deterministic output choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } + sort.Ints(choiceIndices) @@ if len(accum.toolCalls) > 0 { // Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } + sort.Ints(tcIndices) toolCalls := make([]schemas.ChatAssistantMessageToolCall, 0, len(accum.toolCalls)) for _, tcIdx := range tcIndices { toolCalls = append(toolCalls, accum.toolCalls[tcIdx]) }
🧹 Nitpick comments (3)
plugins/maxim/changelog.md (1)
1-2: Consider consolidating version upgrade entries.Having two separate chore entries for version upgrades is slightly redundant. Consider consolidating into a single entry showing the final versions (core 1.3.0, framework 1.2.0).
🔎 Suggested consolidation
-- chore: upgraded versions of core to 1.2.41 and framework to 1.1.51 -- chore: upgraded versions of core to 1.3.0 and framework to 1.2.0 +- chore: upgraded versions of core to 1.3.0 and framework to 1.2.0framework/tracing/propagation.go (1)
118-121: Consider handling unknown versions per W3C spec.The W3C Trace Context spec recommends that implementations SHOULD try to extract
trace-id,parent-id, and the sampled bit from higher-than-supported versions rather than rejecting them outright. Currently, any version other than "00" returns nil.🔎 Possible enhancement
// Validate version (only 00 is currently supported) - if version != "00" { + if len(version) != 2 || !isHex(version) { return nil } + // For unknown versions > 00, still attempt to parse trace-id and parent-idThis would allow forward compatibility with future W3C versions while maintaining strict validation of the hex format.
framework/tracing/tracer.go (1)
17-20: Consider stopping the streaming accumulator inTracer.Stop()
NewTracerembeds astreaming.Accumulator, which (per its constructor) prewarms pools and starts cleanup/background work.Tracer.Stop()currently only stops the underlyingTraceStore, so any goroutines or timers owned by the accumulator may continue running after shutdown.If
streaming.Accumulatorexposes aStop/Closeor similar, it would be cleaner to invoke it here:func (t *Tracer) Stop() { if t.store != nil { t.store.Stop() } if t.accumulator != nil { t.accumulator.Stop() // or equivalent, if available } }This keeps tracing shutdown self‑contained and avoids potential resource leaks.
Also applies to: 24-29, 539-545
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sum
📒 Files selected for processing (89)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/versionui/app/workspace/logs/views/logDetailsSheet.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsx
💤 Files with no reviewable changes (3)
- framework/configstore/tables/mcp.go
- core/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (3)
- framework/version
- plugins/maxim/version
- core/version
🚧 Files skipped from review as they are similar to previous changes (38)
- core/schemas/context.go
- plugins/governance/version
- ui/components/ui/sheet.tsx
- framework/streaming/types.go
- docs/plugins/getting-started.mdx
- transports/bifrost-http/handlers/integrations.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- transports/bifrost-http/handlers/websocket.go
- transports/bifrost-http/handlers/middlewares_test.go
- plugins/telemetry/version
- plugins/governance/go.mod
- plugins/semanticcache/version
- transports/bifrost-http/handlers/governance.go
- plugins/telemetry/main.go
- framework/streaming/chat.go
- transports/bifrost-http/handlers/providers.go
- .github/workflows/scripts/release-bifrost-http.sh
- framework/streaming/responses.go
- examples/plugins/hello-world/go.mod
- transports/bifrost-http/handlers/ui.go
- transports/version
- transports/bifrost-http/handlers/health.go
- examples/plugins/hello-world/main.go
- transports/bifrost-http/handlers/session.go
- plugins/otel/main.go
- plugins/logging/version
- transports/bifrost-http/handlers/plugins.go
- framework/changelog.md
- docs/docs.json
- transports/bifrost-http/handlers/cache.go
- plugins/otel/changelog.md
- core/schemas/trace.go
- framework/streaming/audio.go
- core/bifrost_test.go
- .github/workflows/scripts/push-mintlify-changelog.sh
- ui/components/ui/switch.tsx
- plugins/jsonparser/version
- transports/bifrost-http/handlers/mcp.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/handlers/logging.goplugins/mocker/main.goframework/plugins/dynamicplugin_test.gotransports/bifrost-http/handlers/config.godocs/plugins/writing-plugin.mdxplugins/mocker/versionframework/plugins/dynamicplugin.goplugins/logging/utils.godocs/plugins/migration-guide.mdxplugins/governance/main.gocore/schemas/plugin.gocore/changelog.mdtransports/changelog.mdtransports/bifrost-http/lib/middleware.goplugins/logging/main.goui/components/sidebar.tsxtransports/bifrost-http/integrations/router.gocore/providers/utils/utils.goframework/streaming/accumulator.goplugins/telemetry/changelog.mdplugins/semanticcache/changelog.mdplugins/maxim/changelog.mdframework/streaming/transcription.goframework/tracing/propagation.goplugins/maxim/main.goframework/tracing/helpers.goframework/tracing/store.gotransports/bifrost-http/handlers/mcpserver.goplugins/mocker/changelog.mdplugins/semanticcache/main.goplugins/otel/converter.gocore/bifrost.goplugins/jsonparser/main.gotransports/bifrost-http/handlers/middlewares.gocore/schemas/bifrost.gocore/schemas/tracer.goplugins/jsonparser/changelog.mdtransports/bifrost-http/handlers/inference.goframework/tracing/tracer.goframework/tracing/llmspan.goplugins/otel/versiontransports/bifrost-http/server/server.goplugins/governance/changelog.mdplugins/logging/changelog.md
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/logging.goplugins/mocker/main.goframework/plugins/dynamicplugin_test.gotransports/bifrost-http/handlers/config.goframework/plugins/dynamicplugin.goplugins/logging/utils.goplugins/governance/main.gocore/schemas/plugin.gotransports/bifrost-http/lib/middleware.goplugins/logging/main.gotransports/bifrost-http/integrations/router.gocore/providers/utils/utils.goframework/streaming/accumulator.goframework/streaming/transcription.goframework/tracing/propagation.goplugins/maxim/main.goframework/tracing/helpers.goframework/tracing/store.gotransports/bifrost-http/handlers/mcpserver.goplugins/semanticcache/main.goplugins/otel/converter.gocore/bifrost.goplugins/jsonparser/main.gotransports/bifrost-http/handlers/middlewares.gocore/schemas/bifrost.gocore/schemas/tracer.gotransports/bifrost-http/handlers/inference.goframework/tracing/tracer.goframework/tracing/llmspan.gotransports/bifrost-http/server/server.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/server/server.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
🧬 Code graph analysis (25)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin.go (4)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/context.go (1)
BifrostContext(32-42)core/schemas/bifrost.go (2)
BifrostRequest(173-193)BifrostResponse(335-355)examples/plugins/hello-world/main.go (2)
GetName(15-17)HTTPTransportMiddleware(19-27)
plugins/logging/utils.go (2)
core/schemas/bifrost.go (6)
RequestType(89-89)TextCompletionStreamRequest(94-94)ChatCompletionStreamRequest(96-96)ResponsesStreamRequest(98-98)SpeechStreamRequest(101-101)TranscriptionStreamRequest(103-103)framework/streaming/types.go (11)
ProcessedStreamResponse(118-126)StreamType(10-10)StreamTypeText(13-13)StreamTypeChat(14-14)StreamTypeResponses(17-17)StreamTypeAudio(15-15)StreamTypeTranscription(16-16)StreamResponseType(20-20)StreamResponseTypeFinal(24-24)StreamResponseTypeDelta(23-23)AccumulatedData(28-48)
core/schemas/plugin.go (3)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)ui/lib/types/plugins.ts (1)
Plugin(12-19)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/integrations/router.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (9)
ListModelsRequest(92-92)TextCompletionRequest(93-93)ChatCompletionRequest(95-95)ResponsesRequest(97-97)EmbeddingRequest(99-99)SpeechRequest(100-100)TranscriptionRequest(102-102)BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)transports/bifrost-http/handlers/inference.go (5)
ChatRequest(172-176)ResponsesRequest(266-270)EmbeddingRequest(273-277)SpeechRequest(279-283)TranscriptionRequest(285-289)
core/providers/utils/utils.go (5)
core/schemas/provider.go (1)
PostHookRunner(312-312)core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/trace.go (4)
AttrTimeToFirstToken(198-198)AttrTotalChunks(199-199)SpanStatusError(151-151)SpanStatusOk(149-149)framework/tracing/helpers.go (1)
EndSpan(50-56)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
framework/streaming/transcription.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
plugins/maxim/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(25-37)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
framework/tracing/store.go (1)
core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)
transports/bifrost-http/handlers/mcpserver.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (17)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindHTTPRequest(131-131)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
core/bifrost.go (6)
core/schemas/tracer.go (3)
Tracer(38-116)DefaultTracer(181-183)SpanHandle(11-11)framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/trace.go (10)
SpanKindFallback(129-129)AttrProviderName(159-159)AttrRequestModel(160-160)SpanStatusError(151-151)SpanStatusOk(149-149)SpanKind(115-115)SpanKindRetry(127-127)SpanKindLLMCall(121-121)SpanKindInternal(139-139)SpanKindPlugin(123-123)framework/tracing/helpers.go (1)
EndSpan(50-56)core/utils.go (1)
IsStreamRequestType(197-199)core/schemas/context.go (1)
NewBifrostContextWithTimeout(70-73)
core/schemas/bifrost.go (2)
core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)
core/schemas/tracer.go (8)
core/schemas/models.go (1)
Model(109-129)core/schemas/bifrost.go (2)
ModelProvider(33-33)BifrostResponse(335-355)core/schemas/chatcompletions.go (1)
BifrostLLMUsage(845-852)core/schemas/speech.go (1)
BifrostSpeechResponse(22-29)core/schemas/transcriptions.go (1)
BifrostTranscriptionResponse(16-26)core/schemas/trace.go (3)
Trace(10-19)SpanKind(115-115)SpanStatus(143-143)framework/tracing/helpers.go (1)
EndSpan(50-56)framework/tracing/tracer.go (1)
Tracer(17-20)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
framework/tracing/tracer.go (10)
framework/tracing/store.go (1)
TraceStore(25-37)framework/streaming/accumulator.go (2)
Accumulator(14-30)NewAccumulator(440-480)core/schemas/tracer.go (2)
Tracer(38-116)SpanHandle(11-11)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (6)
BifrostContextKeySpanID(144-144)BifrostRequest(173-193)BifrostResponse(335-355)BifrostError(474-483)BifrostContextKeyRequestID(124-124)BifrostContextKeyStreamEndIndicator(131-131)core/schemas/chatcompletions.go (1)
BifrostChatResponse(27-42)core/schemas/models.go (1)
Model(109-129)core/schemas/context.go (2)
BifrostContext(32-42)NewBifrostContext(47-65)framework/streaming/types.go (1)
StreamResponseTypeFinal(24-24)
framework/tracing/llmspan.go (6)
core/schemas/bifrost.go (23)
BifrostRequest(173-193)RequestType(89-89)ChatCompletionRequest(95-95)ChatCompletionStreamRequest(96-96)TextCompletionRequest(93-93)TextCompletionStreamRequest(94-94)EmbeddingRequest(99-99)TranscriptionRequest(102-102)TranscriptionStreamRequest(103-103)SpeechRequest(100-100)SpeechStreamRequest(101-101)ResponsesRequest(97-97)BatchCreateRequest(104-104)BatchListRequest(105-105)BatchRetrieveRequest(106-106)BatchCancelRequest(107-107)BatchResultsRequest(108-108)FileUploadRequest(109-109)FileListRequest(110-110)FileRetrieveRequest(111-111)FileDeleteRequest(112-112)FileContentRequest(113-113)BifrostResponse(335-355)core/schemas/trace.go (9)
AttrProviderName(159-159)AttrRequestModel(160-160)AttrError(216-216)AttrErrorType(217-217)AttrErrorCode(218-218)AttrMaxTokens(163-163)AttrTemperature(164-164)AttrTopP(165-165)AttrStopSequences(166-166)core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)BifrostChatResponse(27-42)core/schemas/textcompletions.go (2)
BifrostTextCompletionRequest(10-17)BifrostTextCompletionResponse(69-77)core/schemas/responses.go (2)
BifrostResponsesRequest(32-39)BifrostResponsesResponse(45-85)core/schemas/files.go (10)
BifrostFileUploadRequest(53-67)BifrostFileListRequest(109-128)BifrostFileRetrieveRequest(146-159)BifrostFileDeleteRequest(186-198)BifrostFileContentRequest(215-227)BifrostFileUploadResponse(90-106)BifrostFileListResponse(136-143)BifrostFileRetrieveResponse(167-183)BifrostFileDeleteResponse(206-212)BifrostFileContentResponse(235-241)
transports/bifrost-http/server/server.go (4)
core/schemas/plugin.go (3)
BifrostHTTPMiddleware(38-38)Plugin(71-97)ObservabilityPlugin(123-137)transports/bifrost-http/handlers/middlewares.go (2)
TransportInterceptorMiddleware(50-70)NewTracingMiddleware(207-215)framework/tracing/store.go (1)
NewTraceStore(40-67)framework/tracing/tracer.go (1)
NewTracer(24-29)
🔇 Additional comments (56)
plugins/otel/version (1)
1-1: LGTM!Version bump to 1.0.52 is appropriate for the HTTPTransportMiddleware migration changes.
.github/workflows/release-pipeline.yml (2)
5-6: LGTM!Adding
v1.4.0to the trigger branches is appropriate for releasing the new HTTPTransportMiddleware changes.
597-597: Good fix for dynamic branch reference.Using
${{ github.ref_name }}instead of hardcodedmainin the Discord notification correctly reflects the actual branch that triggered the pipeline.ui/components/sidebar.tsx (2)
208-208: LGTM!Styling update from
rounded-mdtorounded-smis a minor visual refinement.
243-243: LGTM!Consistent styling update to match the parent menu button's border radius.
core/changelog.md (2)
3-5: LGTM!Feature additions are well documented and align with the PR objectives.
6-35: Well-documented breaking change with clear migration path.The breaking changes section provides comprehensive guidance including:
- Clear before/after API signatures
- Key behavioral changes
- Actionable migration steps
- Reference to the migration guide
One note: The documentation references "core v1.3.0" (lines 12, 17), but the PR targets branch
v1.4.0. Verify these version references are intentional and consistent with your versioning strategy.plugins/jsonparser/main.go (2)
86-89: LGTM!Correctly implements
HTTPTransportMiddleware()returningnilsince this plugin only processes responses viaPostHookand doesn't need HTTP transport interception.
91-102: Minor documentation formatting update.The added blank line in the doc comment is a minor formatting adjustment with no functional impact.
plugins/semanticcache/changelog.md (1)
1-18: LGTM!Changelog comprehensively documents the breaking change from
TransportInterceptortoHTTPTransportMiddlewarewith clear guidance for plugin consumers.plugins/maxim/changelog.md (1)
4-19: LGTM!Breaking changes documentation is consistent with other plugin changelogs and provides clear migration guidance.
plugins/jsonparser/changelog.md (1)
1-18: LGTM!Changelog is well-structured and consistently documents the breaking change migration from
TransportInterceptortoHTTPTransportMiddleware.transports/bifrost-http/handlers/mcpserver.go (1)
73-77: LGTM!The middleware type migration from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddlewareis consistent with the PR-wide refactoring. The route registration logic remains unchanged.core/providers/utils/utils.go (2)
887-899: LGTM!The streaming chunk accumulation and deferred span completion logic is well-structured. The defensive nil checks before type assertions are appropriate.
1472-1484: Post-hook span finalization logic is well-designed.The approach of injecting the deferred span ID into the context before calling the finalizer ensures post-hook spans are correctly parented under the LLM span. The fallback to the original context when no span ID exists is appropriate.
docs/plugins/migration-guide.mdx (1)
1-14: Well-structured migration guide.The documentation clearly explains the API changes, provides before/after code examples, and includes helpful troubleshooting. The comparison table at lines 21-27 effectively summarizes the key differences.
transports/changelog.md (1)
9-31: Breaking changes are well-documented.The changelog clearly communicates the API change, provides a migration summary with old/new signatures, and links to the detailed migration guide. This gives developers the information they need to update their plugins.
core/schemas/bifrost.go (2)
25-25: LGTM!The
Tracerfield addition with thenil = NoOpTracercomment provides a clean opt-in mechanism for distributed tracing while maintaining backward compatibility.
143-149: Context keys are well-documented.The new tracing context keys follow the established naming convention and have clear inline documentation explaining their purpose. The distinction between keys set by Bifrost vs. user-settable keys is clear from the comments.
framework/tracing/propagation.go (2)
10-56: LGTM!The normalization functions properly handle UUID-to-W3C format conversion by stripping hyphens, enforcing lowercase, and validating hex characters. The truncation behavior for span IDs correctly handles longer inputs like UUIDs.
72-84: Implementation is correct per clarification.Based on learnings,
ExtractParentIDintentionally returnsctx.TraceIDbecause it extracts the trace ID from the parent request's traceparent header for trace continuation across service boundaries.transports/bifrost-http/handlers/logging.go (1)
42-50: LGTM!The middleware type migration from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddlewareis consistent with the PR-wide refactoring pattern.framework/streaming/transcription.go (1)
44-58: LGTM!The Time to First Token (TTFT) calculation is correctly implemented and consistent with the pattern used in other streaming files (audio.go, chat.go, responses.go). The defensive check for non-zero timestamps before calculation prevents invalid TTFT values.
transports/bifrost-http/handlers/config.go (1)
55-55: LGTM! Middleware type migration aligns with plugin v2 architecture.The migration from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddlewarecentralizes the middleware type definition in the schemas package, consistent with the broader architectural changes in this PR.plugins/telemetry/changelog.md (1)
1-18: LGTM! Breaking changes are well-documented.The changelog comprehensively documents the migration from
TransportInterceptortoHTTPTransportMiddleware, including:
- Clear API comparison (old vs new)
- Consumer impact and required actions
- Version information
- Reference to the Plugin Migration Guide
This addresses the previous review feedback effectively.
plugins/governance/changelog.md (1)
1-21: LGTM! Comprehensive breaking change documentation.The changelog provides thorough documentation of the plugin interface migration, consistent with other plugin changelogs in this PR. The breaking changes section clearly communicates the impact and migration steps.
plugins/logging/changelog.md (1)
1-19: LGTM! Breaking changes and feature improvements documented.The changelog documents both the breaking API change and the feature improvement (central accumulator for memory optimization), providing complete information for plugin consumers.
transports/bifrost-http/handlers/inference.go (2)
365-365: LGTM! Middleware type migration consistent with plugin v2 architecture.The change from
lib.BifrostHTTPMiddlewaretoschemas.BifrostHTTPMiddlewarealigns with the broader migration to centralize middleware types in the schemas package.
1072-1090: LGTM! Streaming trace completion logic is well-designed.The deferred trace completion implementation correctly:
- Signals to middleware that trace completion should be deferred (line 1074)
- Retrieves the trace completer callback from context (line 1077)
- Ensures
w.Flush()completes before finalizing the trace (line 1084)- Gracefully handles missing trace completer with nil check (line 1087)
The type assertion at line 1077 silently discards the boolean, which is acceptable here since the nil check at line 1087 handles the case where the assertion fails or the value is not set.
Pattern observation: This ensures proper OTEL span lifecycle management where the parent trace (including
llm.callspans) is only finalized after the entire streaming response has been written to the client.plugins/mocker/main.go (1)
481-484: LGTM! Clean migration to HTTPTransportMiddleware.The migration from
TransportInterceptortoHTTPTransportMiddlewareis correct. Returningnilis appropriate for this plugin since it doesn't need HTTP transport-level interception (it uses PreHook/PostHook for request/response mocking instead).docs/plugins/writing-plugin.mdx (2)
65-241: LGTM! Excellent versioned documentation for the breaking change.The documentation effectively communicates the plugin interface migration by:
- Using tabs to provide version-specific examples (v1.4.x+ vs v1.3.x)
- Showing complete code examples with the new HTTPTransportMiddleware pattern
- Explaining the middleware composition pattern (calling
next(ctx))- Including appropriate warnings about transport-specific behavior
- Maintaining backward compatibility documentation for reference
This approach makes it easy for plugin developers to understand both the old and new patterns and migrate their plugins accordingly.
429-444: LGTM! Updated log examples reflect the new middleware pattern.The log output examples correctly show
HTTPTransportMiddleware calledfor v1.4.x+ and maintain the legacyTransportInterceptor calledfor v1.3.x, helping developers verify their plugin is working correctly after migration.plugins/semanticcache/main.go (1)
338-341: LGTM! HTTPTransportMiddleware migration is correct.The migration from
TransportInterceptortoHTTPTransportMiddlewarefollows the same pattern as other plugins. Returningnilis appropriate since this plugin performs semantic caching via PreHook/PostHook and doesn't require HTTP transport-level interception.plugins/logging/utils.go (1)
229-300: LGTM! Solid streaming conversion implementation.The function correctly maps streaming accumulator results to processed stream responses with:
- Proper nil handling
- Comprehensive request type → stream type mapping with safe defaults
- Correct response type determination based on finality
- Safe tool calls extraction with nil guards
plugins/mocker/changelog.md (1)
1-18: Excellent changelog documentation.The breaking changes section now comprehensively documents the TransportInterceptor → HTTPTransportMiddleware migration with clear comparison, consumer guidance, and migration references.
framework/streaming/accumulator.go (3)
104-118: LGTM! Proper timestamp initialization for TTFT.The accumulator now correctly initializes both
TimestampandStartTimestampto provide a baseline for TTFT calculations when chunks arrive.
138-141: LGTM! Consistent FirstChunkTimestamp tracking.First chunk timestamp is now consistently captured across all streaming types (chat, transcription, audio, responses), enabling accurate TTFT calculations.
Also applies to: 162-165, 186-189, 210-213
384-392: LGTM! Thread-safe public accumulator creation.The public
CreateStreamAccumulatorfunction properly uses locking when overriding the start timestamp, maintaining thread safety.transports/bifrost-http/lib/middleware.go (1)
3-23: LGTM! Clean type migration to centralized schemas.The middleware chaining logic is unchanged; only the type reference has been updated to use the centralized
schemas.BifrostHTTPMiddlewaredefinition.framework/plugins/dynamicplugin_test.go (1)
54-84: LGTM! Test correctly validates new middleware API.The test properly exercises the new
HTTPTransportMiddlewareflow:
- Obtains middleware function from plugin
- Wraps a next handler using standard middleware pattern
- Uses
fasthttp.RequestCtxas expected- Verifies next handler invocation
core/schemas/plugin.go (3)
36-38: LGTM! Standard middleware type definition.The
BifrostHTTPMiddlewaretype follows the standard fasthttp middleware pattern: receives next handler, returns wrapped handler.
75-79: LGTM! Clear middleware method documentation.The
HTTPTransportMiddlewaremethod is well-documented with clear usage notes and transport-specific applicability.
109-137: LGTM! Clean observability extension pattern.The
ObservabilityPlugininterface extendsPluginand adds trace injection capability for observability backends. The documentation clearly explains async invocation and implementation requirements.plugins/maxim/main.go (1)
124-127: LGTM! Appropriate nil return for unused middleware.Returning
nilfromHTTPTransportMiddlewarecorrectly indicates this plugin doesn't require HTTP transport interception.plugins/governance/main.go (1)
258-309: Well-structured HTTP middleware implementation.The middleware correctly:
- Parses virtual key from multiple header sources
- Validates virtual key state
- Augments headers with MCP tool configurations
- Performs load balancing with weighted provider selection
- Marshals/unmarshals request body using sonic
The implementation follows the standard middleware pattern and integrates well with the governance store.
transports/bifrost-http/integrations/router.go (2)
69-73: Middleware type migration toschemas.BifrostHTTPMiddlewarelooks consistent
ExtensionRouter.RegisterRoutesandGenericRouter.RegisterRoutesnow accept...schemas.BifrostHTTPMiddlewareand still chain vialib.ChainMiddlewares, which matches the new plugin v2 middleware API and the updated handlers/server wiring.Also applies to: 318-367
1105-1121: Streaming trace deferral and completion wiring looks correctSetting
BifrostContextKeyDeferTraceCompletionand using atraceCompletercallback that runs after the SSE writer finishes ensures:
- The tracing middleware doesn’t flush traces prematurely for streaming requests, and
- Traces are completed only after the full stream (including final chunks) is written.
This aligns with the new tracing/observability design and keeps SSE semantics intact.
Also applies to: 1134-1143, 1336-1347
framework/tracing/store.go (1)
24-37: TraceStore concurrency and lifecycle management look solid
TraceStoreusessync.Map+ pools for allocation, guards background cleanup withstopOnce, and cleanly shuts down viaStop(). Streaming helpers (AppendStreamingChunk/GetAccumulatedData) are nil‑safe and won’t panic even if no deferred span exists. This addresses the earlier double‑close panic risk while keeping the store safe for repeated shutdowns.Also applies to: 39-67, 313-371
plugins/logging/main.go (1)
96-111: Logging plugin’s integration with central tracer and middleware API looks correct
HTTPTransportMiddlewarecorrectly returnsnil, so this plugin doesn’t participate in HTTP middleware chaining.- Streaming requests now create a stream accumulator via the tracer in
PreHook, andPostHookdelegates streaming aggregation totracer.ProcessStreamingChunk, only updating logs on final chunks.- Cleanup of stream accumulators is explicitly left to the tracing middleware, while
Cleanup()only stops the internal ticker/worker.This matches the new plugin v2 + tracing design without introducing new lifecycle issues.
Also applies to: 113-156, 194-197, 225-233, 384-390, 423-470, 614-625
transports/bifrost-http/server/server.go (1)
29-29: HTTP middleware type migration and tracing bootstrap wiring look coherent
RegisterInferenceRoutes,RegisterAPIRoutes, andRegisterUIRoutesnow consistently accept...schemas.BifrostHTTPMiddleware, aligning with the new plugin v2 middleware API and updated handlers.PrepareCommonMiddlewaresreturns[]schemas.BifrostHTTPMiddlewareand uses the Prometheus plugin’sHTTPMiddlewaredirectly.- In
Bootstrap, inference middlewares are composed as: tracing → transport interceptor → common (telemetry) → optional auth, with tracer initialization gated on observability/logging plugins and wired into the Bifrost client.This is a clean integration of the new tracing + middleware architecture.
Also applies to: 939-947, 949-1033, 1035-1039, 1095-1107, 1213-1217, 1231-1233, 1236-1238, 1240-1264
transports/bifrost-http/handlers/middlewares.go (2)
3-18: Middleware type migration for CORS, plugin transport, and auth looks good
CorsMiddleware,TransportInterceptorMiddleware, andAuthMiddlewarenow all returnschemas.BifrostHTTPMiddleware, matching the new core middleware type.TransportInterceptorMiddlewarecorrectly collectsplugin.HTTPTransportMiddleware()values, skipsnil, and chains them vialib.ChainMiddlewareswhile preserving plugin order.- Auth semantics remain unchanged while adopting the new middleware type.
No functional regressions spotted here.
Also applies to: 20-47, 49-70, 84-189
281-307: Trace completion and observability forwarding logic is well‑structured
completeAndFlushTracecorrectly:
- Cleans up the stream accumulator for the trace,
- Calls
EndTraceto obtain the completed trace,- Iterates observability plugins and calls
Injecton a fresh background context, and- Returns the trace to the pool via
ReleaseTrace.
GetObservabilityPluginsprovides a simple helper for extracting observability plugins via type assertion. This matches the rest of the tracing/OTEL integration.Also applies to: 309-329
framework/tracing/helpers.go (1)
1-83: Context helpers for trace/span operations are safe and minimalThese helpers:
- Safely derive the current trace ID from context,
- No‑op when there is no trace ID or trace/span present, and
- Delegate span creation/ending/attribute/event operations to
TraceStore.This keeps call sites simple while avoiding panics when tracing is disabled or unavailable.
plugins/otel/converter.go (1)
72-87: OTEL trace/span conversion is comprehensive and type‑safe
convertTraceToResourceSpanbuilds a singleResourceSpanwith appropriate resource attributes and instrumentation scope, converting all Bifrost spans viaconvertSpanToOTELSpan.convertSpanToOTELSpancorrectly maps trace/span IDs, timestamps, attributes, events, status, and parent span ID.anyToKeyValueandconvertAttributesToKeyValueshandle common scalar, slice, and map types while skipping nil/empty values, andconvertSpanKind/convertSpanStatus/convertSpanEventsmap Bifrost enums and events into the corresponding OTEL representations.This should produce well‑formed OTLP traces for downstream observability backends.
Also applies to: 90-110, 112-131, 133-229, 231-257, 259-269, 271-285
core/schemas/tracer.go (1)
13-116: Tracing core abstractions look consistent and safeThe
Tracerinterface,StreamAccumulatorResult, andNoOpTracerare well-aligned with how tracing is consumed elsewhere (Bifrost + framework tracer). No-op implementations avoid panics even when spans/handles are nil, and the interface surface is coherent for both streaming and non‑streaming flows.Also applies to: 118-187
framework/tracing/llmspan.go (1)
377-383: The current implementation is correct—no change neededThe
Embeddingfield inEmbeddingInputis typed as[]int, not[]float64. The%dformat specifier is appropriate for integer values. The suggested change to%gis not applicable here.Likely an incorrect or invalid review comment.
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
18b4142 to
4d9d0fc
Compare
There was a problem hiding this comment.
Actionable comments posted: 9
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
ui/components/ui/switch.tsx (1)
1-36: Address breaking changes to Switch component styling.This PR includes changes to the Switch component's default size and border-radius styling:
- Default size change (line 13): Changing the default from
"default"to"md"is a breaking change. Any existing<Switch>instances without an explicitsizeprop will render smaller than before.- Border radius change (lines 16, 26): Changing from
rounded-fulltorounded-smchanges the visual appearance from fully rounded to slightly rounded corners. This is a breaking visual change across all Switch instances.Both changes should be evaluated for backward compatibility impact. If these changes are intentional as part of the larger refactor, consider documenting the breaking changes in a changelog or migration guide.
♻️ Duplicate comments (12)
ui/components/ui/switch.tsx (1)
13-13: Breaking change: default size reduction affects existing usages.As flagged in the previous review, changing the default from
"default"to"md"will cause all existing Switch components without an explicit size prop to render smaller (h-5 w-9 instead of h-6 w-11). This is a breaking visual change that impacts multiple files across the codebase.🔎 Revert to preserve existing behavior
- ({ className, size = "md", ...props }, ref) => ( + ({ className, size = "default", ...props }, ref) => (Alternatively, if the smaller default is intentional, audit all Switch usages and explicitly add
size="default"where the larger appearance must be preserved.plugins/mocker/version (1)
1-1: Version inconsistency was previously flagged.The concern about patch-level bump for a breaking change was already raised in a prior review. See the existing comment for details on aligning with semantic versioning.
transports/changelog.md (1)
16-23: Clarify version boundaries for the API migration.The version labels "v1.3.x (removed)" and "v1.4.x+ (new)" conflict with other changelogs that indicate the new
HTTPTransportMiddlewareAPI was introduced in core v1.3.0. This inconsistency may confuse developers about which versions support which API.If the new API is available in v1.3.0+, consider updating the labels to reflect that
TransportInterceptorwas removed in v1.3.0 (not v1.3.x) and the new API is available in v1.3.0+ (not v1.4.x+). Alternatively, if v1.4.x truly introduces the breaking change, clarify this across all changelogs.🔎 Suggested version alignment
Based on the pattern in plugin changelogs referencing "core v1.3.0 and framework v1.2.0":
**Migration summary:** - ``` - // v1.3.x (removed) + ```text + // v1.2.x and earlier (removed) TransportInterceptor(ctx *BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) - // v1.4.x+ (new) + // v1.3.0+ (new) HTTPTransportMiddleware() BifrostHTTPMiddleware // where BifrostHTTPMiddleware = func(next fasthttp.RequestHandler) fasthttp.RequestHandler</details> </blockquote></details> <details> <summary>transports/bifrost-http/integrations/router.go (1)</summary><blockquote> `502-607`: **Non‑streaming handlers should use the converted Bifrost context, not `*fasthttp.RequestCtx`.** In `handleNonStreamingRequest`, all Bifrost client calls still pass `ctx` (a `*fasthttp.RequestCtx`) instead of the cancellable/enriched context from `ConvertToBifrostContext`: ```go listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, bifrostReq.TextCompletionRequest) chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, bifrostReq.ChatRequest) responsesResponse, bifrostErr := g.client.ResponsesRequest(ctx, bifrostReq.ResponsesRequest) embeddingResponse, bifrostErr := g.client.EmbeddingRequest(ctx, bifrostReq.EmbeddingRequest) speechResponse, bifrostErr := g.client.SpeechRequest(ctx, bifrostReq.SpeechRequest) transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(ctx, bifrostReq.TranscriptionRequest)This contradicts the comment about using the cancellable context, and (assuming the Bifrost client expects
context.Context) is both a compile‑time type mismatch and a functional break for tracing, plugin context, and timeouts.These calls should use
*bifrostCtx(or a localrequestCtx := *bifrostCtx) exactly like the batch/file handlers:Suggested fix
-func (g *GenericRouter) handleNonStreamingRequest(ctx *fasthttp.RequestCtx, config RouteConfig, req interface{}, bifrostReq *schemas.BifrostRequest, bifrostCtx *context.Context) { - // Use the cancellable context from ConvertToBifrostContext +func (g *GenericRouter) handleNonStreamingRequest(ctx *fasthttp.RequestCtx, config RouteConfig, req interface{}, bifrostReq *schemas.BifrostRequest, bifrostCtx *context.Context) { + // Use the cancellable context from ConvertToBifrostContext @@ - case bifrostReq.ListModelsRequest != nil: - listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) + case bifrostReq.ListModelsRequest != nil: + listModelsResponse, bifrostErr := g.client.ListModelsRequest(*bifrostCtx, bifrostReq.ListModelsRequest) @@ - case bifrostReq.TextCompletionRequest != nil: - textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, bifrostReq.TextCompletionRequest) + case bifrostReq.TextCompletionRequest != nil: + textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(*bifrostCtx, bifrostReq.TextCompletionRequest) @@ - case bifrostReq.ChatRequest != nil: - chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, bifrostReq.ChatRequest) + case bifrostReq.ChatRequest != nil: + chatResponse, bifrostErr := g.client.ChatCompletionRequest(*bifrostCtx, bifrostReq.ChatRequest) @@ - case bifrostReq.ResponsesRequest != nil: - responsesResponse, bifrostErr := g.client.ResponsesRequest(ctx, bifrostReq.ResponsesRequest) + case bifrostReq.ResponsesRequest != nil: + responsesResponse, bifrostErr := g.client.ResponsesRequest(*bifrostCtx, bifrostReq.ResponsesRequest) @@ - case bifrostReq.EmbeddingRequest != nil: - embeddingResponse, bifrostErr := g.client.EmbeddingRequest(ctx, bifrostReq.EmbeddingRequest) + case bifrostReq.EmbeddingRequest != nil: + embeddingResponse, bifrostErr := g.client.EmbeddingRequest(*bifrostCtx, bifrostReq.EmbeddingRequest) @@ - case bifrostReq.SpeechRequest != nil: - speechResponse, bifrostErr := g.client.SpeechRequest(ctx, bifrostReq.SpeechRequest) + case bifrostReq.SpeechRequest != nil: + speechResponse, bifrostErr := g.client.SpeechRequest(*bifrostCtx, bifrostReq.SpeechRequest) @@ - case bifrostReq.TranscriptionRequest != nil: - transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(ctx, bifrostReq.TranscriptionRequest) + case bifrostReq.TranscriptionRequest != nil: + transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(*bifrostCtx, bifrostReq.TranscriptionRequest)Also applies to: 660-693
core/providers/utils/utils.go (1)
1412-1501: Passcontext.Contextby value incompleteDeferredSpan(repeat of earlier review).
completeDeferredSpan(ctx *context.Context, ...)is still taking a pointer tocontext.Contextand then dereferencing for everyValuelookup. Go convention (andcontext.Context’s design) is to always pass contexts by value; using a pointer buys nothing and complicates call sites (completeDeferredSpan(&ctx, ...)).This was already flagged previously; you can simplify and make the API more idiomatic by:
- Changing the signature to
func completeDeferredSpan(ctx context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError).- Removing the nil‑pointer check and using
ctx.Value(...)directly.- Updating all callers to pass
ctx(not&ctx).Proposed refactor
-func completeDeferredSpan(ctx *context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { - if ctx == nil { - return - } - - // Get the trace ID from context (this IS available in the provider's goroutine) - traceID, ok := (*ctx).Value(schemas.BifrostContextKeyTraceID).(string) +func completeDeferredSpan(ctx context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { + // Get the trace ID from context (this IS available in the provider's goroutine) + traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string) @@ - tracerVal := (*ctx).Value(schemas.BifrostContextKeyTracer) + tracerVal := ctx.Value(schemas.BifrostContextKeyTracer) @@ - if finalizer, ok := (*ctx).Value(schemas.BifrostContextKeyPostHookSpanFinalizer).(func(context.Context)); ok && finalizer != nil { + if finalizer, ok := ctx.Value(schemas.BifrostContextKeyPostHookSpanFinalizer).(func(context.Context)); ok && finalizer != nil { @@ - finalizerCtx := context.WithValue(*ctx, schemas.BifrostContextKeySpanID, spanID) + finalizerCtx := context.WithValue(ctx, schemas.BifrostContextKeySpanID, spanID) finalizer(finalizerCtx) } else { - finalizer(*ctx) + finalizer(ctx) } } @@ - tracer.ClearDeferredSpan(traceID) + tracer.ClearDeferredSpan(traceID)And adjust call sites, e.g.:
- completeDeferredSpan(&ctx, response, nil) + completeDeferredSpan(ctx, response, nil)- completeDeferredSpan(&ctx, nil, bifrostErr) + completeDeferredSpan(ctx, nil, bifrostErr)docs/plugins/migration-guide.mdx (1)
247-250: Ensure@v1.4.0exists before publishing this command.The migration guide suggests:
go get github.com/maximhq/bifrost/[email protected]If the
v1.4.0tag is not created by the time this guide is used, this command will fail for users. Either:
- Guarantee that
v1.4.0is tagged before releasing these docs, or- Update the example to reference an existing tag or a commit hash, and adjust when the tag is cut.
examples/plugins/hello-world/main.go (1)
19-27: Example no longer demonstrates transport→hook context propagation.
HTTPTransportMiddlewarestores"hello-world-plugin-transport-interceptor"as a user value on*fasthttp.RequestCtx, butPreHookandPostHookread that key from*schemas.BifrostContext. Without explicitly copying this value into the Bifrost context,value1will always lognil, which undercuts the example’s intent.Either:
- Bridge the value into
BifrostContext(e.g., fetch*schemas.BifrostContextfromctx.UserValueand callSetValue), or- Remove the
value1reads fromPreHook/PostHookif you don’t want to demo this propagation anymore.Also applies to: 29-44
plugins/governance/main.go (2)
233-256: Validatex-bf-vkheader withVirtualKeyPrefixlike other headers.
parseVirtualKeycurrently returns thex-bf-vkheader value verbatim:vkHeader := ctx.Request.Header.Peek("x-bf-vk") if string(vkHeader) != "" { return bifrost.Ptr(string(vkHeader)) }but the
Authorizationandx-api-keypaths both enforceVirtualKeyPrefixon the key. This inconsistency allows arbitraryx-bf-vkvalues to be treated as virtual keys, bypassing the prefix guard used elsewhere.You should normalize and enforce the prefix for
x-bf-vkas well, e.g.:Suggested fix
func parseVirtualKey(ctx *fasthttp.RequestCtx) *string { var virtualKeyValue string vkHeader := ctx.Request.Header.Peek("x-bf-vk") - if string(vkHeader) != "" { - return bifrost.Ptr(string(vkHeader)) - } + if raw := strings.TrimSpace(string(vkHeader)); raw != "" { + if strings.HasPrefix(strings.ToLower(raw), VirtualKeyPrefix) { + return bifrost.Ptr(raw) + } + } @@ - xAPIKey := string(ctx.Request.Header.Peek("x-api-key")) - if xAPIKey != "" && strings.HasPrefix(strings.ToLower(xAPIKey), VirtualKeyPrefix) { - return bifrost.Ptr(xAPIKey) + xAPIKey := strings.TrimSpace(string(ctx.Request.Header.Peek("x-api-key"))) + if xAPIKey != "" && strings.HasPrefix(strings.ToLower(xAPIKey), VirtualKeyPrefix) { + return bifrost.Ptr(xAPIKey) } return nil }
258-307: Fix log messages around JSON (un)marshaling in HTTP middleware.Within
HTTPTransportMiddleware, both error logs say “failed to marshal request body to check for virtual key”:err = sonic.Unmarshal(ctx.Request.Body(), &payload) if err != nil { p.logger.Error("failed to marshal request body to check for virtual key: %v", err) ... } ... body, err := sonic.Marshal(payload) if err != nil { p.logger.Error("failed to marshal request body to check for virtual key: %v", err) ... }The first is an unmarshal failure, and by the second point you’re no longer “checking for virtual key” but rebuilding the body after governance‑driven changes.
Consider tightening these messages for accuracy:
Suggested wording
- err = sonic.Unmarshal(ctx.Request.Body(), &payload) - if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + err = sonic.Unmarshal(ctx.Request.Body(), &payload) + if err != nil { + p.logger.Error("failed to unmarshal request body for governance HTTP middleware: %v", err) @@ - body, err := sonic.Marshal(payload) - if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + body, err := sonic.Marshal(payload) + if err != nil { + p.logger.Error("failed to marshal updated request body after governance routing: %v", err)core/schemas/trace.go (1)
40-49: GuardTrace.Reset,Span.End, andSpan.Resetwith the same mutex used for other mutations.
Trace.AddSpan/GetSpanandSpan.SetAttribute/AddEventusemu, but:
Trace.ResetmutatesSpansand other fields without locking.Span.EndwritesEndTime,Status, andStatusMsgwithout locking.Span.Resetclears all fields without locking.If these are ever called while other goroutines still hold references (e.g., via a pool or concurrent readers), you get data races despite the mutex presence.
Align them with the rest of the API by locking around the mutations:
Proposed adjustment
func (t *Trace) Reset() { - t.TraceID = "" - t.ParentID = "" - t.RootSpan = nil - t.Spans = t.Spans[:0] - t.StartTime = time.Time{} - t.EndTime = time.Time{} - t.Attributes = nil + t.mu.Lock() + defer t.mu.Unlock() + t.TraceID = "" + t.ParentID = "" + t.RootSpan = nil + t.Spans = t.Spans[:0] + t.StartTime = time.Time{} + t.EndTime = time.Time{} + t.Attributes = nil } func (s *Span) End(status SpanStatus, statusMsg string) { - s.EndTime = time.Now() - s.Status = status - s.StatusMsg = statusMsg + s.mu.Lock() + defer s.mu.Unlock() + s.EndTime = time.Now() + s.Status = status + s.StatusMsg = statusMsg } func (s *Span) Reset() { - s.SpanID = "" - s.ParentID = "" - s.TraceID = "" - s.Name = "" - s.Kind = SpanKindUnspecified - s.StartTime = time.Time{} - s.EndTime = time.Time{} - s.Status = SpanStatusUnset - s.StatusMsg = "" - s.Attributes = nil - s.Events = s.Events[:0] + s.mu.Lock() + defer s.mu.Unlock() + s.SpanID = "" + s.ParentID = "" + s.TraceID = "" + s.Name = "" + s.Kind = SpanKindUnspecified + s.StartTime = time.Time{} + s.EndTime = time.Time{} + s.Status = SpanStatusUnset + s.StatusMsg = "" + s.Attributes = nil + s.Events = s.Events[:0] }Also applies to: 84-104
framework/tracing/tracer.go (1)
393-455: Make choice and tool‑call ordering deterministic inbuildCompleteResponseFromChunks.
choiceMapand eachacc.toolCallsare maps; you buildchoiceIndicesandtcIndicesslices but never sort them, so:
- Choice order in
result.ChatResponse.Choicesis nondeterministic.- Tool‑call order per message is also nondeterministic.
The comments already say “Sort … for deterministic output”, so this looks like an omission.
Add sorting before iteration:
Proposed fix
// Build final choices from accumulated data // Sort choice indices for deterministic output choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } + sort.Ints(choiceIndices) @@ if len(accum.toolCalls) > 0 { // Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } + sort.Ints(tcIndices)Also applies to: 433-441
framework/tracing/llmspan.go (1)
190-193: NamespaceExtraParamsattributes to avoid collisions with standard keys.Across multiple helpers,
ExtraParamsare written directly intoattrs:
- Chat/Text/Embedding/Responses requests
- Batch and file request types
This allows user‑supplied keys to overwrite standard ones (e.g.,
gen_ai.request.model), which can corrupt telemetry.Consider prefixing these keys consistently, e.g.:
const extraPrefix = "gen_ai.extra." for k, v := range req.Params.ExtraParams { attrs[extraPrefix+k] = fmt.Sprintf("%v", v) }and apply the same pattern to all
ExtraParamsloops.Also applies to: 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 769-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029
🧹 Nitpick comments (4)
transports/bifrost-http/lib/middleware.go (1)
19-21: Consider guarding against nil middlewares.If a plugin returns
nilfromHTTPTransportMiddleware(), callingmiddlewares[i](chained)will panic. Consider filtering nil entries or documenting that callers must exclude nil middlewares before chaining.🔎 Proposed fix to guard against nil middlewares
func ChainMiddlewares(handler fasthttp.RequestHandler, middlewares ...schemas.BifrostHTTPMiddleware) fasthttp.RequestHandler { // If no middlewares, return the original handler if len(middlewares) == 0 { return handler } // Build the chain from right to left (last middleware wraps the handler) // This ensures execution order is left to right (first middleware executes first) chained := handler for i := len(middlewares) - 1; i >= 0; i-- { + if middlewares[i] == nil { + continue + } chained = middlewares[i](chained) } return chained }plugins/maxim/main.go (1)
124-127: No‑op HTTP middleware is fine; ensure callers handle nil safely.For Maxim, returning
nilfromHTTPTransportMiddlewareis a reasonable way to opt out of HTTP‑layer interception, given all useful work happens in PreHook/PostHook.Just ensure the middleware wiring (
lib.ChainMiddlewares/ router registration) checks for nil before invoking middleware functions so a nil return doesn’t result in a panic when chaining.plugins/governance/main.go (1)
42-49:BaseGovernancePluginstill exposesTransportInterceptor; consider updating or removing.
BaseGovernancePlugincontinues to declare:type BaseGovernancePlugin interface { GetName() string TransportInterceptor(...) PreHook(...) PostHook(...) Cleanup() error GetGovernanceStore() GovernanceStore }while the concrete
GovernancePluginnow implementsHTTPTransportMiddleware()instead and no longer providesTransportInterceptor. If anything in the codebase still relies onBaseGovernancePlugin, this mismatch will either cause compile‑time errors when used withGovernancePluginor mislead future contributors about the supported transport hook.If
BaseGovernancePluginis no longer used, it’s better to remove it; otherwise, update it to exposeHTTPTransportMiddleware()instead ofTransportInterceptorto stay in sync with the new plugin API.Also applies to: 258-307
framework/tracing/helpers.go (1)
58-69: Consider nil store check for consistency.While other functions implicitly handle nil store via
GetTracereturning nil,SetSpanAttributeandAddSpanEventwould benefit from an explicit nil store check for clarity.🔎 Optional improvement
func SetSpanAttribute(ctx context.Context, store *TraceStore, spanID, key string, value any) { + if store == nil { + return + } trace := GetTrace(ctx, store)
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sum
📒 Files selected for processing (89)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/versionui/app/workspace/logs/views/logDetailsSheet.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsx
💤 Files with no reviewable changes (3)
- core/mcp.go
- framework/configstore/tables/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (2)
- plugins/semanticcache/version
- plugins/maxim/version
🚧 Files skipped from review as they are similar to previous changes (27)
- .github/workflows/scripts/release-bifrost-http.sh
- transports/bifrost-http/handlers/ui.go
- framework/streaming/types.go
- transports/bifrost-http/handlers/config.go
- .github/workflows/scripts/push-mintlify-changelog.sh
- transports/bifrost-http/handlers/cache.go
- core/bifrost_test.go
- core/schemas/context.go
- framework/version
- framework/streaming/audio.go
- transports/bifrost-http/handlers/websocket.go
- plugins/governance/go.mod
- transports/bifrost-http/handlers/mcpserver.go
- transports/bifrost-http/handlers/governance.go
- plugins/governance/version
- plugins/logging/utils.go
- plugins/semanticcache/main.go
- plugins/jsonparser/version
- plugins/logging/version
- transports/version
- framework/plugins/dynamicplugin_test.go
- docs/plugins/getting-started.mdx
- plugins/logging/changelog.md
- transports/bifrost-http/handlers/middlewares_test.go
- ui/components/sidebar.tsx
- plugins/otel/version
- core/version
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/maxim/changelog.mdplugins/mocker/changelog.mdplugins/maxim/main.godocs/plugins/writing-plugin.mdxframework/streaming/transcription.goframework/streaming/responses.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/mcp.goplugins/mocker/main.gotransports/bifrost-http/lib/middleware.goplugins/semanticcache/changelog.mdcore/schemas/plugin.goui/app/workspace/logs/views/logDetailsSheet.tsxtransports/bifrost-http/handlers/session.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/providers.goexamples/plugins/hello-world/go.modtransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/middlewares.goplugins/governance/changelog.mdtransports/bifrost-http/handlers/inference.goplugins/logging/main.gocore/changelog.mdframework/tracing/propagation.goplugins/telemetry/main.goui/components/ui/switch.tsxplugins/jsonparser/main.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/logging.goplugins/telemetry/changelog.mdplugins/telemetry/versionplugins/otel/main.goframework/streaming/chat.gocore/schemas/trace.goplugins/jsonparser/changelog.mdexamples/plugins/hello-world/main.goframework/changelog.mdplugins/mocker/versiondocs/plugins/migration-guide.mdxcore/bifrost.gocore/schemas/tracer.goframework/streaming/accumulator.goplugins/otel/changelog.mdtransports/bifrost-http/handlers/health.goframework/tracing/helpers.goplugins/otel/converter.gocore/schemas/bifrost.goframework/tracing/store.goframework/tracing/llmspan.goframework/tracing/tracer.gocore/providers/utils/utils.godocs/docs.jsonui/components/ui/sheet.tsxtransports/bifrost-http/server/server.gotransports/changelog.mdplugins/governance/main.go
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
plugins/maxim/main.goframework/streaming/transcription.goframework/streaming/responses.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/mcp.goplugins/mocker/main.gotransports/bifrost-http/lib/middleware.gocore/schemas/plugin.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/inference.goplugins/logging/main.goframework/tracing/propagation.goplugins/telemetry/main.goplugins/jsonparser/main.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/logging.goplugins/otel/main.goframework/streaming/chat.gocore/schemas/trace.goexamples/plugins/hello-world/main.gocore/bifrost.gocore/schemas/tracer.goframework/streaming/accumulator.gotransports/bifrost-http/handlers/health.goframework/tracing/helpers.goplugins/otel/converter.gocore/schemas/bifrost.goframework/tracing/store.goframework/tracing/llmspan.goframework/tracing/tracer.gocore/providers/utils/utils.gotransports/bifrost-http/server/server.goplugins/governance/main.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/server/server.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
🧬 Code graph analysis (30)
plugins/maxim/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
framework/streaming/transcription.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
framework/streaming/responses.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/mcp.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/plugin.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/trace.go (1)
Trace(10-19)
ui/app/workspace/logs/views/logDetailsSheet.tsx (4)
ui/components/ui/sheet.tsx (1)
SheetContent(137-137)ui/components/ui/alertDialog.tsx (9)
AlertDialog(83-83)AlertDialogTrigger(93-93)AlertDialogContent(86-86)AlertDialogHeader(89-89)AlertDialogTitle(92-92)AlertDialogDescription(87-87)AlertDialogFooter(88-88)AlertDialogCancel(85-85)AlertDialogAction(84-84)ui/components/ui/dropdownMenu.tsx (5)
DropdownMenu(193-193)DropdownMenuTrigger(207-207)DropdownMenuContent(195-195)DropdownMenuItem(197-197)DropdownMenuSeparator(202-202)ui/components/ui/button.tsx (1)
Button(70-70)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/integrations/router.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (6)
ResponsesRequest(97-97)EmbeddingRequest(99-99)SpeechRequest(100-100)TranscriptionRequest(102-102)BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)transports/bifrost-http/handlers/inference.go (5)
ChatRequest(172-176)ResponsesRequest(266-270)EmbeddingRequest(273-277)SpeechRequest(279-283)TranscriptionRequest(285-289)
transports/bifrost-http/handlers/providers.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/plugins.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/middlewares.go (5)
core/schemas/plugin.go (3)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)Plugin(71-97)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/propagation.go (1)
ExtractParentID(74-84)core/schemas/bifrost.go (4)
BifrostContextKeyTraceID(143-143)BifrostContextKeyTraceCompleter(148-148)BifrostContextKeySpanID(144-144)BifrostContextKeyDeferTraceCompletion(147-147)
plugins/logging/main.go (3)
core/utils.go (1)
IsStreamRequestType(197-199)core/schemas/bifrost.go (4)
RequestType(89-89)BifrostContextKeyTracer(146-146)BifrostContextKeyTraceID(143-143)TranscriptionRequest(102-102)core/schemas/tracer.go (1)
Tracer(38-116)
plugins/telemetry/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (2)
GetName(15-17)HTTPTransportMiddleware(19-27)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/main.go (4)
core/schemas/plugin.go (1)
ObservabilityPlugin(123-137)core/schemas/context.go (1)
BifrostContext(32-42)core/schemas/bifrost.go (3)
BifrostRequest(173-193)BifrostResponse(335-355)BifrostError(474-483)core/schemas/trace.go (1)
Trace(10-19)
framework/streaming/chat.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
core/schemas/trace.go (1)
framework/tracing/helpers.go (1)
AddSpan(32-38)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
transports/bifrost-http/handlers/health.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(25-37)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (14)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindHTTPRequest(131-131)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanEvent(107-111)
core/schemas/bifrost.go (2)
core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)
framework/tracing/store.go (2)
core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
framework/tracing/llmspan.go (8)
core/schemas/bifrost.go (18)
BifrostRequest(173-193)RequestType(89-89)TextCompletionRequest(93-93)EmbeddingRequest(99-99)TranscriptionRequest(102-102)SpeechRequest(100-100)ResponsesRequest(97-97)BatchCreateRequest(104-104)BatchListRequest(105-105)BatchRetrieveRequest(106-106)BatchCancelRequest(107-107)BatchResultsRequest(108-108)FileUploadRequest(109-109)FileListRequest(110-110)FileRetrieveRequest(111-111)FileDeleteRequest(112-112)FileContentRequest(113-113)BifrostResponse(335-355)core/schemas/trace.go (34)
AttrProviderName(159-159)AttrRequestModel(160-160)AttrError(216-216)AttrErrorType(217-217)AttrErrorCode(218-218)AttrVirtualKeyID(228-228)AttrVirtualKeyName(229-229)AttrSelectedKeyID(230-230)AttrSelectedKeyName(231-231)AttrTeamID(232-232)AttrTeamName(233-233)AttrCustomerID(234-234)AttrCustomerName(235-235)AttrNumberOfRetries(236-236)AttrFallbackIndex(237-237)AttrMaxTokens(163-163)AttrTemperature(164-164)AttrTopP(165-165)AttrStopSequences(166-166)AttrPresencePenalty(167-167)AttrFrequencyPenalty(168-168)AttrParallelToolCall(169-169)AttrRequestUser(170-170)AttrInputMessages(222-222)AttrResponseID(191-191)AttrResponseModel(192-192)AttrObject(197-197)AttrSystemFprint(194-194)AttrCreated(196-196)AttrServiceTier(195-195)AttrOutputMessages(225-225)AttrFinishReason(193-193)AttrN(175-175)AttrUsageCost(213-213)core/schemas/chatcompletions.go (1)
BifrostChatRequest(12-19)core/schemas/textcompletions.go (2)
BifrostTextCompletionRequest(10-17)BifrostTextCompletionResponse(69-77)core/schemas/embedding.go (2)
BifrostEmbeddingRequest(9-16)BifrostEmbeddingResponse(22-28)core/schemas/responses.go (2)
BifrostResponsesRequest(32-39)BifrostResponsesResponse(45-85)core/schemas/batch.go (5)
BifrostBatchCreateRequest(65-83)BifrostBatchListRequest(118-133)BifrostBatchRetrieveRequest(150-159)BifrostBatchCancelRequest(205-214)BifrostBatchResultsRequest(234-246)core/schemas/files.go (5)
BifrostFileUploadRequest(53-67)BifrostFileListRequest(109-128)BifrostFileRetrieveRequest(146-159)BifrostFileDeleteRequest(186-198)BifrostFileContentRequest(215-227)
🪛 markdownlint-cli2 (0.18.1)
transports/changelog.md
16-16: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
4d9d0fc to
810ac1d
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
transports/bifrost-http/lib/config.go (1)
400-409: DereferencePricingSyncIntervalsafely and apply defaults.In
initFrameworkConfigFromFile, the else-if branch dereferencesconfigData.FrameworkConfig.Pricing.PricingSyncIntervalwithout first checking if it is nil:syncDuration := time.Duration(*configData.FrameworkConfig.Pricing.PricingSyncInterval) * time.Second pricingConfig.PricingSyncInterval = &syncDurationIf
pricing_sync_intervalis omitted in the JSON config, this will panic. Additionally, the code should fall back to sensible defaults when no pricing configuration is available, similar toinitDefaultFrameworkConfig:if frameworkConfig != nil && frameworkConfig.PricingSyncInterval != nil && *frameworkConfig.PricingSyncInterval > 0 { syncDuration := time.Duration(*frameworkConfig.PricingSyncInterval) * time.Second pricingConfig.PricingSyncInterval = &syncDuration } else { pricingConfig.PricingSyncInterval = bifrost.Ptr(modelcatalog.DefaultPricingSyncInterval) }Add nil checks before dereferencing and apply the same default fallback pattern used elsewhere.
core/bifrost.go (1)
3034-3072: ProcessAndSendError missing pipeline release for streaming error paths.The streaming pipeline lifecycle has a resource leak:
When a scanner error occurs during streaming (e.g., network issues),
ProcessAndSendError()is called but it does not invokecompleteDeferredSpan(), which is responsible for calling the stored finalizer and releasing the pipeline.Leak scenario:
- Pipeline created and finalizer stored in context (lines 3037–3052)
- Scanner error occurs during streaming →
ProcessAndSendError()calledProcessAndSendError()has nocompleteDeferredSpan()call → finalizer never invoked- Back in bifrost.go (line 3070):
bifrostErrorisnil(error was sent through channel, not returned), so the release condition fails- Pipeline is never released
Solution: Add the same
completeDeferredSpan()check toProcessAndSendError()that exists inProcessAndSendBifrostError()(lines 941–946 in core/providers/utils/utils.go).
♻️ Duplicate comments (15)
ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
183-187: Addaria-labelfor accessibility.The icon-only button should include an
aria-labelto provide context for screen reader users. This was flagged in a previous review.🔎 Suggested fix
<DropdownMenuTrigger asChild> - <Button variant="ghost" size="icon"> + <Button variant="ghost" size="icon" aria-label="Log actions"> <MoreVertical className="h-3 w-3" /> </Button> </DropdownMenuTrigger>framework/tracing/llmspan.go (2)
190-193: ExtraParams keys can collide with standard attribute names.User-provided keys from
ExtraParamsare written directly to the attributes map. If a user sets a key like"gen_ai.request.model", it would overwrite the standard attribute. This pattern repeats throughout the file (lines 297-300, 364-367, 574-577, 707-710, 737-740, 752-755, 767-770, 782-785, 957-960, 981-984, 996-999, 1011-1014, 1026-1029).
376-382: Use a float-appropriate formatter for embedding values.The code uses
%d(integer format) for embedding values, but embeddings are typically float slices. This will truncate values or produce incorrect output.core/providers/utils/utils.go (3)
880-925: Defer span completion until after post-hooks for accurate tracing.The current implementation calls
completeDeferredSpan(line 897) beforepostHookRunner(line 902), which means:
- LLM span attributes reflect the pre-post-hook response, not what the client actually received
- Post-hook span aggregation completes before the final PostHook runs for the last chunk
For accurate end-to-end tracing, restructure to:
- Run
postHookRunnerfirst to getprocessedResponseandprocessedError- Send the processed result to
responseChan- Then call
completeDeferredSpanwith the processed values if this is the final chunkThis ensures spans capture the fully processed streaming response.
🔎 Suggested restructuring
func ProcessAndSendResponse(...) { // Accumulate chunk for tracing (common for all providers) if tracer, ok := ctx.Value(schemas.BifrostContextKeyTracer).(schemas.Tracer); ok && tracer != nil { if traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string); ok && traceID != "" { tracer.AddStreamingChunk(traceID, response) } } - // Check if this is the final chunk and complete deferred span if so - if isFinalChunk := ctx.Value(schemas.BifrostContextKeyStreamEndIndicator); isFinalChunk != nil { - if final, ok := isFinalChunk.(bool); ok && final { - completeDeferredSpan(&ctx, response, nil) - } - } - // Run post hooks on the response processedResponse, processedError := postHookRunner(&ctx, response, nil) if HandleStreamControlSkip(processedError) { return } + + // Complete deferred span AFTER post-hooks with processed values + if isFinalChunk := ctx.Value(schemas.BifrostContextKeyStreamEndIndicator); isFinalChunk != nil { + if final, ok := isFinalChunk.(bool); ok && final { + completeDeferredSpan(&ctx, processedResponse, processedError) + } + } streamResponse := &schemas.BifrostStream{} // ... rest of function }
932-969: Apply the same fix: defer span completion until after post-hooks.
ProcessAndSendBifrostErrorhas the same ordering issue asProcessAndSendResponse:completeDeferredSpan(line 942) is called beforepostHookRunner(line 947).Apply the same restructuring here to ensure error spans reflect the post-processed error state.
1416-1501: Pass context by value, not by pointer (Go convention).The function signature
completeDeferredSpan(ctx *context.Context, ...)violates Go conventions. Contexts should always be passed by value, never by pointer:
context.Contextis designed for safe concurrent use when passed by value- Passing by pointer is an anti-pattern in Go
Update the signature and all call sites:
🔎 Proposed fix
-func completeDeferredSpan(ctx *context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { - if ctx == nil { +func completeDeferredSpan(ctx context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { + if ctx == nil || ctx == context.Background() || ctx == context.TODO() { return } - traceID, ok := (*ctx).Value(schemas.BifrostContextKeyTraceID).(string) + traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string) if !ok || traceID == "" { return } - tracerVal := (*ctx).Value(schemas.BifrostContextKeyTracer) + tracerVal := ctx.Value(schemas.BifrostContextKeyTracer) // ... rest of function with ctx instead of (*ctx)Update all call sites:
-completeDeferredSpan(&ctx, response, nil) +completeDeferredSpan(ctx, response, nil)examples/plugins/hello-world/main.go (1)
19-44: Fix broken context value propagation in example.The middleware sets a value on
fasthttp.RequestCtx(line 23) butPreHook(line 30) andPostHook(line 39) try to read it from*schemas.BifrostContext. These are different context objects, so the value will always benil.To demonstrate proper context value flow between middleware and hooks:
Option 1: Remove the context reads from PreHook/PostHook if propagation demo is no longer needed:
🔎 Option 1: Remove broken context reads
func PreHook(ctx *schemas.BifrostContext, req *schemas.BifrostRequest) (*schemas.BifrostRequest, *schemas.PluginShortCircuit, error) { - value1 := ctx.Value(schemas.BifrostContextKey("hello-world-plugin-transport-interceptor")) - fmt.Println("value1:", value1) ctx.SetValue(schemas.BifrostContextKey("hello-world-plugin-pre-hook"), "pre-hook-value") fmt.Println("PreHook called") return req, nil, nil } func PostHook(ctx *schemas.BifrostContext, resp *schemas.BifrostResponse, bifrostErr *schemas.BifrostError) (*schemas.BifrostResponse, *schemas.BifrostError, error) { fmt.Println("PostHook called") - value1 := ctx.Value(schemas.BifrostContextKey("hello-world-plugin-transport-interceptor")) - fmt.Println("value1:", value1) value2 := ctx.Value(schemas.BifrostContextKey("hello-world-plugin-pre-hook")) fmt.Println("value2:", value2) return resp, bifrostErr, nil }Option 2: Document that middleware and core hooks use different contexts (if this is intentional to show the separation of concerns).
transports/changelog.md (1)
11-31: Add language specifier to code block and verify version references.Two minor issues to address:
Markdown linting: The code block at line 16 needs a language specifier (e.g.,
goortext) to fix the MD040 violation.Version consistency: The migration summary shows "v1.3.x (removed)" and "v1.4.x+ (new)", but the core changelog (core/changelog.md) indicates the new API is available in "core v1.3.0+". Please verify and align version references across both changelogs to avoid developer confusion.
🔎 Suggested fix for markdown
**Migration summary:** - ``` + ```go // v1.3.x (removed) TransportInterceptor(ctx *BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error)transports/bifrost-http/integrations/router.go (1)
502-693: Non-streaming handlers using wrong context (duplicate critical issue).Lines 514, 534, 557, 580, 603, 626, and 660 pass the raw
*fasthttp.RequestCtx(ctx) to Bifrost client methods instead of the converted/cancellable Bifrost context (*bifrostCtx). This breaks:
- Tracing and plugin context propagation (all
BifrostContextKey*values live onbifrostCtx)- Potential context cancellation
- May cause type mismatches if Bifrost methods expect
context.ContextThe comment on line 503 explicitly says "Use the cancellable context from ConvertToBifrostContext", but the code doesn't follow it.
🔎 Proposed fix for all non-streaming calls
case bifrostReq.ListModelsRequest != nil: - listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) + listModelsResponse, bifrostErr := g.client.ListModelsRequest(*bifrostCtx, bifrostReq.ListModelsRequest) case bifrostReq.TextCompletionRequest != nil: - textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, bifrostReq.TextCompletionRequest) + textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(*bifrostCtx, bifrostReq.TextCompletionRequest) case bifrostReq.ChatRequest != nil: - chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, bifrostReq.ChatRequest) + chatResponse, bifrostErr := g.client.ChatCompletionRequest(*bifrostCtx, bifrostReq.ChatRequest) case bifrostReq.ResponsesRequest != nil: - responsesResponse, bifrostErr := g.client.ResponsesRequest(ctx, bifrostReq.ResponsesRequest) + responsesResponse, bifrostErr := g.client.ResponsesRequest(*bifrostCtx, bifrostReq.ResponsesRequest) case bifrostReq.EmbeddingRequest != nil: - embeddingResponse, bifrostErr := g.client.EmbeddingRequest(ctx, bifrostReq.EmbeddingRequest) + embeddingResponse, bifrostErr := g.client.EmbeddingRequest(*bifrostCtx, bifrostReq.EmbeddingRequest) case bifrostReq.SpeechRequest != nil: - speechResponse, bifrostErr := g.client.SpeechRequest(ctx, bifrostReq.SpeechRequest) + speechResponse, bifrostErr := g.client.SpeechRequest(*bifrostCtx, bifrostReq.SpeechRequest) case bifrostReq.TranscriptionRequest != nil: - transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(ctx, bifrostReq.TranscriptionRequest) + transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(*bifrostCtx, bifrostReq.TranscriptionRequest)plugins/governance/main.go (2)
233-256: Missing prefix validation for x-bf-vk header (duplicate issue).The
x-bf-vkheader on lines 236-238 is returned without validating that it starts withVirtualKeyPrefix, while theAuthorization(Bearer) andx-api-keyheaders on lines 243 and 252 correctly validate the prefix. This inconsistency could allow invalid virtual keys to pass through when provided via thex-bf-vkheader.🔎 Proposed fix
vkHeader := ctx.Request.Header.Peek("x-bf-vk") if string(vkHeader) != "" { + vkValue := string(vkHeader) + if strings.HasPrefix(strings.ToLower(vkValue), VirtualKeyPrefix) { - return bifrost.Ptr(string(vkHeader)) + return bifrost.Ptr(vkValue) + } }
287-292: Fix error message accuracy (duplicate issue).Line 289: The error message says "failed to marshal" but the operation is unmarshaling the request body.
🔎 Proposed fix
- p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body: %v", err)framework/plugins/dynamicplugin.go (1)
140-147: Type mismatch: dynamic plugin lookup will fail (duplicate critical issue).The example plugin at
examples/plugins/hello-world/main.go(lines 18-26) exportsHTTPTransportMiddlewareas a factory function:func HTTPTransportMiddleware() schemas.BifrostHTTPMiddleware { return func(next fasthttp.RequestHandler) fasthttp.RequestHandler { ... } }However, line 145 attempts to cast the symbol directly to
schemas.BifrostHTTPMiddleware, which will always fail because the symbol is a function that returns the middleware, not the middleware itself.🔎 Proposed fix
// Looking up for HTTPTransportMiddleware method httpTransportMiddlewareSym, err := plugin.Lookup("HTTPTransportMiddleware") if err != nil { return nil, err } - if dp.httpTransportMiddleware, ok = httpTransportMiddlewareSym.(schemas.BifrostHTTPMiddleware); !ok { - return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func(next fasthttp.RequestHandler) fasthttp.RequestHandler") + httpTransportMiddlewareFunc, ok := httpTransportMiddlewareSym.(func() schemas.BifrostHTTPMiddleware) + if !ok { + return nil, fmt.Errorf("failed to cast HTTPTransportMiddleware to func() schemas.BifrostHTTPMiddleware") } + dp.httpTransportMiddleware = httpTransportMiddlewareFunc() + if dp.httpTransportMiddleware == nil { + return nil, fmt.Errorf("HTTPTransportMiddleware factory returned nil") + }transports/bifrost-http/server/server.go (1)
950-1018: UseGetGovernancePluginName()when deciding whether to register governance routes.
RegisterAPIRoutescurrently does:governancePlugin, _ := FindPluginByName[schemas.Plugin](s.Plugins, governance.PluginName)This bypasses
OSSToEnterprisePluginNameOverridesand theGetGovernancePluginName()helper you already use ingetGovernancePlugin(). In an enterprise deployment where the governance plugin is renamed, the plugin will be loaded, but the API routes won’t be registered because this lookup uses the hard‑coded OSS name.Switch the lookup to:
governancePlugin, _ := FindPluginByName[schemas.Plugin](s.Plugins, s.GetGovernancePluginName())so routing honors the same override mechanism as the rest of the server.
transports/bifrost-http/handlers/middlewares.go (1)
191-279:TracingMiddlewaremust pass acontext.ContexttoStartSpan, not*fasthttp.RequestCtx.Here you call:
spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)but
ctxis*fasthttp.RequestCtx. The tracer’sStartSpanAPI is defined overcontext.Context, and prior work in this repo already introducedlib.ConvertToBifrostContextfor this exact adaptation.You should:
- Convert the fasthttp context to a
context.Context(e.g.bctx := lib.ConvertToBifrostContext(ctx)), and- Pass that into
StartSpan, then use the returnedspanCtxfor any subsequent context‑based operations.This is currently a type/semantic mismatch and will either fail to compile or break trace context propagation.
core/bifrost.go (1)
3893-3898: Critical: Fix tracer type assertion in Shutdown (regression or incomplete fix).The type assertion at line 3895 is incorrect and will always fail.
bifrost.tracer.Load()returns a*tracerWrapper, not aschemas.Tracer, so the assertion will never succeed andt.Stop()is never called. This causes background goroutines in the tracer to leak.🔎 Proposed fix
// Stop the tracer to clean up background goroutines - if tracer := bifrost.tracer.Load(); tracer != nil { - if t, ok := tracer.(schemas.Tracer); ok { - t.Stop() - } - } + if v := bifrost.tracer.Load(); v != nil { + if tw, ok := v.(*tracerWrapper); ok && tw != nil && tw.tracer != nil { + tw.tracer.Stop() + } + }Note: A past review comment indicated this was addressed in commit 4d9d0fc, but the incorrect type assertion remains in the current code. Please verify whether this is a regression or if the fix was not fully applied.
🧹 Nitpick comments (7)
framework/streaming/chat.go (1)
164-178: LGTM! AccumulatedData initialization is complete.All fields are explicitly initialized including the new
TimeToFirstTokenfield. The initialization pattern is consistent with the audio streaming implementation, which promotes maintainability across the codebase.Optional observation: The TTFT calculation logic (lines 157-161) is duplicated across multiple streaming files (
audio.go,chat.go, and likelyresponses.goandtranscription.gobased on the PR context). Consider extracting this into a shared helper function to reduce duplication, though the current implementation is clear and correct.🔎 Optional refactor to reduce duplication
You could add a helper method to the
AccumulatororStreamAccumulator:// calculateTTFT calculates Time To First Token in milliseconds func (a *StreamAccumulator) calculateTTFT() int64 { if a.StartTimestamp.IsZero() || a.FirstChunkTimestamp.IsZero() { return 0 } return a.FirstChunkTimestamp.Sub(a.StartTimestamp).Nanoseconds() / 1e6 }Then use it in all streaming files:
- // Calculate Time to First Token (TTFT) in milliseconds - var ttft int64 - if !accumulator.StartTimestamp.IsZero() && !accumulator.FirstChunkTimestamp.IsZero() { - ttft = accumulator.FirstChunkTimestamp.Sub(accumulator.StartTimestamp).Nanoseconds() / 1e6 - } + ttft := accumulator.calculateTTFT()transports/bifrost-http/lib/config_test.go (1)
6675-12270: LoadConfig callsites correctly updated to new 2‑arg signatureAll the updated invocations of
LoadConfigin the SQLite integration tests now matchfunc LoadConfig(ctx context.Context, configDirPath string) (*Config, error), and the surrounding usage ofctx := context.Background()andtempDiris unchanged, so behavior should be identical aside from dropping the unused third parameter. If you find yourself touching this area again, you could optionally DRY up the repeatedctx/tempDir → LoadConfigpattern with a small helper, but it isn’t required for this change.plugins/otel/converter.go (1)
165-170: Potential integer overflow when convertinguint64toint64.Converting
uint64toint64can overflow for values greater thanmath.MaxInt64, resulting in negative values being reported in traces.🔎 Suggested fix: clamp large uint64 values
case uint64: - return kvInt(key, int64(v)) + if v > math.MaxInt64 { + return kvInt(key, math.MaxInt64) + } + return kvInt(key, int64(v))You'll need to import
mathif not already imported.transports/bifrost-http/handlers/middlewares.go (1)
49-68: Plugin HTTP transport middleware chaining is correct; consider caching the chain.
TransportInterceptorMiddlewarenow:
- Pulls the current plugin snapshot via
config.GetLoadedPlugins().- Collects non‑nil
plugin.HTTPTransportMiddleware()functions.- Chains them via
lib.ChainMiddlewares.This matches the new HTTP middleware model and preserves ordering. If this ever shows up in profiles, you could later cache the assembled plugin chain and rebuild it only when the loaded plugin set changes, instead of per‑request.
framework/tracing/store.go (1)
13-371: TraceStore design (TTL cleanup, pooling, deferred spans) looks solid; only minor polish possible.The
TraceStoreimplementation correctly:
- Uses
sync.Map+ pools for traces/spans.- Manages TTL‑based cleanup in a background goroutine with a guarded
Stop()(stopOnce).- Provides simple APIs for deferred spans and streaming accumulation with proper locking.
- Returns traces/spans to pools via
ReleaseTrace/releaseSpan.The only minor nit is that
DeferredSpanInfo.TracerandRequestIDare currently unused; if they’re not needed by the tracer, consider removing them or wiring them up in future work.core/bifrost.go (2)
2860-2871: Consider chaining context.WithValue calls for readability.The current pattern of sequential context pointer updates works correctly but could be more idiomatic:
🔎 Suggested refactor
- // Update context with span ID - *ctx = spanCtx - // Store tracer in context BEFORE calling requestHandler, so streaming goroutines // have access to it for completing deferred spans when the stream ends. // The streaming goroutine captures the context when it starts, so these values // must be set before requestHandler() is called. - *ctx = context.WithValue(*ctx, schemas.BifrostContextKeyTracer, tracer) - - // Record stream start time for TTFT calculation (only for streaming requests) - // This is also used by RunPostHooks to detect streaming mode - if IsStreamRequestType(requestType) { - streamStartTime := time.Now() - *ctx = context.WithValue(*ctx, schemas.BifrostContextKeyStreamStartTime, streamStartTime) - } + *ctx = spanCtx + *ctx = context.WithValue(*ctx, schemas.BifrostContextKeyTracer, tracer) + if IsStreamRequestType(requestType) { + *ctx = context.WithValue(*ctx, schemas.BifrostContextKeyStreamStartTime, time.Now()) + }This reduces temporary variable allocations and keeps the context updates together.
3287-3293: Consider extracting span context propagation helper.The pattern of propagating span context to pluginCtx is repeated in both
RunPreHooksandRunPostHooks(non-streaming path):if spanCtx != nil { if spanID, ok := spanCtx.Value(schemas.BifrostContextKeySpanID).(string); ok { pluginCtx.SetValue(schemas.BifrostContextKeySpanID, spanID) } }🔎 Suggested refactor
Extract to a helper method:
// propagateSpanContext copies span ID from source context to target context func propagateSpanContext(sourceCtx context.Context, targetCtx *schemas.BifrostContext) { if sourceCtx == nil { return } if spanID, ok := sourceCtx.Value(schemas.BifrostContextKeySpanID).(string); ok { targetCtx.SetValue(schemas.BifrostContextKeySpanID, spanID) } }Then use:
propagateSpanContext(spanCtx, pluginCtx)This reduces duplication and makes the intent clearer.
Also applies to: 3356-3362
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sum
📒 Files selected for processing (91)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/versionui/app/workspace/logs/views/logDetailsSheet.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsx
💤 Files with no reviewable changes (3)
- framework/configstore/tables/mcp.go
- core/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (2)
- plugins/maxim/version
- plugins/governance/version
🚧 Files skipped from review as they are similar to previous changes (38)
- framework/streaming/responses.go
- core/bifrost_test.go
- docs/docs.json
- transports/bifrost-http/handlers/providers.go
- transports/version
- plugins/semanticcache/version
- transports/bifrost-http/handlers/session.go
- transports/bifrost-http/handlers/plugins.go
- framework/version
- plugins/telemetry/main.go
- plugins/maxim/main.go
- plugins/logging/utils.go
- transports/bifrost-http/handlers/websocket.go
- transports/bifrost-http/handlers/logging.go
- transports/bifrost-http/handlers/health.go
- framework/streaming/types.go
- .github/workflows/scripts/release-bifrost-http.sh
- plugins/otel/version
- transports/bifrost-http/handlers/governance.go
- core/schemas/bifrost.go
- docs/plugins/migration-guide.mdx
- framework/tracing/propagation.go
- plugins/logging/version
- transports/bifrost-http/handlers/mcpserver.go
- framework/streaming/transcription.go
- framework/streaming/accumulator.go
- ui/components/ui/switch.tsx
- ui/components/ui/sheet.tsx
- plugins/jsonparser/version
- transports/bifrost-http/handlers/mcp.go
- plugins/mocker/version
- framework/changelog.md
- plugins/telemetry/version
- framework/tracing/tracer.go
- examples/plugins/hello-world/go.mod
- docs/plugins/getting-started.mdx
- plugins/governance/go.mod
- docs/plugins/writing-plugin.mdx
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/maxim/changelog.mdcore/schemas/plugin.goplugins/jsonparser/changelog.mdtransports/bifrost-http/handlers/inference.goplugins/semanticcache/changelog.mdplugins/otel/changelog.mdui/app/workspace/logs/views/logDetailsSheet.tsxtransports/bifrost-http/handlers/ui.goexamples/plugins/hello-world/main.goplugins/logging/main.gotransports/bifrost-http/handlers/middlewares_test.goframework/streaming/chat.goframework/tracing/store.goplugins/governance/changelog.mdtransports/bifrost-http/integrations/router.goplugins/otel/main.goui/components/sidebar.tsxframework/plugins/dynamicplugin_test.goframework/streaming/audio.gotransports/bifrost-http/handlers/cache.goplugins/governance/main.gotransports/bifrost-http/server/server.goplugins/mocker/main.gotransports/bifrost-http/lib/config_test.gocore/versioncore/changelog.mdplugins/semanticcache/main.gotransports/bifrost-http/handlers/config.goplugins/otel/converter.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/integrations.goplugins/mocker/changelog.mdcore/schemas/tracer.goframework/plugins/dynamicplugin.gocore/schemas/context.gocore/providers/utils/utils.goplugins/logging/changelog.mdcore/bifrost.gotransports/bifrost-http/handlers/middlewares.goframework/tracing/llmspan.goframework/tracing/helpers.gotransports/bifrost-http/lib/middleware.gocore/schemas/trace.gotransports/changelog.mdplugins/jsonparser/main.goplugins/telemetry/changelog.md
🧠 Learnings (4)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
core/schemas/plugin.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/ui.goexamples/plugins/hello-world/main.goplugins/logging/main.gotransports/bifrost-http/handlers/middlewares_test.goframework/streaming/chat.goframework/tracing/store.gotransports/bifrost-http/integrations/router.goplugins/otel/main.goframework/plugins/dynamicplugin_test.goframework/streaming/audio.gotransports/bifrost-http/handlers/cache.goplugins/governance/main.gotransports/bifrost-http/server/server.goplugins/mocker/main.gotransports/bifrost-http/lib/config_test.goplugins/semanticcache/main.gotransports/bifrost-http/handlers/config.goplugins/otel/converter.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/integrations.gocore/schemas/tracer.goframework/plugins/dynamicplugin.gocore/schemas/context.gocore/providers/utils/utils.gocore/bifrost.gotransports/bifrost-http/handlers/middlewares.goframework/tracing/llmspan.goframework/tracing/helpers.gotransports/bifrost-http/lib/middleware.gocore/schemas/trace.goplugins/jsonparser/main.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/server/server.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
🧬 Code graph analysis (25)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/logging/main.go (6)
core/utils.go (1)
IsStreamRequestType(197-199)core/schemas/bifrost.go (4)
RequestType(89-89)BifrostContextKeyTracer(146-146)BifrostContextKeyTraceID(143-143)TranscriptionRequest(102-102)ui/lib/types/config.ts (1)
RequestType(138-160)core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)framework/streaming/types.go (1)
ProcessedStreamResponse(118-126)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/store.go (3)
framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
transports/bifrost-http/integrations/router.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
plugins/otel/main.go (3)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)core/schemas/context.go (1)
BifrostContext(32-42)core/schemas/trace.go (1)
Trace(10-19)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
framework/streaming/audio.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/handlers/cache.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/server/server.go (5)
core/schemas/plugin.go (3)
BifrostHTTPMiddleware(38-38)Plugin(71-97)ObservabilityPlugin(123-137)plugins/governance/main.go (2)
PluginName(23-23)Config(34-36)plugins/logging/main.go (2)
PluginName(22-22)Config(92-94)plugins/telemetry/main.go (2)
PluginName(21-21)Config(64-66)transports/bifrost-http/lib/config.go (2)
Config(189-220)LoadConfig(276-356)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/lib/config_test.go (1)
transports/bifrost-http/lib/config.go (1)
LoadConfig(276-356)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (2)
GetName(15-17)HTTPTransportMiddleware(19-27)
core/schemas/context.go (1)
core/schemas/bifrost.go (1)
BifrostContextKeyDeferTraceCompletion(147-147)
core/providers/utils/utils.go (3)
core/schemas/bifrost.go (6)
BifrostResponse(335-355)BifrostContextKeyTraceID(143-143)BifrostContextKeyStreamEndIndicator(131-131)BifrostError(474-483)BifrostContextKeyPostHookSpanFinalizer(149-149)BifrostContextKeySpanID(144-144)framework/tracing/tracer.go (1)
Tracer(17-20)framework/tracing/helpers.go (1)
EndSpan(50-56)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(25-37)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/trace.go (2)
framework/tracing/helpers.go (1)
AddSpan(32-38)ui/lib/constants/logs.ts (1)
Status(161-161)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
🪛 markdownlint-cli2 (0.18.1)
transports/changelog.md
16-16: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
810ac1d to
330f7f3
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
transports/bifrost-http/lib/config.go (1)
1509-1535: Avoid nil dereference when FrameworkConfig.PricingSyncInterval is omittedIn
initFrameworkConfigFromFile, whenconfig.ConfigStore == nilandconfigData.FrameworkConfig.Pricingis set, you unconditionally dereferenceconfigData.FrameworkConfig.Pricing.PricingSyncInterval:syncDuration := time.Duration(*configData.FrameworkConfig.Pricing.PricingSyncInterval) * time.Second pricingConfig.PricingSyncInterval = &syncDurationIf the JSON includes
pricingwith onlypricing_url(nopricing_sync_interval), this will panic during startup.Guard the pointer and optionally fall back to a default:
Suggested nil-safe handling
} else if configData.FrameworkConfig != nil && configData.FrameworkConfig.Pricing != nil { - pricingConfig.PricingURL = configData.FrameworkConfig.Pricing.PricingURL - syncDuration := time.Duration(*configData.FrameworkConfig.Pricing.PricingSyncInterval) * time.Second - pricingConfig.PricingSyncInterval = &syncDuration + pricingConfig.PricingURL = configData.FrameworkConfig.Pricing.PricingURL + if configData.FrameworkConfig.Pricing.PricingSyncInterval != nil { + syncDuration := time.Duration(*configData.FrameworkConfig.Pricing.PricingSyncInterval) * time.Second + pricingConfig.PricingSyncInterval = &syncDuration + } + // Optionally: else leave nil so modelcatalog.Init can apply its own default. }
♻️ Duplicate comments (17)
ui/components/ui/sheet.tsx (2)
103-103: Past review suggestion still not applied - verify if intentional.The className logic still differs from the fix suggested in the past review (marked as addressed in commit 80eb07d). The current implementation always applies
mb-6, whereas the suggested fix would only apply it whenexpandableis false:
- Current:
cn("mb-6 flex items-center", sheetContext?.expandable ? "p-0" : "")
→mb-6is always present- Suggested:
cn("flex items-center", sheetContext?.expandable ? "p-0" : "mb-6")
→mb-6only when not expandableIf the current behavior is intentional (wanting margin-bottom even when expandable), please add a brief comment explaining the decision so future reviewers understand this was deliberate. Otherwise, apply the originally suggested fix.
🔎 Apply the originally suggested fix
- <div data-slot="sheet-header" className={cn("mb-6 flex items-center", sheetContext?.expandable ? "p-0" : "")} {...props}> + <div data-slot="sheet-header" className={cn("flex items-center", sheetContext?.expandable ? "p-0" : "mb-6")} {...props}>
114-114: Past review suggestion still not applied - makeml-3conditional.The
ml-3class is applied unconditionally, but the expand/collapse button only renders whenexpandableis true ANDsideis "right". When the button is absent, this creates unwanted left margin.🔎 Suggested fix
- <div className={cn("flex h-full min-w-0 flex-1 flex-row items-center ml-3", className)}>{children}</div> + <div className={cn("flex h-full min-w-0 flex-1 flex-row items-center", sheetContext?.expandable && sheetContext?.side === "right" ? "ml-3" : "", className)}>{children}</div>ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
183-187: Add aria-label for accessibility.The icon-only button lacks an accessible name for screen reader users. Include an
aria-labelto describe the button's purpose.🔎 Suggested fix
<DropdownMenuTrigger asChild> - <Button variant="ghost" size="icon"> + <Button variant="ghost" size="icon" aria-label="Log actions"> <MoreVertical className="h-3 w-3" /> </Button> </DropdownMenuTrigger>ui/components/ui/switch.tsx (1)
13-13: Breaking changes already comprehensively flagged in previous review.The previous review comments thoroughly identified and analyzed the breaking changes introduced by:
- Changing default
sizefrom"default"to"md"(affects all Switch instances without explicit size prop)- Changing border radius from
rounded-fulltorounded-sm(affects visual appearance of all switches)Both changes impact existing UI across the codebase and should be carefully considered. The previous feedback includes detailed impact analysis and fix suggestions.
If these changes are intentional as part of a design system update, please:
- Document the rationale in the PR description
- Update visual regression tests or screenshots
- Consider providing a migration guide or codemod for consuming code
Also applies to: 16-16, 26-26
framework/tracing/llmspan.go (2)
190-193: NamespaceExtraParamsto avoid collisions with standard tracing attributesAcross the helpers you write
ExtraParamsdirectly intoattrswith raw keys:for k, v := range req.Params.ExtraParams { attrs[k] = fmt.Sprintf("%v", v) }This lets user-supplied keys overwrite core attributes like
gen_ai.request.modelor any future standard keys.Consider namespacing and/or collision‑checking, e.g.:
Example namespacing approach
- for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) - } + const extraPrefix = "gen_ai.extra." + for k, v := range req.Params.ExtraParams { + key := extraPrefix + k + // Only set if not already present to avoid clobbering core attributes + if _, exists := attrs[key]; !exists { + attrs[key] = fmt.Sprintf("%v", v) + } + }Apply the same pattern to all the ExtraParams loops in this file.
Also applies to: 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 768-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029
377-381: Use float-safe formatting for embedding values instead of%d
PopulateEmbeddingRequestAttributesstringifies embedding elements with an integer verb:embedding[i] = fmt.Sprintf("%d", v)If
Embeddingis a float slice (typical for embeddings), this produces%!d(float64=…)strings and makes the attribute hard to consume.Prefer a float-aware conversion:
Suggested change
- } else if req.Input.Embedding != nil { - embedding := make([]string, len(req.Input.Embedding)) - for i, v := range req.Input.Embedding { - embedding[i] = fmt.Sprintf("%d", v) - } - attrs[schemas.AttrInputEmbedding] = strings.Join(embedding, ",") - } + } else if req.Input.Embedding != nil { + embedding := make([]string, len(req.Input.Embedding)) + for i, v := range req.Input.Embedding { + // Adjust precision/bitSize to match the actual element type if needed. + embedding[i] = fmt.Sprintf("%g", v) + } + attrs[schemas.AttrInputEmbedding] = strings.Join(embedding, ",") + }transports/bifrost-http/integrations/router.go (1)
502-693: Critical: Non-streaming handlers are using the wrong context for Bifrost calls.All non-streaming calls (lines 514, 534, 557, 580, 603, 626, 660) pass
ctx(*fasthttp.RequestCtx) to the Bifrost client instead of the convertedbifrostCtx:listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, bifrostReq.TextCompletionRequest) chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, bifrostReq.ChatRequest) // ... etcThe comment at line 503 says "Use the cancellable context from ConvertToBifrostContext", and you already computed
bifrostCtxearlier. Passing the HTTP request context here instead of the enriched Bifrost context:
- Breaks tracing and plugin context propagation (all
BifrostContextKey*values live onbifrostCtx)- May not compile if Bifrost methods expect
context.Contextrather than*fasthttp.RequestCtxThese calls should use the converted/cancellable context (
*bifrostCtx), just like the streaming path does.🔎 Proposed fix
func (g *GenericRouter) handleNonStreamingRequest(ctx *fasthttp.RequestCtx, config RouteConfig, req interface{}, bifrostReq *schemas.BifrostRequest, bifrostCtx *context.Context) { // Use the cancellable context from ConvertToBifrostContext var response interface{} var err error switch { case bifrostReq.ListModelsRequest != nil: - listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) + listModelsResponse, bifrostErr := g.client.ListModelsRequest(*bifrostCtx, bifrostReq.ListModelsRequest) // ... case bifrostReq.TextCompletionRequest != nil: - textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, bifrostReq.TextCompletionRequest) + textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(*bifrostCtx, bifrostReq.TextCompletionRequest) // ... case bifrostReq.ChatRequest != nil: - chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, bifrostReq.ChatRequest) + chatResponse, bifrostErr := g.client.ChatCompletionRequest(*bifrostCtx, bifrostReq.ChatRequest) // ... case bifrostReq.ResponsesRequest != nil: - responsesResponse, bifrostErr := g.client.ResponsesRequest(ctx, bifrostReq.ResponsesRequest) + responsesResponse, bifrostErr := g.client.ResponsesRequest(*bifrostCtx, bifrostReq.ResponsesRequest) // ... case bifrostReq.EmbeddingRequest != nil: - embeddingResponse, bifrostErr := g.client.EmbeddingRequest(ctx, bifrostReq.EmbeddingRequest) + embeddingResponse, bifrostErr := g.client.EmbeddingRequest(*bifrostCtx, bifrostReq.EmbeddingRequest) // ... case bifrostReq.SpeechRequest != nil: - speechResponse, bifrostErr := g.client.SpeechRequest(ctx, bifrostReq.SpeechRequest) + speechResponse, bifrostErr := g.client.SpeechRequest(*bifrostCtx, bifrostReq.SpeechRequest) // ... case bifrostReq.TranscriptionRequest != nil: - transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(ctx, bifrostReq.TranscriptionRequest) + transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(*bifrostCtx, bifrostReq.TranscriptionRequest)Please rebuild and test non-streaming requests (chat, embeddings, etc.) after fixing to verify tracing, plugin hooks, and timeouts work correctly.
transports/changelog.md (2)
16-23: Version references are inconsistent with core changelog.The migration summary states
v1.3.x (removed)andv1.4.x+ (new), but according to the core changelog (core/changelog.md), the new API is available incore v1.3.0+. This inconsistency could confuse developers about which versions to target.Please align with core changelog:
- Change "v1.4.x+ (new)" to "v1.3.0+ (new)"
- Adjust "v1.3.x (removed)" to "v1.2.x (removed)" to match the version where the old API existed
16-16: Add language specifier to fenced code block.The fenced code block is missing a language identifier, which prevents proper syntax highlighting.
🔎 Suggested fix
**Migration summary:** - ``` + ```text // v1.3.x (removed) TransportInterceptor(ctx *BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error)core/providers/utils/utils.go (1)
1428-1432: Refactor to pass context by value, not by pointer.The function signature
completeDeferredSpan(ctx *context.Context, ...)violates Go conventions. Contexts should always be passed by value, never by pointer. Thecontext.Contexttype is designed to be passed by value and is safe for concurrent use.🔎 Proposed fix
-func completeDeferredSpan(ctx *context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { - if ctx == nil { +func completeDeferredSpan(ctx context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { + if ctx == context.Background() || ctx == context.TODO() { return } - // Get the trace ID from context (this IS available in the provider's goroutine) - traceID, ok := (*ctx).Value(schemas.BifrostContextKeyTraceID).(string) + traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string) if !ok || traceID == "" { return } - // Get the tracer from context - tracerVal := (*ctx).Value(schemas.BifrostContextKeyTracer) + tracerVal := ctx.Value(schemas.BifrostContextKeyTracer) if tracerVal == nil { return }And update all call sites:
- completeDeferredSpan(&ctx, response, nil) + completeDeferredSpan(ctx, response, nil)plugins/governance/main.go (2)
233-256: Validatex-bf-vkprefix and addx-goog-api-keysupport inparseVirtualKey
parseVirtualKeystill:
- Accepts any non‑empty
x-bf-vkwithout validating theVirtualKeyPrefix, unlikeAuthorizationandx-api-key.- Ignores
x-goog-api-key, even though the rest of the stack treats it as another virtual-key carrier.This leads to inconsistent enforcement and allows malformed
x-bf-vkvalues to hitGetVirtualKey, and also means governance/MCP headers won’t be applied for requests authenticating solely viax-goog-api-key.Consider:
- Adding the same
VirtualKeyPrefixcheck forx-bf-vk(trim + lower-case, thenHasPrefix).- Adding a
x-goog-api-keybranch mirroring thex-api-keylogic, so all supported VK headers are treated consistently.Example adjustment
func parseVirtualKey(ctx *fasthttp.RequestCtx) *string { - var virtualKeyValue string - vkHeader := ctx.Request.Header.Peek("x-bf-vk") - if string(vkHeader) != "" { - return bifrost.Ptr(string(vkHeader)) - } + var virtualKeyValue string + if raw := string(ctx.Request.Header.Peek("x-bf-vk")); raw != "" { + v := strings.TrimSpace(raw) + if strings.HasPrefix(strings.ToLower(v), VirtualKeyPrefix) { + return bifrost.Ptr(v) + } + } // ... existing Authorization handling ... - xAPIKey := string(ctx.Request.Header.Peek("x-api-key")) - if xAPIKey != "" && strings.HasPrefix(strings.ToLower(xAPIKey), VirtualKeyPrefix) { - return bifrost.Ptr(xAPIKey) - } + if v := string(ctx.Request.Header.Peek("x-api-key")); v != "" && + strings.HasPrefix(strings.ToLower(v), VirtualKeyPrefix) { + return bifrost.Ptr(v) + } + if v := string(ctx.Request.Header.Peek("x-goog-api-key")); v != "" && + strings.HasPrefix(strings.ToLower(v), VirtualKeyPrefix) { + return bifrost.Ptr(v) + } return nil }#!/bin/bash # Verify all virtual-key sources are handled consistently rg -n "BifrostContextKeyVirtualKey|x-bf-vk|x-api-key|x-goog-api-key" -S
258-306: Fix log message:UnmarshalvsMarshalin middleware body handlingIn
HTTPTransportMiddleware:
- Line 287:
sonic.Unmarshalis used, but the error log says"failed to marshal request body...".- Line 301:
sonic.Marshaluses the same message, which is accurate there.To avoid confusion when debugging, update the first log message to say “unmarshal”:
Suggested tweak
- err = sonic.Unmarshal(ctx.Request.Body(), &payload) - if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + err = sonic.Unmarshal(ctx.Request.Body(), &payload) + if err != nil { + p.logger.Error("failed to unmarshal request body to check for virtual key: %v", err)transports/bifrost-http/handlers/middlewares.go (1)
201-215:TracingMiddleware.Middlewaremust pass acontext.ContexttoStartSpan, not*fasthttp.RequestCtxHere:
spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)
ctxis*fasthttp.RequestCtx, butStartSpanis defined on the tracer as:StartSpan(ctx context.Context, name string, kind schemas.SpanKind) (context.Context, schemas.SpanHandle)
*fasthttp.RequestCtxdoes not implementcontext.Context, so this will not compile and, even if coerced, would break the tracer’s expectations aroundValue/deadline propagation. This is essentially the same issue previously flagged and marked as addressed.You should convert the HTTP context into a
context.Contextfirst (e.g. via your existinglib.ConvertToBifrostContext) and pass that intoStartSpan, then propagate the resulting span ID back into thefasthttpcontext:Suggested fix sketch
- // Create root span for the HTTP request - spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) + // Create a Bifrost/context.Context from the fasthttp context + bctx := lib.ConvertToBifrostContext(ctx) + // Create root span for the HTTP request + spanCtx, rootSpan := m.tracer.Load().StartSpan(bctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) if rootSpan != nil { m.tracer.Load().SetAttribute(rootSpan, "http.method", string(ctx.Method())) m.tracer.Load().SetAttribute(rootSpan, "http.url", string(ctx.RequestURI())) m.tracer.Load().SetAttribute(rootSpan, "http.user_agent", string(ctx.Request.Header.UserAgent())) - // Set root span ID in context for child span creation - if spanID, ok := spanCtx.Value(schemas.BifrostContextKeySpanID).(string); ok { - ctx.SetUserValue(schemas.BifrostContextKeySpanID, spanID) - } + // Set root span ID in the fasthttp context for child span creation + if spanID, ok := spanCtx.Value(schemas.BifrostContextKeySpanID).(string); ok { + ctx.SetUserValue(schemas.BifrostContextKeySpanID, spanID) + } }(Adapt
lib.ConvertToBifrostContextusage to match its actual signature.)This keeps the tracer API correctly fed with a real
context.Contextwhile still exposing the span ID throughRequestCtx.UserValue.#!/bin/bash # Show StartSpan signature and its usages to confirm the type mismatch rg -n "StartSpan\(ctx context.Context" core/schemas/tracer.go -n rg -n "StartSpan\(" transports/bifrost-http/handlers/middlewares.go -n -C2Also applies to: 217-225, 227-279
core/schemas/trace.go (3)
41-49:Trace.Reset()lacks mutex protection.This method modifies all fields including
Spanswithout holding the mutex, creating a potential data race ifResetis called while another goroutine is accessing the trace (e.g., viaAddSpanorGetSpan). Add mutex protection or document thatResetmust only be called when exclusive access is guaranteed.🔎 Proposed fix
// Reset clears the trace for reuse from pool func (t *Trace) Reset() { + t.mu.Lock() + defer t.mu.Unlock() t.TraceID = "" t.ParentID = "" t.RootSpan = nil t.Spans = t.Spans[:0] t.StartTime = time.Time{} t.EndTime = time.Time{} t.Attributes = nil }
85-89:Span.End()method lacks mutex protection.Unlike
SetAttributeandAddEvent, theEndmethod modifiesEndTime,Status, andStatusMsgwithout holding the mutex. IfEndis called concurrently with reads or writes to these fields, a data race may occur.🔎 Proposed fix
// End marks the span as complete with the given status func (s *Span) End(status SpanStatus, statusMsg string) { + s.mu.Lock() + defer s.mu.Unlock() s.EndTime = time.Now() s.Status = status s.StatusMsg = statusMsg }
92-104:Span.Reset()lacks mutex protection.Similar to
Trace.Reset(), this method modifies fields without holding the mutex. If reused from a pool while another goroutine still holds a reference, a data race could occur.🔎 Proposed fix
// Reset clears the span for reuse from pool func (s *Span) Reset() { + s.mu.Lock() + defer s.mu.Unlock() s.SpanID = "" s.ParentID = "" s.TraceID = "" s.Name = "" s.Kind = SpanKindUnspecified s.StartTime = time.Time{} s.EndTime = time.Time{} s.Status = SpanStatusUnset s.StatusMsg = "" s.Attributes = nil s.Events = s.Events[:0] }framework/tracing/tracer.go (1)
394-440: Missing sort calls for deterministic ordering inbuildCompleteResponseFromChunks.The comments at lines 395 and 434 claim to "Sort choice indices for deterministic output" and "Sort tool calls by index", but the actual
sort.Ints()calls are missing. Without sorting, the order of choices and tool calls will be nondeterministic because map iteration order is randomized in Go.Add
sort.Ints(choiceIndices)after line 398 andsort.Ints(tcIndices)after line 437 to ensure deterministic, reproducible output.🔎 Proposed fix
Add the sort import if not already present:
import ( "context" + "sort" "time" "github.com/maximhq/bifrost/core/schemas" "github.com/maximhq/bifrost/framework/modelcatalog" "github.com/maximhq/bifrost/framework/streaming" )Then add the sorting calls:
// Build final choices from accumulated data // Sort choice indices for deterministic output choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } + sort.Ints(choiceIndices) for _, idx := range choiceIndices { accum := choiceMap[idx] ... if len(accum.toolCalls) > 0 { // Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } + sort.Ints(tcIndices) toolCalls := make([]schemas.ChatAssistantMessageToolCall, 0, len(accum.toolCalls)) for _, tcIdx := range tcIndices {
🧹 Nitpick comments (5)
framework/tracing/propagation.go (1)
10-29: Treat all‑zero trace/span IDs as invalid to fully align with W3C Trace Context
normalizeTraceID,normalizeSpanID, andParseTraceparentvalidate hex and length but accept all‑zero IDs, which the W3C Trace Context spec marks as invalid for bothtrace-idandparent-id. That can yield headers some tracers will reject.Consider rejecting all‑zero IDs during normalization and parsing so invalid upstream values don’t silently propagate:
Suggested guard for all‑zero IDs
func normalizeTraceID(traceID string) string { normalized := strings.ReplaceAll(traceID, "-", "") normalized = strings.ToLower(normalized) if len(normalized) != 32 { return "" } - if !isHex(normalized) { + if !isHex(normalized) || normalized == strings.Repeat("0", 32) { return "" } return normalized } func normalizeSpanID(spanID string) string { normalized := strings.ReplaceAll(spanID, "-", "") normalized = strings.ToLower(normalized) if len(normalized) > 16 { normalized = normalized[:16] } if len(normalized) != 16 { return "" } - if !isHex(normalized) { + if !isHex(normalized) || normalized == strings.Repeat("0", 16) { return "" } return normalized } func ParseTraceparent(traceparent string) *W3CTraceContext { parts := strings.Split(traceparent, "-") if len(parts) != 4 { return nil } version, traceID, parentID, traceFlags := parts[0], parts[1], parts[2], parts[3] if version != "00" { return nil } - if len(traceID) != 32 || !isHex(traceID) { + if len(traceID) != 32 || !isHex(traceID) || traceID == strings.Repeat("0", 32) { return nil } - if len(parentID) != 16 || !isHex(parentID) { + if len(parentID) != 16 || !isHex(parentID) || parentID == strings.Repeat("0", 16) { return nil } if len(traceFlags) != 2 || !isHex(traceFlags) { return nil } ... }Also applies to: 31-56, 123-136
core/bifrost.go (3)
2350-2379: Fallback span instrumentation is correct but duplicatedThe new fallback spans in both
handleRequestandhandleStreamRequestcorrectly capture provider/model/index and outcome (ok vs error) onSpanKindFallbackspans. The logic is essentially duplicated between the two methods though; consider a small helper (e.g.withFallbackSpan(...)) to reduce drift if you tweak attributes or naming later.Also applies to: 2449-2477
3322-3384: Streaming post‑hook aggregation works, but comment about pre‑hook hierarchy is slightly misleadingThe streaming path in
RunPostHookscorrectly switches from per‑plugin spans to timing accumulation, andFinalizeStreamingPostHookSpansemits one aggregatedplugin.<name>.posthookspan per plugin with invocation/error counts and avg/total durations. However:
postHookPluginOrderis built in post‑hook execution order (reverse of pre‑hook order), so the nested spans reflect the post‑hook call stack, not strictly the pre‑hook hierarchy as the comment suggests.This is mostly a documentation nit, but if you really want “pre‑hook hierarchy” semantics, you’d need to either record pre‑hook order separately or reverse
postHookPluginOrderwhen building aggregated spans.Also applies to: 3399-3410, 3412-3484
3893-3896: Optional: guardShutdowntracer access defensively
Shutdowncurrently doesbifrost.tracer.Load().(*tracerWrapper)without checking for a nil value. Under normal construction viaInitthis is fine, but a zero‑valueBifrostor misuse ofSetTracerin tests would panic here.Consider a small guard:
Suggested defensive tweak
- if tracerWrapper := bifrost.tracer.Load().(*tracerWrapper); tracerWrapper != nil && tracerWrapper.tracer != nil { - tracerWrapper.tracer.Stop() - } + if v := bifrost.tracer.Load(); v != nil { + if tw, ok := v.(*tracerWrapper); ok && tw != nil && tw.tracer != nil { + tw.tracer.Stop() + } + }framework/tracing/store.go (1)
197-242: Consider documenting the span hierarchy behavior.
StartSpanalways sets the parent to the root span if one exists (lines 231-236), meaning it can only create the root span or direct children of the root. For deeper span hierarchies, callers must useStartChildSpan. This design is likely intentional to keep trace-level span creation simple, but it would be clearer with a brief comment explaining the hierarchy constraint.📝 Suggested documentation addition
-// StartSpan creates a new span and adds it to the trace +// StartSpan creates a new span and adds it to the trace. +// If a root span exists, the new span becomes a child of the root. +// For deeper span hierarchies, use StartChildSpan to specify a parent span ID. func (s *TraceStore) StartSpan(traceID, name string, kind schemas.SpanKind) *schemas.Span {
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sum
📒 Files selected for processing (91)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/versionui/app/workspace/logs/views/logDetailsSheet.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsx
💤 Files with no reviewable changes (3)
- plugins/otel/ttlsyncmap.go
- framework/configstore/tables/mcp.go
- core/mcp.go
✅ Files skipped from review due to trivial changes (2)
- plugins/otel/version
- plugins/jsonparser/version
🚧 Files skipped from review as they are similar to previous changes (39)
- .github/workflows/scripts/release-bifrost-http.sh
- framework/streaming/audio.go
- .github/workflows/scripts/push-mintlify-changelog.sh
- framework/streaming/types.go
- ui/components/sidebar.tsx
- plugins/telemetry/main.go
- plugins/maxim/version
- plugins/semanticcache/version
- core/schemas/context.go
- plugins/semanticcache/main.go
- docs/plugins/migration-guide.mdx
- framework/streaming/chat.go
- transports/bifrost-http/handlers/middlewares_test.go
- plugins/telemetry/version
- transports/bifrost-http/handlers/cache.go
- plugins/logging/utils.go
- plugins/mocker/version
- plugins/semanticcache/changelog.md
- transports/bifrost-http/lib/config_test.go
- framework/tracing/helpers.go
- plugins/mocker/changelog.md
- framework/plugins/dynamicplugin.go
- transports/bifrost-http/handlers/inference.go
- core/schemas/plugin.go
- transports/bifrost-http/handlers/mcp.go
- transports/bifrost-http/handlers/health.go
- docs/plugins/writing-plugin.mdx
- framework/streaming/transcription.go
- core/version
- transports/bifrost-http/handlers/governance.go
- examples/plugins/hello-world/go.mod
- core/bifrost_test.go
- transports/bifrost-http/lib/middleware.go
- plugins/maxim/main.go
- plugins/maxim/changelog.md
- plugins/jsonparser/main.go
- transports/bifrost-http/handlers/plugins.go
- framework/version
- examples/plugins/hello-world/main.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
ui/app/workspace/logs/views/logDetailsSheet.tsxtransports/bifrost-http/handlers/websocket.goframework/streaming/responses.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/logging.goplugins/mocker/main.goplugins/governance/changelog.mdtransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/integrations.goplugins/jsonparser/changelog.mdplugins/logging/changelog.mdplugins/governance/versiontransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/integrations/router.goplugins/governance/main.goframework/tracing/propagation.goplugins/telemetry/changelog.mddocs/plugins/getting-started.mdxcore/schemas/trace.goframework/tracing/store.goframework/streaming/accumulator.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/tracer.goplugins/logging/main.gotransports/changelog.mdtransports/bifrost-http/lib/config.goplugins/otel/main.goplugins/logging/versiontransports/bifrost-http/handlers/mcpserver.goui/components/ui/sheet.tsxplugins/otel/converter.godocs/docs.jsonplugins/governance/go.modplugins/otel/changelog.mdtransports/bifrost-http/handlers/config.gotransports/bifrost-http/server/server.goframework/plugins/dynamicplugin_test.gocore/changelog.mdui/components/ui/switch.tsxframework/tracing/tracer.goframework/changelog.mdframework/tracing/llmspan.gotransports/versioncore/bifrost.go
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/websocket.goframework/streaming/responses.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/logging.goplugins/mocker/main.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/integrations/router.goplugins/governance/main.goframework/tracing/propagation.gocore/schemas/trace.goframework/tracing/store.goframework/streaming/accumulator.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/tracer.goplugins/logging/main.gotransports/bifrost-http/lib/config.goplugins/otel/main.gotransports/bifrost-http/handlers/mcpserver.goplugins/otel/converter.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/server/server.goframework/plugins/dynamicplugin_test.goframework/tracing/tracer.goframework/tracing/llmspan.gocore/bifrost.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/server/server.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
🧬 Code graph analysis (25)
transports/bifrost-http/handlers/websocket.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/responses.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/providers.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/middlewares.go (5)
core/schemas/plugin.go (2)
ObservabilityPlugin(123-137)Plugin(71-97)core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/propagation.go (1)
ExtractParentID(74-84)core/schemas/bifrost.go (4)
BifrostContextKeyTraceID(143-143)BifrostContextKeyTraceCompleter(148-148)BifrostContextKeySpanID(144-144)BifrostContextKeyDeferTraceCompletion(147-147)core/schemas/trace.go (3)
SpanKindHTTPRequest(131-131)SpanStatusError(151-151)SpanStatusOk(149-149)
transports/bifrost-http/integrations/router.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/trace.go (2)
framework/tracing/helpers.go (1)
AddSpan(32-38)ui/lib/constants/logs.ts (1)
Status(161-161)
framework/tracing/store.go (3)
framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
core/providers/utils/utils.go (2)
framework/tracing/tracer.go (1)
Tracer(17-20)framework/tracing/helpers.go (1)
EndSpan(50-56)
core/schemas/bifrost.go (2)
core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)
core/schemas/tracer.go (8)
core/schemas/bifrost.go (4)
ModelProvider(33-33)BifrostError(474-483)BifrostRequest(173-193)BifrostResponse(335-355)core/schemas/chatcompletions.go (1)
BifrostLLMUsage(845-852)core/schemas/speech.go (1)
BifrostSpeechResponse(22-29)core/schemas/transcriptions.go (1)
BifrostTranscriptionResponse(16-26)core/schemas/trace.go (3)
Trace(10-19)SpanKind(115-115)SpanStatus(143-143)framework/tracing/helpers.go (1)
EndSpan(50-56)core/schemas/context.go (1)
BifrostContext(32-42)framework/tracing/tracer.go (1)
Tracer(17-20)
plugins/otel/main.go (3)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)core/schemas/context.go (1)
BifrostContext(32-42)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/handlers/mcpserver.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (17)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindHTTPRequest(131-131)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
framework/tracing/tracer.go (10)
framework/tracing/store.go (1)
TraceStore(25-37)framework/streaming/accumulator.go (2)
Accumulator(14-30)NewAccumulator(440-480)core/schemas/tracer.go (3)
Tracer(38-116)SpanHandle(11-11)StreamAccumulatorResult(15-33)core/schemas/trace.go (4)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (6)
BifrostContextKeySpanID(144-144)BifrostRequest(173-193)BifrostResponse(335-355)BifrostError(474-483)BifrostContextKeyRequestID(124-124)BifrostContextKeyStreamEndIndicator(131-131)framework/tracing/llmspan.go (3)
PopulateRequestAttributes(14-60)PopulateResponseAttributes(64-106)PopulateErrorAttributes(109-124)core/schemas/chatcompletions.go (1)
BifrostChatResponse(27-42)core/schemas/models.go (1)
Model(109-129)framework/streaming/types.go (1)
StreamResponseTypeFinal(24-24)
framework/tracing/llmspan.go (5)
core/schemas/bifrost.go (18)
BifrostRequest(173-193)RequestType(89-89)TextCompletionRequest(93-93)EmbeddingRequest(99-99)TranscriptionRequest(102-102)SpeechRequest(100-100)ResponsesRequest(97-97)BatchCreateRequest(104-104)BatchListRequest(105-105)BatchRetrieveRequest(106-106)BatchCancelRequest(107-107)BatchResultsRequest(108-108)FileUploadRequest(109-109)FileListRequest(110-110)FileRetrieveRequest(111-111)FileDeleteRequest(112-112)FileContentRequest(113-113)BifrostResponse(335-355)core/schemas/trace.go (11)
AttrProviderName(159-159)AttrRequestModel(160-160)AttrMaxTokens(163-163)AttrTemperature(164-164)AttrTopP(165-165)AttrStopSequences(166-166)AttrPresencePenalty(167-167)AttrFrequencyPenalty(168-168)AttrParallelToolCall(169-169)AttrRequestUser(170-170)AttrUsageCost(213-213)core/schemas/chatcompletions.go (2)
BifrostChatRequest(12-19)BifrostChatResponse(27-42)core/schemas/files.go (5)
BifrostFileUploadRequest(53-67)BifrostFileListRequest(109-128)BifrostFileRetrieveRequest(146-159)BifrostFileDeleteRequest(186-198)BifrostFileContentRequest(215-227)framework/modelcatalog/main.go (1)
ModelCatalog(23-46)
core/bifrost.go (5)
core/schemas/tracer.go (3)
Tracer(38-116)DefaultTracer(181-183)SpanHandle(11-11)core/schemas/bifrost.go (9)
BifrostRequest(173-193)BifrostContextKeyTracer(146-146)BifrostContextKeyStreamStartTime(145-145)BifrostContextKeyTraceID(143-143)BifrostResponse(335-355)RequestType(89-89)BifrostContextKeyPostHookSpanFinalizer(149-149)BifrostError(474-483)BifrostContextKeySpanID(144-144)framework/tracing/helpers.go (1)
EndSpan(50-56)core/utils.go (1)
IsStreamRequestType(197-199)core/schemas/context.go (1)
NewBifrostContextWithTimeout(70-73)
🪛 markdownlint-cli2 (0.18.1)
transports/changelog.md
16-16: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
2 times, most recently
from
6fb3318 to
3f130fd
Compare
There was a problem hiding this comment.
Actionable comments posted: 6
♻️ Duplicate comments (17)
plugins/jsonparser/version (1)
1-1: Patch version increment inconsistent with breaking changes.This concern was already raised in a previous review. The changelog documents breaking changes (TransportInterceptor → HTTPTransportMiddleware), but the version increment is only a patch bump (1.3.52 → 1.3.53). Per semantic versioning, breaking changes warrant at minimum a minor version bump.
plugins/otel/changelog.md (1)
19-19: Verify the migration guide link exists.The changelog references
/docs/plugins/migration-guideat line 19. A previous review comment flagged that this file doesn't exist in the repository. Ensure the migration guide is created before merging, or update this link to point to the correct location.#!/bin/bash # Description: Verify the migration guide exists and check alternative paths # Check if the exact path exists if fd -t f "migration-guide" docs/; then echo "Migration guide found:" fd -t f "migration-guide" docs/ else echo "Migration guide not found at expected path" fi # Also check for plugin documentation structure echo "Plugin docs structure:" fd -t f . docs/plugins/ 2>/dev/null || echo "docs/plugins/ directory not found" fd -t f . docs/features/plugins/ 2>/dev/null || echo "docs/features/plugins/ directory not found"ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
183-187: Add aria-label for screen reader accessibility.The icon-only button at line 184 lacks an accessible name for screen reader users. Add an
aria-labelattribute to improve accessibility.🔎 Proposed fix
<DropdownMenuTrigger asChild> - <Button variant="ghost" size="icon"> + <Button variant="ghost" size="icon" aria-label="Log actions"> <MoreVertical className="h-3 w-3" /> </Button> </DropdownMenuTrigger>core/providers/utils/utils.go (1)
1424-1513: Refactor to pass context by value.The function signature
completeDeferredSpan(ctx *context.Context, ...)violates Go conventions. Thecontext.Contextinterface should always be passed by value, never by pointer, as it's designed for safe concurrent use and value semantics.🔎 Proposed fix
Update the function signature and body:
-func completeDeferredSpan(ctx *context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { +func completeDeferredSpan(ctx context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { if ctx == nil { return } - traceID, ok := (*ctx).Value(schemas.BifrostContextKeyTraceID).(string) + traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string) if !ok || traceID == "" { return } - tracerVal := (*ctx).Value(schemas.BifrostContextKeyTracer) + tracerVal := ctx.Value(schemas.BifrostContextKeyTracer) // ... rest of function updated similarlyUpdate all call sites:
-completeDeferredSpan(&ctx, processedResponse, processedError) +completeDeferredSpan(ctx, processedResponse, processedError)framework/streaming/chat.go (1)
157-171: TTFT calculation is correct but duplicated.The TTFT calculation logic is correct and properly guards against zero timestamps. However, this is identical to the implementation in
framework/streaming/responses.go. See the refactoring suggestion in that file to extract this into a shared helper function.transports/bifrost-http/integrations/router.go (1)
508-514: Non-streaming path passes wrong context to Bifrost client.The non-streaming request handlers pass
ctx(*fasthttp.RequestCtx) to Bifrost client methods (Lines 514, 534, 557, 580, 603, 626, 660), but the comment at Line 503-507 states "Use the cancellable context from ConvertToBifrostContext". ThebifrostCtxcontains trace propagation, plugin context keys, and timeout handling that would be lost.This appears to be the same issue flagged in a previous review. Passing
*bifrostCtx(dereferenced) instead ofctxwould ensure tracing and plugin hooks work correctly.🔎 Suggested fix
case bifrostReq.ListModelsRequest != nil: - listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) + listModelsResponse, bifrostErr := g.client.ListModelsRequest(*bifrostCtx, bifrostReq.ListModelsRequest)Apply similar changes to all other non-streaming request type cases (TextCompletionRequest, ChatCompletionRequest, ResponsesRequest, EmbeddingRequest, SpeechRequest, TranscriptionRequest).
transports/changelog.md (1)
16-23: Add language specifier to fenced code block.The fenced code block lacks a language identifier, which prevents proper syntax highlighting. Add
goortextafter the opening backticks.🔎 Suggested fix
**Migration summary:** - ``` + ```go // v1.3.x (removed) TransportInterceptor(ctx *BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error)plugins/governance/main.go (1)
233-256: Missing prefix validation for x-bf-vk header.The
parseVirtualKeyfunction validates that values fromAuthorization(Bearer) andx-api-keyheaders start withVirtualKeyPrefix, but thex-bf-vkheader at lines 235-238 is returned without prefix validation. This inconsistency could allow invalid virtual keys to pass through.🔎 Proposed fix
vkHeader := ctx.Request.Header.Peek("x-bf-vk") if string(vkHeader) != "" { + vkValue := string(vkHeader) + if strings.HasPrefix(strings.ToLower(vkValue), VirtualKeyPrefix) { - return bifrost.Ptr(string(vkHeader)) + return bifrost.Ptr(vkValue) + } }framework/tracing/store.go (1)
13-22: Trim unused fields fromDeferredSpanInfoand clear streaming state on TTL cleanup
DeferredSpanInfo.TracerandDeferredSpanInfo.RequestIDare never assigned or read; they just bloat each deferred entry. This was called out in an earlier review and still appears unused here.cleanupOldTracesevicts expired traces fromtracesand releases them, but does not delete any corresponding entries indeferredSpans. For long‑running processes with abandoned streaming traces, this can leak accumulated chunk buffers.Consider:
- Dropping the unused fields from
DeferredSpanInfo, or wiring them up if you really need them.- In
cleanupOldTraces, deleting the deferred span entry for each removed trace (e.g. usingtrace.TraceIDfrom the deleted value).Example adjustments
type DeferredSpanInfo struct { SpanID string StartTime time.Time - Tracer schemas.Tracer // Reference to tracer for completing the span - RequestID string // Request ID for accumulator lookup FirstChunkTime time.Time // Timestamp of first chunk (for TTFT calculation) AccumulatedChunks []*schemas.BifrostResponse // Accumulated streaming chunks mu sync.Mutex // Mutex for thread-safe chunk accumulation @@ func (s *TraceStore) cleanupOldTraces() { cutoff := time.Now().Add(-s.ttl) @@ - s.traces.Range(func(key, value any) bool { - trace := value.(*schemas.Trace) + s.traces.Range(func(key, value any) bool { + trace := value.(*schemas.Trace) if trace.StartTime.Before(cutoff) { if deleted, ok := s.traces.LoadAndDelete(key); ok { - s.ReleaseTrace(deleted.(*schemas.Trace)) + t := deleted.(*schemas.Trace) + // Drop any deferred streaming state for this trace + s.ClearDeferredSpan(t.TraceID) + s.ReleaseTrace(t) count++ }Also applies to: 119-177, 341-360
transports/bifrost-http/handlers/middlewares.go (1)
191-307: FixTracingMiddlewareto pass acontext.ContextintoStartSpaninstead of*fasthttp.RequestCtx
Tracer.StartSpanis defined againstcontext.Context, butMiddleware()currently calls:spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)where
ctxis*fasthttp.RequestCtx. This does not satisfycontext.Contextand will not compile; even if it did, the tracer implementation expects a realcontext.Context(it callsValueon it, etc.).Use the existing adapter pattern (as in the inference handlers) to convert the fasthttp context to a Bifrost context first, and then pass that into
StartSpan, while still propagating the resulting span ID back into the fasthttpRequestCtx:Suggested fix using `lib.ConvertToBifrostContext`
func (m *TracingMiddleware) Middleware() schemas.BifrostHTTPMiddleware { return func(next fasthttp.RequestHandler) fasthttp.RequestHandler { return func(ctx *fasthttp.RequestCtx) { @@ - // Create root span for the HTTP request - spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) + // Create root span for the HTTP request using a standard context + bfCtx := lib.ConvertToBifrostContext(ctx) + spanCtx, rootSpan := m.tracer.Load().StartSpan(bfCtx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) if rootSpan != nil { m.tracer.Load().SetAttribute(rootSpan, "http.method", string(ctx.Method())) m.tracer.Load().SetAttribute(rootSpan, "http.url", string(ctx.RequestURI())) m.tracer.Load().SetAttribute(rootSpan, "http.user_agent", string(ctx.Request.Header.UserAgent())) // Set root span ID in context for child span creation if spanID, ok := spanCtx.Value(schemas.BifrostContextKeySpanID).(string); ok { ctx.SetUserValue(schemas.BifrostContextKeySpanID, spanID) } }This keeps tracing consistent with the rest of the stack and avoids a type mismatch at compile time.
core/schemas/trace.go (3)
40-49:Trace.Reset()lacks mutex protection — potential data race.This method modifies all fields including
Spanswithout holding the mutex, creating a data race ifResetis called while another goroutine is accessing the trace (e.g., viaAddSpanorGetSpan).This was flagged in a previous review. Add mutex protection:
🔎 Proposed fix
// Reset clears the trace for reuse from pool func (t *Trace) Reset() { + t.mu.Lock() + defer t.mu.Unlock() t.TraceID = "" t.ParentID = "" t.RootSpan = nil t.Spans = t.Spans[:0] t.StartTime = time.Time{} t.EndTime = time.Time{} t.Attributes = nil }
84-89:Span.End()lacks mutex protection — potential data race.Unlike
SetAttributeandAddEvent, theEndmethod modifiesEndTime,Status, andStatusMsgwithout holding the mutex.🔎 Proposed fix
// End marks the span as complete with the given status func (s *Span) End(status SpanStatus, statusMsg string) { + s.mu.Lock() + defer s.mu.Unlock() s.EndTime = time.Now() s.Status = status s.StatusMsg = statusMsg }
91-104:Span.Reset()lacks mutex protection — potential data race.Similar to
Trace.Reset(), this method modifies fields without holding the mutex.🔎 Proposed fix
// Reset clears the span for reuse from pool func (s *Span) Reset() { + s.mu.Lock() + defer s.mu.Unlock() s.SpanID = "" s.ParentID = "" // ... rest of fields }framework/tracing/tracer.go (2)
393-398: Choice indices are not sorted despite the comment.The comment at line 394 says "Sort choice indices for deterministic output" but no sorting is performed. The iteration order over a map in Go is non-deterministic, so the resulting choices may appear in different orders across runs.
This was flagged in a previous review and marked as addressed, but the sorting code is still missing.
🔎 Proposed fix
+import "sort" // Build final choices from accumulated data // Sort choice indices for deterministic output choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } +sort.Ints(choiceIndices)
433-441: Tool call indices are not sorted despite the comment.Similarly, the comment at line 433 says "Sort tool calls by index" but no sorting is performed. The iteration order will be non-deterministic.
🔎 Proposed fix
// Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } +sort.Ints(tcIndices) toolCalls := make([]schemas.ChatAssistantMessageToolCall, 0, len(accum.toolCalls))framework/tracing/llmspan.go (2)
190-193: ExtraParams keys may collide with standard attribute names.User-provided keys from
ExtraParamsare written directly to the attributes map without namespacing. If a user sets a key like"gen_ai.request.model"in ExtraParams, it would overwrite the standard attribute set earlier.This pattern repeats at lines 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 768-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029.
🔎 Suggested approach: namespace ExtraParams
// ExtraParams for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) + attrs["gen_ai.extra."+k] = fmt.Sprintf("%v", v) }
376-382: Use a float-appropriate formatter for embedding values.The code uses
%dformat verb for embedding values:embedding[i] = fmt.Sprintf("%d", v)If
Embeddingis a float slice (typical for embeddings),%dis incorrect and will truncate values or produce unexpected results.🔎 Proposed fix
} else if req.Input.Embedding != nil { embedding := make([]string, len(req.Input.Embedding)) for i, v := range req.Input.Embedding { - embedding[i] = fmt.Sprintf("%d", v) + embedding[i] = fmt.Sprintf("%g", v) } attrs[schemas.AttrInputEmbedding] = strings.Join(embedding, ",") }
🧹 Nitpick comments (8)
framework/streaming/responses.go (1)
742-756: Extract duplicated TTFT calculation logic.The TTFT calculation logic here is identical to the implementation in
framework/streaming/chat.go(lines 157-171). This violates the DRY principle and creates maintenance burden.🔎 Proposed refactor to extract common TTFT calculation
Add a helper function in
framework/streaming/types.goorframework/streaming/accumulator.go:// CalculateTTFT calculates Time to First Token in milliseconds from start and first chunk timestamps. // Returns 0 if either timestamp is zero. func CalculateTTFT(startTimestamp, firstChunkTimestamp time.Time) int64 { if startTimestamp.IsZero() || firstChunkTimestamp.IsZero() { return 0 } return firstChunkTimestamp.Sub(startTimestamp).Nanoseconds() / 1e6 }Then update this file:
- // Calculate Time to First Token (TTFT) in milliseconds - var ttft int64 - if !accumulator.StartTimestamp.IsZero() && !accumulator.FirstChunkTimestamp.IsZero() { - ttft = accumulator.FirstChunkTimestamp.Sub(accumulator.StartTimestamp).Nanoseconds() / 1e6 - } + // Calculate Time to First Token (TTFT) in milliseconds + ttft := CalculateTTFT(accumulator.StartTimestamp, accumulator.FirstChunkTimestamp) // Initialize accumulated data data := &AccumulatedData{Apply the same change to
framework/streaming/chat.go,framework/streaming/audio.go, andframework/streaming/transcription.go.ui/components/devProfiler.tsx (2)
17-24: Handle edge cases informatBytesfor negative or extremely large values.The function could produce unexpected results for edge cases:
Math.logof a negative number returnsNaN, and very large numbers could exceed thesizesarray bounds.🔎 Suggested improvement
function formatBytes (bytes: number): string { if (bytes === 0) return '0 B' + if (bytes < 0) return `-${formatBytes(-bytes)}` const k = 1024 const sizes = ['B', 'KB', 'MB', 'GB'] - const i = Math.floor(Math.log(bytes) / Math.log(k)) + const i = Math.min(Math.floor(Math.log(bytes) / Math.log(k)), sizes.length - 1) return `${(bytes / Math.pow(k, i)).toFixed(1)} ${sizes[i]}` }
369-394: Consider using a stable key for allocation items.Using array index as key is generally discouraged for dynamic lists, but since this list is refreshed as a whole from the API and doesn't support reordering/insertion, it's acceptable here. If allocations gain unique identifiers in the future, prefer using those.
framework/streaming/accumulator.go (1)
138-141: Consider extracting duplicated FirstChunkTimestamp logic.The same
FirstChunkTimestamptracking logic is repeated across all fouradd*StreamChunkmethods. While functional, this could be extracted into a helper method to reduce duplication.🔎 Example helper extraction
// trackFirstChunk sets FirstChunkTimestamp if not already set. // Must be called while holding accumulator.mu lock. func (acc *StreamAccumulator) trackFirstChunk(chunkTimestamp time.Time) { if acc.FirstChunkTimestamp.IsZero() { acc.FirstChunkTimestamp = chunkTimestamp } }Then in each add method:
accumulator.trackFirstChunk(chunk.Timestamp)Also applies to: 162-165, 186-189, 210-213
framework/tracing/propagation.go (1)
118-121: Consider allowing higher version parsing per W3C spec.The W3C Trace Context spec recommends that implementations attempt to parse higher versions by extracting trace-id, parent-id, and the sampled bit, rather than rejecting them outright. However, strictly supporting only version "00" is a reasonable conservative choice for now.
core/bifrost.go (1)
70-71: Tightentraceratomic access and fix stale field comment
- The field comment on Line 70 says
atomic.Value“stores schemas.Tracer”, but you now always store*tracerWrapper. This is misleading for future maintainers.- Both
getTracer()andShutdown()currently do an unchecked.(*tracerWrapper)onatomic.Value.Load(). IfBifrostwere ever constructed withoutInit/SetTracer(tests, future code, or partial initialization), these would panic.Consider:
- Updating the comment to reflect
*tracerWrapper.- Making
getTracerandShutdowndefensively handle a nil/incorrect value and fall back toschemas.DefaultTracer()or simply no-op inShutdown.Example defensive tweak
- tracer atomic.Value // tracer for distributed tracing (stores schemas.Tracer, NoOpTracer if not configured) + tracer atomic.Value // stores *tracerWrapper wrapping a schemas.Tracer (NoOpTracer if not configured) func (bifrost *Bifrost) getTracer() schemas.Tracer { - return bifrost.tracer.Load().(*tracerWrapper).tracer + if v := bifrost.tracer.Load(); v != nil { + if tw, ok := v.(*tracerWrapper); ok && tw != nil && tw.tracer != nil { + return tw.tracer + } + } + return schemas.DefaultTracer() } func (bifrost *Bifrost) Shutdown() { @@ - // Stop the tracerWrapper to clean up background goroutines - if tracerWrapper := bifrost.tracer.Load().(*tracerWrapper); tracerWrapper != nil && tracerWrapper.tracer != nil { - tracerWrapper.tracer.Stop() - } + // Stop the tracerWrapper to clean up background goroutines + if v := bifrost.tracer.Load(); v != nil { + if tw, ok := v.(*tracerWrapper); ok && tw != nil && tw.tracer != nil { + tw.tracer.Stop() + } + }Also applies to: 95-107, 129-147, 253-265, 3893-3896
framework/tracing/helpers.go (1)
10-82: Helpers are clean; optionally guard against nilTraceStoreThe context-based helpers are straightforward and defensive on missing trace/ID/span. The only additional hardening you might consider is an early
if store == nil { return }inGetTrace,AddSpan,AddChildSpan,EndSpan,SetSpanAttribute, andAddSpanEventto avoid panics if a nil store is ever passed in.transports/bifrost-http/handlers/devpprof.go (1)
270-368:getTopAllocationsprovides synthetic data, not actual allocation hotspots.The function attempts to provide allocation data but:
runtime.Caller(i+2)captures the call stack ofgetTopAllocationsitself, not actual allocation sites- The "distributed estimate" calculation (
HeapAlloc / (i+1)) produces arbitrary numbers- The fallback to hardcoded runtime function names with unrelated memory stats doesn't reflect real allocation patterns
This is acceptable for a dev-only diagnostic tool, but the data should be clearly labeled as estimates in the API response or documentation to avoid misinterpretation.
Consider adding a comment in the
PprofDatastruct or API documentation indicating thatTopAllocationsprovides estimated/synthetic data rather than actual allocation profiling results. For accurate heap profiling, users should usego tool pprofdirectly.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sum
📒 Files selected for processing (96)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/versionui/app/clientLayout.tsxui/app/workspace/logs/views/logDetailsSheet.tsxui/components/devProfiler.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsxui/lib/store/apis/devApi.tsui/lib/store/apis/index.ts
💤 Files with no reviewable changes (3)
- core/mcp.go
- plugins/otel/ttlsyncmap.go
- framework/configstore/tables/mcp.go
✅ Files skipped from review due to trivial changes (2)
- framework/version
- plugins/governance/version
🚧 Files skipped from review as they are similar to previous changes (35)
- transports/bifrost-http/handlers/plugins.go
- plugins/semanticcache/changelog.md
- ui/components/ui/switch.tsx
- examples/plugins/hello-world/go.mod
- plugins/mocker/version
- docs/plugins/getting-started.mdx
- .github/workflows/scripts/push-mintlify-changelog.sh
- plugins/mocker/main.go
- ui/components/ui/sheet.tsx
- core/version
- plugins/logging/utils.go
- plugins/governance/go.mod
- core/bifrost_test.go
- transports/bifrost-http/handlers/governance.go
- plugins/semanticcache/main.go
- core/schemas/context.go
- .github/workflows/scripts/release-bifrost-http.sh
- transports/bifrost-http/handlers/mcp.go
- docs/plugins/migration-guide.mdx
- transports/bifrost-http/handlers/websocket.go
- plugins/logging/changelog.md
- plugins/telemetry/version
- transports/bifrost-http/handlers/providers.go
- plugins/maxim/main.go
- transports/version
- examples/plugins/hello-world/main.go
- transports/bifrost-http/handlers/logging.go
- transports/bifrost-http/lib/config.go
- plugins/semanticcache/version
- framework/streaming/transcription.go
- plugins/jsonparser/main.go
- docs/docs.json
- plugins/otel/version
- framework/streaming/audio.go
- plugins/logging/version
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
framework/streaming/responses.goui/lib/store/apis/index.tsplugins/mocker/changelog.mdtransports/bifrost-http/handlers/ui.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/health.goframework/streaming/types.goplugins/maxim/versionplugins/jsonparser/changelog.mdframework/plugins/dynamicplugin_test.gotransports/bifrost-http/integrations/router.goframework/changelog.mdcore/providers/utils/utils.goplugins/otel/changelog.mdui/lib/store/apis/devApi.tsplugins/telemetry/changelog.mddocs/plugins/writing-plugin.mdxplugins/governance/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.goui/components/devProfiler.tsxtransports/changelog.mdtransports/bifrost-http/handlers/config.gocore/schemas/bifrost.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/integrations.goui/components/sidebar.tsxframework/tracing/helpers.goplugins/governance/changelog.mdplugins/otel/main.goframework/tracing/store.goframework/plugins/dynamicplugin.gocore/schemas/tracer.gotransports/bifrost-http/handlers/inference.gocore/bifrost.gotransports/bifrost-http/handlers/session.goplugins/otel/converter.gotransports/bifrost-http/server/server.goframework/streaming/accumulator.gocore/schemas/plugin.goframework/tracing/propagation.gotransports/bifrost-http/lib/config_test.goui/app/workspace/logs/views/logDetailsSheet.tsxplugins/logging/main.gocore/schemas/trace.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/mcpserver.goplugins/maxim/changelog.mdframework/streaming/chat.goframework/tracing/llmspan.goframework/tracing/tracer.goui/app/clientLayout.tsxtransports/bifrost-http/handlers/devpprof.gocore/changelog.mdplugins/jsonparser/version
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
framework/streaming/responses.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/health.goframework/streaming/types.goframework/plugins/dynamicplugin_test.gotransports/bifrost-http/integrations/router.gocore/providers/utils/utils.goplugins/governance/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gocore/schemas/bifrost.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/integrations.goframework/tracing/helpers.goplugins/otel/main.goframework/tracing/store.goframework/plugins/dynamicplugin.gocore/schemas/tracer.gotransports/bifrost-http/handlers/inference.gocore/bifrost.gotransports/bifrost-http/handlers/session.goplugins/otel/converter.gotransports/bifrost-http/server/server.goframework/streaming/accumulator.gocore/schemas/plugin.goframework/tracing/propagation.gotransports/bifrost-http/lib/config_test.goplugins/logging/main.gocore/schemas/trace.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/mcpserver.goframework/streaming/chat.goframework/tracing/llmspan.goframework/tracing/tracer.gotransports/bifrost-http/handlers/devpprof.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/ui.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/server/server.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/devpprof.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
🧬 Code graph analysis (31)
framework/streaming/responses.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/health.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
transports/bifrost-http/integrations/router.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/telemetry/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/cache.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
ui/components/devProfiler.tsx (3)
npx/bin.js (3)
k(178-178)sizes(179-179)data(172-172)ui/lib/utils/port.ts (1)
isDevelopmentMode(110-112)ui/components/ui/tooltip.tsx (1)
Tooltip(43-43)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/schemas/bifrost.go (2)
core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(25-37)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
plugins/otel/main.go (2)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)core/schemas/trace.go (1)
Trace(10-19)
framework/tracing/store.go (3)
core/schemas/tracer.go (1)
Tracer(38-116)core/schemas/bifrost.go (1)
BifrostResponse(335-355)core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)
core/schemas/tracer.go (3)
core/schemas/bifrost.go (4)
ModelProvider(33-33)BifrostError(474-483)BifrostRequest(173-193)BifrostResponse(335-355)core/schemas/chatcompletions.go (1)
BifrostLLMUsage(845-852)core/schemas/trace.go (2)
SpanKind(115-115)SpanStatus(143-143)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
core/bifrost.go (6)
core/schemas/tracer.go (3)
Tracer(38-116)DefaultTracer(181-183)SpanHandle(11-11)framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/bifrost.go (10)
ListModelsRequest(92-92)BifrostRequest(173-193)BifrostContextKeyTracer(146-146)BifrostContextKeyStreamStartTime(145-145)BifrostContextKeyTraceID(143-143)BifrostResponse(335-355)RequestType(89-89)BifrostContextKeyPostHookSpanFinalizer(149-149)BifrostError(474-483)BifrostContextKeySpanID(144-144)framework/tracing/helpers.go (1)
EndSpan(50-56)core/utils.go (1)
IsStreamRequestType(197-199)core/schemas/context.go (1)
NewBifrostContextWithTimeout(70-73)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (15)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
transports/bifrost-http/lib/config_test.go (1)
transports/bifrost-http/lib/config.go (1)
LoadConfig(276-356)
ui/app/workspace/logs/views/logDetailsSheet.tsx (4)
ui/components/ui/sheet.tsx (1)
SheetContent(137-137)ui/components/ui/alertDialog.tsx (9)
AlertDialog(83-83)AlertDialogTrigger(93-93)AlertDialogContent(86-86)AlertDialogHeader(89-89)AlertDialogTitle(92-92)AlertDialogDescription(87-87)AlertDialogFooter(88-88)AlertDialogCancel(85-85)AlertDialogAction(84-84)ui/components/ui/dropdownMenu.tsx (5)
DropdownMenu(193-193)DropdownMenuTrigger(207-207)DropdownMenuContent(195-195)DropdownMenuItem(197-197)DropdownMenuSeparator(202-202)ui/components/ui/button.tsx (1)
Button(70-70)
plugins/logging/main.go (3)
core/schemas/bifrost.go (3)
RequestType(89-89)BifrostContextKeyTracer(146-146)BifrostContextKeyTraceID(143-143)core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)
core/schemas/trace.go (1)
framework/tracing/helpers.go (1)
AddSpan(32-38)
transports/bifrost-http/handlers/middlewares.go (6)
core/schemas/plugin.go (3)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)Plugin(71-97)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)framework/tracing/tracer.go (1)
Tracer(17-20)framework/tracing/propagation.go (1)
ExtractParentID(74-84)core/schemas/bifrost.go (4)
BifrostContextKeyTraceID(143-143)BifrostContextKeyTraceCompleter(148-148)BifrostContextKeySpanID(144-144)BifrostContextKeyDeferTraceCompletion(147-147)core/schemas/trace.go (3)
SpanKindHTTPRequest(131-131)SpanStatusError(151-151)SpanStatusOk(149-149)
transports/bifrost-http/handlers/mcpserver.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/chat.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
ui/app/clientLayout.tsx (1)
ui/components/devProfiler.tsx (1)
DevProfiler(55-422)
🪛 markdownlint-cli2 (0.18.1)
transports/changelog.md
16-16: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
3f130fd to
e8ca91d
Compare
There was a problem hiding this comment.
Actionable comments posted: 8
♻️ Duplicate comments (18)
framework/tracing/llmspan.go (1)
190-194: NamespaceExtraParamskeys to avoid clobbering standard trace attributesAcross the various
Populate*RequestAttributeshelpers,ExtraParamsis currently merged intoattrswith user-supplied keys (e.g.,attrs[k] = fmt.Sprintf("%v", v)). That means a caller can accidentally overwrite standard keys likegen_ai.request.model,gen_ai.usage.*, etc., by choosing the same key name inExtraParams.To keep the core attribute namespace stable while still exposing user data, consider:
- Prefixing all extra keys (for example,
attrs["gen_ai.extra."+k] = fmt.Sprintf("%v", v)), and- Applying this consistently in all the ExtraParams loops (chat, text, embedding, responses, batch, and file operations).
Optionally, you can also guard against duplicate prefixed keys if you want to avoid overwriting on collision.
Also applies to: 298-301, 365-368, 575-577, 708-710, 739-741, 753-756, 768-771, 783-786, 958-961, 982-985, 997-1000, 1012-1015, 1027-1030
core/schemas/trace.go (1)
40-49: GuardTrace.Reset,Span.End, andSpan.Resetwith their mutexes to avoid data racesThese methods mutate shared state (
Tracefields andSpantiming/status/collections) without acquiring the corresponding mutex, while other methods (AddSpan,SetAttribute,AddEvent) do take the lock. If traces/spans are used concurrently (which tracing callers commonly do), this will lead to data races when spans are ended or traces/spans are reset for pool reuse.Consider wrapping each of these in the appropriate lock/unlock:
Trace.Reset: lockt.mufor the duration of the field/slice resets.Span.End: locks.muwhile settingEndTime,Status, andStatusMsg.Span.Reset: locks.muwhile clearing fields and truncatingEvents.This keeps the concurrency model consistent with the rest of the API and avoids subtle pool-related races.
Proposed change
func (t *Trace) Reset() { - t.TraceID = "" - t.ParentID = "" - t.RootSpan = nil - t.Spans = t.Spans[:0] - t.StartTime = time.Time{} - t.EndTime = time.Time{} - t.Attributes = nil + t.mu.Lock() + defer t.mu.Unlock() + t.TraceID = "" + t.ParentID = "" + t.RootSpan = nil + t.Spans = t.Spans[:0] + t.StartTime = time.Time{} + t.EndTime = time.Time{} + t.Attributes = nil } // End marks the span as complete with the given status func (s *Span) End(status SpanStatus, statusMsg string) { - s.EndTime = time.Now() - s.Status = status - s.StatusMsg = statusMsg + s.mu.Lock() + defer s.mu.Unlock() + s.EndTime = time.Now() + s.Status = status + s.StatusMsg = statusMsg } // Reset clears the span for reuse from pool func (s *Span) Reset() { - s.SpanID = "" - s.ParentID = "" - s.TraceID = "" - s.Name = "" - s.Kind = SpanKindUnspecified - s.StartTime = time.Time{} - s.EndTime = time.Time{} - s.Status = SpanStatusUnset - s.StatusMsg = "" - s.Attributes = nil - s.Events = s.Events[:0] + s.mu.Lock() + defer s.mu.Unlock() + s.SpanID = "" + s.ParentID = "" + s.TraceID = "" + s.Name = "" + s.Kind = SpanKindUnspecified + s.StartTime = time.Time{} + s.EndTime = time.Time{} + s.Status = SpanStatusUnset + s.StatusMsg = "" + s.Attributes = nil + s.Events = s.Events[:0] }Also applies to: 84-104
docs/plugins/migration-guide.mdx (1)
247-250: Verify v1.4.0 tag will exist at merge time.The
go getcommand references@v1.4.0, but this tag may not exist in the repository yet. Ensure the v1.4.0 tag is created before merging this PR, or update the documentation to use an existing version tag, a specific commit hash, or add a note indicating this is for an upcoming release.Run this script to verify the tag status:
#!/bin/bash # Check if v1.4.0 tag exists and show latest tags git tag -l "v1.4.0" echo "---" echo "Latest tags:" git tag -l --sort=-version:refname | head -10transports/changelog.md (1)
16-16: Add language specifier to code fence.The fenced code block is missing a language identifier, which prevents syntax highlighting and violates markdownlint rule MD040.
🔎 Proposed fix
**Migration summary:** - ``` + ```go // v1.3.x (removed) TransportInterceptor(ctx *BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error)examples/plugins/hello-world/main.go (1)
19-27: HTTPTransportMiddleware sets fasthttp context value but PreHook/PostHook read from BifrostContext.The middleware on line 23 sets
ctx.SetUserValue(...)onfasthttp.RequestCtx, but PreHook (line 30) and PostHook (lines 39-40) attempt to read from*schemas.BifrostContext. These are separate context types, so the value will never propagate. This breaks the example's demonstration of data flow between hooks and middleware.transports/bifrost-http/integrations/router.go (1)
514-514: Non-streaming handlers pass fasthttp.RequestCtx instead of bifrostCtx to Bifrost client.All non-streaming client calls (lines 514, 534, 557, 580, 603, 626, 660) pass
ctx(the*fasthttp.RequestCtx) instead of the converted*bifrostCtx. This breaks tracing, plugin context propagation, and may cause type errors if the client expectscontext.Context. The comment on line 503 even says "Use the cancellable context from ConvertToBifrostContext" but the code doesn't follow through.Also applies to: 534-534, 557-557, 580-580, 603-603, 626-626, 660-660
framework/plugins/dynamicplugin.go (1)
140-147: Type mismatch: HTTPTransportMiddleware symbol is a factory function, not the middleware itself.Line 145 attempts to cast the symbol directly to
schemas.BifrostHTTPMiddleware, but the example plugin atexamples/plugins/hello-world/main.go(lines 19-27) exportsHTTPTransportMiddlewareas a factory function:func HTTPTransportMiddleware() schemas.BifrostHTTPMiddlewareThe type assertion will fail at runtime. The symbol should be cast to
func() schemas.BifrostHTTPMiddleware, then called to obtain the middleware.core/providers/utils/utils.go (1)
1428-1428: Pass context by value, not by pointer.The function signature
completeDeferredSpan(ctx *context.Context, ...)violates Go conventions. Contexts should always be passed by value. Thecontext.Contexttype is designed to be passed by value and is safe for concurrent use.plugins/logging/main.go (1)
225-233: Revisit PreHook’s tracer-based accumulator creation; likely dead with current tracer injection order.In
PreHookyou gateCreateStreamAccumulatoron:if bifrost.IsStreamRequestType(req.RequestType) { if tracer, ok := ctx.Value(schemas.BifrostContextKeyTracer).(schemas.Tracer); ok && tracer != nil { if traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string); ok && traceID != "" { tracer.CreateStreamAccumulator(traceID, createdTimestamp) } } }But
BifrostContextKeyTraceris injected by core insideexecuteRequestWithRetries, which runs on the worker side after pluginPreHooks have already executed. Unless another component setsBifrostContextKeyTracerearlier, this condition will almost never be true in the normal HTTP→Bifrost path, so the PreHook accumulator creation is effectively dead code.Given that:
- streaming aggregation now flows through
tracer.ProcessStreamingChunkinPostHook, and- cleanup is explicitly delegated to the tracing middleware/tracer,
it would be cleaner and less confusing to either:
- remove the PreHook
CreateStreamAccumulatorblock and let the tracer lazily create accumulators insideProcessStreamingChunk, or- move tracer injection earlier (before
RunPreHooks) if you really want PreHook to own creation.Right now the lifecycle is split in a way that’s hard to reason about and easy to drift.
Also applies to: 382-471, 615-625
plugins/governance/main.go (2)
233-261: Validatex-bf-vkwithVirtualKeyPrefixlike the other headers.
parseVirtualKeycurrently returns thex-bf-vkheader verbatim without checking that it starts withVirtualKeyPrefix, whileAuthorization,x-api-key, andx-goog-api-keyall enforce that prefix. This inconsistency allows malformed or non-virtual keys viax-bf-vkto bypass the basic format check.Consider normalizing and validating
x-bf-vksimilarly before returning:Suggested adjustment
func parseVirtualKey(ctx *fasthttp.RequestCtx) *string { var virtualKeyValue string vkHeader := ctx.Request.Header.Peek("x-bf-vk") - if string(vkHeader) != "" { - return bifrost.Ptr(string(vkHeader)) - } + if raw := strings.TrimSpace(string(vkHeader)); raw != "" { + if strings.HasPrefix(strings.ToLower(raw), VirtualKeyPrefix) { + return bifrost.Ptr(raw) + } + }This keeps all header paths aligned and avoids treating
x-bf-vkas a privileged bypass.
291-307: Fix log message: this block is doing unmarshal, not marshal.In the HTTP middleware body handling, the error log on
sonic.Unmarshalstill says "failed to marshal request body", which is misleading for debugging.Suggested fix
- err = sonic.Unmarshal(ctx.Request.Body(), &payload) - if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + err = sonic.Unmarshal(ctx.Request.Body(), &payload) + if err != nil { + p.logger.Error("failed to unmarshal request body to check for virtual key: %v", err)core/bifrost.go (1)
70-71: Harden tracer storage/access to avoid panics from nil or unexpected values.
traceris stored in anatomic.Valueas*tracerWrapper, but bothgetTracer()andShutdown()unconditionally type‑assert:return bifrost.tracer.Load().(*tracerWrapper).tracer // ... if tracerWrapper := bifrost.tracer.Load().(*tracerWrapper); tracerWrapper != nil && tracerWrapper.tracer != nil { tracerWrapper.tracer.Stop() }If
Load()ever returnsnilor a non‑*tracerWrapper(e.g., a misconfigured test or external code constructingBifrostwithout callingInitproperly), these assertions will panic. Earlier feedback on tracer storage raised a similar concern.Consider adding defensive guards both when setting and when reading:
- Ensure
SetTraceralways stores a non‑nil*tracerWrapper.- Make
getTracerandShutdownhandleLoad()==nilor wrong types by falling back toschemas.DefaultTracer()(for reads) and skippingStop()(for shutdown), optionally with a debug/warn log.Illustrative adjustment
func (bifrost *Bifrost) SetTracer(tracer schemas.Tracer) { if tracer == nil { tracer = schemas.DefaultTracer() } - bifrost.tracer.Store(&tracerWrapper{tracer: tracer}) + bifrost.tracer.Store(&tracerWrapper{tracer: tracer}) } func (bifrost *Bifrost) getTracer() schemas.Tracer { - return bifrost.tracer.Load().(*tracerWrapper).tracer + if v := bifrost.tracer.Load(); v != nil { + if tw, ok := v.(*tracerWrapper); ok && tw != nil && tw.tracer != nil { + return tw.tracer + } + } + return schemas.DefaultTracer() } func (bifrost *Bifrost) Shutdown() { // ... - if tracerWrapper := bifrost.tracer.Load().(*tracerWrapper); tracerWrapper != nil && tracerWrapper.tracer != nil { - tracerWrapper.tracer.Stop() - } + if v := bifrost.tracer.Load(); v != nil { + if tw, ok := v.(*tracerWrapper); ok && tw != nil && tw.tracer != nil { + tw.tracer.Stop() + } + } // ... }This keeps the wrapper approach while making the public API resilient to accidental misuse and future refactors around tracer initialization.
Also applies to: 102-107, 118-147, 253-265, 3893-3896
transports/bifrost-http/handlers/devpprof.go (3)
142-150:Stop()can panic if called twice due to closing an already-closed channel.The
Stop()method closesc.stopChwithout protection against double-close. IfStop()is called twice, it will panic. Additionally,Start()afterStop()will fail becausestopChremains closed.🔎 Proposed fix using select to guard the close
func (c *MetricsCollector) Stop() { c.mu.Lock() defer c.mu.Unlock() - if c.started { + if c.started && c.stopCh != nil { + select { + case <-c.stopCh: + // Already closed + default: close(c.stopCh) + } c.started = false + c.stopCh = nil } }
176-190:getCPUSampleuses Unix-onlysyscall.Getrusageand will not compile on Windows.
syscall.Getrusageis only available on Unix platforms. Since the project supports Windows (per platform-specific handling inserver.go), this file needs a build constraint or platform-specific implementation.Add a build constraint at the top of the file:
//go:build !windows // +build !windowsOr create
devpprof_unix.goanddevpprof_windows.gowith appropriate stubs.
370-376:devPprofHandleris created locally and never cleaned up, causing goroutine leaks.In
server.go(line 1022),devPprofHandleris instantiated as a local variable andRegisterRoutesstarts the collector (line 373). Since the handler isn't stored onBifrostHTTPServer, itsCleanup()method is never called during shutdown, leaving the collector goroutine running indefinitely.Store
devPprofHandleras a field inBifrostHTTPServerand callCleanup()in the shutdown sequence.framework/tracing/store.go (1)
13-22: Remove unusedTracerandRequestIDfields fromDeferredSpanInfo.The
Tracer(line 17) andRequestID(line 18) fields are never assigned or used.StoreDeferredSpan()(lines 120-125) only initializesSpanIDandStartTime. In contrast,FirstChunkTimeandAccumulatedChunksare actively used inAppendStreamingChunk().🔎 Proposed fix
type DeferredSpanInfo struct { SpanID string StartTime time.Time - Tracer schemas.Tracer // Reference to tracer for completing the span - RequestID string // Request ID for accumulator lookup FirstChunkTime time.Time // Timestamp of first chunk (for TTFT calculation) AccumulatedChunks []*schemas.BifrostResponse // Accumulated streaming chunks mu sync.Mutex // Mutex for thread-safe chunk accumulation }transports/bifrost-http/server/server.go (1)
959-959: Uses.GetGovernancePluginName()instead of hardcodedgovernance.PluginName.Line 959 uses
governance.PluginNamedirectly, bypassing the enterprise plugin name override mechanism. Lines 510 and 657 correctly uses.GetGovernancePluginName()for the same operation.🔎 Proposed fix
- governancePlugin, _ := FindPluginByName[schemas.Plugin](s.Plugins, governance.PluginName) + governancePlugin, _ := FindPluginByName[schemas.Plugin](s.Plugins, s.GetGovernancePluginName())framework/tracing/tracer.go (1)
393-398: Choice and tool-call indices are not sorted, causing nondeterministic output.The comments say "Sort choice indices for deterministic output" (line 394) and "Sort tool calls by index" (line 433), but no actual sorting is performed. The iteration order over maps in Go is nondeterministic, so the output order will vary between runs.
🔎 Proposed fix - add sorting
+import "sort" // Build final choices from accumulated data // Sort choice indices for deterministic output choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } +sort.Ints(choiceIndices) // ... later ... // Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } +sort.Ints(tcIndices)Also applies to: 433-437
🧹 Nitpick comments (5)
.github/workflows/scripts/release-bifrost-http.sh (1)
266-266: Consider sourcing the PostgreSQL password dynamically.The password
bifrost_passwordis hardcoded in thedocker execcommand. While this is acceptable for test environments, it creates a maintenance burden if the password indocker-compose.ymlchanges.🔎 Suggested improvement
Extract the password from the docker-compose file or environment:
+# Extract PostgreSQL password from docker-compose or environment +POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-bifrost_password}" + docker exec -e PGPASSWORD=bifrost_password "$(docker compose -f "$CONFIGS_DIR/docker-compose.yml" ps -q postgres)" \Or parse it from the compose file:
# Extract from docker-compose.yml POSTGRES_PASSWORD=$(grep -A 5 'POSTGRES_PASSWORD' "$CONFIGS_DIR/docker-compose.yml" | grep -oP ':\s*\K\S+' | head -1) POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-bifrost_password}"framework/streaming/accumulator.go (1)
104-127: ClarifyStartTimestampsemantics and consider trimming redundantIsZerochecks
createStreamAccumulatornow always initializesStartTimestamptotime.Now(), while theadd*StreamChunkmethods still conditionally setStartTimestampwhen it’s zero and separately trackFirstChunkTimestampfor TTFT. In normal flows where accumulators are always created viacreateStreamAccumulator/CreateStreamAccumulator, thoseStartTimestamp.IsZero()branches will rarely, if ever, fire.If accumulators aren’t constructed elsewhere with a zero
StartTimestamp, consider either:
- Removing the
StartTimestamp.IsZero()blocks in theadd*StreamChunkmethods, or- Updating the comment above
createStreamAccumulatorto explicitly describe the “default to now, optionally overridden byCreateStreamAccumulator” behavior and that per-chunk code won’t back-fillStartTimestampanymore.This would make the TTFT/latency behavior easier to reason about and avoid dead code paths.
Also applies to: 129-223
framework/tracing/propagation.go (1)
72-84: ClarifyExtractParentIDdocstring to match what’s returnedThe implementation intentionally returns the trace-id component (
ctx.TraceID) from the incomingtraceparent, not the parent/span ID. The current comment (“extracts the parent trace ID from W3C traceparent header”) can be read as span‑ID semantics; consider rephrasing to something like “extracts the trace ID from the parent request’s W3Ctraceparentheader” to avoid confusion.transports/bifrost-http/handlers/middlewares.go (1)
281-307:completeAndFlushTraceruns in a goroutine without error aggregation or timeout.The trace completion and plugin injection runs asynchronously, which is good for not blocking the response. However:
- If
plugin.Injecthangs, the goroutine will leak- Errors are only logged, with no way to track injection failures across plugins
Consider adding a timeout context for plugin injection:
go func() { ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) defer cancel() // ... existing cleanup code ... for _, plugin := range *m.obsPlugins.Load() { if plugin == nil { continue } if err := plugin.Inject(ctx, completedTrace); err != nil { logger.Warn("observability plugin %s failed to inject trace: %v", plugin.GetName(), err) } } // ... }()core/schemas/tracer.go (1)
9-11:SpanHandleas empty interface limits type safety.Using
interface{}forSpanHandlemeans any value can be passed, which could lead to runtime errors if incorrect types are used. This is intentional for flexibility across implementations but worth noting.Consider documenting the expected handle structure or adding runtime type checks in implementations to fail fast on misuse. The current approach is acceptable given the interface-based design.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sum
📒 Files selected for processing (96)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/versionui/app/clientLayout.tsxui/app/workspace/logs/views/logDetailsSheet.tsxui/components/devProfiler.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsxui/lib/store/apis/devApi.tsui/lib/store/apis/index.ts
💤 Files with no reviewable changes (3)
- framework/configstore/tables/mcp.go
- core/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (1)
- plugins/telemetry/version
🚧 Files skipped from review as they are similar to previous changes (40)
- framework/streaming/types.go
- ui/lib/store/apis/index.ts
- plugins/governance/version
- transports/bifrost-http/handlers/ui.go
- transports/version
- plugins/otel/version
- core/schemas/context.go
- framework/streaming/audio.go
- transports/bifrost-http/handlers/mcp.go
- plugins/mocker/main.go
- framework/version
- plugins/logging/version
- transports/bifrost-http/handlers/health.go
- transports/bifrost-http/handlers/plugins.go
- plugins/mocker/changelog.md
- framework/plugins/dynamicplugin_test.go
- .github/workflows/release-pipeline.yml
- framework/streaming/responses.go
- transports/bifrost-http/handlers/providers.go
- plugins/maxim/changelog.md
- .github/workflows/scripts/push-mintlify-changelog.sh
- plugins/mocker/version
- ui/lib/store/apis/devApi.ts
- ui/components/ui/switch.tsx
- plugins/maxim/main.go
- core/schemas/plugin.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- plugins/logging/utils.go
- docs/plugins/writing-plugin.mdx
- ui/components/ui/sheet.tsx
- ui/app/clientLayout.tsx
- transports/bifrost-http/handlers/websocket.go
- transports/bifrost-http/handlers/config.go
- transports/bifrost-http/handlers/cache.go
- transports/bifrost-http/handlers/logging.go
- plugins/jsonparser/version
- framework/streaming/chat.go
- docs/docs.json
- plugins/telemetry/main.go
- plugins/semanticcache/version
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/handlers/session.goplugins/semanticcache/changelog.mdplugins/jsonparser/changelog.mdtransports/bifrost-http/handlers/inference.goplugins/governance/go.modtransports/changelog.mdplugins/semanticcache/main.goframework/tracing/propagation.gotransports/bifrost-http/lib/config_test.gocore/changelog.mdframework/plugins/dynamicplugin.goplugins/logging/changelog.mdplugins/maxim/versiontransports/bifrost-http/handlers/integrations.gocore/providers/utils/utils.godocs/plugins/getting-started.mdxtransports/bifrost-http/lib/middleware.goplugins/otel/changelog.mdframework/tracing/helpers.gocore/schemas/tracer.goplugins/otel/main.gotransports/bifrost-http/integrations/router.goplugins/telemetry/changelog.mdexamples/plugins/hello-world/go.modtransports/bifrost-http/handlers/devpprof.goframework/tracing/store.gocore/schemas/trace.goui/components/sidebar.tsxtransports/bifrost-http/handlers/mcpserver.goplugins/governance/changelog.mdtransports/bifrost-http/handlers/governance.goplugins/otel/converter.gocore/schemas/bifrost.gotransports/bifrost-http/handlers/middlewares_test.goexamples/plugins/hello-world/main.goframework/tracing/tracer.goplugins/governance/main.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/middlewares.gocore/versionframework/tracing/llmspan.gotransports/bifrost-http/server/server.gocore/bifrost.goplugins/logging/main.goui/components/devProfiler.tsxframework/streaming/transcription.gocore/bifrost_test.godocs/plugins/migration-guide.mdxframework/changelog.mdframework/streaming/accumulator.goplugins/jsonparser/main.go
🧠 Learnings (6)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/inference.goplugins/semanticcache/main.goframework/tracing/propagation.gotransports/bifrost-http/lib/config_test.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/integrations.gocore/providers/utils/utils.gotransports/bifrost-http/lib/middleware.goframework/tracing/helpers.gocore/schemas/tracer.goplugins/otel/main.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/devpprof.goframework/tracing/store.gocore/schemas/trace.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/governance.goplugins/otel/converter.gocore/schemas/bifrost.gotransports/bifrost-http/handlers/middlewares_test.goexamples/plugins/hello-world/main.goframework/tracing/tracer.goplugins/governance/main.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/middlewares.goframework/tracing/llmspan.gotransports/bifrost-http/server/server.gocore/bifrost.goplugins/logging/main.goframework/streaming/transcription.gocore/bifrost_test.goframework/streaming/accumulator.goplugins/jsonparser/main.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/server/server.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
📚 Learning: 2025-12-24T04:36:57.698Z
Learnt from: danpiths
Repo: maximhq/bifrost PR: 1169
File: transports/bifrost-http/handlers/governance.go:1708-1878
Timestamp: 2025-12-24T04:36:57.698Z
Learning: In governance update handlers (e.g., updateModelConfig, updateProviderGovernance), design updates to support clearing individual fields by sending null/empty values (e.g., {"rate_limit": {"token_max_limit": null}} clears token_max_limit). Follow this pattern for partial updates so users can remove specific governance settings without deleting the whole entity. Ensure budget updates follow the same approach using direct field assignment. Review input validation, JSON decoding (e.g., pointers vs values in Go structs), and API documentation to reflect nullable fields and expected behavior.
Applied to files:
transports/bifrost-http/handlers/governance.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
🧬 Code graph analysis (26)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
transports/bifrost-http/lib/config_test.go (1)
transports/bifrost-http/lib/config.go (1)
LoadConfig(276-356)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (2)
GetName(15-17)HTTPTransportMiddleware(19-27)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(25-37)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
transports/bifrost-http/integrations/router.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
transports/bifrost-http/handlers/devpprof.go (4)
ui/lib/store/apis/devApi.ts (6)
MemoryStats(4-10)CPUStats(13-17)RuntimeStats(20-26)AllocationInfo(29-35)HistoryPoint(38-45)PprofData(48-55)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)transports/bifrost-http/handlers/utils.go (1)
SendJSON(16-22)
framework/tracing/store.go (3)
framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
core/schemas/trace.go (2)
framework/tracing/helpers.go (1)
AddSpan(32-38)ui/lib/constants/logs.ts (1)
Status(161-161)
transports/bifrost-http/handlers/mcpserver.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/governance.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (15)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindHTTPRequest(131-131)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanEvent(107-111)
core/schemas/bifrost.go (2)
core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/tracer.go (6)
framework/tracing/store.go (1)
TraceStore(25-37)framework/streaming/accumulator.go (2)
Accumulator(14-30)NewAccumulator(440-480)core/schemas/tracer.go (3)
Tracer(38-116)SpanHandle(11-11)StreamAccumulatorResult(15-33)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (6)
BifrostContextKeySpanID(144-144)BifrostRequest(173-193)BifrostResponse(335-355)BifrostError(474-483)BifrostContextKeyRequestID(124-124)BifrostContextKeyStreamEndIndicator(131-131)core/schemas/context.go (2)
BifrostContext(32-42)NewBifrostContext(47-65)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/llmspan.go (3)
core/schemas/bifrost.go (18)
BifrostRequest(173-193)RequestType(89-89)TextCompletionRequest(93-93)EmbeddingRequest(99-99)TranscriptionRequest(102-102)SpeechRequest(100-100)ResponsesRequest(97-97)BatchCreateRequest(104-104)BatchListRequest(105-105)BatchRetrieveRequest(106-106)BatchCancelRequest(107-107)BatchResultsRequest(108-108)FileUploadRequest(109-109)FileListRequest(110-110)FileRetrieveRequest(111-111)FileDeleteRequest(112-112)FileContentRequest(113-113)BifrostResponse(335-355)core/schemas/trace.go (19)
AttrProviderName(159-159)AttrRequestModel(160-160)AttrError(216-216)AttrErrorType(217-217)AttrErrorCode(218-218)AttrInputMessages(222-222)AttrResponseID(191-191)AttrResponseModel(192-192)AttrObject(197-197)AttrSystemFprint(194-194)AttrCreated(196-196)AttrServiceTier(195-195)AttrOutputMessages(225-225)AttrFinishReason(193-193)AttrPromptTokens(208-208)AttrCompletionTokens(209-209)AttrTotalTokens(210-210)AttrPrompt(181-181)AttrUsageCost(213-213)core/schemas/chatcompletions.go (1)
BifrostChatRequest(12-19)
core/bifrost.go (6)
core/schemas/tracer.go (3)
Tracer(38-116)DefaultTracer(181-183)SpanHandle(11-11)framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/bifrost.go (5)
BifrostRequest(173-193)BifrostContextKeyTracer(146-146)BifrostResponse(335-355)RequestType(89-89)BifrostError(474-483)framework/tracing/helpers.go (1)
EndSpan(50-56)core/utils.go (1)
IsStreamRequestType(197-199)core/schemas/context.go (1)
NewBifrostContextWithTimeout(70-73)
ui/components/devProfiler.tsx (1)
ui/lib/utils/port.ts (1)
isDevelopmentMode(110-112)
framework/streaming/transcription.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
core/bifrost_test.go (1)
core/schemas/tracer.go (1)
DefaultTracer(181-183)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
🪛 markdownlint-cli2 (0.18.1)
transports/changelog.md
16-16: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
plugins/otel/main.go (1)
87-112: Fix OTEL resource attributes parsing:strings.SplitSeq+rangeusage won’t compileTwo problems here:
strings.SplitSeqdoes not exist in the Go stdlib (and no alternative import is present), so this line won’t compile.for attribute := range ...iterates indexes, butstrings.TrimSpace(attribute)expects a string; that’s a type mismatch.This block should use
strings.Splitand capture the value, e.g.:Proposed fix
- if attributes, ok := os.LookupEnv(OTELResponseAttributesEnvKey); ok { - // We will split the attributes by , and then split each attribute by = - for attribute := range strings.SplitSeq(attributes, ",") { - attributeParts := strings.Split(strings.TrimSpace(attribute), "=") + if attributes, ok := os.LookupEnv(OTELResponseAttributesEnvKey); ok { + // We will split the attributes by , and then split each attribute by = + for _, attribute := range strings.Split(attributes, ",") { + attributeParts := strings.Split(strings.TrimSpace(attribute), "=") if len(attributeParts) == 2 { attributesFromEnvironment = append( attributesFromEnvironment, kvStr(strings.TrimSpace(attributeParts[0]), strings.TrimSpace(attributeParts[1])), ) } } }
♻️ Duplicate comments (24)
examples/plugins/hello-world/go.mod (1)
10-22: Update transitive dependency versions to match the plugin stack.The indirect dependencies still use outdated versions compared to the governance, logging, and telemetry plugins:
github.com/klauspost/compressshould be v1.18.2 (line 16, currently v1.18.1)golang.org/x/archshould be v0.23.0 (line 20, currently v0.22.0)golang.org/x/sysshould be v0.39.0 (line 21, currently v0.38.0)Update these versions and run
go mod tidyto ensure consistency across the plugin stack.🔎 Proposed fix
require ( github.com/andybalholm/brotli v1.2.0 // indirect github.com/bytedance/gopkg v0.1.3 // indirect github.com/bytedance/sonic v1.14.1 // indirect github.com/bytedance/sonic/loader v0.3.0 // indirect github.com/cloudwego/base64x v0.1.6 // indirect - github.com/klauspost/compress v1.18.1 // indirect + github.com/klauspost/compress v1.18.2 // indirect github.com/klauspost/cpuid/v2 v2.3.0 // indirect github.com/twitchyliquid64/golang-asm v0.15.1 // indirect github.com/valyala/bytebufferpool v1.0.0 // indirect - golang.org/x/arch v0.22.0 // indirect - golang.org/x/sys v0.38.0 // indirect + golang.org/x/arch v0.23.0 // indirect + golang.org/x/sys v0.39.0 // indirect )Then run:
cd examples/plugins/hello-world && go mod tidytransports/bifrost-http/handlers/devpprof.go (5)
1-15: Missing Unix-only build constraint for syscall.Getrusage usage.The file imports
syscalland usessyscall.Getrusage(line 178), which is only available on Unix platforms. Without build constraints, this code will fail to compile on Windows. As noted in previous reviews, the project supports Windows (evidenced byserver.go).This is a duplicate of the previous review comment on lines 176-190, but the issue has not been addressed.
Add build constraints at the top of the file:
+//go:build !windows +// +build !windows + package handlersOr split into platform-specific files (
devpprof_unix.gowith the constraint anddevpprof_windows.gowith a Windows-compatible implementation).
142-150: Stop() doesn't recreate stopCh, breaking Start() after Stop().This is a duplicate of the previous review comment. The issue persists:
Stop()closesstopChbut never recreates it. IfStart()is called afterStop(), the goroutine will exit immediately becausestopChis already closed.Apply the previously suggested fix to make
Stop()idempotent and allowStart()afterStop():🔎 Proposed fix
func (c *MetricsCollector) Stop() { c.mu.Lock() defer c.mu.Unlock() - if c.started { + if c.started && c.stopCh != nil { + select { + case <-c.stopCh: + // Already closed + default: close(c.stopCh) + } c.started = false + c.stopCh = nil } }And in
Start(), recreate the channel:func (c *MetricsCollector) Start() { c.mu.Lock() if c.started { c.mu.Unlock() return } + if c.stopCh == nil { + c.stopCh = make(chan struct{}) + } c.started = true c.mu.Unlock() go c.collectLoop() }
175-190: Unix-specific syscall usage breaks Windows compilation.This is a duplicate of the previous review comment. The issue persists:
syscall.Getrusageis only available on Unix platforms and will cause compilation failures on Windows.This was already flagged in the review of lines 1-15. Apply the build constraint solution mentioned there.
270-368: getTopAllocations provides misleading placeholder data.This is a duplicate of the previous review comment on lines 293-311. The issue persists: the function claims to return "top allocation sites" but uses
runtime.Caller(which captures the current call stack, not allocation hotspots) and arbitrary divisions of heap statistics.As previously suggested, either:
- Document clearly that this is placeholder/approximated data
- Use
runtime/pprofto parse real heap profile data- Remove
top_allocationsuntil accurate data can be providedThe current implementation may confuse users who expect real profiling data.
370-376: Collector started but never stopped, causing goroutine leak.This is a duplicate of the previous review comment on lines 370-376. The issue persists:
RegisterRoutesstarts the collector goroutine, but since the handler is created as a local variable inserver.go, itsCleanup()method is never called during server shutdown, leaking the background goroutine.As previously suggested, store
devPprofHandleras a field inBifrostHTTPServerand call itsCleanup()method in the shutdown sequence (around line 1330-1348 inserver.go).transports/changelog.md (1)
16-23: Add language specifier to fenced code block.The code block is missing a language identifier, which prevents syntax highlighting and violates MD040. This was flagged in a previous review but remains unaddressed.
🔎 Add language specifier
**Migration summary:** - ``` + ```go // v1.3.x (removed) TransportInterceptor(ctx *BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error)As per static analysis hints.
core/providers/utils/utils.go (1)
1424-1513: RefactorcompleteDeferredSpanto takecontext.Contextby value instead of*context.Context
completeDeferredSpanonly reads from the context (trace ID, tracer, post‑hook span finalizer) and does not need to mutate it. Passing*context.Contexthere is unconventional in Go and unnecessary, since the localctxvalue in the callers is already being updated via the pointer passed topostHookRunner.You can simplify and align with Go context best practices by changing the helper to take
ctx context.Contextand updating the call sites accordingly.Proposed refactor
-// completeDeferredSpan completes the deferred LLM span for streaming requests. -// This is called when the final chunk is processed (when StreamEndIndicator is true). -// It retrieves the deferred span handle from TraceStore using the trace ID from context, -// populates response attributes from accumulated chunks, and ends the span. -func completeDeferredSpan(ctx *context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { - if ctx == nil { - return - } +// completeDeferredSpan completes the deferred LLM span for streaming requests. +// This is called when the final chunk is processed (when StreamEndIndicator is true). +// It retrieves the deferred span handle from TraceStore using the trace ID from context, +// populates response attributes from accumulated chunks, and ends the span. +func completeDeferredSpan(ctx context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { + if ctx == nil { + return + } // Get the trace ID from context (this IS available in the provider's goroutine) - traceID, ok := (*ctx).Value(schemas.BifrostContextKeyTraceID).(string) + traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string) @@ - tracerVal := (*ctx).Value(schemas.BifrostContextKeyTracer) + tracerVal := ctx.Value(schemas.BifrostContextKeyTracer) @@ - if finalizer, ok := (*ctx).Value(schemas.BifrostContextKeyPostHookSpanFinalizer).(func(context.Context)); ok && finalizer != nil { + if finalizer, ok := ctx.Value(schemas.BifrostContextKeyPostHookSpanFinalizer).(func(context.Context)); ok && finalizer != nil { @@ - finalizerCtx := context.WithValue(*ctx, schemas.BifrostContextKeySpanID, spanID) + finalizerCtx := context.WithValue(ctx, schemas.BifrostContextKeySpanID, spanID) finalizer(finalizerCtx) } else { - finalizer(*ctx) + finalizer(ctx) } } @@ - tracer.ClearDeferredSpan(traceID) + tracer.ClearDeferredSpan(traceID) }And adjust the call sites:
- completeDeferredSpan(&ctx, processedResponse, processedError) + completeDeferredSpan(ctx, processedResponse, processedError)(similarly for all other
completeDeferredSpan(&ctx, ...)calls).In Go, what are the recommended conventions for passing `context.Context`—by value or by pointer—and are there any official guidelines discouraging `*context.Context` parameters?plugins/jsonparser/version (1)
1-1: Re‑check version bump level vs documented breaking changesThe version is bumped to
1.3.53even though the plugin API changed fromTransportInterceptortoHTTPTransportMiddleware, which is called out as breaking in docs. If you intend to follow semver for this plugin, consider promoting this to at least a minor bump (e.g.,1.4.0) for clarity, and keep it consistent with other plugins in this stack.#!/bin/bash # Compare version files across plugins to ensure consistent bump strategy for the middleware v2 change. fd --type f 'version$' plugins | xargs -I{} sh -c 'echo "{}: $(cat "{}")'docs/plugins/migration-guide.mdx (1)
247-250: Version reference issue (duplicate concern).Line 249 references
@v1.4.0, but as noted in previous review comments, this tag does not exist in the repository yet. Before merging, either:
- Create the v1.4.0 release tag, or
- Update the documentation to reference an existing version tag, or
- Use a commit hash instead
This ensures the
go getcommand works when users follow the guide.framework/changelog.md (1)
1-1: Fix grammar error in the tracing feature entry.The phrase "to enabling e2e tracing" should be "to enable e2e tracing".
🔎 Proposed fix
-- feat: adds new tracing framework for allowing plugins to enabling e2e tracing +- feat: adds new tracing framework for allowing plugins to enable e2e tracingexamples/plugins/hello-world/main.go (1)
19-27: Context value propagation is broken between middleware and hooks.The middleware sets a value on
fasthttp.RequestCtxviaSetUserValue, butPreHook(line 30) andPostHook(line 39) read fromschemas.BifrostContextusingctx.Value(). These are different context types, so the value will always benilin the hooks.Either propagate the value through the Bifrost context (if the framework supports extracting it from
RequestCtxuser values), or remove the context reads from the hooks if the data flow demonstration is no longer needed.transports/bifrost-http/integrations/router.go (1)
508-508: Non-streaming path uses rawctxinstead of enrichedbifrostCtx.The non-streaming client calls pass
ctx(*fasthttp.RequestCtx) while the comment at line 503 says "Use the cancellable context from ConvertToBifrostContext". This inconsistency was flagged in a previous review. The streaming and batch/file handlers correctly use the Bifrost context, but non-streaming inference calls do not.Also applies to: 514-514, 534-534, 557-557, 580-580, 603-603, 626-626, 660-660
plugins/governance/main.go (1)
292-297: Fix error message: "marshal" should be "unmarshal".Line 294 logs "failed to marshal" but the operation on line 292 is
sonic.Unmarshal. The error message should reflect the actual operation.🔎 Proposed fix
err = sonic.Unmarshal(ctx.Request.Body(), &payload) if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body: %v", err) next(ctx) return }framework/tracing/store.go (1)
13-177: TraceStore implementation and streaming accumulation look correct; consider trimming unused fieldsThe TraceStore’s responsibilities — pooled trace/span allocation, TTL-based cleanup via ticker +
stopOnce, deferred span storage, and TTFT calculation fromStartTime/FirstChunkTime— are implemented in a thread-safe and straightforward way. TheStop()fix withsync.Oncealso avoids double-close panics.One minor nit: in
DeferredSpanInfo, theTracerandRequestIDfields are still never written or read anywhere in this file. Unless other packages rely on them (none were found in prior analysis), you can safely remove them to reduce confusion and per-entry footprint.Also applies to: 244-371
plugins/otel/converter.go (1)
148-171: Guarduint64→int64cast inanyToKeyValueto avoid overflowThe
case uint64branch does a directint64(v)cast, which will overflow for values larger thanmath.MaxInt64. Even if unlikely, this can silently corrupt attributes for large counters or IDs.Consider clamping or logging when overflow would occur, e.g.:
Proposed fix
+import "math" @@ func anyToKeyValue(key string, value any) *KeyValue { - case uint64: - return kvInt(key, int64(v)) + case uint64: + if v > math.MaxInt64 { + // Optionally: log here via plugin logger + return kvInt(key, math.MaxInt64) + } + return kvInt(key, int64(v))transports/bifrost-http/handlers/middlewares.go (1)
191-279: FixTracingMiddlewareto pass acontext.ContextintoStartSpan(currently uses*fasthttp.RequestCtx)
TracingMiddleware.Middlewarecurrently does:spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)Here
ctxis*fasthttp.RequestCtx, butStartSpanis defined on the tracer as:StartSpan(ctx context.Context, name string, kind SpanKind) (context.Context, SpanHandle)
*fasthttp.RequestCtxdoes not implementcontext.Context, so this will not compile and also breaks theGetTraceID-based lookup that expects a realcontext.Context.A safe pattern is to build a proper
context.Contextseeded with the new trace ID and use that for the span, while still storing the trace ID on the fasthttp ctx for downstream handlers:Proposed fix
@@ func (m *TracingMiddleware) Middleware() schemas.BifrostHTTPMiddleware { - // Extract parent trace ID from W3C headers (if present) - parentID := tracing.ExtractParentID(&ctx.Request.Header) - // Create trace in store - only ID returned (trace data stays in store) - traceID := m.tracer.Load().CreateTrace(parentID) - // Only trace ID goes into context (lightweight, no bloat) - ctx.SetUserValue(schemas.BifrostContextKeyTraceID, traceID) + // Extract parent trace ID from W3C headers (if present) + parentID := tracing.ExtractParentID(&ctx.Request.Header) + // Create trace in store - only ID returned (trace data stays in store) + traceID := m.tracer.Load().CreateTrace(parentID) + // Store trace ID on fasthttp context for downstream handlers + ctx.SetUserValue(schemas.BifrostContextKeyTraceID, traceID) @@ - // Create root span for the HTTP request - spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) + // Create root span for the HTTP request using a proper context.Context + spanBaseCtx := context.WithValue(context.Background(), schemas.BifrostContextKeyTraceID, traceID) + tr := m.tracer.Load() + spanCtx, rootSpan := tr.StartSpan(spanBaseCtx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) @@ - if spanID, ok := spanCtx.Value(schemas.BifrostContextKeySpanID).(string); ok { + if spanID, ok := spanCtx.Value(schemas.BifrostContextKeySpanID).(string); ok { ctx.SetUserValue(schemas.BifrostContextKeySpanID, spanID) } }Also consider caching
tr := m.tracer.Load()once at the top of the handler and reusing it to avoid repeated atomic loads and to simplify nil checks.#!/bin/bash # Verify Tracer.StartSpan signature and ensure it expects context.Context rg -n "type Tracer interface" core/schemas/tracer.go -n -A10 rg -n "func (t \*Tracer) StartSpan" framework/tracing/tracer.go -n -A5Also applies to: 281-307
framework/tracing/llmspan.go (1)
190-193: ExtraParams keys may collide with standard attribute names.User-provided keys from
ExtraParamsare written directly to the attributes map without namespacing. If a user sets a key like"gen_ai.request.model"in ExtraParams, it would overwrite the standard attribute set earlier in the function.This pattern repeats throughout the file at lines 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 768-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, and 1027-1029.
🔎 Suggested approach
Consider namespacing ExtraParams keys to prevent collisions:
// ExtraParams for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) + attrs["gen_ai.extra."+k] = fmt.Sprintf("%v", v) }Apply this pattern consistently across all ExtraParams handling locations.
core/schemas/trace.go (3)
40-49:Trace.Reset()lacks mutex protection.This method modifies all fields including
Spanswithout holding the mutex, creating a potential data race ifResetis called while another goroutine is accessing the trace (e.g., viaAddSpanorGetSpan).
84-89:Span.End()method lacks mutex protection.Unlike
SetAttributeandAddEvent, theEndmethod modifiesEndTime,Status, andStatusMsgwithout holding the mutex. IfEndis called concurrently with reads or writes to these fields, a data race may occur.
91-104:Span.Reset()lacks mutex protection.Similar to
Trace.Reset(), this method modifies fields without holding the mutex. If reused from a pool while another goroutine still holds a reference, a data race could occur.transports/bifrost-http/server/server.go (2)
1021-1026: Dev pprof handler created locally and never cleaned up.When
IsDevMode()is true,devPprofHandleris created locally and itsRegisterRoutesstarts the collector goroutine. Since the handler isn't stored on the server,Cleanup()is never called during shutdown, causing a resource leak.
1243-1252: Add TraceStore cleanup during server shutdown.The
TraceStoreis initialized with a cleanup goroutine (line 1245) but is never cleaned up during shutdown. ThetraceStorevariable is local-scoped and the underlyingTraceStore.Stop()method is not called. This cleanup goroutine and its associated ticker will continue running indefinitely, causing a resource leak.Store
traceStorein the server struct and callStop()in the shutdown sequence (lines 1299-1334), or add a public cleanup method toTracerthat callsstore.Stop()and invoke it fromBifrost.Shutdown().framework/tracing/tracer.go (1)
394-442: Choice and tool-call indices are not sorted, causing non-deterministic output.Lines 394-398 build
choiceIndicesfrom map keys with a comment "Sort choice indices for deterministic output" but no sorting is performed. Similarly, lines 434-437 buildtcIndiceswith a comment "Sort tool calls by index" but again no sorting occurs.Map iteration order in Go is random, so the output order of choices and tool calls will vary across runs, making traces inconsistent.
🔎 Proposed fix
+ sort.Ints(choiceIndices) + for _, idx := range choiceIndices { accum := choiceMap[idx] // Build message msg := &schemas.ChatMessage{ Role: accum.role, } // Set content if accum.content != "" { msg.Content = &schemas.ChatMessageContent{ ContentStr: &accum.content, } } // Build assistant message fields if accum.refusal != "" || accum.reasoning != "" || len(accum.reasoningDetails) > 0 || accum.audio != nil || len(accum.toolCalls) > 0 { msg.ChatAssistantMessage = &schemas.ChatAssistantMessage{} if accum.refusal != "" { msg.ChatAssistantMessage.Refusal = &accum.refusal } if accum.reasoning != "" { msg.ChatAssistantMessage.Reasoning = &accum.reasoning } if len(accum.reasoningDetails) > 0 { msg.ChatAssistantMessage.ReasoningDetails = accum.reasoningDetails } if accum.audio != nil { msg.ChatAssistantMessage.Audio = accum.audio } if len(accum.toolCalls) > 0 { // Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } + sort.Ints(tcIndices) toolCalls := make([]schemas.ChatAssistantMessageToolCall, 0, len(accum.toolCalls)) for _, tcIdx := range tcIndices { toolCalls = append(toolCalls, accum.toolCalls[tcIdx]) }
🧹 Nitpick comments (4)
ui/lib/store/apis/devApi.ts (1)
59-59: Clarify polling configuration in comment.The comment "polls every 10 seconds" is misleading—the endpoint itself doesn't configure polling. The polling interval is set by the consumer when calling the hook (e.g.,
useGetDevPprofQuery({}, { pollingInterval: 10000 })).🔎 Suggested comment clarification
- // Get dev pprof data - polls every 10 seconds + // Get dev pprof data - consumers can configure polling via pollingInterval option getDevPprof: builder.query<PprofData, void>({plugins/logging/changelog.md (1)
1-19: Changelog clearly documents the HTTPTransportMiddleware breaking changeThe description of the old vs new signatures and the required core/framework versions is accurate and aligned with the updated
Plugininterface. If you expect some consumers to have calledTransportInterceptordirectly, you might soften “no code changes are required” to clarify it applies to users configuring the plugin via Bifrost rather than invoking its methods directly.framework/tracing/propagation.go (1)
150-162: Add validation fortraceFlagsparameter.
FormatTraceparentdefaults emptytraceFlagsto"00"but doesn't validate non-empty inputs. Per W3C Trace Context spec, trace flags must be exactly 2 hex characters. InvalidtraceFlagsvalues will produce non-compliant headers.🔎 Proposed fix
func FormatTraceparent(traceID, spanID, traceFlags string) string { normalizedTraceID := normalizeTraceID(traceID) normalizedSpanID := normalizeSpanID(spanID) if normalizedTraceID == "" || normalizedSpanID == "" { return "" } - if traceFlags == "" { - traceFlags = "00" // Default: not sampled + // Validate and normalize traceFlags + if traceFlags == "" || len(traceFlags) != 2 || !isHex(traceFlags) { + traceFlags = "00" // Default: not sampled (or invalid input) } + traceFlags = strings.ToLower(traceFlags) return "00-" + normalizedTraceID + "-" + normalizedSpanID + "-" + traceFlags }core/bifrost.go (1)
70-82: Tracer storage viaatomic.Value+tracerWrapperis consistent; consider minor defensive checksWiring the tracer as:
tracerfield usingatomic.Value,- storing a single concrete type
*tracerWrapper,- defaulting to
schemas.DefaultTracer()when unset,is a clean way to avoid
atomic.Value’s concrete-type panics and ensure a non-nil tracer implementation is always available.SetTracerandgetTracer()are consistent with that pattern, andShutdownnow correctly callsStop()on the underlying tracer.If you expect
Bifrostinstances to ever be constructed outsideInit, you might defensively:
- Guard
bifrost.tracer.Load()againstnilingetTracer()/Shutdown, and- Log when falling back to
DefaultTracer().As written, it’s fine for the current Init-only construction flow.
Also applies to: 102-147, 253-265, 3893-3896
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sum
📒 Files selected for processing (96)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/versionui/app/clientLayout.tsxui/app/workspace/logs/views/logDetailsSheet.tsxui/components/devProfiler.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsxui/lib/store/apis/devApi.tsui/lib/store/apis/index.ts
💤 Files with no reviewable changes (3)
- core/mcp.go
- framework/configstore/tables/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (2)
- plugins/otel/version
- framework/version
🚧 Files skipped from review as they are similar to previous changes (35)
- .github/workflows/release-pipeline.yml
- .github/workflows/scripts/push-mintlify-changelog.sh
- framework/streaming/audio.go
- transports/bifrost-http/handlers/session.go
- transports/bifrost-http/handlers/mcp.go
- framework/streaming/types.go
- transports/bifrost-http/handlers/health.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- transports/bifrost-http/handlers/websocket.go
- plugins/governance/version
- transports/bifrost-http/handlers/governance.go
- transports/bifrost-http/handlers/providers.go
- transports/bifrost-http/handlers/config.go
- transports/bifrost-http/handlers/plugins.go
- plugins/semanticcache/version
- docs/docs.json
- framework/tracing/helpers.go
- framework/plugins/dynamicplugin_test.go
- plugins/telemetry/main.go
- plugins/telemetry/version
- core/schemas/bifrost.go
- core/version
- ui/components/ui/sheet.tsx
- .github/workflows/scripts/release-bifrost-http.sh
- plugins/mocker/version
- transports/bifrost-http/lib/config_test.go
- ui/components/devProfiler.tsx
- plugins/governance/go.mod
- ui/components/sidebar.tsx
- plugins/maxim/version
- plugins/jsonparser/main.go
- framework/streaming/chat.go
- ui/components/ui/switch.tsx
- transports/bifrost-http/handlers/ui.go
- plugins/telemetry/changelog.md
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
ui/lib/store/apis/index.tsplugins/mocker/main.gotransports/bifrost-http/handlers/cache.goplugins/logging/versiontransports/changelog.mdtransports/bifrost-http/integrations/router.godocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxplugins/semanticcache/main.goplugins/otel/converter.gotransports/bifrost-http/handlers/devpprof.goplugins/maxim/changelog.mdplugins/jsonparser/changelog.mdtransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.goframework/tracing/store.gocore/providers/utils/utils.gocore/schemas/tracer.gotransports/bifrost-http/handlers/middlewares_test.goui/app/clientLayout.tsxplugins/semanticcache/changelog.mdframework/plugins/dynamicplugin.goplugins/jsonparser/versionplugins/logging/main.goplugins/governance/main.goframework/streaming/transcription.gocore/changelog.mdtransports/bifrost-http/handlers/inference.goplugins/logging/utils.goframework/tracing/propagation.goui/lib/store/apis/devApi.tsdocs/plugins/writing-plugin.mdxplugins/governance/changelog.mdframework/streaming/accumulator.goplugins/mocker/changelog.mdplugins/otel/changelog.mdtransports/bifrost-http/handlers/integrations.gocore/bifrost.gotransports/bifrost-http/handlers/logging.goexamples/plugins/hello-world/main.gocore/schemas/trace.gocore/bifrost_test.gotransports/bifrost-http/handlers/mcpserver.goframework/tracing/tracer.goexamples/plugins/hello-world/go.modtransports/bifrost-http/lib/config.goplugins/otel/main.goplugins/logging/changelog.mdtransports/versioncore/schemas/plugin.gocore/schemas/context.goplugins/maxim/main.goframework/tracing/llmspan.goframework/streaming/responses.goframework/changelog.mdtransports/bifrost-http/server/server.go
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
plugins/mocker/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/integrations/router.goplugins/semanticcache/main.goplugins/otel/converter.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.goframework/tracing/store.gocore/providers/utils/utils.gocore/schemas/tracer.gotransports/bifrost-http/handlers/middlewares_test.goframework/plugins/dynamicplugin.goplugins/logging/main.goplugins/governance/main.goframework/streaming/transcription.gotransports/bifrost-http/handlers/inference.goplugins/logging/utils.goframework/tracing/propagation.goframework/streaming/accumulator.gotransports/bifrost-http/handlers/integrations.gocore/bifrost.gotransports/bifrost-http/handlers/logging.goexamples/plugins/hello-world/main.gocore/schemas/trace.gocore/bifrost_test.gotransports/bifrost-http/handlers/mcpserver.goframework/tracing/tracer.gotransports/bifrost-http/lib/config.goplugins/otel/main.gocore/schemas/plugin.gocore/schemas/context.goplugins/maxim/main.goframework/tracing/llmspan.goframework/streaming/responses.gotransports/bifrost-http/server/server.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/cache.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/server/server.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
🧬 Code graph analysis (28)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/cache.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
plugins/otel/converter.go (1)
core/schemas/trace.go (16)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
transports/bifrost-http/handlers/devpprof.go (4)
ui/lib/store/apis/devApi.ts (6)
MemoryStats(4-10)CPUStats(13-17)RuntimeStats(20-26)AllocationInfo(29-35)HistoryPoint(38-45)PprofData(48-55)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)transports/bifrost-http/handlers/utils.go (1)
SendJSON(16-22)
transports/bifrost-http/handlers/middlewares.go (7)
core/schemas/plugin.go (3)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)Plugin(71-97)core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)framework/tracing/propagation.go (1)
ExtractParentID(74-84)core/schemas/bifrost.go (4)
BifrostContextKeyTraceID(143-143)BifrostContextKeyTraceCompleter(148-148)BifrostContextKeySpanID(144-144)BifrostContextKeyDeferTraceCompletion(147-147)core/schemas/trace.go (3)
SpanKindHTTPRequest(131-131)SpanStatusError(151-151)SpanStatusOk(149-149)framework/tracing/helpers.go (1)
EndSpan(50-56)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/store.go (5)
core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)core/schemas/bifrost.go (1)
BifrostResponse(335-355)core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
core/schemas/tracer.go (6)
core/schemas/bifrost.go (4)
ModelProvider(33-33)BifrostError(474-483)BifrostRequest(173-193)BifrostResponse(335-355)core/schemas/chatcompletions.go (1)
BifrostLLMUsage(845-852)core/schemas/speech.go (1)
BifrostSpeechResponse(22-29)core/schemas/transcriptions.go (1)
BifrostTranscriptionResponse(16-26)core/schemas/trace.go (3)
Trace(10-19)SpanKind(115-115)SpanStatus(143-143)core/schemas/context.go (1)
BifrostContext(32-42)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
ui/app/clientLayout.tsx (1)
ui/components/devProfiler.tsx (1)
DevProfiler(55-422)
plugins/logging/main.go (5)
core/utils.go (1)
IsStreamRequestType(197-199)core/schemas/bifrost.go (3)
RequestType(89-89)BifrostContextKeyTracer(146-146)BifrostContextKeyTraceID(143-143)core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/tracer.go (1)
Tracer(17-20)framework/streaming/types.go (1)
ProcessedStreamResponse(118-126)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/transcription.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
ui/lib/store/apis/devApi.ts (1)
transports/bifrost-http/handlers/devpprof.go (6)
MemoryStats(27-33)CPUStats(36-40)RuntimeStats(43-49)AllocationInfo(52-58)HistoryPoint(61-68)PprofData(71-78)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
core/bifrost_test.go (1)
core/schemas/tracer.go (1)
DefaultTracer(181-183)
transports/bifrost-http/handlers/mcpserver.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/main.go (2)
core/schemas/plugin.go (1)
ObservabilityPlugin(123-137)core/schemas/trace.go (1)
Trace(10-19)
core/schemas/plugin.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/trace.go (1)
Trace(10-19)
core/schemas/context.go (1)
core/schemas/bifrost.go (1)
BifrostContextKeyDeferTraceCompletion(147-147)
plugins/maxim/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
framework/streaming/responses.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/server/server.go (8)
core/schemas/plugin.go (2)
Plugin(71-97)ObservabilityPlugin(123-137)plugins/governance/main.go (1)
PluginName(23-23)plugins/logging/main.go (1)
PluginName(22-22)plugins/telemetry/main.go (1)
PluginName(21-21)transports/bifrost-http/handlers/devpprof.go (2)
IsDevMode(107-109)NewDevPprofHandler(123-127)transports/bifrost-http/handlers/middlewares.go (1)
NewTracingMiddleware(207-215)framework/tracing/store.go (1)
NewTraceStore(40-67)framework/tracing/tracer.go (1)
NewTracer(24-29)
🪛 markdownlint-cli2 (0.18.1)
transports/changelog.md
16-16: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
e8ca91d to
5186ad3
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (18)
plugins/jsonparser/version (1)
1-1: Semver violation remains unresolved; stack targets v1.4.0.The version
1.3.53still represents a patch increment despite the breaking change (TransportInterceptor → HTTPTransportMiddleware) documented in the changelog. This was previously flagged and remains unaddressed.Given that this PR stack targets the v1.4.0 branch and introduces framework-wide breaking changes across multiple plugins, consider whether:
- The jsonparser plugin version should align with the Bifrost v1.4.0 release version, or
- Individual plugin versions should independently reflect their breaking changes (e.g., bump to 2.0.0 or align with 1.4.0)
Verify the project's plugin versioning strategy across the entire PR stack (#1170, #1169, #1121, #1120, #1119, #888) to ensure consistency.
ui/components/ui/sheet.tsx (2)
103-103: Past review suggestion remains unaddressed.The past review suggested moving
mb-6to the false branch of the ternary to avoid applying it whenexpandableis true (wherep-0could conflict). The current implementation still appliesmb-6unconditionally. If this deviation from the suggested fix is intentional, consider adding a brief comment explaining the decision.🔎 Original suggested fix for reference
- <div data-slot="sheet-header" className={cn("mb-6 flex items-center", sheetContext?.expandable ? "p-0" : "")} {...props}> + <div data-slot="sheet-header" className={cn("flex items-center", sheetContext?.expandable ? "p-0" : "mb-6")} {...props}>
114-114: Consider making left margin conditional on button presence.The unconditional
ml-3creates left spacing even when the expand/collapse button isn't rendered (i.e., whenexpandableis false orsideis not "right"). If consistent spacing is intentional regardless of button presence, the current implementation is fine. Otherwise, consider the conditional approach from the past review.🔎 Original suggested fix for reference
- <div className={cn("flex h-full min-w-0 flex-1 flex-row items-center ml-3", className)}>{children}</div> + <div className={cn("flex h-full min-w-0 flex-1 flex-row items-center", sheetContext?.expandable && sheetContext?.side === "right" ? "ml-3" : "", className)}>{children}</div>plugins/otel/changelog.md (1)
19-19: Verify the migration guide file exists in this PR.The changelog references
/docs/plugins/migration-guide. Based on the AI summary,docs/plugins/migration-guide.mdxappears to be added in this PR. Ensure this file is included and the link resolves correctly after merge.#!/bin/bash # Verify migration guide file exists fd "migration-guide" docs/transports/changelog.md (2)
16-23: Version reference inconsistency with plugin changelogs.The migration summary states
v1.3.x (removed)andv1.4.x+ (new), but the plugin changelogs (maxim, jsonparser, semanticcache) consistently referencecore v1.3.0+for the new API. This inconsistency could confuse developers about which version to target for migration.Consider aligning with the plugin changelogs: change
v1.4.x+ (new)tov1.3.0+ (new)and adjustv1.3.x (removed)to the prior version series (e.g.,v1.2.x (removed)).
16-16: Add language specifier to fenced code block.The fenced code block lacks a language identifier, preventing proper syntax highlighting and violating MD040. Add
textorgoafter the opening backticks:🔎 Suggested fix
**Migration summary:** - ``` + ```text // v1.3.x (removed)framework/changelog.md (1)
1-1: Fix grammar and expand tracing framework description.Line 1 contains a grammar error: "to enabling e2e tracing" should be "to enable e2e tracing". Additionally, consider expanding this entry to briefly explain what the tracing framework provides to plugins (e.g., "enables plugins to report spans and traces for distributed tracing integration").
core/providers/utils/utils.go (1)
876-931: Streaming span completion logic looks correct; consider future cleanup of pointer‑context pattern.The updated
ProcessAndSendResponse/ProcessAndSendBifrostErrornow:
- Record chunks into the tracer before post‑hooks.
- Run
postHookRunnerfirst socompleteDeferredSpansees post‑processed data.- Complete the deferred span both on normal final chunks and on skip‑stream control, using the processed response/error.
- Use
tracer.GetAccumulatedChunksto populate response attributes, TTFT, and total chunks, then finalize plugin post‑hook spans before ending thellm.callspan.
ProviderSendsDoneMarkernow treating HuggingFace as non‑[DONE] is consistent with prior guidance for providers that terminate without the sentinel. Based on learnings, this is the desired behavior.The remaining use of
*context.ContextincompleteDeferredSpanfollows the existing PostHookRunner pattern; if you revisit that API later, passingcontext.Contextby value would better align with Go conventions, but it’s not blocking for this change.Also applies to: 938-981, 1424-1513
transports/bifrost-http/integrations/router.go (1)
502-685: Use the converted Bifrost context in non‑streaming client calls, not the HTTP request context.
handleNonStreamingRequeststill passes the raw*fasthttp.RequestCtx(ctx) into Bifrost client methods:listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, ...) textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, ...) chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, ...) responsesResponse, bifrostErr := g.client.ResponsesRequest(ctx, ...) embeddingResponse, bifrostErr := g.client.EmbeddingRequest(ctx, ...) speechResponse, bifrostErr := g.client.SpeechRequest(ctx, ...) transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(ctx, ...)Yet the comment above and the rest of this file show the intent is to use the cancellable context from
ConvertToBifrostContext:
- Streaming path correctly uses
streamCtx := *bifrostCtxfor all*StreamRequestcalls.- Batch and file handlers use
requestCtx := *bifrostCtxfor their Bifrost calls.Passing
ctxhere bypasses the enriched Bifrost context (trace IDs, plugin values, timeouts), so non‑streaming requests won’t participate fully in tracing/governance/plugin flows and may not respect cancellation semantics.Consider updating these calls to use
*bifrostCtxinstead:- listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) + listModelsResponse, bifrostErr := g.client.ListModelsRequest(*bifrostCtx, bifrostReq.ListModelsRequest) - textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, bifrostReq.TextCompletionRequest) + textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(*bifrostCtx, bifrostReq.TextCompletionRequest) - chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, bifrostReq.ChatRequest) + chatResponse, bifrostErr := g.client.ChatCompletionRequest(*bifrostCtx, bifrostReq.ChatRequest) - responsesResponse, bifrostErr := g.client.ResponsesRequest(ctx, bifrostReq.ResponsesRequest) + responsesResponse, bifrostErr := g.client.ResponsesRequest(*bifrostCtx, bifrostReq.ResponsesRequest) - embeddingResponse, bifrostErr := g.client.EmbeddingRequest(ctx, bifrostReq.EmbeddingRequest) + embeddingResponse, bifrostErr := g.client.EmbeddingRequest(*bifrostCtx, bifrostReq.EmbeddingRequest) - speechResponse, bifrostErr := g.client.SpeechRequest(ctx, bifrostReq.SpeechRequest) + speechResponse, bifrostErr := g.client.SpeechRequest(*bifrostCtx, bifrostReq.SpeechRequest) - transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(ctx, bifrostReq.TranscriptionRequest) + transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(*bifrostCtx, bifrostReq.TranscriptionRequest)This keeps non‑streaming behavior consistent with streaming, batch, and file flows, and ensures all observability/governance context travels through Bifrost.
framework/tracing/propagation.go (1)
10-29: W3C trace context helpers are well‑structured and align with the intended propagation semantics.
normalizeTraceID/normalizeSpanIDenforce hex formatting and lengths, handling UUID inputs cleanly.ParseTraceparentandFormatTraceparentcorrectly validate IDs and now normalizetraceFlags, defaulting to"00"when invalid, which avoids emitting malformed headers.ExtractParentIDintentionally returns the parsed trace ID from the incomingtraceparentfor trace continuation, which matches the stored guidance for this file.Overall this module provides a solid basis for W3C trace context handling in the tracing stack.
Also applies to: 58-84, 104-143, 145-182
plugins/governance/main.go (1)
233-261: Align virtual‑key validation across headers and fix misleading error logs in HTTPTransportMiddleware.Two issues here:
x-bf-vk is not validated against
VirtualKeyPrefix.
parseVirtualKeyreturnsx-bf-vkdirectly if non‑empty, without checking that it starts withVirtualKeyPrefix(unlike the Authorization / x-api-key / x-goog-api-key branches).- This makes VK validation inconsistent and can let malformed values bypass the usual prefix check, affecting governance enforcement and store lookups.
Consider tightening this branch to mirror the other header sources:
func parseVirtualKey(ctx *fasthttp.RequestCtx) *string { var virtualKeyValue string
vkHeader := ctx.Request.Header.Peek("x-bf-vk")if string(vkHeader) != "" {return bifrost.Ptr(string(vkHeader))}
vkHeader := string(ctx.Request.Header.Peek("x-bf-vk"))if vkHeader != "" {v := strings.TrimSpace(vkHeader)if strings.HasPrefix(strings.ToLower(v), VirtualKeyPrefix) {return bifrost.Ptr(v)}} // ... existing Authorization / x-api-key / x-goog-api-key logic ...
Unmarshal log message says "marshal".
In
HTTPTransportMiddleware, the first error log in this block:err = sonic.Unmarshal(ctx.Request.Body(), &payload) if err != nil { p.logger.Error("failed to marshal request body to check for virtual key: %v", err) }is performing an unmarshal, not a marshal. This was already raised earlier; the wording is still misleading.
A minimal fix:
- p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body to check for virtual key: %v", err)The rest of the middleware (MCP tool header injection and provider load‑balancing) looks reasonable and safely falls back to
next(ctx)on errors.Also applies to: 263-314
framework/tracing/store.go (1)
13-22: Drop unusedTracerandRequestIDfields fromDeferredSpanInfo
DeferredSpanInfo.Tracerand.RequestIDare not read or written anywhere in this file; onlySpanID,StartTime,FirstChunkTime,AccumulatedChunks, andmuare used. Keeping dead fields here increases memory footprint and obscures the actual responsibilities of the struct.Unless there is concrete usage elsewhere in the stack, consider removing these fields and updating the comment to match.
plugins/logging/main.go (1)
225-233: Verify thatBifrostContextKeyTraceris actually present duringPreHookfor streaming
PreHookcreates a stream accumulator only when both tracer and trace ID are found in the context:if bifrost.IsStreamRequestType(req.RequestType) { if tracer, ok := ctx.Value(schemas.BifrostContextKeyTracer).(schemas.Tracer); ok && tracer != nil { if traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string); ok && traceID != "" { tracer.CreateStreamAccumulator(traceID, createdTimestamp) } } }From
core/bifrost.go,BifrostContextKeyTraceris set insideexecuteRequestWithRetries, which runs after pluginPreHooks. Unless some upstream layer injects the tracer into the context earlier, this block will never run on the standard HTTP → Bifrost path, andProcessStreamingChunkwill operate without aCreateStreamAccumulatorhaving been called (or will rely on lazy creation inside the tracer).Either:
- Move accumulator initialization into the tracer (e.g. inside
ProcessStreamingChunkif missing), or- Ensure
BifrostContextKeyTracer(and trace ID) are set beforeRunPreHooks, or- Remove this
PreHookbranch if the tracer now owns accumulator lifecycle.Also applies to: 423-438
plugins/otel/converter.go (1)
148-170: Avoid unsafeuint64→int64cast inanyToKeyValueThe
uint64branch casts directly toint64:case uint64: return kvInt(key, int64(v))For values >
math.MaxInt64, this overflows and will produce a negative or otherwise invalid value in OTEL attributes.A simple guard (clamp or log-and-cap) would make this safe.
Proposed fix
import ( "encoding/hex" "fmt" "strings" + "math" @@ case uint32: return kvInt(key, int64(v)) case uint64: - return kvInt(key, int64(v)) + if v > math.MaxInt64 { + // Clamp to MaxInt64 to avoid overflow + return kvInt(key, math.MaxInt64) + } + return kvInt(key, int64(v))transports/bifrost-http/handlers/middlewares.go (1)
201-279:TracingMiddlewareshould callStartSpanwithcontext.Context, not*fasthttp.RequestCtx
TracingMiddleware.Middlewarecurrently does:spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)where
ctxis*fasthttp.RequestCtx. However, the tracer implementation used elsewhere in the stack implementscore/schemas.Tracer, whoseStartSpanexpects acontext.Context.*fasthttp.RequestCtxdoes not implementcontext.Context, so this will not type-check unless the tracer has a different signature here.You likely want to:
- Build a proper
context.Context(e.g.baseCtx := context.Background()or one derived from the request/trace ID), and- Pass that to
StartSpan, then- Use the returned
spanCtxonly for span propagation (e.g. extractingBifrostContextKeySpanIDas you already do).For example:
baseCtx := context.Background() spanCtx, rootSpan := m.tracer.Load().StartSpan(baseCtx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)and continue to read
BifrostContextKeySpanIDfromspanCtxas you do today.core/schemas/trace.go (1)
40-49: ProtectTrace.Reset,Span.End, andSpan.Resetwith the existing mutexes to avoid data races.
TraceandSpanboth havemu sync.Mutex, and mutation helpers (AddSpan,SetAttribute,AddEvent) correctly lock it. However:
Trace.Reset()clearsRootSpan,Spans, and timing/attribute fields without acquiringt.mu.Span.End()updatesEndTime,Status, andStatusMsgwithout acquirings.mu.Span.Reset()clears all span fields, includingEvents, without acquirings.mu.If traces/spans are reused from pools or accessed concurrently (e.g., via the tracer and plugins) while these methods run, Go’s race detector will flag data races and behavior becomes nondeterministic.
Consider wrapping these mutating methods in the same mutex used elsewhere, e.g.:
func (t *Trace) Reset() { - t.TraceID = "" - t.ParentID = "" - t.RootSpan = nil - t.Spans = t.Spans[:0] - t.StartTime = time.Time{} - t.EndTime = time.Time{} - t.Attributes = nil + t.mu.Lock() + defer t.mu.Unlock() + t.TraceID = "" + t.ParentID = "" + t.RootSpan = nil + t.Spans = t.Spans[:0] + t.StartTime = time.Time{} + t.EndTime = time.Time{} + t.Attributes = nil } func (s *Span) End(status SpanStatus, statusMsg string) { - s.EndTime = time.Now() - s.Status = status - s.StatusMsg = statusMsg + s.mu.Lock() + defer s.mu.Unlock() + s.EndTime = time.Now() + s.Status = status + s.StatusMsg = statusMsg } func (s *Span) Reset() { - s.SpanID = "" - s.ParentID = "" - s.TraceID = "" - s.Name = "" - s.Kind = SpanKindUnspecified - s.StartTime = time.Time{} - s.EndTime = time.Time{} - s.Status = SpanStatusUnset - s.StatusMsg = "" - s.Attributes = nil - s.Events = s.Events[:0] + s.mu.Lock() + defer s.mu.Unlock() + s.SpanID = "" + s.ParentID = "" + s.TraceID = "" + s.Name = "" + s.Kind = SpanKindUnspecified + s.StartTime = time.Time{} + s.EndTime = time.Time{} + s.Status = SpanStatusUnset + s.StatusMsg = "" + s.Attributes = nil + s.Events = s.Events[:0] }This matches the locking strategy already used by the other methods and keeps reuse via pools safe.
Also applies to: 84-104
framework/tracing/tracer.go (1)
233-258: MakebuildCompleteResponseFromChunksdeterministic by sorting choice and tool‑call indices.The reconstruction logic still builds indices from maps but never sorts them:
choiceIndicesis populated fromchoiceMap(map[int]*choiceAccumulator) and iterated in map order.tcIndicesis populated fromacc.toolCalls(map[int]ChatAssistantMessageToolCall) and likewise iterated in map order.Despite comments about “Sort choice indices for deterministic output” and “Sort tool calls by index”, no
sort.Ints(...)calls are present. This means the order ofChoicesandToolCallsin the finalChatResponseis nondeterministic, which can make logs and tests flaky.Consider:
import "sort" ... choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } - for _, idx := range choiceIndices { + sort.Ints(choiceIndices) + for _, idx := range choiceIndices { ... if len(accum.toolCalls) > 0 { - tcIndices := make([]int, 0, len(accum.toolCalls)) - for tcIdx := range accum.toolCalls { - tcIndices = append(tcIndices, tcIdx) - } + tcIndices := make([]int, 0, len(accum.toolCalls)) + for tcIdx := range accum.toolCalls { + tcIndices = append(tcIndices, tcIdx) + } + sort.Ints(tcIndices) toolCalls := make([]schemas.ChatAssistantMessageToolCall, 0, len(accum.toolCalls)) for _, tcIdx := range tcIndices { toolCalls = append(toolCalls, accum.toolCalls[tcIdx]) }This preserves existing behavior while making the output stable across runs.
Also applies to: 271-293, 393-455
framework/tracing/llmspan.go (1)
190-193: ExtraParams written directly to attrs map may cause attribute key collisions.User-provided keys from
ExtraParamsare written directly to the attributes map without namespacing. If a user sets a key like"gen_ai.request.model"in ExtraParams, it would overwrite the standard attribute set on line 22.This pattern repeats throughout the file at lines 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 768-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, and 1027-1029.
🧹 Nitpick comments (12)
.github/workflows/scripts/push-mintlify-changelog.sh (2)
239-244: Branch resolution logic is sound for CI environments.The dynamic branch detection with detached HEAD handling is appropriate. The fallback chain (current branch →
GITHUB_REF_NAME→main) correctly handles common CI scenarios where the repository is checked out in detached HEAD state.💡 Optional: Add explanatory comment
For future maintainers, consider adding a brief comment:
# Pulling again before committing +# Determine current branch; in CI detached HEAD scenarios, use GITHUB_REF_NAME CURRENT_BRANCH="$(git rev-parse --abbrev-ref HEAD)" if [ "$CURRENT_BRANCH" = "HEAD" ]; then # In detached HEAD state (common in CI), use GITHUB_REF_NAME or default to main CURRENT_BRANCH="${GITHUB_REF_NAME:-main}" fi
244-255: Add error handling for git operations.The script lacks error handling for critical git operations (
pull,commit,push). If any of these commands fail, the script continues silently, which can lead to incomplete changelog deployments in CI/CD workflows.🔎 Recommended improvements
Option 1: Enable strict error handling at script level
Add at the top of the script (after the shebang):
#!/usr/bin/env bash +set -e # Exit on error +set -o pipefail # Exit on pipe failureOption 2: Add explicit error checking for critical operations
-git pull origin "$CURRENT_BRANCH" +if ! git pull origin "$CURRENT_BRANCH"; then + echo "❌ Failed to pull from $CURRENT_BRANCH" + exit 1 +fi + # Commit and push changes git add docs/changelogs/$VERSION.mdx git add docs/docs.json # Add all cleaned changelog files for file in "${CLEANED_CHANGELOG_FILES[@]}"; do git add "$file" done git config user.name "github-actions[bot]" git config user.email "41898282+github-actions[bot]@users.noreply.github.com" -git commit -m "Adds changelog for $VERSION --skip-pipeline" -git push origin "$CURRENT_BRANCH" +if git commit -m "Adds changelog for $VERSION --skip-pipeline"; then + if ! git push origin "$CURRENT_BRANCH"; then + echo "❌ Failed to push to $CURRENT_BRANCH" + exit 1 + fi + echo "✅ Successfully pushed changelog for $VERSION" +else + echo "⚠️ No changes to commit" +fiui/components/devProfiler.tsx (1)
139-145: Consider differentiating the minimize button icon.Both the "Collapse" toggle (Line 134) and "Minimize" button (Line 144) use
<ChevronDown>when expanded, which may confuse users about their different behaviors. Consider using a distinct icon for minimize (e.g.,Minusfrom lucide-react).ui/lib/store/apis/devApi.ts (1)
59-59: Nitpick: Comment is slightly misleading.The comment "polls every 10 seconds" describes consumer behavior, not endpoint behavior. The polling interval is configured at the call site in
devProfiler.tsx, not in this endpoint definition.🔎 Suggested clarification
endpoints: (builder) => ({ - // Get dev pprof data - polls every 10 seconds + // Get dev pprof data (consumers typically poll this endpoint) getDevPprof: builder.query<PprofData, void>({framework/plugins/dynamicplugin.go (1)
24-28: Dynamic plugin now correctly treatsHTTPTransportMiddlewareas a factory; only error text is slightly stale.The change to store a
func() schemas.BifrostHTTPMiddlewareand assert the symbol to that type fixes the earlier runtime type‑assertion issue when loading plugins that export a factory. The accessorHTTPTransportMiddleware()then correctly returns the constructed middleware.As a minor polish, the error message at line 146 still mentions
func(next fasthttp.RequestHandler) fasthttp.RequestHandler; updating it to reflectfunc() schemas.BifrostHTTPMiddlewarewould make debugging clearer.Also applies to: 36-39, 140-147
plugins/logging/main.go (1)
96-111: MakeCleanupidempotent to avoid double-close panics ondone
Cleanupstops the ticker and unconditionally closesp.done. IfCleanupis ever called twice for the sameLoggerPlugininstance (e.g., via tests or misuse), the secondclose(p.done)will panic.Wrapping the shutdown path in a
sync.Oncekeeps the API safer without changing normal behavior.Proposed diff
type LoggerPlugin struct { ctx context.Context store logstore.LogStore disableContentLogging *bool pricingManager *modelcatalog.ModelCatalog mu sync.Mutex done chan struct{} wg sync.WaitGroup + cleanupOnce sync.Once logger schemas.Logger logCallback LogCallback droppedRequests atomic.Int64 cleanupTicker *time.Ticker // Ticker for cleaning up old processing logs logMsgPool sync.Pool // Pool for reusing LogMessage structs updateDataPool sync.Pool // Pool for reusing UpdateLogData structs @@ func (p *LoggerPlugin) Cleanup() error { - // Stop the cleanup ticker - if p.cleanupTicker != nil { - p.cleanupTicker.Stop() - } - // Signal the background worker to stop - close(p.done) - // Wait for the background worker to finish processing remaining items - p.wg.Wait() - // Note: Accumulator cleanup is handled by the tracer, not the logging plugin - // GORM handles connection cleanup automatically - return nil + p.cleanupOnce.Do(func() { + // Stop the cleanup ticker + if p.cleanupTicker != nil { + p.cleanupTicker.Stop() + } + // Signal the background worker to stop + close(p.done) + // Wait for the background worker to finish processing remaining items + p.wg.Wait() + // Note: Accumulator cleanup is handled by the tracer, not the logging plugin + // GORM handles connection cleanup automatically + }) + return nil }Also applies to: 144-156, 158-180, 614-625
framework/tracing/store.go (1)
341-360: Be explicit that TTL cleanup may reclaim still-active traces
cleanupOldTracesdeletes any trace whoseStartTimeis older thannow - ttl, regardless of whether it’s still in active use:cutoff := time.Now().Add(-s.ttl) s.traces.Range(func(key, value any) bool { trace := value.(*schemas.Trace) if trace.StartTime.Before(cutoff) { if deleted, ok := s.traces.LoadAndDelete(key); ok { s.ReleaseTrace(deleted.(*schemas.Trace)) count++ } } return true })If a very long-lived request exceeds
ttl, its trace could be reset and returned to the pool while spans are still being added, leading to missing telemetry or potential data races.At minimum, this should be clearly documented and
ttlconfigured well above any realistic request duration. Alternatively, consider tightening the predicate (e.g., only cleaning traces that have been explicitly completed or have anEndTimeset).transports/bifrost-http/lib/config.go (2)
276-356: LoadConfig signature and default/file paths look consistent with the new API.The removal of
EnterpriseOverridesand the updated calls toloadConfigFromDefaultsandloadConfigFromFileare internally consistent, andLoadConfignow has a simpler surface that matches howBootstrapuses it. Just ensure any other call sites in the stack have been updated to the new(ctx, configDirPath)signature.
1509-1542: Pricing manager initialization now uniformly uses defaultmodelcatalog.Init(no overrides).Both
initFrameworkConfigFromFileandinitDefaultFrameworkConfiginitializepricingConfigand callmodelcatalog.Init(ctx, pricingConfig, config.ConfigStore, nil, logger), which aligns with the removal of enterprise overrides and withLoadPricingManagerin the server. This keeps pricing behavior consistent across file-based and default config paths; the warning‑only handling of init errors is unchanged and acceptable given pricing is ancillary to core routing.Also applies to: 1862-1915
transports/bifrost-http/server/server.go (1)
214-324: Plugin loading, middleware type migration, tracing, and dev‑pprof lifecycle wiring look consistent.
LoadPlugin/LoadPluginscorrectly use generics to return either concrete plugin types orschemas.Plugin, and built‑in cases (telemetry, logging, governance, maxim, semanticcache, otel) are wired through a single helper withoutEnterpriseOverrides.- All HTTP middleware signatures have been migrated to
schemas.BifrostHTTPMiddleware(inRegisterInferenceRoutes,RegisterAPIRoutes,RegisterUIRoutes, andPrepareCommonMiddlewares), aligning server wiring with the new plugin v2 middleware type.Bootstrapnow useslib.LoadConfig(ctx, configDir)andLoadPlugins(ctx, s.Config), then initializes tracing only when observability/logging plugins are present, injectingTracingMiddlewareahead of the inference middlewares and setting the tracer on the client.- The dev pprof handler is now stored on the server, registered only in dev mode, and cleaned up during shutdown, which fixes the earlier goroutine/lifecycle leak.
These changes are cohesive with the stated PR goal of a unified HTTP middleware pattern plus improved observability, with no additional correctness issues in the shown paths.
Also applies to: 327-443, 942-1042, 1084-1096, 1110-1163, 1220-1253, 1299-1333
framework/tracing/tracer.go (1)
13-52: Overall Tracer implementation and streaming integration look solid and align with the new tracing architecture.
Tracercleanly wrapsTraceStoreand delegates trace/span lifecycle operations (CreateTrace,EndTrace,ReleaseTrace,StartSpan,EndSpan, attribute/event setters) via thespanHandleabstraction, matching theschemas.Tracerinterface (enforced by the compile‑time assertion).- LLM request/response/error attribute population uses helper functions from
llmspan.go, keeping schema logic centralized while keeping tracer lightweight.- Deferred span support (
StoreDeferredSpan,GetDeferredSpanHandle,ClearDeferredSpan,GetDeferredSpanID) correctly keys by traceID and pairs with the store’s deferred span map.- Streaming integration via the embedded
streaming.Accumulator(CreateStreamAccumulator,CleanupStreamAccumulator,ProcessStreamingChunk,GetAccumulator) is well‑designed:ProcessStreamingChunkwraps the originalBifrostContextto inject traceID as requestID and preserve the stream end indicator, then maps accumulator results intoschemas.StreamAccumulatorResultfor plugins.Aside from the accumulator cleanup and deterministic ordering concerns noted separately, this file is a good fit for the new tracing+plugins v2 stack.
Also applies to: 61-171, 172-231, 460-531
framework/tracing/llmspan.go (1)
1279-1283: Consider using strings.Builder for string concatenation in loops.For consistency with line 1315 (which already uses
strings.Builder), consider refactoring the string concatenation loops at lines 1279-1283 and 1289-1292 to usestrings.Builderas well.While the performance impact is likely minimal for typical workloads (small numbers of blocks), using
strings.Builderthroughout provides consistency and better performance characteristics for edge cases with many blocks.🔎 Example refactor for lines 1279-1283
} else if msg.Content.ContentBlocks != nil { + var builder strings.Builder for _, block := range msg.Content.ContentBlocks { if block.Text != nil { - content += *block.Text + builder.WriteString(*block.Text) } } + content = builder.String() }Also applies to: 1289-1292
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sumtransports/go.sumis excluded by!**/*.sum
📒 Files selected for processing (100)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.sh.github/workflows/scripts/release-framework.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/devpprof_unix.gotransports/bifrost-http/handlers/devpprof_windows.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/go.modtransports/versionui/app/clientLayout.tsxui/app/workspace/logs/views/logDetailsSheet.tsxui/components/devProfiler.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsxui/lib/store/apis/devApi.tsui/lib/store/apis/index.ts
💤 Files with no reviewable changes (3)
- framework/configstore/tables/mcp.go
- core/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (3)
- plugins/semanticcache/version
- plugins/otel/version
- plugins/maxim/version
🚧 Files skipped from review as they are similar to previous changes (35)
- transports/bifrost-http/handlers/governance.go
- transports/bifrost-http/handlers/cache.go
- transports/bifrost-http/handlers/mcp.go
- ui/components/ui/switch.tsx
- plugins/telemetry/version
- framework/streaming/audio.go
- plugins/governance/version
- core/bifrost_test.go
- .github/workflows/scripts/release-bifrost-http.sh
- plugins/mocker/version
- framework/version
- plugins/mocker/main.go
- transports/bifrost-http/handlers/health.go
- ui/app/clientLayout.tsx
- plugins/logging/utils.go
- transports/version
- transports/bifrost-http/handlers/session.go
- core/schemas/context.go
- transports/bifrost-http/handlers/plugins.go
- framework/streaming/chat.go
- examples/plugins/hello-world/main.go
- plugins/jsonparser/main.go
- framework/streaming/types.go
- ui/app/workspace/logs/views/logDetailsSheet.tsx
- plugins/governance/go.mod
- transports/bifrost-http/handlers/middlewares_test.go
- examples/plugins/hello-world/go.mod
- framework/streaming/transcription.go
- transports/bifrost-http/lib/config_test.go
- core/version
- plugins/logging/version
- core/schemas/bifrost.go
- transports/bifrost-http/lib/middleware.go
- docs/docs.json
- transports/bifrost-http/handlers/integrations.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
docs/plugins/migration-guide.mdxui/lib/store/apis/index.tsui/components/sidebar.tsxtransports/bifrost-http/handlers/ui.goplugins/maxim/main.gocore/schemas/plugin.gocore/changelog.mdtransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/devpprof_unix.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/mcpserver.goplugins/governance/main.gotransports/bifrost-http/integrations/router.goframework/plugins/dynamicplugin_test.gocore/providers/utils/utils.goui/components/devProfiler.tsxplugins/semanticcache/main.gotransports/bifrost-http/handlers/devpprof.goplugins/otel/main.goplugins/governance/changelog.mdui/lib/store/apis/devApi.tsplugins/semanticcache/changelog.mdplugins/telemetry/changelog.mdplugins/telemetry/main.gotransports/bifrost-http/handlers/providers.goplugins/jsonparser/changelog.mdframework/tracing/store.goframework/streaming/accumulator.gotransports/bifrost-http/handlers/inference.goplugins/maxim/changelog.mdframework/tracing/propagation.gocore/bifrost.goplugins/otel/converter.goframework/tracing/helpers.gotransports/bifrost-http/handlers/middlewares.gocore/schemas/trace.gotransports/go.modui/components/ui/sheet.tsxtransports/bifrost-http/server/server.goplugins/mocker/changelog.mddocs/plugins/getting-started.mdxframework/streaming/responses.goplugins/logging/changelog.mdtransports/changelog.mdcore/schemas/tracer.goframework/tracing/tracer.goplugins/logging/main.goplugins/otel/changelog.mddocs/plugins/writing-plugin.mdxtransports/bifrost-http/handlers/devpprof_windows.goframework/changelog.mdtransports/bifrost-http/handlers/config.goframework/tracing/llmspan.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/websocket.goplugins/jsonparser/version
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/ui.goplugins/maxim/main.gocore/schemas/plugin.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/devpprof_unix.goframework/plugins/dynamicplugin.gotransports/bifrost-http/handlers/mcpserver.goplugins/governance/main.gotransports/bifrost-http/integrations/router.goframework/plugins/dynamicplugin_test.gocore/providers/utils/utils.goplugins/semanticcache/main.gotransports/bifrost-http/handlers/devpprof.goplugins/otel/main.goplugins/telemetry/main.gotransports/bifrost-http/handlers/providers.goframework/tracing/store.goframework/streaming/accumulator.gotransports/bifrost-http/handlers/inference.goframework/tracing/propagation.gocore/bifrost.goplugins/otel/converter.goframework/tracing/helpers.gotransports/bifrost-http/handlers/middlewares.gocore/schemas/trace.gotransports/bifrost-http/server/server.goframework/streaming/responses.gocore/schemas/tracer.goframework/tracing/tracer.goplugins/logging/main.gotransports/bifrost-http/handlers/devpprof_windows.gotransports/bifrost-http/handlers/config.goframework/tracing/llmspan.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/websocket.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/devpprof_unix.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/devpprof_windows.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/websocket.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
🧬 Code graph analysis (28)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/maxim/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
core/schemas/plugin.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/handlers/logging.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (2)
GetName(15-17)HTTPTransportMiddleware(19-27)
transports/bifrost-http/handlers/mcpserver.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/integrations/router.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (6)
ResponsesRequest(97-97)EmbeddingRequest(99-99)SpeechRequest(100-100)TranscriptionRequest(102-102)BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)transports/bifrost-http/handlers/inference.go (5)
ChatRequest(172-176)ResponsesRequest(266-270)EmbeddingRequest(273-277)SpeechRequest(279-283)TranscriptionRequest(285-289)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
ui/components/devProfiler.tsx (1)
ui/lib/utils/port.ts (1)
isDevelopmentMode(110-112)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
transports/bifrost-http/handlers/devpprof.go (3)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)transports/bifrost-http/handlers/utils.go (1)
SendJSON(16-22)
plugins/otel/main.go (5)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)core/schemas/context.go (1)
BifrostContext(32-42)core/schemas/bifrost.go (3)
BifrostRequest(173-193)BifrostResponse(335-355)BifrostError(474-483)core/schemas/trace.go (1)
Trace(10-19)
plugins/telemetry/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/providers.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/store.go (2)
core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(147-147)BifrostContextKeyTraceCompleter(148-148)
core/bifrost.go (5)
core/schemas/tracer.go (1)
DefaultTracer(181-183)core/schemas/trace.go (12)
SpanKindFallback(129-129)AttrProviderName(159-159)AttrRequestModel(160-160)SpanStatusError(151-151)SpanStatusOk(149-149)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindPlugin(123-123)AttrPluginInvocations(202-202)AttrPluginAvgDurationMs(203-203)AttrPluginTotalDurationMs(204-204)AttrPluginErrorCount(205-205)framework/tracing/helpers.go (1)
EndSpan(50-56)core/utils.go (1)
IsStreamRequestType(197-199)core/schemas/context.go (1)
NewBifrostContextWithTimeout(70-73)
plugins/otel/converter.go (2)
plugins/otel/main.go (1)
OtelPlugin(58-75)core/schemas/trace.go (17)
Trace(10-19)Span(52-65)SpanKind(115-115)SpanKindLLMCall(121-121)SpanKindHTTPRequest(131-131)SpanKindPlugin(123-123)SpanKindInternal(139-139)SpanKindRetry(127-127)SpanKindFallback(129-129)SpanKindMCPTool(125-125)SpanKindEmbedding(133-133)SpanKindSpeech(135-135)SpanKindTranscription(137-137)SpanStatus(143-143)SpanStatusOk(149-149)SpanStatusError(151-151)SpanEvent(107-111)
framework/tracing/helpers.go (3)
core/schemas/bifrost.go (1)
BifrostContextKeyTraceID(143-143)framework/tracing/store.go (1)
TraceStore(25-37)core/schemas/trace.go (5)
Trace(10-19)SpanKind(115-115)Span(52-65)SpanStatus(143-143)SpanEvent(107-111)
transports/bifrost-http/handlers/middlewares.go (5)
core/schemas/plugin.go (3)
BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)Plugin(71-97)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)framework/tracing/propagation.go (1)
ExtractParentID(74-84)core/schemas/bifrost.go (4)
BifrostContextKeyTraceID(143-143)BifrostContextKeyTraceCompleter(148-148)BifrostContextKeySpanID(144-144)BifrostContextKeyDeferTraceCompletion(147-147)core/schemas/trace.go (3)
SpanKindHTTPRequest(131-131)SpanStatusError(151-151)SpanStatusOk(149-149)
core/schemas/trace.go (2)
framework/tracing/helpers.go (1)
AddSpan(32-38)ui/lib/constants/logs.ts (1)
Status(161-161)
framework/streaming/responses.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
core/schemas/tracer.go (6)
core/schemas/chatcompletions.go (1)
BifrostLLMUsage(845-852)core/schemas/speech.go (1)
BifrostSpeechResponse(22-29)core/schemas/transcriptions.go (1)
BifrostTranscriptionResponse(16-26)core/schemas/trace.go (2)
SpanKind(115-115)SpanStatus(143-143)core/schemas/bifrost.go (2)
BifrostRequest(173-193)BifrostResponse(335-355)core/schemas/context.go (1)
BifrostContext(32-42)
framework/tracing/tracer.go (9)
framework/tracing/store.go (1)
TraceStore(25-37)framework/streaming/accumulator.go (2)
Accumulator(14-30)NewAccumulator(440-480)core/schemas/tracer.go (1)
StreamAccumulatorResult(15-33)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (5)
BifrostContextKeySpanID(144-144)BifrostRequest(173-193)BifrostResponse(335-355)BifrostContextKeyRequestID(124-124)BifrostContextKeyStreamEndIndicator(131-131)framework/tracing/llmspan.go (3)
PopulateRequestAttributes(14-60)PopulateResponseAttributes(64-106)PopulateErrorAttributes(109-124)core/schemas/chatcompletions.go (6)
BifrostResponseChoice(736-744)ChatReasoningDetails(755-762)ChatAssistantMessageToolCall(710-716)ChatStreamResponseChoice(783-785)ChatAssistantMessage(649-656)ChatNonStreamResponseChoice(777-780)core/schemas/context.go (2)
BifrostContext(32-42)NewBifrostContext(47-65)framework/streaming/types.go (1)
StreamResponseTypeFinal(24-24)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/websocket.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
🪛 markdownlint-cli2 (0.18.1)
transports/changelog.md
16-16: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
5186ad3 to
b12158e
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
transports/bifrost-http/server/server.go (1)
203-212: Avoid returningConfig.Providersmap outside the lock (concurrent map access risk).
GovernanceInMemoryStore.GetConfiguredProvidersreturnss.Config.Providerswhile only briefly holdingConfig.Mu.RLock. Callers keep a reference to the underlying map after the lock is released, but writers (e.g., AddProvider/UpdateProviderConfig/RemoveProvider) mutate the same map underConfig.Mu. Concurrent map read/write will panic at runtime.Return a copy instead of the shared map, e.g.:
Proposed fix
func (s *GovernanceInMemoryStore) GetConfiguredProviders() map[schemas.ModelProvider]configstore.ProviderConfig { - // Use read lock for thread-safe access - no need to copy on hot path - s.Config.Mu.RLock() - defer s.Config.Mu.RUnlock() - return s.Config.Providers + s.Config.Mu.RLock() + defer s.Config.Mu.RUnlock() + + out := make(map[schemas.ModelProvider]configstore.ProviderConfig, len(s.Config.Providers)) + for k, v := range s.Config.Providers { + out[k] = v + } + return out }transports/bifrost-http/lib/config.go (1)
1509-1527: Guard against nilPricingSyncIntervalininitFrameworkConfigFromFile.When loading from file and no framework config exists in the store, you do:
if configData.FrameworkConfig != nil && configData.FrameworkConfig.Pricing != nil { pricingConfig.PricingURL = configData.FrameworkConfig.Pricing.PricingURL syncDuration := time.Duration(*configData.FrameworkConfig.Pricing.PricingSyncInterval) * time.Second pricingConfig.PricingSyncInterval = &syncDuration }If
pricing.pricing_sync_intervalis omitted fromconfig.json,PricingSyncIntervalwill be nil and this panics at startup.Align with the safer pattern you already use in
initDefaultFrameworkConfigby checking for nil (and optionally >0) before dereferencing, e.g.:Proposed fix
-} else if configData.FrameworkConfig != nil && configData.FrameworkConfig.Pricing != nil { - pricingConfig.PricingURL = configData.FrameworkConfig.Pricing.PricingURL - syncDuration := time.Duration(*configData.FrameworkConfig.Pricing.PricingSyncInterval) * time.Second - pricingConfig.PricingSyncInterval = &syncDuration -} +} else if cfg := configData.FrameworkConfig; cfg != nil && cfg.Pricing != nil { + pricingConfig.PricingURL = cfg.Pricing.PricingURL + if cfg.Pricing.PricingSyncInterval != nil && *cfg.Pricing.PricingSyncInterval > 0 { + syncDuration := time.Duration(*cfg.Pricing.PricingSyncInterval) * time.Second + pricingConfig.PricingSyncInterval = &syncDuration + } +}
♻️ Duplicate comments (16)
plugins/logging/version (1)
1-1: Version bump inconsistent with breaking changes in this PR stack.The patch-level bump to
1.3.53does not reflect the breaking change introduced by the HTTPTransportMiddleware migration. Given that:
- The transports package bumped to
1.4.0-prerelease1- This PR targets the v1.4.0 branch
- Plugins implementing
TransportInterceptormust be rewritten to useHTTPTransportMiddlewareBump to
1.4.0or1.4.0-prerelease1to signal the breaking change and maintain version consistency across the stack.ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
183-186: Add aria-label for accessibility.The icon-only button lacks an accessible name for screen readers. Add an
aria-labelto describe the button's purpose.🔎 Proposed fix
<DropdownMenuTrigger asChild> - <Button variant="ghost" size="icon"> + <Button variant="ghost" size="icon" aria-label="Open actions menu"> <MoreVertical className="h-3 w-3" /> </Button> </DropdownMenuTrigger>plugins/otel/changelog.md (1)
1-20: Migration guide link requires verification.The changelog references
/docs/plugins/migration-guideon line 19. This has been flagged in previous reviews—ensure the migration guide exists at the referenced path or update the link to point to the correct documentation location.core/schemas/trace.go (3)
40-49: Critical concurrency issue:Trace.Reset()still lacks mutex protection.This method modifies all fields including the
Spansslice without holding the mutex, creating a data race if called concurrently withAddSpanorGetSpan. This issue was previously flagged but remains unresolved.🔎 Proposed fix
// Reset clears the trace for reuse from pool func (t *Trace) Reset() { + t.mu.Lock() + defer t.mu.Unlock() t.TraceID = "" t.ParentID = "" t.RootSpan = nil t.Spans = t.Spans[:0] t.StartTime = time.Time{} t.EndTime = time.Time{} t.Attributes = nil }
84-89: Critical concurrency issue:Span.End()lacks mutex protection.Unlike
SetAttributeandAddEvent, theEndmethod modifiesEndTime,Status, andStatusMsgwithout holding the mutex. This creates a data race if called concurrently with other span operations. This issue was previously flagged but remains unresolved.🔎 Proposed fix
// End marks the span as complete with the given status func (s *Span) End(status SpanStatus, statusMsg string) { + s.mu.Lock() + defer s.mu.Unlock() s.EndTime = time.Now() s.Status = status s.StatusMsg = statusMsg }
91-104: Critical concurrency issue:Span.Reset()lacks mutex protection.Similar to
Trace.Reset(), this method modifies fields without acquiring the mutex, creating a data race when a span is reused from a pool while another goroutine holds a reference. This issue was previously flagged but remains unresolved.🔎 Proposed fix
// Reset clears the span for reuse from pool func (s *Span) Reset() { + s.mu.Lock() + defer s.mu.Unlock() s.SpanID = "" s.ParentID = "" s.TraceID = "" s.Name = "" s.Kind = SpanKindUnspecified s.StartTime = time.Time{} s.EndTime = time.Time{} s.Status = SpanStatusUnset s.StatusMsg = "" s.Attributes = nil s.Events = s.Events[:0] }examples/plugins/hello-world/main.go (1)
19-27: Example demonstrates broken data flow between middleware and hooks.The middleware sets a value on
fasthttp.RequestCtx(line 23) butPreHookandPostHook(lines 30, 39-40) attempt to read from*schemas.BifrostContext. These are different context types, so the value will always benil.Either remove the context value reads from the hooks (since the propagation doesn't work), or add a comment explaining that middleware operates on the HTTP transport context while hooks operate on the Bifrost internal context.
🔎 Suggested fix to remove broken context reads
func PreHook(ctx *schemas.BifrostContext, req *schemas.BifrostRequest) (*schemas.BifrostRequest, *schemas.PluginShortCircuit, error) { - value1 := ctx.Value(schemas.BifrostContextKey("hello-world-plugin-transport-interceptor")) - fmt.Println("value1:", value1) ctx.SetValue(schemas.BifrostContextKey("hello-world-plugin-pre-hook"), "pre-hook-value") fmt.Println("PreHook called") return req, nil, nil } func PostHook(ctx *schemas.BifrostContext, resp *schemas.BifrostResponse, bifrostErr *schemas.BifrostError) (*schemas.BifrostResponse, *schemas.BifrostError, error) { fmt.Println("PostHook called") - value1 := ctx.Value(schemas.BifrostContextKey("hello-world-plugin-transport-interceptor")) - fmt.Println("value1:", value1) value2 := ctx.Value(schemas.BifrostContextKey("hello-world-plugin-pre-hook")) fmt.Println("value2:", value2) return resp, bifrostErr, nil }framework/changelog.md (1)
1-1: Fix grammar and expand tracing framework description.Line 1 contains a grammar error: "to enabling e2e tracing" should be "to enable e2e tracing".
Additionally, consider briefly explaining what the tracing framework provides (e.g., "enables plugins to report spans and traces for distributed tracing integration") to match the detail level of the HTTPTransportMiddleware breaking change documentation below.
🔎 Suggested fix
-- feat: adds new tracing framework for allowing plugins to enable e2e tracing +- feat: adds new tracing framework to enable e2e tracing. The framework allows plugins to report spans and traces for distributed tracing integration with observability backends.docs/plugins/migration-guide.mdx (1)
247-250: Non-existent version tag will break user migration.Line 249 references
@v1.4.0which doesn't exist yet (this PR is open and targets the v1.4.0 branch). Users following this guide will encounter an error.Before merging, either:
- Create and push the
v1.4.0tag, or- Update to reference an existing version/commit, or
- Add a note: "Replace
@v1.4.0with the latest release version after v1.4.0 is published"🔎 Suggested fix with placeholder
2. **Update Bifrost to v1.4.x:** ```bash - go get github.com/maximhq/bifrost/[email protected] + # Replace @v1.4.0 with the actual release version once published + go get github.com/maximhq/bifrost/[email protected] ```Or reference the target branch until release:
- go get github.com/maximhq/bifrost/[email protected] + go get github.com/maximhq/bifrost/core@latesttransports/bifrost-http/integrations/router.go (1)
502-693: Non-streaming requests use wrong context - breaks tracing and plugin propagation.All non-streaming Bifrost client calls (lines 514, 534, 557, 580, 603, 626, 660) pass
ctx(*fasthttp.RequestCtx) instead of the converted Bifrost context (*bifrostCtx). The comment at line 503 acknowledges that the "cancellable context from ConvertToBifrostContext" should be used, but the code doesn't follow this.This breaks:
- Tracing (all
BifrostContextKey*values are onbifrostCtx, notctx)- Plugin context propagation
- Timeout and cancellation behavior
🔎 Proposed fix
Apply this pattern to all non-streaming calls:
case bifrostReq.ListModelsRequest != nil: - listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, bifrostReq.ListModelsRequest) + listModelsResponse, bifrostErr := g.client.ListModelsRequest(*bifrostCtx, bifrostReq.ListModelsRequest) case bifrostReq.TextCompletionRequest != nil: - textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, bifrostReq.TextCompletionRequest) + textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(*bifrostCtx, bifrostReq.TextCompletionRequest) case bifrostReq.ChatRequest != nil: - chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, bifrostReq.ChatRequest) + chatResponse, bifrostErr := g.client.ChatCompletionRequest(*bifrostCtx, bifrostReq.ChatRequest) case bifrostReq.ResponsesRequest != nil: - responsesResponse, bifrostErr := g.client.ResponsesRequest(ctx, bifrostReq.ResponsesRequest) + responsesResponse, bifrostErr := g.client.ResponsesRequest(*bifrostCtx, bifrostReq.ResponsesRequest) case bifrostReq.EmbeddingRequest != nil: - embeddingResponse, bifrostErr := g.client.EmbeddingRequest(ctx, bifrostReq.EmbeddingRequest) + embeddingResponse, bifrostErr := g.client.EmbeddingRequest(*bifrostCtx, bifrostReq.EmbeddingRequest) case bifrostReq.SpeechRequest != nil: - speechResponse, bifrostErr := g.client.SpeechRequest(ctx, bifrostReq.SpeechRequest) + speechResponse, bifrostErr := g.client.SpeechRequest(*bifrostCtx, bifrostReq.SpeechRequest) case bifrostReq.TranscriptionRequest != nil: - transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(ctx, bifrostReq.TranscriptionRequest) + transcriptionResponse, bifrostErr := g.client.TranscriptionRequest(*bifrostCtx, bifrostReq.TranscriptionRequest)core/providers/utils/utils.go (1)
1424-1513: Pass context by value, not by pointer (Go convention).The function signature
completeDeferredSpan(ctx *context.Context, ...)violates Go conventions. Contexts should always be passed by value, never by pointer. Thecontext.Contexttype is designed to be passed by value and is safe for concurrent use.🔎 Proposed fix
-func completeDeferredSpan(ctx *context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { - if ctx == nil { +func completeDeferredSpan(ctx context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { + if ctx == nil || ctx == context.Background() || ctx == context.TODO() { return } - traceID, ok := (*ctx).Value(schemas.BifrostContextKeyTraceID).(string) + traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string) if !ok || traceID == "" { return } - tracerVal := (*ctx).Value(schemas.BifrostContextKeyTracer) + tracerVal := ctx.Value(schemas.BifrostContextKeyTracer) // ... continue pattern for all (*ctx).Value callsAnd update all call sites (lines 901, 928, 952, 978):
- completeDeferredSpan(&ctx, processedResponse, processedError) + completeDeferredSpan(ctx, processedResponse, processedError)plugins/governance/main.go (1)
233-261: Add prefix validation for x-bf-vk header for consistency.The
x-bf-vkheader on lines 236-238 is returned without validating that it starts withVirtualKeyPrefix, unlike the other headers (Authorization,x-api-key,x-goog-api-key) which all validate the prefix. This inconsistency could allow invalid virtual keys to pass through when provided via thex-bf-vkheader.🔎 Proposed fix
vkHeader := ctx.Request.Header.Peek("x-bf-vk") if string(vkHeader) != "" { - return bifrost.Ptr(string(vkHeader)) + vkValue := string(vkHeader) + if strings.HasPrefix(strings.ToLower(vkValue), VirtualKeyPrefix) { + return bifrost.Ptr(vkValue) + } }transports/bifrost-http/handlers/middlewares.go (1)
247-256:TracingMiddlewarestill passes*fasthttp.RequestCtxintoStartSpan, which expectscontext.Context
StartSpanincore/schemas.Tracertakes acontext.Context, but here you call:spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)where
ctxis*fasthttp.RequestCtx. This won’t type‑check and also preventsStartSpanfrom seeing the trace ID viactx.Value(...).You likely want to construct a proper
context.Contextthat carries the W3C trace ID, for example:goCtx := context.Background() if v := ctx.UserValue(schemas.BifrostContextKeyTraceID); v != nil { if traceID, ok := v.(string); ok && traceID != "" { goCtx = context.WithValue(goCtx, schemas.BifrostContextKeyTraceID, traceID) } } spanCtx, rootSpan := m.tracer.Load().StartSpan(goCtx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)and then use
spanCtxonly for span propagation (e.g., to extract the span ID), keeping*fasthttp.RequestCtxfor HTTP.plugins/otel/converter.go (1)
148-171: Avoid unsafeuint64→int64cast inanyToKeyValueThe
uint64branch does:case uint64: return kvInt(key, int64(v))If
vexceedsmath.MaxInt64, this overflows and produces a negative OTEL attribute value. Even if unlikely, it makes debugging harder.Consider clamping or preserving unsigned semantics, for example:
case uint64: if v > math.MaxInt64 { // optionally log a warning return kvInt(key, math.MaxInt64) } return kvInt(key, int64(v))framework/tracing/llmspan.go (1)
191-193: NamespaceExtraParamskeys to avoid clobbering core attributes.Across the request helpers you write user‑supplied
ExtraParamsdirectly intoattrs:for k, v := range req.Params.ExtraParams { attrs[k] = fmt.Sprintf("%v", v) }This allows callers to overwrite standard attributes (e.g.,
gen_ai.request.model,gen_ai.provider) just by choosing the same key, which was already noted in a previous review.Consider prefixing these keys (and doing it consistently in all helpers) to avoid collisions, e.g.:
Proposed pattern
- for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) - } + for k, v := range req.Params.ExtraParams { + key := "gen_ai.extra." + k // or a similar, well-documented prefix + if _, exists := attrs[key]; !exists { + attrs[key] = fmt.Sprintf("%v", v) + } + }Apply the same pattern to
ExtraParamsin chat, text, embedding, responses, batch, and file helpers.Also applies to: 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 769-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029
framework/tracing/tracer.go (1)
393-455: MakebuildCompleteResponseFromChunksdeterministic by sorting indices.The comments say “Sort choice indices for deterministic output” and “Sort tool calls by index”, but:
choiceIndicesis built from a map and then iterated without sorting.tcIndicesis also built from a map and iterated without sorting.Map iteration order is random, so choice and tool‑call ordering will vary between runs, which can make traces and logs flaky.
Add explicit sorts before the loops:
Proposed fix
import ( "context" "time" "github.com/maximhq/bifrost/core/schemas" "github.com/maximhq/bifrost/framework/modelcatalog" "github.com/maximhq/bifrost/framework/streaming" + "sort" ) @@ // Build final choices from accumulated data // Sort choice indices for deterministic output choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } + sort.Ints(choiceIndices) @@ if len(accum.toolCalls) > 0 { // Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } + sort.Ints(tcIndices) toolCalls := make([]schemas.ChatAssistantMessageToolCall, 0, len(accum.toolCalls)) for _, tcIdx := range tcIndices { toolCalls = append(toolCalls, accum.toolCalls[tcIdx]) } msg.ChatAssistantMessage.ToolCalls = toolCallsAlso consider clarifying the function comment that this reconstruction currently only handles chat streaming responses (it bails out when
lastChunk.ChatResponse == nil).
🧹 Nitpick comments (4)
transports/bifrost-http/lib/config_test.go (1)
6663-12271: LoadConfig signature updates across SQLite integration tests look correctAll updated call sites now use
LoadConfig(ctx, tempDir)consistently, checkerrimmediately, and ensureConfigStore.Close(ctx)is invoked (either explicitly between loads or viadeferon the final load). There’s no remaining dependency on the old third parameter, and the tests’ semantics around hash/merge behavior and DB state are unchanged.If you touch this area again, you could optionally DRY up the repeated
ctx := context.Background()+LoadConfig(ctx, tempDir)+Closepattern into a small helper (e.g.,mustLoadConfig(t, dir)returning the config and registeringt.Cleanup), but that’s purely a readability/maintenance nicety, not required for this change.transports/bifrost-http/handlers/devpprof.go (1)
272-313: Defensively handle missingalloc_objects/alloc_spacesample types ingetTopAllocationsRight now
allocObjectsIdx/allocSpaceIdxdefault to zero and are used unconditionally. If the parsed profile ever lacks those sample types, you’ll silently read whatever is at index 0, which makes the “top allocations” inaccurate.Consider tracking booleans when you discover each index and returning an empty slice (or skipping aggregation) if either type is not found, instead of assuming index 0 is correct.
framework/tracing/store.go (1)
13-22: Clarify or drop unusedTracer/RequestIDonDeferredSpanInfoWithin this file, only
SpanID,StartTime,FirstChunkTime,AccumulatedChunks, andmuare used. IfTracerandRequestIDare no longer part of the streaming/deferred‑span design, consider removing them (or explicitly marking them as reserved) to keep the struct lean and avoid confusion about lifecycles.plugins/logging/main.go (1)
614-625: MakeCleanupidempotent to avoid double‑close panics ondone
Cleanupstops the ticker and unconditionallyclose(p.done)then waits. IfCleanupis ever called twice (tests, repeated shutdown, or RemovePlugin + Shutdown wiring), the second call will panic on closing an already‑closed channel.Consider guarding with a
sync.Onceor an atomic flag so the close + wait sequence runs only once.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumplugins/governance/go.sumis excluded by!**/*.sumtransports/go.sumis excluded by!**/*.sum
📒 Files selected for processing (100)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.sh.github/workflows/scripts/release-framework.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/governance/versionplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/jsonparser/versionplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/logging/versionplugins/maxim/changelog.mdplugins/maxim/main.goplugins/maxim/versionplugins/mocker/changelog.mdplugins/mocker/main.goplugins/mocker/versionplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/otel/versionplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/semanticcache/versionplugins/telemetry/changelog.mdplugins/telemetry/main.goplugins/telemetry/versiontransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/devpprof_unix.gotransports/bifrost-http/handlers/devpprof_windows.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/go.modtransports/versionui/app/clientLayout.tsxui/app/workspace/logs/views/logDetailsSheet.tsxui/components/devProfiler.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsxui/lib/store/apis/devApi.tsui/lib/store/apis/index.ts
💤 Files with no reviewable changes (3)
- core/mcp.go
- framework/configstore/tables/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (2)
- plugins/mocker/version
- plugins/governance/version
🚧 Files skipped from review as they are similar to previous changes (30)
- core/version
- .github/workflows/release-pipeline.yml
- ui/components/sidebar.tsx
- .github/workflows/scripts/release-bifrost-http.sh
- transports/bifrost-http/handlers/plugins.go
- transports/bifrost-http/handlers/middlewares_test.go
- framework/version
- transports/bifrost-http/handlers/health.go
- transports/bifrost-http/handlers/cache.go
- core/schemas/context.go
- plugins/maxim/main.go
- plugins/governance/changelog.md
- ui/components/devProfiler.tsx
- transports/bifrost-http/handlers/config.go
- .github/workflows/scripts/release-framework.sh
- docs/docs.json
- ui/components/ui/sheet.tsx
- transports/go.mod
- docs/plugins/writing-plugin.mdx
- plugins/maxim/version
- transports/version
- transports/changelog.md
- ui/lib/store/apis/devApi.ts
- plugins/logging/utils.go
- transports/bifrost-http/handlers/devpprof_unix.go
- ui/components/ui/switch.tsx
- plugins/jsonparser/version
- .github/workflows/scripts/push-mintlify-changelog.sh
- plugins/semanticcache/version
- ui/lib/store/apis/index.ts
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
transports/bifrost-http/handlers/devpprof_windows.gotransports/bifrost-http/handlers/mcp.goexamples/plugins/hello-world/main.goexamples/plugins/hello-world/go.modframework/streaming/transcription.goplugins/maxim/changelog.mdplugins/mocker/main.goplugins/semanticcache/changelog.mdtransports/bifrost-http/lib/middleware.godocs/plugins/getting-started.mdxtransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/providers.goframework/streaming/types.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/inference.goframework/streaming/chat.gocore/bifrost.goplugins/telemetry/main.gotransports/bifrost-http/handlers/mcpserver.gocore/schemas/plugin.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/websocket.goui/app/workspace/logs/views/logDetailsSheet.tsxframework/changelog.mdplugins/governance/main.goplugins/semanticcache/main.gocore/changelog.mdplugins/logging/main.goframework/plugins/dynamicplugin_test.goplugins/otel/versionframework/streaming/audio.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/devpprof.goplugins/jsonparser/changelog.mdplugins/logging/versionframework/streaming/accumulator.goplugins/otel/changelog.mdtransports/bifrost-http/handlers/session.goplugins/telemetry/versioncore/schemas/tracer.goplugins/mocker/changelog.mddocs/plugins/migration-guide.mdxcore/providers/utils/utils.goplugins/jsonparser/main.gocore/schemas/bifrost.goframework/tracing/store.goframework/streaming/responses.goframework/plugins/dynamicplugin.gotransports/bifrost-http/lib/config_test.goplugins/governance/go.modplugins/otel/converter.goplugins/telemetry/changelog.mdframework/tracing/helpers.goplugins/logging/changelog.mdui/app/clientLayout.tsxplugins/otel/main.gotransports/bifrost-http/integrations/router.gocore/schemas/trace.goframework/tracing/llmspan.gotransports/bifrost-http/server/server.goframework/tracing/tracer.gotransports/bifrost-http/lib/config.goframework/tracing/propagation.gocore/bifrost_test.go
🧠 Learnings (6)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/devpprof_windows.gotransports/bifrost-http/handlers/mcp.goexamples/plugins/hello-world/main.goframework/streaming/transcription.goplugins/mocker/main.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/providers.goframework/streaming/types.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/inference.goframework/streaming/chat.gocore/bifrost.goplugins/telemetry/main.gotransports/bifrost-http/handlers/mcpserver.gocore/schemas/plugin.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/websocket.goplugins/governance/main.goplugins/semanticcache/main.goplugins/logging/main.goframework/plugins/dynamicplugin_test.goframework/streaming/audio.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/devpprof.goframework/streaming/accumulator.gotransports/bifrost-http/handlers/session.gocore/schemas/tracer.gocore/providers/utils/utils.goplugins/jsonparser/main.gocore/schemas/bifrost.goframework/tracing/store.goframework/streaming/responses.goframework/plugins/dynamicplugin.gotransports/bifrost-http/lib/config_test.goplugins/otel/converter.goframework/tracing/helpers.goplugins/otel/main.gotransports/bifrost-http/integrations/router.gocore/schemas/trace.goframework/tracing/llmspan.gotransports/bifrost-http/server/server.goframework/tracing/tracer.gotransports/bifrost-http/lib/config.goframework/tracing/propagation.gocore/bifrost_test.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/devpprof_windows.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/server/server.gotransports/bifrost-http/lib/config.go
📚 Learning: 2025-12-24T04:36:57.698Z
Learnt from: danpiths
Repo: maximhq/bifrost PR: 1169
File: transports/bifrost-http/handlers/governance.go:1708-1878
Timestamp: 2025-12-24T04:36:57.698Z
Learning: In governance update handlers (e.g., updateModelConfig, updateProviderGovernance), design updates to support clearing individual fields by sending null/empty values (e.g., {"rate_limit": {"token_max_limit": null}} clears token_max_limit). Follow this pattern for partial updates so users can remove specific governance settings without deleting the whole entity. Ensure budget updates follow the same approach using direct field assignment. Review input validation, JSON decoding (e.g., pointers vs values in Go structs), and API documentation to reflect nullable fields and expected behavior.
Applied to files:
transports/bifrost-http/handlers/governance.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
akshaydeo
force-pushed
the
11-19-plugins_v2_architecture
branch
from
b12158e to
828d3d7
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
plugins/governance/main.go (1)
42-49: RemoveTransportInterceptorfromBaseGovernancePlugininterface or implement it inGovernancePlugin.The
BaseGovernancePlugininterface declaresTransportInterceptor(line 44), butGovernancePluginimplementsHTTPTransportMiddleware(line 264) instead. The interface is actively used intransports/bifrost-http/server/server.gofor type assertions and method calls. TheTransportInterceptormethod in the interface is never called and appears to be a leftover from the migration to the new v2 plugin architecture.transports/bifrost-http/server/server.go (1)
203-212: Avoid returning the shared providers map after releasing the lock (data race risk).
GovernanceInMemoryStore.GetConfiguredProvidersreturnss.Config.Providersdirectly while only holdingMuduring the read of the field. Callers then iterate/use the same map after the lock is released, while other goroutines (e.g., AddProvider/UpdateProviderConfig/RemoveProvider) may mutate it underMu. That’s a classic concurrent map read/write hazard.Make this method return a copy of the map (or a safely immutable view) constructed under the read lock, so callers never share the live internal map.
Proposed fix: copy the providers map under the lock
type GovernanceInMemoryStore struct { Config *lib.Config } func (s *GovernanceInMemoryStore) GetConfiguredProviders() map[schemas.ModelProvider]configstore.ProviderConfig { - // Use read lock for thread-safe access - no need to copy on hot path - s.Config.Mu.RLock() - defer s.Config.Mu.RUnlock() - return s.Config.Providers + // Copy under read lock to avoid exposing the shared map to concurrent writers + s.Config.Mu.RLock() + defer s.Config.Mu.RUnlock() + + out := make(map[schemas.ModelProvider]configstore.ProviderConfig, len(s.Config.Providers)) + for k, v := range s.Config.Providers { + out[k] = v + } + return out }transports/bifrost-http/lib/config.go (1)
1509-1527: Guard against nilPricingSyncIntervalwhen initializing framework config from file.In
initFrameworkConfigFromFile, the else-branch:} else if configData.FrameworkConfig != nil && configData.FrameworkConfig.Pricing != nil { pricingConfig.PricingURL = configData.FrameworkConfig.Pricing.PricingURL syncDuration := time.Duration(*configData.FrameworkConfig.Pricing.PricingSyncInterval) * time.Second pricingConfig.PricingSyncInterval = &syncDuration }assumes
PricingSyncIntervalis non-nil. If a user providesframework.pricingwithoutpricing_sync_interval, this will panic during config load.Add a nil (and optional >0) guard, and either fall back to defaults or leave the field nil so
modelcatalog.Initcan apply its defaults.Suggested defensive change
func initFrameworkConfigFromFile(ctx context.Context, config *Config, configData *ConfigData) { pricingConfig := &modelcatalog.Config{} @@ - } else if configData.FrameworkConfig != nil && configData.FrameworkConfig.Pricing != nil { - pricingConfig.PricingURL = configData.FrameworkConfig.Pricing.PricingURL - syncDuration := time.Duration(*configData.FrameworkConfig.Pricing.PricingSyncInterval) * time.Second - pricingConfig.PricingSyncInterval = &syncDuration + } else if configData.FrameworkConfig != nil && configData.FrameworkConfig.Pricing != nil { + pricingConfig.PricingURL = configData.FrameworkConfig.Pricing.PricingURL + if configData.FrameworkConfig.Pricing.PricingSyncInterval != nil && + *configData.FrameworkConfig.Pricing.PricingSyncInterval > 0 { + syncDuration := time.Duration(*configData.FrameworkConfig.Pricing.PricingSyncInterval) * time.Second + pricingConfig.PricingSyncInterval = &syncDuration + } }If you want to enforce a default when nil, you can set
pricingConfig.PricingSyncIntervaltobifrost.Ptr(modelcatalog.DefaultPricingSyncInterval)in theelsecase.
♻️ Duplicate comments (15)
ui/components/ui/sheet.tsx (1)
114-114: Make left margin conditional on button presence.The
ml-3class is applied unconditionally, but the expand/collapse button only renders whenexpandableis true ANDsideis "right". When the button is absent, this creates unwanted left margin.🔎 Suggested fix
- <div className={cn("flex h-full min-w-0 flex-1 flex-row items-center ml-3", className)}>{children}</div> + <div className={cn("flex h-full min-w-0 flex-1 flex-row items-center", sheetContext?.expandable && sheetContext?.side === "right" ? "ml-3" : "", className)}>{children}</div>transports/changelog.md (1)
15-15: Add language specifier to fenced code block.The fenced code block is missing a language specifier, which prevents proper syntax highlighting.
🔎 Suggested fix
**Migration summary:** - ``` + ```go // v1.3.x (removed) TransportInterceptor(ctx *BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error)Based on static analysis hints.
core/providers/utils/utils.go (1)
1428-1431: Context should be passed by value, not by pointer.The function signature
completeDeferredSpan(ctx *context.Context, ...)violates Go conventions. Thecontext.Contexttype is designed to be passed by value and is safe for concurrent use. Passing it by pointer can lead to subtle bugs and is considered an anti-pattern.🔎 Proposed fix
-func completeDeferredSpan(ctx *context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { - if ctx == nil { +func completeDeferredSpan(ctx context.Context, result *schemas.BifrostResponse, err *schemas.BifrostError) { + if ctx == nil { return } - traceID, ok := (*ctx).Value(schemas.BifrostContextKeyTraceID).(string) + traceID, ok := ctx.Value(schemas.BifrostContextKeyTraceID).(string)Update all call sites accordingly:
-completeDeferredSpan(&ctx, processedResponse, processedError) +completeDeferredSpan(ctx, processedResponse, processedError)plugins/otel/changelog.md (1)
1-19: LGTM: Changelog comprehensively documents the breaking change.The changelog clearly documents the migration from
TransportInterceptortoHTTPTransportMiddlewarewith old/new API signatures and consumer guidance. The past review flagged the missing migration guide link, which should be addressed separately.framework/changelog.md (1)
1-1: Grammar and detail issues in tracing feature entry remain unaddressed.The changelog entry still has awkward phrasing ("for allowing plugins to enable") and lacks detail about what the tracing framework provides to plugins. The breaking change section below is much more detailed, creating an imbalance.
Consider revising to:
-- feat: adds new tracing framework for allowing plugins to enable e2e tracing +- feat: adds tracing framework to enable end-to-end distributed tracing. Plugins can now report spans and traces for observability integration via the ObservabilityPlugin interface.This improves grammar and briefly describes the capability without being overly verbose.
examples/plugins/hello-world/main.go (1)
19-27: Context value propagation still broken between middleware and hooks.The HTTPTransportMiddleware sets a value on the
fasthttp.RequestCtx(line 23), butPreHook(line 30) andPostHook(lines 39-40) attempt to read from*schemas.BifrostContext. These are different context types, so the value will always benilin the hooks, breaking the example's demonstration of data flow.As noted in the previous review, you need to either:
- Extract the
*schemas.BifrostContextfrom the fasthttp context and set values on it, or- Remove the context value reads from PreHook/PostHook if cross-hook data flow is not needed for this example
ui/app/workspace/logs/views/logDetailsSheet.tsx (1)
183-187: Accessibility: Icon-only dropdown trigger still missing aria-label.The icon-only Button inside the DropdownMenuTrigger lacks an accessible name for screen reader users. This issue was flagged in a previous review and remains unaddressed.
🔎 Add aria-label for accessibility
<DropdownMenuTrigger asChild> - <Button variant="ghost" size="icon"> + <Button variant="ghost" size="icon" aria-label="Open log actions menu"> <MoreVertical className="h-3 w-3" /> </Button> </DropdownMenuTrigger>transports/bifrost-http/integrations/router.go (1)
517-519: Non-streaming calls pass wrong context type — breaks tracing and plugin propagation.The non-streaming request handlers call Bifrost client methods with
ctx(a*fasthttp.RequestCtx) instead of the enriched*bifrostCtx. The comment on line 508-512 says "Use the cancellable context from ConvertToBifrostContext", yet all the calls below passctx:listModelsResponse, bifrostErr := g.client.ListModelsRequest(ctx, ...) textCompletionResponse, bifrostErr := g.client.TextCompletionRequest(ctx, ...) chatResponse, bifrostErr := g.client.ChatCompletionRequest(ctx, ...) // ... etcThis breaks tracing context propagation and plugin hooks since all
BifrostContextKey*values live onbifrostCtx. The streaming path correctly usesstreamCtx := *bifrostCtx.This issue was flagged in a previous review. Please verify that these calls are updated to use
*bifrostCtxinstead ofctx.plugins/governance/main.go (1)
292-297: Fix error message accuracy.Line 294: The error message says "failed to marshal" but the operation is unmarshaling (
sonic.Unmarshal).🔎 Proposed fix
err = sonic.Unmarshal(ctx.Request.Body(), &payload) if err != nil { - p.logger.Error("failed to marshal request body to check for virtual key: %v", err) + p.logger.Error("failed to unmarshal request body: %v", err) next(ctx) return }framework/tracing/store.go (1)
13-22: Trim unused fields onDeferredSpanInfoand consider cleaningdeferredSpansin TTL sweep.
DeferredSpanInfo.TracerandDeferredSpanInfo.RequestID(lines 17–18) are never set or read anywhere; they add complexity and memory overhead without value. Either wire them up or remove them.cleanupOldTracesonly evicts fromtraces(lines 341–355) but leaves any correspondingdeferredSpansentries alive; for long‑running processes this can accumulate stale streaming state for traces that have already been TTL‑purged.You can:
- Drop the unused fields from
DeferredSpanInfo, and- In
cleanupOldTraces, alsos.deferredSpans.Delete(trace.TraceID)when a trace is evicted.Example cleanup tweaks
type DeferredSpanInfo struct { SpanID string StartTime time.Time - Tracer schemas.Tracer // Reference to tracer for completing the span - RequestID string // Request ID for accumulator lookup FirstChunkTime time.Time // Timestamp of first chunk (for TTFT calculation) AccumulatedChunks []*schemas.BifrostResponse // Accumulated streaming chunks mu sync.Mutex // Mutex for thread-safe chunk accumulation } @@ func (s *TraceStore) cleanupOldTraces() { cutoff := time.Now().Add(-s.ttl) @@ s.traces.Range(func(key, value any) bool { trace := value.(*schemas.Trace) if trace.StartTime.Before(cutoff) { if deleted, ok := s.traces.LoadAndDelete(key); ok { + // Best-effort cleanup of any deferred span for this trace + if t := deleted.(*schemas.Trace); t != nil { + s.deferredSpans.Delete(t.TraceID) + } s.ReleaseTrace(deleted.(*schemas.Trace)) count++ } } return true })Also applies to: 119-177, 341-360
transports/bifrost-http/handlers/middlewares.go (1)
191-279:TracingMiddleware.StartSpanstill passes*fasthttp.RequestCtxwherecontext.Contextis required.
Middleware()calls:spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest)Here
ctxis*fasthttp.RequestCtx, butStartSpanis defined as:StartSpan(ctx context.Context, name string, kind schemas.SpanKind) (context.Context, schemas.SpanHandle)
*fasthttp.RequestCtxdoes not implementcontext.Context, so this will not compile. You need to construct a realcontext.Context(and attach the trace ID) before callingStartSpan, then propagate the returnedspanCtxas needed.Suggested fix using a Go context carrying the trace ID
func (m *TracingMiddleware) Middleware() schemas.BifrostHTTPMiddleware { return func(next fasthttp.RequestHandler) fasthttp.RequestHandler { return func(ctx *fasthttp.RequestCtx) { @@ - // Create trace in store - only ID returned (trace data stays in store) - traceID := m.tracer.Load().CreateTrace(parentID) - // Only trace ID goes into context (lightweight, no bloat) - ctx.SetUserValue(schemas.BifrostContextKeyTraceID, traceID) + // Create trace in store - only ID returned (trace data stays in store) + traceID := m.tracer.Load().CreateTrace(parentID) + // Expose trace ID to downstream handlers via fasthttp user values + ctx.SetUserValue(schemas.BifrostContextKeyTraceID, traceID) @@ - // Create root span for the HTTP request - spanCtx, rootSpan := m.tracer.Load().StartSpan(ctx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) + // Build a standard context for tracer usage, seeded with the trace ID + baseCtx := context.WithValue(context.Background(), schemas.BifrostContextKeyTraceID, traceID) + // Create root span for the HTTP request + spanCtx, rootSpan := m.tracer.Load().StartSpan(baseCtx, string(ctx.RequestURI()), schemas.SpanKindHTTPRequest) @@ - // Set root span ID in context for child span creation + // Set root span ID in fasthttp context for child span creation downstream if spanID, ok := spanCtx.Value(schemas.BifrostContextKeySpanID).(string); ok { ctx.SetUserValue(schemas.BifrostContextKeySpanID, spanID) }This keeps tracing on a proper
context.Contextwhile still exposing IDs viactx.UserValuefor HTTP handlers and Bifrost to consume.core/schemas/trace.go (1)
40-49: GuardTrace.Reset,Span.End, andSpan.Resetwith mutexes or document exclusive-use assumptions.
Trace.Reset(lines 40–49) andSpan.End/Span.Reset(lines 84–104) mutate shared state without taking their respective mutexes, while other mutators (AddSpan,SetAttribute,AddEvent) are locked. If spans or traces are read/updated concurrently (e.g., while exporters or middleware run), this can lead to data races.Either:
- Protect these methods with the existing mutexes, or
- Clearly document that they may only be called once no other goroutine can access the trace/span and ensure all call sites respect that contract.
Illustrative locking changes
func (t *Trace) Reset() { - t.TraceID = "" - t.ParentID = "" - t.RootSpan = nil - t.Spans = t.Spans[:0] - t.StartTime = time.Time{} - t.EndTime = time.Time{} - t.Attributes = nil + t.mu.Lock() + defer t.mu.Unlock() + t.TraceID = "" + t.ParentID = "" + t.RootSpan = nil + t.Spans = t.Spans[:0] + t.StartTime = time.Time{} + t.EndTime = time.Time{} + t.Attributes = nil } @@ func (s *Span) End(status SpanStatus, statusMsg string) { - s.EndTime = time.Now() - s.Status = status - s.StatusMsg = statusMsg + s.mu.Lock() + defer s.mu.Unlock() + s.EndTime = time.Now() + s.Status = status + s.StatusMsg = statusMsg } @@ func (s *Span) Reset() { - s.SpanID = "" - s.ParentID = "" - s.TraceID = "" - s.Name = "" - s.Kind = SpanKindUnspecified - s.StartTime = time.Time{} - s.EndTime = time.Time{} - s.Status = SpanStatusUnset - s.StatusMsg = "" - s.Attributes = nil - s.Events = s.Events[:0] + s.mu.Lock() + defer s.mu.Unlock() + s.SpanID = "" + s.ParentID = "" + s.TraceID = "" + s.Name = "" + s.Kind = SpanKindUnspecified + s.StartTime = time.Time{} + s.EndTime = time.Time{} + s.Status = SpanStatusUnset + s.StatusMsg = "" + s.Attributes = nil + s.Events = s.Events[:0] }Also applies to: 84-104
plugins/otel/converter.go (1)
148-228: Guard againstuint64→int64overflow inanyToKeyValue.The
uint64case inanyToKeyValuecasts directly toint64:case uint64: return kvInt(key, int64(v))For values >
math.MaxInt64this overflows and will produce negative numbers. That’s rare but possible for counters/IDs.Example overflow-safe handling
import ( "encoding/hex" "fmt" "strings" + "math" @@ case uint64: - return kvInt(key, int64(v)) + if v > math.MaxInt64 { + // Clamp and preserve sign; optionally log via plugin logger if available + return kvInt(key, math.MaxInt64) + } + return kvInt(key, int64(v))If you’d rather not clamp silently, you could instead convert large
uint64values to strings (kvStr) to preserve the full value.In Go, does converting a uint64 value larger than math.MaxInt64 to int64 overflow, and is clamping or string-encoding preferred when exporting such values to OpenTelemetry integer attributes?framework/tracing/tracer.go (1)
233-455: Ensure deterministic ordering of choices and tool calls inbuildCompleteResponseFromChunks.You build
choiceIndicesandtcIndicesfrom maps but never sort them, so choice order and tool-call order will vary between runs despite the “Sort … for deterministic output” comments.Sorting these index slices before iterating will make reconstructed responses stable (useful for tests and for downstream observability tooling that expects consistent ordering).
Proposed ordering fix
@@ // Build final choices from accumulated data // Sort choice indices for deterministic output choiceIndices := make([]int, 0, len(choiceMap)) for idx := range choiceMap { choiceIndices = append(choiceIndices, idx) } + sort.Ints(choiceIndices) @@ if len(accum.toolCalls) > 0 { // Sort tool calls by index tcIndices := make([]int, 0, len(accum.toolCalls)) for tcIdx := range accum.toolCalls { tcIndices = append(tcIndices, tcIdx) } + sort.Ints(tcIndices) toolCalls := make([]schemas.ChatAssistantMessageToolCall, 0, len(accum.toolCalls)) for _, tcIdx := range tcIndices { toolCalls = append(toolCalls, accum.toolCalls[tcIdx]) }Remember to add:
import "sort"at the top of the file.
framework/tracing/llmspan.go (1)
190-193: ExtraParams still written without namespacing (duplicate concern).User-provided keys from
ExtraParamscontinue to be written directly into the attributes map across all request types without a namespace prefix. This can cause collisions with standard attribute keys (e.g., if a user sets"gen_ai.request.model"in ExtraParams, it would overwrite line 22's value).This pattern appears 14 times throughout the file in all request attribute population functions.
🔎 Suggested approach: namespace ExtraParams consistently
Apply a prefix to all ExtraParams keys before insertion. Example for one location:
// ExtraParams for k, v := range req.Params.ExtraParams { - attrs[k] = fmt.Sprintf("%v", v) + attrs["gen_ai.extra."+k] = fmt.Sprintf("%v", v) }Repeat this pattern at all 14 occurrences to prevent attribute key collisions.
Also applies to: 298-300, 365-367, 575-577, 708-710, 738-740, 753-755, 768-770, 783-785, 958-960, 982-984, 997-999, 1012-1014, 1027-1029
🧹 Nitpick comments (3)
ui/lib/store/apis/devApi.ts (1)
59-59: Misleading comment about polling.The comment "polls every 10 seconds" suggests automatic polling, but RTK Query's
builder.querydoesn't configure polling by default. Polling is typically configured at the hook usage site via thepollingIntervaloption. Consider updating the comment to reflect this, or remove it to avoid confusion.🔎 Proposed fix
endpoints: (builder) => ({ - // Get dev pprof data - polls every 10 seconds + // Get dev pprof data getDevPprof: builder.query<PprofData, void>({transports/bifrost-http/handlers/devpprof.go (1)
104-122: Global collector singleton is safe but consider handler lifecycle.The
sync.Oncepattern ensures thread-safe singleton initialization. However, sinceglobalCollectoris package-level, multipleDevPprofHandlerinstances will share the same collector. This is likely intentional but worth noting.Consider documenting this behavior or ensuring
Cleanup()is only called once during server shutdown.framework/tracing/llmspan.go (1)
1279-1293: Consider strings.Builder for string concatenation in loops (optional optimization).Lines 1279-1283 and 1289-1293 use
+=for string concatenation in loops, which can be less efficient for multiple blocks. Usingstrings.Builder(as done inextractMessageContentat lines 1315-1321) would be more efficient.However, given typical message sizes in practice, the current approach is acceptable.
🔎 Optional refactor using strings.Builder
content := "" + var contentBuilder strings.Builder if msg.Content != nil { if msg.Content.ContentStr != nil && *msg.Content.ContentStr != "" { content = *msg.Content.ContentStr } else if msg.Content.ContentBlocks != nil { for _, block := range msg.Content.ContentBlocks { if block.Text != nil { - content += *block.Text + contentBuilder.WriteString(*block.Text) } } + content = contentBuilder.String() } } // Extract reasoning text reasoning := "" + var reasoningBuilder strings.Builder if msg.ResponsesReasoning != nil && msg.ResponsesReasoning.Summary != nil { for _, block := range msg.ResponsesReasoning.Summary { if block.Text != "" { - reasoning += block.Text + reasoningBuilder.WriteString(block.Text) } } + reasoning = reasoningBuilder.String() }
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
examples/plugins/hello-world/go.sumis excluded by!**/*.sumtransports/go.sumis excluded by!**/*.sum
📒 Files selected for processing (92)
.github/workflows/release-pipeline.yml.github/workflows/scripts/push-mintlify-changelog.sh.github/workflows/scripts/release-bifrost-http.sh.github/workflows/scripts/release-framework.shcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/mcp.gocore/providers/utils/utils.gocore/schemas/bifrost.gocore/schemas/context.gocore/schemas/plugin.gocore/schemas/trace.gocore/schemas/tracer.gocore/versiondocs/docs.jsondocs/plugins/getting-started.mdxdocs/plugins/migration-guide.mdxdocs/plugins/writing-plugin.mdxexamples/plugins/hello-world/go.modexamples/plugins/hello-world/main.goframework/changelog.mdframework/configstore/tables/mcp.goframework/plugins/dynamicplugin.goframework/plugins/dynamicplugin_test.goframework/streaming/accumulator.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/responses.goframework/streaming/transcription.goframework/streaming/types.goframework/tracing/helpers.goframework/tracing/llmspan.goframework/tracing/propagation.goframework/tracing/store.goframework/tracing/tracer.goframework/versionplugins/governance/changelog.mdplugins/governance/go.modplugins/governance/main.goplugins/jsonparser/changelog.mdplugins/jsonparser/main.goplugins/logging/changelog.mdplugins/logging/main.goplugins/logging/utils.goplugins/maxim/changelog.mdplugins/maxim/main.goplugins/mocker/changelog.mdplugins/mocker/main.goplugins/otel/changelog.mdplugins/otel/converter.goplugins/otel/main.goplugins/otel/ttlsyncmap.goplugins/semanticcache/changelog.mdplugins/semanticcache/main.goplugins/telemetry/changelog.mdplugins/telemetry/main.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/devpprof_unix.gotransports/bifrost-http/handlers/devpprof_windows.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/health.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/handlers/logging.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/handlers/mcpserver.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/plugins.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/websocket.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/changelog.mdtransports/go.modtransports/versionui/app/clientLayout.tsxui/app/workspace/logs/views/logDetailsSheet.tsxui/components/devProfiler.tsxui/components/sidebar.tsxui/components/ui/sheet.tsxui/components/ui/switch.tsxui/lib/store/apis/devApi.tsui/lib/store/apis/index.ts
💤 Files with no reviewable changes (3)
- core/mcp.go
- framework/configstore/tables/mcp.go
- plugins/otel/ttlsyncmap.go
✅ Files skipped from review due to trivial changes (1)
- docs/plugins/migration-guide.mdx
🚧 Files skipped from review as they are similar to previous changes (26)
- core/version
- ui/components/sidebar.tsx
- .github/workflows/scripts/release-framework.sh
- framework/streaming/responses.go
- transports/bifrost-http/handlers/governance.go
- transports/bifrost-http/handlers/logging.go
- transports/version
- transports/bifrost-http/handlers/providers.go
- transports/go.mod
- transports/bifrost-http/handlers/devpprof_windows.go
- core/bifrost_test.go
- docs/plugins/writing-plugin.mdx
- core/schemas/bifrost.go
- .github/workflows/scripts/release-bifrost-http.sh
- framework/streaming/chat.go
- plugins/maxim/main.go
- docs/docs.json
- plugins/telemetry/main.go
- framework/version
- transports/bifrost-http/handlers/plugins.go
- framework/plugins/dynamicplugin.go
- framework/tracing/helpers.go
- transports/bifrost-http/handlers/websocket.go
- transports/bifrost-http/handlers/health.go
- transports/bifrost-http/handlers/mcpserver.go
- transports/bifrost-http/handlers/devpprof_unix.go
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
always check the stack if there is one for the current PR. do not give localized reviews for the PR, always see all changes in the light of the whole stack of PRs (if there is a stack, if there is no stack you can continue to make localized suggestions/reviews)
Files:
plugins/mocker/changelog.mdcore/changelog.mdui/components/devProfiler.tsxtransports/bifrost-http/handlers/config.goframework/streaming/transcription.gotransports/bifrost-http/handlers/middlewares_test.goplugins/logging/utils.goui/app/workspace/logs/views/logDetailsSheet.tsxframework/tracing/propagation.goplugins/logging/changelog.mdplugins/governance/main.gocore/providers/utils/utils.gotransports/changelog.mdtransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/ui.goplugins/otel/changelog.mdtransports/bifrost-http/handlers/cache.goplugins/logging/main.godocs/plugins/getting-started.mdxtransports/bifrost-http/handlers/session.goframework/streaming/audio.goframework/changelog.mdui/components/ui/sheet.tsxtransports/bifrost-http/integrations/router.goplugins/mocker/main.gotransports/bifrost-http/handlers/inference.goplugins/semanticcache/changelog.mdtransports/bifrost-http/lib/middleware.goexamples/plugins/hello-world/go.modframework/plugins/dynamicplugin_test.gocore/schemas/tracer.gotransports/bifrost-http/server/server.goplugins/maxim/changelog.mdui/components/ui/switch.tsxplugins/jsonparser/main.goplugins/jsonparser/changelog.mdplugins/otel/main.goexamples/plugins/hello-world/main.gocore/bifrost.goplugins/governance/go.modtransports/bifrost-http/handlers/mcp.goui/app/clientLayout.tsxframework/tracing/store.goui/lib/store/apis/devApi.tscore/schemas/trace.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/middlewares.goframework/streaming/types.goplugins/telemetry/changelog.mdtransports/bifrost-http/handlers/integrations.goframework/tracing/tracer.gocore/schemas/plugin.goplugins/governance/changelog.mdplugins/otel/converter.gotransports/bifrost-http/lib/config_test.goframework/tracing/llmspan.goframework/streaming/accumulator.gocore/schemas/context.goplugins/semanticcache/main.goui/lib/store/apis/index.ts
🧠 Learnings (5)
📚 Learning: 2025-12-09T17:07:42.007Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/schemas/account.go:9-18
Timestamp: 2025-12-09T17:07:42.007Z
Learning: In core/schemas/account.go, the HuggingFaceKeyConfig field within the Key struct is currently unused and reserved for future Hugging Face inference endpoint deployments. Do not flag this field as missing from OpenAPI documentation or require its presence in the API spec until the feature is actively implemented and used. When the feature is added, update the OpenAPI docs accordingly; otherwise, treat this field as non-breaking and not part of the current API surface.
Applied to files:
transports/bifrost-http/handlers/config.goframework/streaming/transcription.gotransports/bifrost-http/handlers/middlewares_test.goplugins/logging/utils.goframework/tracing/propagation.goplugins/governance/main.gocore/providers/utils/utils.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/cache.goplugins/logging/main.gotransports/bifrost-http/handlers/session.goframework/streaming/audio.gotransports/bifrost-http/integrations/router.goplugins/mocker/main.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/lib/middleware.goframework/plugins/dynamicplugin_test.gocore/schemas/tracer.gotransports/bifrost-http/server/server.goplugins/jsonparser/main.goplugins/otel/main.goexamples/plugins/hello-world/main.gocore/bifrost.gotransports/bifrost-http/handlers/mcp.goframework/tracing/store.gocore/schemas/trace.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/middlewares.goframework/streaming/types.gotransports/bifrost-http/handlers/integrations.goframework/tracing/tracer.gocore/schemas/plugin.goplugins/otel/converter.gotransports/bifrost-http/lib/config_test.goframework/tracing/llmspan.goframework/streaming/accumulator.gocore/schemas/context.goplugins/semanticcache/main.go
📚 Learning: 2025-12-12T08:25:02.629Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1000
File: transports/bifrost-http/integrations/router.go:709-712
Timestamp: 2025-12-12T08:25:02.629Z
Learning: In transports/bifrost-http/**/*.go, update streaming response handling to align with OpenAI Responses API: use typed SSE events such as response.created, response.output_text.delta, response.done, etc., and do not rely on the legacy data: [DONE] termination marker. Note that data: [DONE] is only used by the older Chat Completions and Text Completions streaming APIs. Ensure parsers, writers, and tests distinguish SSE events from the [DONE] sentinel and handle each event type accordingly for correct stream termination and progress updates.
Applied to files:
transports/bifrost-http/handlers/config.gotransports/bifrost-http/handlers/middlewares_test.gotransports/bifrost-http/handlers/devpprof.gotransports/bifrost-http/handlers/ui.gotransports/bifrost-http/handlers/cache.gotransports/bifrost-http/handlers/session.gotransports/bifrost-http/integrations/router.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/lib/middleware.gotransports/bifrost-http/server/server.gotransports/bifrost-http/handlers/mcp.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/handlers/middlewares.gotransports/bifrost-http/handlers/integrations.gotransports/bifrost-http/lib/config_test.go
📚 Learning: 2025-12-24T10:55:31.424Z
Learnt from: akshaydeo
Repo: maximhq/bifrost PR: 888
File: framework/tracing/propagation.go:24-36
Timestamp: 2025-12-24T10:55:31.424Z
Learning: In framework/tracing/propagation.go, ensure ExtractParentID() returns ctx.TraceID (not ctx.ParentID) because the parent request's traceparent header carries the trace ID used for continuing the trace.
Applied to files:
framework/tracing/propagation.go
📚 Learning: 2025-12-22T10:50:40.990Z
Learnt from: Pratham-Mishra04
Repo: maximhq/bifrost PR: 1154
File: plugins/governance/store.go:1165-1186
Timestamp: 2025-12-22T10:50:40.990Z
Learning: In the Bifrost governance plugin, budgets and rate limits have 1:1 relationships with their parent entities (virtual keys, teams, customers). Do not assume sharing; ensure cascade deletion logic only deletes budgets/rate limits when there are no shared references. Enforce invariants in code and add tests to verify no cross-entity sharing and that cascade deletes only remove the specific child of the parent. If a counterexample arises, adjust data model or add guards.
Applied to files:
plugins/governance/main.go
📚 Learning: 2025-12-19T09:26:54.961Z
Learnt from: qwerty-dvorak
Repo: maximhq/bifrost PR: 1006
File: core/providers/utils/utils.go:1050-1051
Timestamp: 2025-12-19T09:26:54.961Z
Learning: Update streaming end-marker handling so HuggingFace is treated as a non-[DONE] provider for backends that do not emit a DONE marker (e.g., meta llama on novita). In core/providers/utils/utils.go, adjust ProviderSendsDoneMarker() (or related logic) to detect providers that may not emit DONE and avoid relying on DONE as the sole end signal. Add tests to cover both DONE-emitting and non-DONE backends, with clear documentation in code comments explaining the rationale and any fallback behavior.
Applied to files:
core/providers/utils/utils.go
🧬 Code graph analysis (33)
ui/components/devProfiler.tsx (3)
npx/bin.js (3)
k(178-178)sizes(179-179)data(172-172)ui/lib/utils/port.ts (1)
isDevelopmentMode(110-112)ui/components/ui/tooltip.tsx (1)
Tooltip(43-43)
transports/bifrost-http/handlers/config.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/transcription.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/handlers/middlewares_test.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
ui/app/workspace/logs/views/logDetailsSheet.tsx (4)
ui/components/ui/sheet.tsx (1)
SheetContent(137-137)ui/components/ui/alertDialog.tsx (9)
AlertDialog(83-83)AlertDialogTrigger(93-93)AlertDialogContent(86-86)AlertDialogHeader(89-89)AlertDialogTitle(92-92)AlertDialogDescription(87-87)AlertDialogFooter(88-88)AlertDialogCancel(85-85)AlertDialogAction(84-84)ui/components/ui/dropdownMenu.tsx (5)
DropdownMenu(193-193)DropdownMenuTrigger(207-207)DropdownMenuContent(195-195)DropdownMenuItem(197-197)DropdownMenuSeparator(202-202)ui/components/ui/button.tsx (1)
Button(70-70)
plugins/governance/main.go (3)
core/utils.go (1)
Ptr(56-58)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/devpprof.go (4)
ui/lib/types/logs.ts (1)
Function(152-157)ui/lib/store/apis/devApi.ts (6)
MemoryStats(4-10)CPUStats(13-17)RuntimeStats(20-26)AllocationInfo(29-35)HistoryPoint(38-45)PprofData(48-55)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)transports/bifrost-http/handlers/utils.go (1)
SendJSON(16-22)
transports/bifrost-http/handlers/ui.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/cache.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/session.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/streaming/audio.go (1)
framework/streaming/types.go (1)
AccumulatedData(28-48)
transports/bifrost-http/integrations/router.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (2)
BifrostContextKeyDeferTraceCompletion(148-148)BifrostContextKeyTraceCompleter(149-149)
plugins/mocker/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/inference.go (2)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)core/schemas/bifrost.go (1)
BifrostContextKeyDeferTraceCompletion(148-148)
transports/bifrost-http/lib/middleware.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/plugins/dynamicplugin_test.go (1)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
core/schemas/tracer.go (7)
core/schemas/bifrost.go (2)
ModelProvider(33-33)BifrostError(490-499)core/schemas/chatcompletions.go (1)
BifrostLLMUsage(854-861)core/schemas/speech.go (1)
BifrostSpeechResponse(22-29)core/schemas/transcriptions.go (1)
BifrostTranscriptionResponse(16-26)core/schemas/trace.go (3)
Trace(10-19)SpanKind(115-115)SpanStatus(143-143)core/schemas/context.go (1)
BifrostContext(32-42)framework/tracing/tracer.go (1)
Tracer(17-20)
transports/bifrost-http/server/server.go (8)
transports/bifrost-http/handlers/devpprof.go (3)
DevPprofHandler(100-102)IsDevMode(109-111)NewDevPprofHandler(125-129)core/schemas/plugin.go (3)
Plugin(71-97)BifrostHTTPMiddleware(38-38)ObservabilityPlugin(123-137)plugins/governance/main.go (2)
Config(34-36)PluginName(23-23)plugins/telemetry/main.go (2)
Config(64-66)PluginName(21-21)transports/bifrost-http/lib/config.go (2)
Config(189-220)LoadConfig(276-356)framework/configstore/clientconfig.go (1)
ClientConfig(37-54)framework/tracing/store.go (1)
NewTraceStore(40-67)framework/tracing/tracer.go (1)
NewTracer(24-29)
plugins/jsonparser/main.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
plugins/otel/main.go (2)
core/schemas/plugin.go (1)
ObservabilityPlugin(123-137)core/schemas/trace.go (1)
Trace(10-19)
examples/plugins/hello-world/main.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
transports/bifrost-http/handlers/mcp.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
ui/app/clientLayout.tsx (1)
ui/components/devProfiler.tsx (1)
DevProfiler(55-422)
framework/tracing/store.go (4)
core/schemas/tracer.go (1)
Tracer(38-116)core/schemas/bifrost.go (1)
BifrostResponse(348-369)core/schemas/trace.go (6)
Trace(10-19)Span(52-65)SpanEvent(107-111)SpanKind(115-115)SpanStatusUnset(147-147)SpanStatus(143-143)framework/tracing/helpers.go (3)
GetTrace(23-29)AddSpan(32-38)EndSpan(50-56)
core/schemas/trace.go (2)
framework/tracing/helpers.go (1)
AddSpan(32-38)ui/lib/constants/logs.ts (1)
Status(164-164)
transports/bifrost-http/handlers/middlewares.go (5)
core/schemas/plugin.go (2)
BifrostHTTPMiddleware(38-38)Plugin(71-97)transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(11-23)core/schemas/tracer.go (1)
Tracer(38-116)framework/tracing/propagation.go (1)
ExtractParentID(74-84)core/schemas/trace.go (3)
SpanKindHTTPRequest(131-131)SpanStatusError(151-151)SpanStatusOk(149-149)
transports/bifrost-http/handlers/integrations.go (1)
core/schemas/plugin.go (1)
BifrostHTTPMiddleware(38-38)
framework/tracing/tracer.go (7)
framework/tracing/store.go (1)
TraceStore(25-37)framework/streaming/accumulator.go (2)
Accumulator(14-30)NewAccumulator(440-480)core/schemas/tracer.go (3)
Tracer(38-116)SpanHandle(11-11)StreamAccumulatorResult(15-33)framework/tracing/helpers.go (3)
GetTraceID(11-20)EndSpan(50-56)GetTrace(23-29)core/schemas/bifrost.go (5)
BifrostContextKeySpanID(145-145)BifrostRequest(175-196)BifrostResponse(348-369)BifrostError(490-499)BifrostContextKeyRequestID(125-125)framework/tracing/llmspan.go (1)
PopulateRequestAttributes(14-60)framework/streaming/types.go (1)
StreamResponseTypeFinal(24-24)
core/schemas/plugin.go (2)
examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)core/schemas/trace.go (1)
Trace(10-19)
transports/bifrost-http/lib/config_test.go (1)
transports/bifrost-http/lib/config.go (1)
LoadConfig(276-356)
framework/streaming/accumulator.go (1)
framework/streaming/types.go (3)
StreamAccumulator(103-115)ChatStreamChunk(77-87)ResponsesStreamChunk(90-100)
core/schemas/context.go (1)
core/schemas/bifrost.go (1)
BifrostContextKeyDeferTraceCompletion(148-148)
plugins/semanticcache/main.go (2)
core/schemas/plugin.go (2)
Plugin(71-97)BifrostHTTPMiddleware(38-38)examples/plugins/hello-world/main.go (1)
HTTPTransportMiddleware(19-27)
🪛 markdownlint-cli2 (0.18.1)
transports/changelog.md
15-15: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
| golang.org/x/arch v0.23.0 // indirect | ||
| github.com/valyala/bytebufferpool v1.0.0 // indirect | ||
| golang.org/x/arch v0.22.0 // indirect |
There was a problem hiding this comment.
Duplicate golang.org/x/arch dependency entries.
The go.mod file contains two conflicting entries for golang.org/x/arch:
- Line 19:
golang.org/x/arch v0.23.0 // indirect - Line 21:
golang.org/x/arch v0.22.0 // indirect
This is invalid go.mod syntax and will cause module resolution errors. Run go mod tidy to clean up the dependency graph and remove the duplicate.
🔎 Fix by running go mod tidy
cd examples/plugins/hello-world && go mod tidyThis will resolve the conflict and keep only one version of the dependency.
🤖 Prompt for AI Agents
In examples/plugins/hello-world/go.mod around lines 19 to 21 there are
duplicate/conflicting entries for golang.org/x/arch (v0.23.0 and v0.22.0) which
breaks module resolution; run `cd examples/plugins/hello-world && go mod tidy`
to let Go reconcile and remove the duplicate, or manually delete the
older/incorrect golang.org/x/arch line so only the resolved version remains and
then run `go mod tidy` to update go.sum.
| // spanHandle is the concrete implementation of schemas.SpanHandle for StoreTracer. | ||
| // It contains the trace and span IDs needed to reference the span in the store. | ||
| type spanHandle struct { | ||
| traceID string | ||
| spanID string | ||
| } |
There was a problem hiding this comment.
Update spanHandle comment to match the actual type name.
The comment still refers to “StoreTracer” but the type is Tracer. This is minor, but it’s confusing when skimming the file; please rename it to reference Tracer.
Suggested doc tweak
-// spanHandle is the concrete implementation of schemas.SpanHandle for StoreTracer.
+// spanHandle is the concrete implementation of schemas.SpanHandle for Tracer.
// It contains the trace and span IDs needed to reference the span in the store.
type spanHandle struct {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // spanHandle is the concrete implementation of schemas.SpanHandle for StoreTracer. | |
| // It contains the trace and span IDs needed to reference the span in the store. | |
| type spanHandle struct { | |
| traceID string | |
| spanID string | |
| } | |
| // spanHandle is the concrete implementation of schemas.SpanHandle for Tracer. | |
| // It contains the trace and span IDs needed to reference the span in the store. | |
| type spanHandle struct { | |
| traceID string | |
| spanID string | |
| } |
🤖 Prompt for AI Agents
In framework/tracing/tracer.go around lines 54 to 59, the type comment for
spanHandle incorrectly references "StoreTracer"; update the doc comment to
reference the actual type name "Tracer" (or make it generic) so it matches the
type declaration and clearly describes that spanHandle is the concrete
implementation of schemas.SpanHandle for Tracer and contains the trace and span
IDs used to reference the span.
Summary
Replace the
TransportInterceptorplugin interface with a more flexibleHTTPTransportMiddlewareapproach that follows the standard middleware pattern used in HTTP servers.Changes
TransportInterceptormethod in thePlugininterface withHTTPTransportMiddlewarethat returns a middleware functionBifrostHTTPMiddlewaretype in the core schemas packageType of change
Affected areas
How to test
Test that plugins can still intercept and modify HTTP requests:
Then configure Bifrost to use this plugin and verify that the middleware is called during HTTP requests.
Breaking changes
This is a breaking change for plugin developers. Any custom plugins that implement the
TransportInterceptormethod will need to be updated to implement the newHTTPTransportMiddlewaremethod instead.Migration instructions:
TransportInterceptorwithHTTPTransportMiddlewarein your pluginSecurity considerations
The new middleware pattern provides better isolation between plugins and more control over the request/response lifecycle, which could improve security by reducing the risk of plugins interfering with each other.
Checklist