Skip to content

tokenizer: propagate trace context on vLLM render requests - #2245

Merged
vMaroon merged 2 commits into
llm-d:mainfrom
mschulist:mschulist-tokenzier-tracing
Aug 1, 2026
Merged

tokenizer: propagate trace context on vLLM render requests#2245
vMaroon merged 2 commits into
llm-d:mainfrom
mschulist:mschulist-tokenzier-tracing

Conversation

@mschulist

@mschulist mschulist commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Wrap the render HTTP transport with otelhttp.NewTransport so that outbound /render requests carry the W3C traceparent header. This links the tokenizer-to-vLLM hop into the same trace span as the EPP request, matching the existing pattern used by the sidecar proxy transport.

Copilot AI review requested due to automatic review settings July 31, 2026 03:51
@mschulist
mschulist requested review from a team, liu-cong, sagearc and vMaroon as code owners July 31, 2026 03:51
@github-actions

Copy link
Copy Markdown
Contributor

🚨 Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

@github-actions github-actions Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jul 31, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR instruments the tokenizer’s vLLM /render HTTP client transport with OpenTelemetry’s otelhttp transport wrapper so outbound render requests propagate W3C trace context (traceparent), aligning tokenizer→vLLM tracing behavior with the existing sidecar proxy transport pattern.

Changes:

  • Wrap the vLLM render http.Client transport with otelhttp.NewTransport(...) so trace context is injected on outbound requests.
  • Add a unit test asserting that a traceparent header is present and carries the expected trace ID.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
pkg/epp/framework/plugins/requestcontrol/dataproducer/tokenizer/vllm_http.go Wraps the render transport with otelhttp.NewTransport to propagate trace context on outbound /render calls.
pkg/epp/framework/plugins/requestcontrol/dataproducer/tokenizer/vllm_http_test.go Adds a test validating traceparent injection for render requests.
Suppressed comments (1)

pkg/epp/framework/plugins/requestcontrol/dataproducer/tokenizer/vllm_http_test.go:329

  • After adding a synchronization channel for the handler, wait for it before asserting on gotTraceparent so the test has a defined happens-before relationship.
	_, _, err = r.Render(ctx, fwkrh.PayloadMap{"prompt": "hello"})
	require.NoError(t, err)

	if gotTraceparent == "" {
		t.Fatal("expected traceparent header to be injected into outbound render request, got none")

@mschulist
mschulist force-pushed the mschulist-tokenzier-tracing branch 2 times, most recently from 91d49ae to 48bbc67 Compare July 31, 2026 04:05
Wrap the render HTTP transport with otelhttp.NewTransport so that
outbound /render requests carry the W3C traceparent header. This links
the tokenizer-to-vLLM hop into the same trace span as the EPP request,
matching the existing pattern used by the sidecar proxy transport.

Signed-off-by: Mark Schulist <mschulist2@gmail.com>
@mschulist
mschulist force-pushed the mschulist-tokenzier-tracing branch from 48bbc67 to cb45b51 Compare July 31, 2026 04:07
@vMaroon
vMaroon merged commit 5393d9f into llm-d:main Aug 1, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants