Skip to content

ref(core): Add more specific event mechanisms and span origins to openAiIntegration #17288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from
Open

ref(core): Add more specific event mechanisms and span origins to openAiIntegration #17288

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
046bc70
ref(core): Add more specific event `mechanism`s to `openAiIntegration`
Lms24 Jul 31, 2025
c3e94e7
update type to trace origin, set origin
Lms24 Aug 1, 2025
4a43a59
fix tests
Lms24 Aug 1, 2025
33edf3d
adjust streaming response mechanism
Lms24 Aug 1, 2025
6b13d3c
fix integration tests
Lms24 Aug 7, 2025
124b977
fix cloudflare integration tests
Lms24 Aug 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion dev-packages/cloudflare-integration-tests/suites/tracing/openai/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ it('traces a basic chat completion request', async () => {
data: expect.objectContaining({
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-3.5-turbo',
'gen_ai.request.temperature': 0.7,
Expand All @@ -31,7 +32,7 @@ it('traces a basic chat completion request', async () => {
}),
description: 'chat gpt-3.5-turbo',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
}),
]),
);
Expand Down
32 changes: 16 additions & 16 deletions dev-packages/node-integration-tests/suites/tracing/openai/openai-tool-calls/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ describe('OpenAI Tool Calls integration', () => {
data: {
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.available_tools': WEATHER_TOOL_DEFINITION,
Expand All @@ -83,15 +83,15 @@ describe('OpenAI Tool Calls integration', () => {
},
description: 'chat gpt-4',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Second span - chat completion with tools and streaming
expect.objectContaining({
data: {
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.stream': true,
Expand All @@ -111,15 +111,15 @@ describe('OpenAI Tool Calls integration', () => {
},
description: 'chat gpt-4 stream-response',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Third span - responses API with tools (non-streaming)
expect.objectContaining({
data: {
'gen_ai.operation.name': 'responses',
'sentry.op': 'gen_ai.responses',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.available_tools': WEATHER_TOOL_DEFINITION,
Expand All @@ -137,15 +137,15 @@ describe('OpenAI Tool Calls integration', () => {
},
description: 'responses gpt-4',
op: 'gen_ai.responses',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Fourth span - responses API with tools and streaming
expect.objectContaining({
data: {
'gen_ai.operation.name': 'responses',
'sentry.op': 'gen_ai.responses',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.stream': true,
Expand All @@ -165,7 +165,7 @@ describe('OpenAI Tool Calls integration', () => {
},
description: 'responses gpt-4 stream-response',
op: 'gen_ai.responses',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
]),
Expand All @@ -179,7 +179,7 @@ describe('OpenAI Tool Calls integration', () => {
data: {
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.messages': '[{"role":"user","content":"What is the weather like in Paris today?"}]',
Expand All @@ -200,15 +200,15 @@ describe('OpenAI Tool Calls integration', () => {
},
description: 'chat gpt-4',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Second span - chat completion with tools and streaming with PII
expect.objectContaining({
data: {
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.stream': true,
Expand All @@ -230,15 +230,15 @@ describe('OpenAI Tool Calls integration', () => {
},
description: 'chat gpt-4 stream-response',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Third span - responses API with tools (non-streaming) with PII
expect.objectContaining({
data: {
'gen_ai.operation.name': 'responses',
'sentry.op': 'gen_ai.responses',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.messages': '[{"role":"user","content":"What is the weather like in Paris today?"}]',
Expand All @@ -258,15 +258,15 @@ describe('OpenAI Tool Calls integration', () => {
},
description: 'responses gpt-4',
op: 'gen_ai.responses',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Fourth span - responses API with tools and streaming with PII
expect.objectContaining({
data: {
'gen_ai.operation.name': 'responses',
'sentry.op': 'gen_ai.responses',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.stream': true,
Expand All @@ -288,7 +288,7 @@ describe('OpenAI Tool Calls integration', () => {
},
description: 'responses gpt-4 stream-response',
op: 'gen_ai.responses',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
]),
Expand Down
48 changes: 24 additions & 24 deletions dev-packages/node-integration-tests/suites/tracing/openai/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ describe('OpenAI integration', () => {
data: {
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-3.5-turbo',
'gen_ai.request.temperature': 0.7,
Expand All @@ -32,15 +32,15 @@ describe('OpenAI integration', () => {
},
description: 'chat gpt-3.5-turbo',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Second span - responses API
expect.objectContaining({
data: {
'gen_ai.operation.name': 'responses',
'sentry.op': 'gen_ai.responses',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-3.5-turbo',
'gen_ai.response.model': 'gpt-3.5-turbo',
Expand All @@ -57,29 +57,29 @@ describe('OpenAI integration', () => {
},
description: 'responses gpt-3.5-turbo',
op: 'gen_ai.responses',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Third span - error handling
expect.objectContaining({
data: {
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'error-model',
},
description: 'chat error-model',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'unknown_error',
}),
// Fourth span - chat completions streaming
expect.objectContaining({
data: {
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.temperature': 0.8,
Expand All @@ -99,15 +99,15 @@ describe('OpenAI integration', () => {
},
description: 'chat gpt-4 stream-response',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Fifth span - responses API streaming
expect.objectContaining({
data: {
'gen_ai.operation.name': 'responses',
'sentry.op': 'gen_ai.responses',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.stream': true,
Expand All @@ -126,7 +126,7 @@ describe('OpenAI integration', () => {
},
description: 'responses gpt-4 stream-response',
op: 'gen_ai.responses',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Sixth span - error handling in streaming context
Expand All @@ -137,11 +137,11 @@ describe('OpenAI integration', () => {
'gen_ai.request.stream': true,
'gen_ai.system': 'openai',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
},
description: 'chat error-model stream-response',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'internal_error',
}),
]),
Expand All @@ -155,7 +155,7 @@ describe('OpenAI integration', () => {
data: {
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-3.5-turbo',
'gen_ai.request.temperature': 0.7,
Expand All @@ -176,15 +176,15 @@ describe('OpenAI integration', () => {
},
description: 'chat gpt-3.5-turbo',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Second span - responses API with PII
expect.objectContaining({
data: {
'gen_ai.operation.name': 'responses',
'sentry.op': 'gen_ai.responses',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-3.5-turbo',
'gen_ai.request.messages': '"Translate this to French: Hello"',
Expand All @@ -203,30 +203,30 @@ describe('OpenAI integration', () => {
},
description: 'responses gpt-3.5-turbo',
op: 'gen_ai.responses',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Third span - error handling with PII
expect.objectContaining({
data: {
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'error-model',
'gen_ai.request.messages': '[{"role":"user","content":"This will fail"}]',
},
description: 'chat error-model',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'unknown_error',
}),
// Fourth span - chat completions streaming with PII
expect.objectContaining({
data: expect.objectContaining({
'gen_ai.operation.name': 'chat',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.temperature': 0.8,
Expand All @@ -249,15 +249,15 @@ describe('OpenAI integration', () => {
}),
description: 'chat gpt-4 stream-response',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Fifth span - responses API streaming with PII
expect.objectContaining({
data: expect.objectContaining({
'gen_ai.operation.name': 'responses',
'sentry.op': 'gen_ai.responses',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
'gen_ai.system': 'openai',
'gen_ai.request.model': 'gpt-4',
'gen_ai.request.stream': true,
Expand All @@ -278,7 +278,7 @@ describe('OpenAI integration', () => {
}),
description: 'responses gpt-4 stream-response',
op: 'gen_ai.responses',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'ok',
}),
// Sixth span - error handling in streaming context with PII
Expand All @@ -290,11 +290,11 @@ describe('OpenAI integration', () => {
'gen_ai.request.messages': '[{"role":"user","content":"This will fail"}]',
'gen_ai.system': 'openai',
'sentry.op': 'gen_ai.chat',
'sentry.origin': 'manual',
'sentry.origin': 'auto.ai.openai',
},
description: 'chat error-model stream-response',
op: 'gen_ai.chat',
origin: 'manual',
origin: 'auto.ai.openai',
status: 'internal_error',
}),
]),
Expand Down
Loading
Loading