Hybrid agent ASGI, DB Traces, and External Traces support #1617

lrafeei · 2025-12-29T22:53:54Z

This PR adds support and testing for the following:

ASGI Web Transactions (SERVER kind)
DB Traces (CLIENT kind)
External Traces (CLIENT kind)

github-actions · 2025-12-29T22:55:27Z

❌MegaLinter analysis: Error

Descriptor	Linter	Files	Fixed	Errors	Elapsed time
✅ ACTION	actionlint	7		0	0.94s
✅ MARKDOWN	markdownlint	7	0	0	1.31s
❌ PYTHON	ruff	994	23	1	1.02s
✅ PYTHON	ruff-format	994	27	0	0.32s
✅ YAML	prettier	15	0	0	1.39s
✅ YAML	v8r	15		0	6.79s
✅ YAML	yamllint	15		0	0.69s

Detailed Issues

❌ PYTHON / ruff - 1 error

::error title=Ruff (RUF005),file=tests/hybridagent_fastapi/test_otel_application.py,line=61,col=24,endLine=61,endColumn=96::tests/hybridagent_fastapi/test_otel_application.py:61:24: RUF005 Consider iterable unpacking instead of concatenation

See detailed reports in MegaLinter artifacts

newrelic/api/opentelemetry.py

codecov-commenter · 2025-12-29T23:12:51Z

Codecov Report

❌ Patch coverage is 91.54930% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.14%. Comparing base (d71dad1) to head (1b5ba1d).
⚠️ Report is 5 commits behind head on develop-hybrid-core-tracing.

Files with missing lines	Patch %	Lines
newrelic/core/database_utils.py	75.00%	2 Missing and 2 partials ⚠️
newrelic/api/opentelemetry.py	97.77%	0 Missing and 1 partial ⚠️
newrelic/hooks/hybridagent_opentelemetry.py	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@                       Coverage Diff                       @@
##           develop-hybrid-core-tracing    #1617      +/-   ##
===============================================================
+ Coverage                        79.93%   80.14%   +0.21%     
===============================================================
  Files                              213      213              
  Lines                            25168    25211      +43     
  Branches                          4001     4016      +15     
===============================================================
+ Hits                             20117    20205      +88     
+ Misses                            3622     3579      -43     
+ Partials                          1429     1427       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

newrelic/api/opentelemetry.py

newrelic/core/database_utils.py

+    partition = "aws"
+    if "amazonaws.cn" in host:
+        partition = "aws-cn"
+    elif "amazonaws-us-gov.com" in host:


In general, to fix incomplete URL substring sanitization, you should parse the URL (if necessary), extract the hostname, and then perform exact or well-scoped suffix checks on that hostname instead of using arbitrary substring searches. For AWS endpoints specifically, you should compare against known, exact domain suffixes (like .amazonaws.cn and .amazonaws-us-gov.com) or match a constrained pattern, not just search for the substring anywhere.

For this code, the most direct and non-breaking fix is to (1) ensure the input is treated strictly as a host/hostname (not a full URL string), and then (2) replace the substring checks with suffix checks that correctly distinguish the AWS partitions:

Treat host as the candidate hostname; if it accidentally contains a scheme or path, first parse it using urllib.parse.urlparse and extract .hostname.

Normalize host to lowercase for case-insensitive comparison.

Check:

If the hostname equals amazonaws.cn or ends with .amazonaws.cn, treat it as the aws-cn partition.

Else if it equals amazonaws-us-gov.com or ends with .amazonaws-us-gov.com, treat it as the aws-us-gov partition.

Otherwise keep the default aws.

This avoids matching cases like evil-amazonaws-us-gov.com.example.org, because such strings do not end with the exact AWS domain suffix. To implement this, we need to import urlparse from urllib.parse, normalize host, handle the case where parsing yields no hostname, and then perform stricter endswith checks in generate_dynamodb_arn. All changes are within newrelic/core/database_utils.py, and no functionality other than safer partition detection is modified.

hmstepanek

A couple suggestions (mainly about the tests)-otherwise looks good.

hmstepanek · 2026-01-01T20:09:56Z

newrelic/api/opentelemetry.py

+        if ("exception.escaped" in self.attributes) or (self.kind in (otel_api_trace.SpanKind.SERVER, otel_api_trace.SpanKind.CONSUMER) and isinstance(current_trace(), Sentinel)):
            # We need to end the transaction as well
-            self.nr_transaction.__exit__(*error)
+            self.nr_transaction.__exit__(*sys.exc_info())


I think you set error = sys.exc_info up above so I think this should still be *error here?

self.nr_transaction.__exit__(*error)

Whoops, you're right. Not sure why I changed it...

newrelic/api/opentelemetry.py

hmstepanek · 2026-01-02T22:43:26Z

newrelic/api/opentelemetry.py

+                host, port = scope["server"] = tuple(scope["server"])
+            else:
+                host, port = None, None
+            request_method = scope["method"]


Should we be using scope.get in case these keys don't exist?

newrelic/core/application.py

newrelic/core/database_utils.py

hmstepanek · 2026-01-02T22:56:18Z

newrelic/hooks/hybridagent_opentelemetry.py

-
-        hybrid_agent_tracer_provider = TracerProvider("hybrid_agent_tracer_provider")
-        _TRACER_PROVIDER = hybrid_agent_tracer_provider
+        _TRACER_PROVIDER = TracerProvider()


Just making sure we don't still need to pass "hybrid_agent_tracer_provider" into here?

Oh, I changed this logic slightly to be more in line with the actual TracerProvider's functionality. I'll flesh it out some more in the future, but basically, the OTel TracerProvider doesn't really take in a name.

hmstepanek · 2026-01-02T23:07:47Z

tests/hybridagent_fastapi/conftest.py

+    app_name="Python Agent Test (Hybrid Agent, FastAPI)", default_settings=_default_settings
+)
+
+os.environ["NEW_RELIC_CONFIG_FILE"] = str(Path(__file__).parent / "newrelic_fastapi.ini")


Maybe take a look at this fixture and see if you can re-use it or do something similar instead of having these .ini files and setting env var variables/etc.

Usage example here.

This is a fair point. I figured this would be going away soon anyway, so I didn't bother to try to clean up that logic. Basically, this will be part of the internal configuration setup and it will not be necessary to do this whole setting of the ini files.

…-client-traces

Co-authored-by: Hannah Stepanek <[email protected]>

lrafeei added 5 commits December 29, 2025 14:05

ASGI (server) support

d90de11

DynamoDB (Client) support

0b04de0

More DB framework tests

0ea3e2c

Requests (Client) support

75a57f4

Tweak cross agent tests

7586edf

github-advanced-security bot found potential problems Dec 29, 2025

View reviewed changes

newrelic/api/opentelemetry.py Fixed Show fixed Hide fixed

mergify bot added the tests-failing Tests failing in CI. label Dec 29, 2025

hmstepanek marked this pull request as ready for review December 31, 2025 20:37

hmstepanek requested a review from a team as a code owner December 31, 2025 20:37

Refactor database attr logic

d95f7a0

hmstepanek reviewed Dec 31, 2025

View reviewed changes

github-advanced-security bot found potential problems Dec 31, 2025

View reviewed changes

Reviewer suggestions and syntax

044e9b9

lrafeei force-pushed the hybrid-agent-asgi-and-client-traces branch from e909db5 to 044e9b9 Compare January 2, 2026 21:05

lrafeei requested a review from hmstepanek January 2, 2026 21:06

Oops, all syntax

1e8adfa

hmstepanek requested changes Jan 2, 2026

View reviewed changes

mergify bot and others added 4 commits January 2, 2026 23:12

Merge branch 'develop-hybrid-core-tracing' into hybrid-agent-asgi-and…

c92ed69

…-client-traces

Apply suggestions from code review

d61b26c

Co-authored-by: Hannah Stepanek <[email protected]>

Reviewer suggestions and linter fixes

2240552

Fix supportability metric merge resolution

1b5ba1d

lrafeei requested a review from hmstepanek January 3, 2026 00:57

hmstepanek approved these changes Jan 3, 2026

View reviewed changes

hmstepanek merged commit 535788d into develop-hybrid-core-tracing Jan 3, 2026
51 of 61 checks passed

hmstepanek deleted the hybrid-agent-asgi-and-client-traces branch January 3, 2026 01:20

mergify bot removed the tests-failing Tests failing in CI. label Jan 3, 2026

@@ -21,6 +21,8 @@
             import re
             import weakref
+            from urllib.parse import urlparse
             from newrelic.core.config import global_settings
             from newrelic.core.internal_metrics import internal_metric
@@ -911,11 +913,19 @@
                 # There are 3 different partition options.
                 # See  https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html for details.
                 partition = "aws"
-                if "amazonaws.cn" in host:
-                    partition = "aws-cn"
-                elif "amazonaws-us-gov.com" in host:
-                    partition = "aws-us-gov"
+                # Normalize and, if necessary, parse host to extract the hostname component.
+                if host:
+                    parsed = urlparse(host)
+                    hostname = parsed.hostname or host
+                    hostname = hostname.lower()
+                    # Match against known AWS partition suffixes, not arbitrary substrings.
+                    if hostname == "amazonaws.cn" or hostname.endswith(".amazonaws.cn"):
+                        partition = "aws-cn"
+                    elif hostname == "amazonaws-us-gov.com" or hostname.endswith(".amazonaws-us-gov.com"):
+                        partition = "aws-us-gov"
                 if partition and region and account_id and target:
                     return f"arn:{partition}:dynamodb:{region}:{account_id:012d}:table/{target}"

Hybrid agent ASGI, DB Traces, and External Traces support #1617

Hybrid agent ASGI, DB Traces, and External Traces support #1617

Uh oh!

Conversation

lrafeei commented Dec 29, 2025

Uh oh!

github-actions bot commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌MegaLinter analysis: Error

Detailed Issues

Uh oh!

Uh oh!

codecov-commenter commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Check failure

Uh oh!

Copilot Autofix

hmstepanek left a comment

Choose a reason for hiding this comment

Uh oh!

hmstepanek Jan 1, 2026

Choose a reason for hiding this comment

Uh oh!

lrafeei Jan 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hmstepanek Jan 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hmstepanek Jan 2, 2026

Choose a reason for hiding this comment

Uh oh!

lrafeei Jan 2, 2026

Choose a reason for hiding this comment

Uh oh!

hmstepanek Jan 2, 2026

Choose a reason for hiding this comment

Uh oh!

lrafeei Jan 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions bot commented Dec 29, 2025 •

edited

Loading

codecov-commenter commented Dec 29, 2025 •

edited

Loading