feat: Add OpenTelemetry instrumentation for ActiveJ HTTP server #13335

kcsurapaneni · 2025-02-18T06:46:43Z

Overview

This PR introduces OpenTelemetry instrumentation for the ActiveJ framework, enabling distributed tracing and context propagation for ActiveJ-based HTTP servers. The changes allow developers to monitor and debug ActiveJ applications in distributed systems by capturing trace context from incoming requests and propagating it through responses.

Key Features

Distributed Tracing: Automatically propagates trace context using the traceparent header.
HTTP Attribute Extraction: Captures detailed HTTP attributes (e.g., method, path, headers, status code) for enriched telemetry data.
Error Handling: Maps exceptions to appropriate HTTP status codes and associates them with spans for better error visibility.

Changes Included

Instrumentation:
- ActiveJHttpServerConnectionInstrumentation: Instruments the serve method of ActiveJ servlets to capture and propagate trace context.
Testing:
- Added test cases to validate tracing behavior and ensure compliance with W3C Trace Context.
Documentation:
- Added README with details about the instrumentation, usage, and configuration.

Testing

All existing tests pass successfully.
New test cases have been added to validate:
- Trace context propagation via the traceparent header.
- Correctness of HTTP attribute extraction.
- Error handling and span enrichment.

Benefits

Observability for ActiveJ applications, making it easier to monitor and debug distributed systems.
Seamless integration with OpenTelemetry exporters (e.g., Jaeger, Zipkin, HyperDX) for visualizing traces.

Related Issues

Closes #13202

Acknowledgments

This implementation was inspired by and adapted from the ActiveJ framework codebase. Special thanks to the ActiveJ community for their excellent work and contributions.

linux-foundation-easycla · 2025-02-18T06:46:48Z

The committers listed above are authorized under a signed CLA.

✅ login: renovate[bot] (f6ef942, aba23a3, 6ec9373, 2c31d8f, bb75dab, 0748348)
✅ login: kcsurapaneni / name: Krishna Chaitanya Surapaneni (a07317e, 1662b23, 4fd797f, b287f78, ce368c1, b241b45, d5bb920, 429ad65, b7aff9d, 7bf1776, e1a29dc, ac95594, 583e063, b4e8e29, f1cfe0e, b52c49d, 1cc6c5f, 5a4d2dd, 65e6d57)
✅ login: trask / name: Trask Stalnaker (20d00a0, 583801e, c098084, c0b67a3, 1420a18)
✅ login: otelbot[bot] (4c149bd, fea36de)
✅ login: laurit / name: Lauri Tulmin (5a92938, 37032d2)
✅ login: zeitlinger / name: Gregor Zeitlinger (c142d7f)

…nabling distributed tracing and context propagation for ActiveJ-based HTTP servers

jaydeluca

we will want integration tests for any new instrumentation

buildscripts/checkstyle.xml

.fossa.yml

...emetry/javaagent/instrumentation/activejhttp/ActiveJHttpServerConnectionInstrumentation.java

…nAsWordInName) at class level

kcsurapaneni · 2025-02-19T16:44:27Z

we will want integration tests for any new instrumentation

@jaydeluca I have added test cases and resolved most of the workflow job failures. But, still there are couple of them are failing and not able to understand the cause after checking the raw logs as well. Could you please help me to understand what causing them to failure so that I can fix them. Thank you!

CC: @trask @laurit

laurit · 2025-02-19T16:51:32Z

See https://scans.gradle.com/s/dy73bc67fptxo/tests/task/:instrumentation:activej-http-6.0:javaagent:test/details/(N%2FA)/Gradle%20Test%20Executor%2045?top-execution=1
Framework seems to require java 17

kcsurapaneni · 2025-02-19T17:03:22Z

See https://scans.gradle.com/s/dy73bc67fptxo/tests/task/:instrumentation:activej-http-6.0:javaagent:test/details/(N%2FA)/Gradle%20Test%20Executor%2045?top-execution=1 Framework seems to require java 17

@laurit Thanks for the link. So, all the agents should support minimum java 8?

laurit · 2025-02-19T17:13:00Z

@laurit Thanks for the link. So, all the agents should support minimum java 8?

If the framework does not support java 8 then you shouldn't run the tests for it with java 8. Add

opentelemetry-java-instrumentation/instrumentation/spring/spring-security-config-6.0/library/build.gradle.kts

Lines 18 to 20 in 4c8f2dc

    
           otelJava { 
        
             minJavaVersionSupported.set(JavaVersion.VERSION_17) 
        
           }

kcsurapaneni · 2025-02-19T17:49:07Z

@laurit Thanks for the link. So, all the agents should support minimum java 8?

If the framework does not support java 8 then you shouldn't run the tests for it with java 8. Add

opentelemetry-java-instrumentation/instrumentation/spring/spring-security-config-6.0/library/build.gradle.kts

Lines 18 to 20 in 4c8f2dc

otelJava {

minJavaVersionSupported.set(JavaVersion.VERSION_17)

}

Thanks for the help! @laurit . Build looks good now. Please review

instrumentation/activej-http-6.0/javaagent/README.md

instrumentation/activej-http-6.0/javaagent/build.gradle.kts

...emetry/javaagent/instrumentation/activejhttp/ActivejHttpServerConnectionInstrumentation.java

.../javaagent/instrumentation/activejhttp/ActivejHttpServerConnectionInstrumentationModule.java

...ain/java/io/opentelemetry/javaagent/instrumentation/activejhttp/ActivejHttpServerHelper.java

...entelemetry/javaagent/instrumentation/activejhttp/ActivejHttpServerHttpAttributesGetter.java

...t/java/io/opentelemetry/javaagent/instrumentation/activejhttp/ActivejRoutingServletTest.java

laurit · 2025-02-20T17:09:13Z

implementation dependencies are bundled into the agent. Replace this with library("io.activej:activej-http:6.0-rc2") library is a custom scope that means compileOnly + testImplementation

@laurit after making the suggested change build job is failed. You can check here https://github.com/open-telemetry/opentelemetry-java-instrumentation/actions/runs/13439272495/job/37549353284?pr=13335

Not sure why import is failing here, any idea? in my local it is working just fine

Latest dep tests try to automatically bump the library dependencies to latest version. You can run them locally by adding -PtestLatestDeps=true. Probably the issue is that the latest version detection ignores milestones and release candidates. Since for 6.x there are only release candidates it probably ends up choosing the latest 5.x version. You can work around it by explicitly setting the version for latest deps with latestDepTestLibrary("io.activej:activej-http:6.0-rc2").

kcsurapaneni · 2025-02-20T18:38:53Z

implementation dependencies are bundled into the agent. Replace this with library("io.activej:activej-http:6.0-rc2") library is a custom scope that means compileOnly + testImplementation

@laurit after making the suggested change build job is failed. You can check here https://github.com/open-telemetry/opentelemetry-java-instrumentation/actions/runs/13439272495/job/37549353284?pr=13335
Not sure why import is failing here, any idea? in my local it is working just fine

Latest dep tests try to automatically bump the library dependencies to latest version. You can run them locally by adding -PtestLatestDeps=true. Probably the issue is that the latest version detection ignores milestones and release candidates. Since for 6.x there are only release candidates it probably ends up choosing the latest 5.x version. You can work around it by explicitly setting the version for latest deps with latestDepTestLibrary("io.activej:activej-http:6.0-rc2").

@laurit now test-latest-deps / testLatestDeps2 is failing. Like you said in my local when I ran ./gradlew :instrumentation:activej-http-6.0:javaagent:test -PtestLatestDeps=true, build was successful.

I don't find the error in the logs as well, so not sure what is happening here. In Raw logs as well, don't see much info and it is really difficult to navigate. Last time you shared an URL like this - https://scans.gradle.com/s/dy73bc67fptxo/tests/task/:instrumentation:activej-http-6.0:javaagent:test/details/(N%2FA)/Gradle%20Test%20Executor%2045?top-execution=1. Can you tell me how to access that in case of failure, so that it will be easy for me in future. Thanks!

trask · 2025-02-20T19:59:29Z

The :instrumentation:spring:spring-webmvc:spring-webmvc-6.0:javaagent:test task failed

this failure looks unrelated to your PR

trask · 2025-02-20T21:27:27Z

this failure looks unrelated to your PR

it's been fixed in main now, should go away whenever you merge latest main into your branch and push

…emetry#13296) Co-authored-by: Lauri Tulmin <[email protected]>

) Co-authored-by: otelbot <[email protected]>

Co-authored-by: otelbot <[email protected]> Co-authored-by: Trask Stalnaker <[email protected]>

…-telemetry#13349) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

…emconv to v1.30.0 (open-telemetry#13351) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Lauri Tulmin <[email protected]>

…en-telemetry#13354) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

…ts to v4-rev20250211-2.0.0 (open-telemetry#13357) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

kcsurapaneni · 2025-02-21T07:05:03Z

this failure looks unrelated to your PR

it's been fixed in main now, should go away whenever you merge latest main into your branch and push

Thanks @trask . Looks good now

…Test

…rter-web to v3.4.3 (open-telemetry#13362) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

…20.5 (open-telemetry#13365) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

kcsurapaneni · 2025-02-21T12:15:05Z

@laurit all the comments are addressed. Please check again. Thanks!

laurit · 2025-02-21T14:11:50Z

...emetry/javaagent/instrumentation/activejhttp/ActivejHttpServerConnectionInstrumentation.java

+      instrumenter()
+          .end(
+              context,
+              httpRequest,
+              responsePromise == null ? null : responsePromise.getResult(),
+              throwable);


I doubt this is the right way. We want to end the span when the request processing ends. For async methods request processing ends when the promise completes not when the called method exits.

@laurit I don't see a better way to do this. I even looked into some of the existing async handling and this is how it is done, here is an example from google-http-client

the linked code in google http client only ends the span on method exit when there was an exception. Did you try what I proposed in #13335 (comment)

laurit · 2025-02-21T14:12:58Z

.../test/java/io/opentelemetry/javaagent/instrumentation/activejhttp/ActivejHttpServerTest.java

+
+public class ActivejHttpServerTest extends AbstractHttpServerTest<HttpServer> {
+
+  @ClassRule public static final EventloopRule eventloopRule = new EventloopRule();


classrule is for junit4 we use junit5

laurit · 2025-02-21T14:13:29Z

.../test/java/io/opentelemetry/javaagent/instrumentation/activejhttp/ActivejHttpServerTest.java

+  @Override
+  protected void assertHighConcurrency(int count) {
+    //
+  }


not implemented

kcsurapaneni requested a review from a team as a code owner February 18, 2025 06:46

kcsurapaneni force-pushed the activej branch from f90bed1 to dfa49e2 Compare February 18, 2025 09:51

Introduces OpenTelemetry instrumentation for the ActiveJ framework, e…

b52c49d

…nabling distributed tracing and context propagation for ActiveJ-based HTTP servers

kcsurapaneni force-pushed the activej branch from dfa49e2 to b52c49d Compare February 18, 2025 09:53

jaydeluca reviewed Feb 18, 2025

View reviewed changes

kcsurapaneni added 8 commits February 18, 2025 06:52

Resolved PR comments

1cc6c5f

instrumentation name is updated with version

b4e8e29

instrumentation name is updated with version

ce368c1

instrumentation name is updated with version

b287f78

checkstyle config is reverted and added @SuppressWarnings(Abbreviatio…

e1a29dc

…nAsWordInName) at class level

renames class files to avoid AbbreviationAsWordInName checkstyle rule

583e063

Test cases are added

b7aff9d

build script dependencies is updated

b241b45

kcsurapaneni force-pushed the activej branch from 7396363 to b241b45 Compare February 19, 2025 07:29

throwable is now read-only parameter

1662b23

base version is set to Java 17

4fd797f