feat: OpenTelemetry context activation

[bantonsson]

TL;DR

This PR is opened as a basis for discussion around providing a more seamless interoperability for users of the OpenTelemetry APIs when using frameworks and libraries that use tracing at their core.

An OpenTelemetry Context that mirrors the active tracing Span is now active on the executing thread while in a tracing Span. Having proper OpenTelemetry context activation will not only make interoperability easier for users of the OpenTelemetry API, but will also open up the possibility for simpler log/trace/baggage correlation and in the future profiling/trace correlation among other things.

Motivation

The aim is to provide users of the OpenTelemetry API a more seamless and consistent experience. One such thing is that calling the OpenTelemetry API Context::current() did not give you a Context that contained an OpenTelemetry Span that represented the current tracing Span. Another thing is propagation of Baggage or user defined types that were not picked up either.

A lot of effort has been put into making the existing OpenTelemetrySpanExt API work the same way to maintain maximum backwards compatibility and to avoid fragmentation by building a separate OpenTelemetry and Tokio Tracing bridge.

Long discussion about OpenTelemetry and Tokio Tracing interoperability, and an issue specifically discussing this POC.

Solution

This solution adds a new feature activate_context that will activate/deactivate an OpenTelemetry Context that mirrors the current tracing Span. This feature is currently on by default, but can be turned off to avoid any performance overhead incurred by the context activation and bookkeeping.

The second big change is that the OpenTelemetry SpanBuilder is lazily consumed and a real OpenTelemetry Span is created when necessary, so there is no longer any need for the PreSampledTracer and the special code in OpenTelemetry that it relies on. This has some implications, like you can no longer set the parent_context after you have entered a Span or called the context method on the Span.

Benchmark numbers are available in this comment, and when the activate_context feature is turned off, the performance stays the same. (There has been one more optimization since the last run there, fixing the regression in the many_events benchmarks)

[djc]

I have been passively maintaining tracing-opentelemetry (mostly by reviewing relatively small PRs) for a few months, but I don't currently have a use case for this crate. Unless some funding for doing this work materializes, I'm not motivated to review larger efforts like this, so I suggest the OpenTelemetry and/or tracing organizations find some other people to move this effort forward.

[mladedav]

I'll take a closer look at everything (including the previous discussions we've had in the other repo) later, but for now, I think the most useful thing would be to get rid of the feature and possibly other unrelated changes (maybe the patched dependencies?) to make it easier to understand which parts here are left because they are needed and which parts are here just for compatibility with the old way.

Generally, I think this would be a step in the right direction, though I'd still want to explore the possibility of providing parents at span creation time and hopefully starting the context then, which would be closer to what opentelemetry specifies.

[bantonsson]

Thanks for the comments @mladedav. I think that getting rid of the feature is a bit risky, since these changes do have performance implications, and it would be good to allow current users to continue.

I'll rebase the PR and try to clean it up a bit.

[bantonsson]

I've updated the PR with the proposed changes except returning the Result from set_parent, since I think it needs more discussion.

[mladedav]

Regarding the feature, would it make sense to have it instead as a config on the layer? Features are notoriously problematic for changing behavior due to feature unification, especially turning off default features since crates tend not to opt out of them. I wouldn't expect this to be a transitive dependency though so it might be fine, but I still think normal configuration would suit it more than a feature.

[bantonsson]

@mladedav I've change the code from a feature to a configuration.

[cijothomas]

@mladedav Would you have bandwidth to continue reviewing this PR? We have delayed cleaning up OTel Tracing API as it'll break tracing-opentelemetry bridge, without this PR merged.

[mladedav]

Sorry, I've created some threads some 23 days ago but did not send the review 🤦

[mladedav]

It's just some minor things so if @bantonsson doesn't have time right now, I can fix those things.

[cijothomas]

Sorry, I've created some threads some 23 days ago but did not send the review 🤦

np! (has happened to me too!). Thanks for your time.

[bantonsson]

@mladedav Just got back from vacation. I'll go through your comments.

[bantonsson]

Hey @mladedav I forgot to comment that I have addressed the review comments. It would be great if you could take a look when you have the time.

[mladedav]

Just a small final cleanup and I think we can merge this.

[bantonsson]

Changed the test

[mladedav]

Thanks for pushing this forward.