Enable OTel instrumentation and propagation

[javiermtorres]

This PR adds optional flags to enable OTel distributed tracing in the httpx/starlette packages.

[javiermtorres]

There are comments in a test showing the use of the new flags. Maybe these should instead be added to one of the cookbooks (or a new one be created).

[njbrake]

Can you provide some more information about why this is required? I don't completely follow why the existing guidance from https://mozilla-ai.github.io/any-agent/tracing/#adding-an-opentelemetry-exporter isn't sufficient.

[javiermtorres]

The key point here is context propagation. Without instrumenting both the (outgoing) httpx library and the (incoming) starlette library, the traces in agents across a2a boundaries will carry different trace ids. With this change, the trace id is carried in the traceparent header (following the W3C specifications) and the spans in both the caller and callee agent can be grouped together in the OpenTelemetry collector, as shown here:

Otherwise, the spans would not be related and we wouldn't be able to get, for example, cost information related to the complete scenario. Note that the first invoke_agent in the picture would belong to the main agent in the test, while the last invoke_agent in the picture would belong to the A2A-served date agent.

[njbrake]

@javiermtorres thanks for the clarification. Are there any alternatives? Like, could we have some mechanism to pass a context id between a server and client?

I don't have a handle on what options are available to solve the problem.

[javiermtorres]

The basic info on distributed tracing can be found here: https://opentelemetry.io/docs/concepts/context-propagation/
As linked there, the main header we use is traceparent: https://www.w3.org/TR/trace-context/#traceparent-header
This issue afaict is often solved at the transport/session level, rather than application.

This solution is pretty standard: we only need to add the appropriate intrumentation packages, instrument a couple of objects, and we're done. It's optional and disabled by default after all, because the casual user won't need to correlate traces on A2A-distributed agents since they can always use https://mozilla-ai.github.io/any-agent/agents/tools/#callables and gather everything locally. But in a distributed setting, this will help.

Maybe I should add a small piece about how to instrument an httpx client so that an A2A served agent can be appropriately traced out of the use of the A2A tool.

[github-actions]

This PR is stale because it has been open 7 days with no activity. Remove stale label or comment or this will be closed in 3 days.

[javiermtorres]

This will be closed since the user is expected to provide their own tracing facilities. A follow up PR will provide the hooks a user would need to instrument the library appropriately.