WIP: Support exemplars in WSGI and ASGI histogram

[felix-hilden]

Description

I'm attempting to fix exemplar support for WSGI and ASGI applications by moving the metric recording to happen within the active span. This will eliminate the need for similar custom middleware metrics that do emit exemplars out of the box already.

Fixes #3031

Type of change

I'm not sure whether this is an omission, or something that was deliberately not developed yet. It has not been explicitly advertised, but I would certainly expect it to behave like this.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Test A

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[linux-foundation-easycla]

• ✅login: felix-hilden / name: Felix Hildén / (dd03138)

The committers listed above are authorized under a signed CLA.

[Chromatius]

I recently came across this issue in the FastAPI instrumentation and tracked it down to the underlying ASGI instrumentation, though it seems to be present in WSGI and Tornado as well as mentioned in the issue. Just thought I would chip in here a bit.

I see two possible solutions:

Option A: defer closing the span until metrics are recorded

This would be what you have started on for WSGI and seems like the more intuitive and clean method. The span end time will become a bit longer with this method but it should be negligible.

Option B: re-create span context after closing the span and pass it to metrics

finally:
    ...
    # Create span context for exemplars
    span_ctx = set_span_in_context(span)
    
    if self.duration_histogram_old:
        duration_attrs_old = _parse_duration_attrs(
            req_attrs, _StabilityMode.DEFAULT, context=span_ctx,
        )
   ...

This is a more minimal and local patch but less clean.

I'd be happy to chip in if you need any assistance with the PR.

[felix-hilden]

Thanks for the help! I properly linked the issue now if there's more discussion here. Like I said in the ticket, I'm not sure if the span context is the issue (unless I have misunderstood something), but let's continue discussing there 🙏 I'd love another set of eyes.

[Chromatius]

I made a PR felix-hilden#1 with my proposed change into your branch including some tests as well. The diff is a bit large since it's based on a more recent version of the upstream repo.