CodSpeed Performance Report

Merging #1456 will not alter performance

_{Comparing lysnikolaou:free-threading-support (dcb3ae1) with master (0db43c6)}

Summary

✅ 99 untouched benchmarks

Codecov Report

Attention: Patch coverage is 95.55556% with 6 lines in your changes missing coverage. Please review.

Project coverage is 95.49%. Comparing base (0db43c6) to head (dcb3ae1).

@@            Coverage Diff             @@
##           master    #1456      +/-   ##
==========================================
- Coverage   95.57%   95.49%   -0.09%     
==========================================
  Files          33       33              
  Lines        6010     6032      +22     
  Branches      364      364              
==========================================
+ Hits         5744     5760      +16     
- Misses        240      246       +6     
  Partials       26       26              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

FTR the CI is failing due to actions/setup-python#1034 and the following fix will need to be implemented here: aio-libs/frozenlist#622 + aio-libs/frozenlist#633.

Out of curiosity, have you checked the importorskip() idea? Does it not work?

Yes. mypy fails if we use importorskip, because it cannot find the BenchmarkFixture attribute. Not sure why that is.

Out of curiosity, have you checked the importorskip() idea? Does it not work?

Yes. mypy fails if we use importorskip, because it cannot find the BenchmarkFixture attribute. Not sure why that is.

The old code referenced the result of importorskip() instead of importing the module, no type checker is going to know what that is. If you wanted to use both, you could use importorskip for the runtime behaviour, then use if TYPE_CHECKING to import the module for type checkers.

i.e.

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from pytest_codspeed import BenchmarkFixture

def test...(benchmark: "BenchmarkFixture"):

If you wanted to use both, you could use importorskip for the runtime behaviour, then use if TYPE_CHECKING to import the module for type checkers.

Thanks for the pointer. Do you feel that this is a better approach than the one currently implemented? The current one does both in a single try/except, so it feels a bit clener to me.

If you wanted to use both, you could use importorskip for the runtime behaviour, then use if TYPE_CHECKING to import the module for type checkers.

Thanks for the pointer. Do you feel that this is a better approach than the one currently implemented? The current one does both in a single try/except, so it feels a bit clener to me.

I don't mind either way. @webknjaz?

FTR the CI is failing due to actions/setup-python#1034 and the following fix will need to be implemented here: aio-libs/frozenlist#622 + aio-libs/frozenlist#633.

One alternative to this would be to use setup-uv instead of setup-python, which could also speed up dependency installation etc. If you feel okay about this, I can try it out in a separate PR. Otherwise, I can also move the custom actions to this repo as well.

FTR the CI is failing due to actions/setup-python#1034 and the following fix will need to be implemented here: aio-libs/frozenlist#622 + aio-libs/frozenlist#633.

One alternative to this would be to use setup-uv instead of setup-python, which could also speed up dependency installation etc. If you feel okay about this, I can try it out in a separate PR. Otherwise, I can also move the custom actions to this repo as well.

I'm still looking into the uv integration, I'm not sure it's ready for prime time just yet. I'll be looking into that separately. But feel free to send a PR with the caching fix separately.

The performance regression in the writer looks real. I can't come up with a better solution though that is thread-safe.

The performance regression in the writer looks real. I can't come up with a better solution though that is thread-safe.

We could try to make it so that the global buffer is used in the non-free-threaded build, but not in the free-threaded one. Cython, however, makes that particularly hard. There's two possible solutions:

• Rewrite the buffer handling in C, where we can use #ifdef Py_GIL_DISABLED.
• Use the Cython compile-time DEF and IF features, that have been deprecated and have no good alternatives.

Both of the solutions are sub-optimal. Thoughts?

The C solution sounds better because it will work long term vs the DEF and IF are guaranteed to break sooner

Are there any plans to expose Py_GIL_DISABLED in Cython? If so, might make more sense to come back and fix it when that happens.

If we do want to write it without Cython, should we consider Rust instead of C?