Investigate removing pytest upper bound pinnings

[vyasr]

When pytest 8 was released it broke a number of popular pytest plugins. As a result, various RAPIDS projects introduced upper bound pinnings, e.g. rapidsai/cudf#14920. We should investigate removing those pinnings and see if we still have issues. IIRC in addition to compatibility with plugins, there were also changes to pytest's import machinery, specifically the --import-mode flag, and we may need to do some work in certain RAPIDS repositories to make their test suite compatible with those changes even if the upstream plugins have been adapted. That work should be done by the relevant teams.

[vyasr]

It looks like there are still issues with pytest 8, particularly relating to xdist I suspect based on the failures I see in rapidsai/cuml#6876 and rapidsai/ucxx#438. I'm guessing that the issues are UCX-specific since the cuml failures all seem to implicate UCX, but that's just a guess based on a quick glance. There may be more to the issues here.

[vyasr]

It looks like cudf has issues as well rapidsai/cudf#19127, but I've traced those to a different root cause.