test: add pytest-randomly to cuda_bindings

[cpcloud]

Add pytest-randomly to cuda_bindings. Tests are randomized by default. pytest-randomly docs

[copy-pr-bot]

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[cpcloud]

/ok to test

[github-actions]

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-1267/
https://nvidia.github.io/cuda-python/pr-preview/pr-1267/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-1267/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-1267/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

[rwgk]

Looks like this particular CI run uncovered 3 types of errors.

It seems to be — random — unsurprisingly I guess, e.g. https://github.com/NVIDIA/cuda-python/actions/runs/19505585558?pr=1268 happened to succeed, and one of the errors here was also triggered under https://github.com/NVIDIA/cuda-python/actions/runs/19505615062?pr=1269

At the initial stage introducing pytest-randomly, maybe it'll help us to start with a fixed seed, e.g.

export RANDOMLY_SEED=12345

?

[cpcloud]

At the initial stage introducing pytest-randomly, maybe it'll help us to start with a fixed seed, e.g.

In practice this leads to never changing that because we'll forget about it, so we should fix the errors and then let it continue to run in a random order. The idea is to be robust to the randomness, acknowledging that this can make things seem flaky.

While I don't like leaving PRs open, this one seems like it might need to be open to get to a point where it's worth merging.

[rwgk]

so we should fix the errors and then let it continue to run in a random order.

This could be exhausting, and we risk not getting this done in a reasonable timeframe.

It could also be distracting in unfortunate ways, e.g. around releases.

because we'll forget about it,

I think we should do this in a controlled way and create a bug to track the stages, roughly:

• Get tests working with fixed random seed.
• After working with that for about a month, change the random seed (but still fixed).
• After no significant distractions for a month, remove the fixed random seed entirely.

[kkraus14]

@cpcloud there's an inherent level of statefulness in CUDA, i.e. driver initialization, context creation and setting it to be current, etc. This statefulness is also quite expensive where we absolutely can't afford to create and tear down contexts for each test for example. Is there a way we can randomize at the module or class level instead of the test level so that we can have more control over some assumptions and control some of these costs?