Tests fail depending on their order

[fayalalebrun]

On the latest revision of the code base, tests will fail around 10-20% of the time, depending on their order. It's hard to trace the reason why, since each time the tests are run, their order is different.

As an idea, it would be useful to be able to choose the order of the tests. And for this order to be printed whenever it's randomly generated.

[NULLx76]

The tests are randomized here, if you want to test a specific order you can make your own test.c file. Using seeded random, allowing to give the seed and printing the seed could be a nicer solution than what currently exists. Though feel free to implement what you deem useful.

It's worth noting that the random order was a conscious choice exactly to find more problems like this.

[fayalalebrun]

Indeed, that does sound like the best solution. I'll get started on working on adding an option to seed the generation as well as printing the seed.

[jdonszelmann]

We made tests randomized precisely because we wanted to find such bugs that only happen in specific situations of memory usage. Most likely cause is something in the memory managers. Luckily it's quite easy to attach the debugger to the kernel