gh-108388: regrtest runs slowest tests first#108391
gh-108388: regrtest runs slowest tests first#108391vstinner wants to merge 3 commits intopython:mainfrom
Conversation
The Python test suite (regrtest) now runs the 20 slowest tests first and then other tests, to better use all available CPUs when running tests in parallel.
This reverts commit 7a6cc3e.
|
I've thought about doing this in the past. It sort of defeats the purpose of the regrtest -r randomization. It also means people running the testsuite themselves don't get a nice feeling of general feedback on their change immediately because the test run just sits there using all 20 of their cores on long tests for a 30-60s before anything is available to run our 400 fast tests that collectively otherwise complete within the first 20 seconds. Perhaps have this look at the -j number and only schedule the 0.7*j slowest tests up front. On Github actions CI systems with basically only one core (two hyperthreads is basically 1.1-1.2 cores), the ordering of fast vs slow tests is not likely to make much difference. buildbots sometimes have more cores though. |
|
It we manage to split enough tests so no test takes longer than 1 minute, it should help a lot of the OS scheduler to better use all available CPUs, and so this change becomes less relevant, or even useless. I close this PR for that. I wanted to give it a try to see how it looks. But when I used it, it was bad that no test completed in the first 30 seconds, as if something was broken :-( So yeah, for users, it's unpleasant :-( We should investigate other approaches to make tests faster and keep them "responsible" (display progress to users ;-)). |
The Python test suite (regrtest) now runs the 20 slowest tests first and
then other tests, to better use all available CPUs when running tests in
parallel.
XXX: for tests, the PR also reverts the commit 7a6cc3e.