Implemented multithreaded test execution #12
Conversation
|
The macos failure is on timeout, so that's odd. |
|
Cool! I'll check it out soon as I can. Maybe the default timeout of 2s is too low for CI. |
| output_file = os.path.join(current_dir, unique_output) | ||
| else: | ||
| output_file = os.path.join(current_dir, step.output) | ||
|
|
There was a problem hiding this comment.
Makes sense for /tmp case. However, if the step specifies an output like output: file.out in the current directory, would that still need conflict resolution?
There was a problem hiding this comment.
I think so because you could have two threads accessing the same file at the same time, I'll run a test.
JustinMeimar
left a comment
JustinMeimar
left a comment
There was a problem hiding this comment.
Passes dragon-runner tests and Gazprea solutions tests on my machine so looks good. I think stdout needs to be flushed after each pass thought, because I see the outputs clumping up and dumping at once.
Sir-NoChill
force-pushed
the
ayrton/multithreaded
branch
from
3ceae40 to
a509395
Compare
|
I sped it up even more. Now we execute full packages in parallel but the dumping is more clumped. Personally I think it's a good enough trade-off, we can reduce test runtime significantly: time dragon-runner config -j8 --> 73.25s user 21.85s system 472% cpu 20.143 total
time dragon-runner config -j1 --> 37.12s user 11.78s system 103% cpu 47.171 totalI'll try to fix the buffer output |
|
Looks good! |
|
Do you want to merge this @Sir-NoChill ? I think it looks good. |
|
I just wanted to fix the output buffering, just haven't gotten around to it |
|
Closing for now since it's been a couple months. There are some architectural changes I'm planning on, so we can reopened this later. |
Multithreading implementation
This PR adds a -j support to the tester. It maintains in-order printing of all output by a queue of outputs and performs multithreading on a sub-package granularity (by subdirectory).
This should work for both individual configs and the grading config, but we may want to test on a larger scale before deployment.