Bug 319307 - Patches to let test suite run in parallel
Summary: Patches to let test suite run in parallel
Status: REPORTED
Alias: None
Product: valgrind
Classification: Developer tools
Component: general (show other bugs)
Version: 3.9.0.SVN
Platform: Compiled Sources Linux
: NOR wishlist
Target Milestone: ---
Assignee: Julian Seward
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-05-04 10:23 UTC by Thomas Rast
Modified: 2024-07-28 14:51 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Mass-edit massif tests to use unique output names (63.50 KB, patch)
2013-05-04 10:24 UTC, Thomas Rast
Details
{call,cache}grind/tests: use 'rm -f' to clean up (7.47 KB, patch)
2013-05-04 10:25 UTC, Thomas Rast
Details
Execute tests after scanning, from toplevel directory (13.92 KB, patch)
2013-05-04 10:25 UTC, Thomas Rast
Details
Implement threaded regression tests (9.84 KB, patch)
2013-05-04 10:25 UTC, Thomas Rast
Details
regtests: use forks instead of threads (1.41 KB, patch)
2013-05-04 10:26 UTC, Thomas Rast
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Rast 2013-05-04 10:23:52 UTC
I got annoyed with the test suite being single-threaded, hence these patches. There are some preliminary "fixes" needed to make the tests work in parallel, because they would otherwise trample over their files or such. Then there is one big patch that would work in theory, but doesn't because Perl is currently broken, and a final band-aid to work around that for now.

You'll need to install the 'forks' module from CPAN for the band-aid to work. Otherwise it just ignores any request for threads.

BEFORE:
$ time perl tests/vg_regtest .
== 636 tests, 1 stderr failure, 0 stdout failures, 0 stderrB failures, 0 stdoutB failures, 0 post failures ==
./memcheck/tests/origin5-bz2             (stderr)

real    6m17.769s
user    4m14.089s
sys     0m33.888s

AFTER:
$ time perl tests/vg_regtest .
== 636 tests, 1 stderr failure, 0 stdout failures, 0 stderrB failures, 0 stdoutB failures, 0 post failures ==
./memcheck/tests/origin5-bz2             (stderr)

real    6m34.197s
user    4m18.843s
sys     0m38.544s

$ time perl tests/vg_regtest -j15 .
== 636 tests, 1 stderr failure, 0 stdout failures, 0 stderrB failures, 0 stdoutB failures, 0 post failures ==
./memcheck/tests/origin5-bz2             (stderr)

real    0m54.682s
user    6m46.404s
sys     0m41.829s

So about 7.3x speedup on a 6-core i7. (I stubbed out the mchelp test here, because it hangs. The origin5-bz2 failure seems to be a glibc-too-new issue.)

Reproducible: Always
Comment 1 Thomas Rast 2013-05-04 10:24:49 UTC
Created attachment 79692 [details]
Mass-edit massif tests to use unique output names
Comment 2 Thomas Rast 2013-05-04 10:25:06 UTC
Created attachment 79693 [details]
{call,cache}grind/tests: use 'rm -f' to clean up
Comment 3 Thomas Rast 2013-05-04 10:25:20 UTC
Created attachment 79694 [details]
Execute tests after scanning, from toplevel directory
Comment 4 Thomas Rast 2013-05-04 10:25:39 UTC
Created attachment 79695 [details]
Implement threaded regression tests
Comment 5 Thomas Rast 2013-05-04 10:26:20 UTC
Created attachment 79696 [details]
regtests: use forks instead of threads
Comment 6 Thomas Rast 2013-05-04 10:28:26 UTC
PS: the third patch actually fixes a directory-changing related bug by no longer changing dirs. If you do this in the current test suite:
$ perl tests/vg_regtest none/tests/amd64 memcheck
it fails to run any memcheck tests:
-- Finished tests in none/tests/amd64 ----------------------------------
`memcheck' neither a directory nor a readable test file/name

This happens because the recursion code uses 'chdir("..")' to go back one level, which is not sufficient if it was called to go more than one level deep (as in the none/tests/amd64 case).