Created attachment 181860 [details] proposed patch In some cases the LTP tests intentionally work with SIGSEGV. This happens e.g. with the mmap18 and select03 testcases. Valgrind detects SIGSEGV and reports that as a failure. Such report can't be suppressed using the suppressions mechanism. That's why this update comes with "output filters". Filters are scripts that read from their stdin, and write filtered output to their stdout. Filters reside in auxprogs/filters. This update comes with 2 filters: For mmap18, and for select03. They are awk scripts.
Hi Martin, (In reply to mcermak from comment #0) > Created attachment 181860 [details] > proposed patch > > In some cases the LTP tests intentionally work with SIGSEGV. This happens > e.g. with the mmap18 and select03 testcases. Valgrind detects SIGSEGV and > reports that as a failure. > > Such report can't be suppressed using the suppressions mechanism. That's > why this update comes with "output filters". Filters are scripts that read > from their stdin, and write filtered output to their stdout. Filters reside > in auxprogs/filters. > > This update comes with 2 filters: For mmap18, and for select03. They are > awk scripts. This looks really useful. > Except for filters, this update also blacklists testcase fork13 > because it is slow. It is possible to add comments prefixed with > the '#' sign (implicitly - because they don't match any testcase > name), and so a comment is added too. Nice trick. We should comment the others too. > This update also introduces new default valgrind arguments VGARGS. > They are: --gen-suppressions=all and --vgdb=no. The former is > there for user convenience, so that a suppression can be easily > captured and reused if needed. I am a little confused by this. Where do those suppressions appear and how/when would you use them? > The latter, --vgdb=no, improves > valgrind behavior with nftw01 nftw6401 setfsgid04 setfsgid03_16 > and symlink03 testcases. These were previously complaining like > this: > > ==22969== could not unlink /tmp/vgdb-pipe-from-vgdb-to-22969-by-root-on ... Aha. Yes, those testcases change uid during their runtime and so they no longer own that file. That seems OK. We aren't really testing the gdbserver functionality here. Also these test must be run with privilege. BTW. setfsgid04 doesn't seem to exist, did you mean setfsgid03?
Created attachment 181995 [details] possible patch (In reply to Mark Wielaard from comment #1) > Hi Martin, > > (In reply to mcermak from comment #0) > > Created attachment 181860 [details] > > proposed patch > > > > In some cases the LTP tests intentionally work with SIGSEGV. This happens > > e.g. with the mmap18 and select03 testcases. Valgrind detects SIGSEGV and > > reports that as a failure. > > > > Such report can't be suppressed using the suppressions mechanism. That's > > why this update comes with "output filters". Filters are scripts that read > > from their stdin, and write filtered output to their stdout. Filters reside > > in auxprogs/filters. > > > > This update comes with 2 filters: For mmap18, and for select03. They are > > awk scripts. > > This looks really useful. > > > Except for filters, this update also blacklists testcase fork13 > > because it is slow. It is possible to add comments prefixed with > > the '#' sign (implicitly - because they don't match any testcase > > name), and so a comment is added too. > > Nice trick. We should comment the others too. > > > This update also introduces new default valgrind arguments VGARGS. > > They are: --gen-suppressions=all and --vgdb=no. The former is > > there for user convenience, so that a suppression can be easily > > captured and reused if needed. > > I am a little confused by this. Where do those suppressions appear and > how/when would you use them? Oh, right.. These would only be produced by --tool=memcheck and would go to the --log-file. I've dropped this part. Thanks for pointing that out. > > The latter, --vgdb=no, improves > > valgrind behavior with nftw01 nftw6401 setfsgid04 setfsgid03_16 > > and symlink03 testcases. These were previously complaining like > > this: > > > > ==22969== could not unlink /tmp/vgdb-pipe-from-vgdb-to-22969-by-root-on ... > > Aha. Yes, those testcases change uid during their runtime and so they no > longer own that file. > That seems OK. We aren't really testing the gdbserver functionality here. > Also these test must be run with privilege. > > BTW. setfsgid04 doesn't seem to exist, did you mean setfsgid03? I think it does exist here: ./auxprogs/auxchecks/ltp-full-20250530/testcases/kernel/syscalls/setfsgid/setfsgid03 ./auxprogs/auxchecks/ltp-full-20250530/testcases/kernel/syscalls/setfsgid/setfsgid03_16 ./auxprogs/auxchecks/ltp-full-20250530/testcases/kernel/syscalls/setfsgid/setfsgid03.c I've referred to setfsgid03_16 in my comment. I've attached updated patch. One important update is the use of updated LTP sources. This LTP version already supports the new LTP_REPRODUCIBLE_OUTPUT feature, so the new version of the tester relies on that. I've dropped the part that was filtering the tail of the testcase logs. The LTP_REPRODUCIBLE_OUTPUT feature seems to do the trick. FInally, per Frank's suggestion, I've enriched the content of the .log files. These now contain summary of all the logs generated by valgrind. Please review the updated patch.
Created attachment 182001 [details] proposed patch (In reply to mcermak from comment #2) > Finally, per Frank's suggestion, I've enriched the content of the .log files... Ugh, the previous patch enriched the log files a bit too much — including even the core files, if those were avail. Attaching an amendment of the previous patch :)
Very nice. The upgrade to LTP 20250530 and LTP_REPRODUCIBLE_OUTPUT=1 and putting the full output/diffs in the logs make the wrapper script more useful. Pushed as: commit b8891a4c5d139b8c327ee20c37066d601fc56b0a Author: Martin Cermak <mcermak@redhat.com> Date: Tue Jun 3 13:43:18 2025 +0200 ltp tests: Use new LTP 20250530, allow for filtering valgrind output Use fresh LTP 20250530. Allow for filtering valgrind output: In some cases the LTP tests intentionally work with SIGSEGV. This happens e.g. with the mmap18 and select03 testcases. Valgrind detects SIGSEGV and reports that as a failure. Such report can't be suppressed using the suppressions mechanism. That's why this update comes with "output filters". Filters are scripts that read from their stdin, and write filtered output to their stdout. Filters reside in auxprogs/filters. This update comes with 2 filters: For mmap18, and for select03. They are awk scripts. Except for filters, this update also blacklists testcase fork13 because it is slow. It is possible to add comments prefixed with the '#' sign (implicitly - because they don't match any testcase name), so a comment is added too. This update also introduces new default valgrind --vgdb=no switch. It improves valgrind behavior with nftw01 nftw6401 setfsgid04 setfsgid03_16 and symlink03 testcases. These were previously complaining like this: ==22969== could not unlink /tmp/vgdb-pipe-from-vgdb-to-22969-by-root-on ...