Bug 504956 - LTP tests: Allow for filtering valgrind output
Summary: LTP tests: Allow for filtering valgrind output
Status: RESOLVED FIXED
Alias: None
Product: valgrind
Classification: Developer tools
Component: general (other bugs)
Version First Reported In: 3.25 GIT
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: mcermak
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-05-29 11:43 UTC by mcermak
Modified: 2025-06-03 17:26 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
proposed patch (6.61 KB, patch)
2025-05-29 11:43 UTC, mcermak
Details
possible patch (9.09 KB, patch)
2025-06-03 12:08 UTC, mcermak
Details
proposed patch (9.12 KB, patch)
2025-06-03 14:31 UTC, mcermak
Details

Note You need to log in before you can comment on or make changes to this bug.
Description mcermak 2025-05-29 11:43:05 UTC
Created attachment 181860 [details]
proposed patch

In some cases the LTP tests intentionally work with SIGSEGV.  This happens e.g. with the mmap18 and select03 testcases.  Valgrind detects SIGSEGV and reports that as a failure.

Such report can't be suppressed using the suppressions mechanism.  That's why this update comes with "output filters". Filters are scripts that read from their stdin, and write filtered output to their stdout. Filters reside in auxprogs/filters.

This update comes with 2 filters: For mmap18, and for select03.  They are awk scripts.
Comment 1 Mark Wielaard 2025-05-31 20:57:14 UTC
Hi Martin,

(In reply to mcermak from comment #0)
> Created attachment 181860 [details]
> proposed patch
> 
> In some cases the LTP tests intentionally work with SIGSEGV.  This happens
> e.g. with the mmap18 and select03 testcases.  Valgrind detects SIGSEGV and
> reports that as a failure.
> 
> Such report can't be suppressed using the suppressions mechanism.  That's
> why this update comes with "output filters". Filters are scripts that read
> from their stdin, and write filtered output to their stdout. Filters reside
> in auxprogs/filters.
> 
> This update comes with 2 filters: For mmap18, and for select03.  They are
> awk scripts.

This looks really useful.

> Except for filters, this update also blacklists testcase fork13
> because it is slow.  It is possible to add comments prefixed with
> the '#' sign (implicitly - because they don't match any testcase
> name), and so a comment is added too.

Nice trick. We should comment the others too.

> This update also introduces new default valgrind arguments VGARGS.
> They are: --gen-suppressions=all and --vgdb=no.  The former is
> there for user convenience, so that a suppression can be easily
> captured and reused if needed. 

I am a little confused by this. Where do those suppressions appear and how/when would you use them?

> The latter, --vgdb=no, improves
> valgrind behavior with nftw01 nftw6401 setfsgid04 setfsgid03_16
> and symlink03 testcases. These were previously complaining like
> this:
> 
> ==22969== could not unlink /tmp/vgdb-pipe-from-vgdb-to-22969-by-root-on ...

Aha. Yes, those testcases change uid during their runtime and so they no longer own that file.
That seems OK. We aren't really testing the gdbserver functionality here.
Also these test must be run with privilege.

BTW. setfsgid04 doesn't seem to exist, did you mean setfsgid03?
Comment 2 mcermak 2025-06-03 12:08:30 UTC
Created attachment 181995 [details]
possible patch

(In reply to Mark Wielaard from comment #1)
> Hi Martin,
> 
> (In reply to mcermak from comment #0)
> > Created attachment 181860 [details]
> > proposed patch
> > 
> > In some cases the LTP tests intentionally work with SIGSEGV.  This happens
> > e.g. with the mmap18 and select03 testcases.  Valgrind detects SIGSEGV and
> > reports that as a failure.
> > 
> > Such report can't be suppressed using the suppressions mechanism.  That's
> > why this update comes with "output filters". Filters are scripts that read
> > from their stdin, and write filtered output to their stdout. Filters reside
> > in auxprogs/filters.
> > 
> > This update comes with 2 filters: For mmap18, and for select03.  They are
> > awk scripts.
> 
> This looks really useful.
> 
> > Except for filters, this update also blacklists testcase fork13
> > because it is slow.  It is possible to add comments prefixed with
> > the '#' sign (implicitly - because they don't match any testcase
> > name), and so a comment is added too.
> 
> Nice trick. We should comment the others too.
> 
> > This update also introduces new default valgrind arguments VGARGS.
> > They are: --gen-suppressions=all and --vgdb=no.  The former is
> > there for user convenience, so that a suppression can be easily
> > captured and reused if needed. 
> 
> I am a little confused by this. Where do those suppressions appear and
> how/when would you use them?

Oh, right..  These would only be produced by --tool=memcheck and would
go to the --log-file.  I've dropped this part.  Thanks for pointing that out.

> > The latter, --vgdb=no, improves
> > valgrind behavior with nftw01 nftw6401 setfsgid04 setfsgid03_16
> > and symlink03 testcases. These were previously complaining like
> > this:
> > 
> > ==22969== could not unlink /tmp/vgdb-pipe-from-vgdb-to-22969-by-root-on ...
> 
> Aha. Yes, those testcases change uid during their runtime and so they no
> longer own that file.
> That seems OK. We aren't really testing the gdbserver functionality here.
> Also these test must be run with privilege.
> 
> BTW. setfsgid04 doesn't seem to exist, did you mean setfsgid03?

I think it does exist here:

./auxprogs/auxchecks/ltp-full-20250530/testcases/kernel/syscalls/setfsgid/setfsgid03
./auxprogs/auxchecks/ltp-full-20250530/testcases/kernel/syscalls/setfsgid/setfsgid03_16
./auxprogs/auxchecks/ltp-full-20250530/testcases/kernel/syscalls/setfsgid/setfsgid03.c

I've referred to setfsgid03_16 in my comment.

I've attached updated patch.  One important update is the use of updated LTP sources.
This LTP version already supports the new LTP_REPRODUCIBLE_OUTPUT feature, so the
new version of the tester relies on that.  I've dropped the part that was filtering the tail
of the testcase logs.  The LTP_REPRODUCIBLE_OUTPUT feature seems to do the trick.

FInally, per Frank's suggestion, I've enriched the content of the .log files.  These now 
contain summary of all the logs generated by valgrind.

Please review the updated patch.
Comment 3 mcermak 2025-06-03 14:31:57 UTC
Created attachment 182001 [details]
proposed patch

(In reply to mcermak from comment #2)
> Finally, per Frank's suggestion, I've enriched the content of the .log files...

Ugh, the previous patch enriched the log files a bit too much — including even the core files, if those were avail.  Attaching an amendment of the previous patch :)
Comment 4 Mark Wielaard 2025-06-03 17:26:50 UTC
Very nice. The upgrade to LTP 20250530 and LTP_REPRODUCIBLE_OUTPUT=1 and putting the full output/diffs in the logs make the wrapper script more useful. Pushed as:

commit b8891a4c5d139b8c327ee20c37066d601fc56b0a
Author: Martin Cermak <mcermak@redhat.com>
Date:   Tue Jun 3 13:43:18 2025 +0200

    ltp tests: Use new LTP 20250530, allow for filtering valgrind output
    
    Use fresh LTP 20250530.
    
    Allow for filtering valgrind output:
    
    In some cases the LTP tests intentionally work with SIGSEGV.  This
    happens e.g. with the mmap18 and select03 testcases.  Valgrind
    detects SIGSEGV and reports that as a failure.
    
    Such report can't be suppressed using the suppressions mechanism.
    That's why this update comes with "output filters". Filters are
    scripts that read from their stdin, and write filtered output to
    their stdout. Filters reside in auxprogs/filters.
    
    This update comes with 2 filters: For mmap18, and for select03.
    They are awk scripts.
    
    Except for filters, this update also blacklists testcase fork13
    because it is slow.  It is possible to add comments prefixed with
    the '#' sign (implicitly - because they don't match any testcase
    name), so a comment is added too.
    
    This update also introduces new default valgrind --vgdb=no switch. It
    improves valgrind behavior with nftw01 nftw6401 setfsgid04 setfsgid03_16
    and symlink03 testcases. These were previously complaining like this:
    
    ==22969== could not unlink /tmp/vgdb-pipe-from-vgdb-to-22969-by-root-on ...