Bug 509590 - Run the LTP tests with LTP_QUIET
Summary: Run the LTP tests with LTP_QUIET
Status: RESOLVED FIXED
Alias: None
Product: valgrind
Classification: Developer tools
Component: general (other bugs)
Version First Reported In: 3.25 GIT
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: mcermak
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-09-17 08:50 UTC by mcermak
Modified: 2025-09-17 21:44 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed/Implemented In:
Sentry Crash Report:


Attachments
proposed patch (4.40 KB, patch)
2025-09-17 08:50 UTC, mcermak
Details
updated patch (6.50 KB, patch)
2025-09-17 15:57 UTC, mcermak
Details
proposed patch (6.95 KB, patch)
2025-09-17 19:12 UTC, mcermak
Details
updated patch (6.92 KB, patch)
2025-09-17 20:44 UTC, mcermak
Details

Note You need to log in before you can comment on or make changes to this bug.
Description mcermak 2025-09-17 08:50:33 UTC
Created attachment 185002 [details]
proposed patch

The attached patch runs the LTP testsuite with LTP_QUIET turned on.  This env var wasn't yet accepted upstream in LTP.  It shorten the test logs and avoids several false positives, specifically with the following testcases: eventfd2_03, shmctl05, mlock03, poll02, prctl09, setsockopt10, and select02.

This update also adds a brief summary for the LTP testsuite, something like the following:
> ...
> [6/7] Testing select02 ...
> [7/7] Testing setsockopt10 ...
>
> Brief LTP test results summary
> -----------------------------------------
> PASS: 6
> FAIL: 1
> -----------------------------------------
>
> TESTING FINISHED, logs in ...
Comment 1 Mark Wielaard 2025-09-17 11:40:01 UTC
I like it. Just some nitpicks.
- Please also add it to NEWS.
- The ltp-patches/0002-Introduce-LTP_QUIET-env-var.patch should be added to auxprogs/Makefile.am (LTP_PATCHES)
  otherwise it won't be added to the dist on release.
- I had some trouble getting it applied because I already had build auxprogs/auxchecks/ltp-full-20250530
  Lets add the patches and the patch script as dependencies.
  And be more aggressive rebuilding. e.g

diff --git a/auxprogs/Makefile.am b/auxprogs/Makefile.am
index 4f9f100c0389..1e09d16c5ec3 100644
--- a/auxprogs/Makefile.am
+++ b/auxprogs/Makefile.am
@@ -21,7 +21,8 @@ LTP_FILTERS = \
        filters/select03
 
 LTP_PATCHES = \
-       ltp-patches/0001-Make-sure-32-bit-powerpc-syscall-defs-don-t-leak-to-.patch
+       ltp-patches/0001-Make-sure-32-bit-powerpc-syscall-defs-don-t-leak-to-.patch \
+       ltp-patches/0002-Introduce-LTP_QUIET-env-var.patch
 
 EXTRA_DIST = \
        docs/valgrind-listener-manpage.xml \
@@ -211,14 +212,14 @@ $(GSL_SRC_DIR)/gsl-patched: $(GSL_TAR)
         autoreconf -f -i -Wnone)
        touch $@
 
-$(LTP_SRC_DIR): $(LTP_TAR)
+$(LTP_SRC_DIR): $(LTP_TAR) ltp-apply-patches.sh $(LTP_PATCHES)
        echo "$(LTP_SHA256_SUM)  $(LTP_TAR)" | @SHA256SUM@ --check -
        (cd $(AUX_CHECK_DIR) && \
         tar Jxf $(LTP_TAR_NAME) && \
         $(abs_top_srcdir)/auxprogs/ltp-apply-patches.sh $(LTP_SRC_DIR) && \
         cd $(LTP_SRC_DIR) && \
         ./configure CC="${CC}" CXX="${CXX}" CFLAGS="$(LTP_CFLAGS)" && \
-        ${MAKE} -j $(nproc) -C testcases/kernel/syscalls)
+        ${MAKE} -B -j $(nproc) -C testcases/kernel/syscalls)
        touch $@
Comment 2 mcermak 2025-09-17 15:57:33 UTC
Created attachment 185011 [details]
updated patch

Thank you Mark for the review.  I'm attaching updated patch.  Please check.
Comment 3 mcermak 2025-09-17 19:12:18 UTC
Created attachment 185023 [details]
proposed patch

One more update re $(nproc) auxprogs/Makefile.am.
Comment 4 Mark Wielaard 2025-09-17 20:11:55 UTC
(In reply to mcermak from comment #3)
> Created attachment 185023 [details]
> proposed patch
> 
> One more update re $(nproc) auxprogs/Makefile.am.

I think the patch is OK. But I don't understand this change.

-	 ${MAKE} -j $(nproc) -C testcases/kernel/syscalls)
+	 ${MAKE} clean && \
+	 ${MAKE} -j $(shell nproc) -C testcases/kernel/syscalls)

Why and what does it do?

I understand the clean part, that is to make sure the next make invocation really builds the syscalls tests.

But why and what does the $(shell ...) part do?

And do really need/want that here?
I understand the -j $(nproc) part was there before.
But now that we are changing it should it really be there?
I though that a sub-make always communicated through the jobserver started by the parent make to control parallelism.
So can we just remove the whole -j ... part?
Comment 5 mcermak 2025-09-17 20:44:30 UTC
Created attachment 185025 [details]
updated patch

Right, I've verified that the value of -j is passed to child make invocations inherently.  It can be seen via $(MAKEFLAGS).

Updated patch attached.  Please check.
Comment 6 Mark Wielaard 2025-09-17 21:44:44 UTC
(In reply to mcermak from comment #5)
> Created attachment 185025 [details]
> updated patch
> 
> Right, I've verified that the value of -j is passed to child make
> invocations inherently.  It can be seen via $(MAKEFLAGS).
> 
> Updated patch attached.  Please check.

Looks good. Lets go with this variant. Then pick up what ends up in upstream ltp when they release their next version.

commit ccf065e7077459d902f6ad221c21b0771bedbb1a
Author: Martin Cermak <mcermak@redhat.com>
Date:   Wed Sep 17 16:27:09 2025 +0200

    Run the LTP tests with LTP_QUIET
    
    Introduce a new LTP_QUIET env var which suppresses certain
    types of LTP log messages, specifically TCONF, TWARN, TINFO,
    and TDEBUG.  This helps us keep the test logs briefer, while
    still keeping the important information in the logs.
    
    This update avoids several false positives, specifically with
    the following testcases: eventfd2_03, shmctl05, mlock03,
    poll02, prctl09, setsockopt10, and select02.
    
    This update also adds a brief summary for the LTP testsuite,
    something like the following:
    
    > ...
    > [6/7] Testing select02 ...
    > [7/7] Testing setsockopt10 ...
    >
    > Brief LTP test results summary
    > -----------------------------------------
    > PASS: 6
    > FAIL: 1
    > -----------------------------------------
    >
    > TESTING FINISHED, logs in ...
    
    Also fix the way -j param spec in auxprogs/Makefile.am.
    
    https://bugs.kde.org/show_bug.cgi?id=509590