Bug 509590

Summary: Run the LTP tests with LTP_QUIET
Product: [Developer tools] valgrind Reporter: mcermak
Component: generalAssignee: mcermak
Status: RESOLVED FIXED    
Severity: normal CC: mark
Priority: NOR    
Version First Reported In: 3.25 GIT   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed/Implemented In:
Sentry Crash Report:
Attachments: proposed patch
updated patch
proposed patch
updated patch

Description mcermak 2025-09-17 08:50:33 UTC
Created attachment 185002 [details]
proposed patch

The attached patch runs the LTP testsuite with LTP_QUIET turned on.  This env var wasn't yet accepted upstream in LTP.  It shorten the test logs and avoids several false positives, specifically with the following testcases: eventfd2_03, shmctl05, mlock03, poll02, prctl09, setsockopt10, and select02.

This update also adds a brief summary for the LTP testsuite, something like the following:
> ...
> [6/7] Testing select02 ...
> [7/7] Testing setsockopt10 ...
>
> Brief LTP test results summary
> -----------------------------------------
> PASS: 6
> FAIL: 1
> -----------------------------------------
>
> TESTING FINISHED, logs in ...
Comment 1 Mark Wielaard 2025-09-17 11:40:01 UTC
I like it. Just some nitpicks.
- Please also add it to NEWS.
- The ltp-patches/0002-Introduce-LTP_QUIET-env-var.patch should be added to auxprogs/Makefile.am (LTP_PATCHES)
  otherwise it won't be added to the dist on release.
- I had some trouble getting it applied because I already had build auxprogs/auxchecks/ltp-full-20250530
  Lets add the patches and the patch script as dependencies.
  And be more aggressive rebuilding. e.g

diff --git a/auxprogs/Makefile.am b/auxprogs/Makefile.am
index 4f9f100c0389..1e09d16c5ec3 100644
--- a/auxprogs/Makefile.am
+++ b/auxprogs/Makefile.am
@@ -21,7 +21,8 @@ LTP_FILTERS = \
        filters/select03
 
 LTP_PATCHES = \
-       ltp-patches/0001-Make-sure-32-bit-powerpc-syscall-defs-don-t-leak-to-.patch
+       ltp-patches/0001-Make-sure-32-bit-powerpc-syscall-defs-don-t-leak-to-.patch \
+       ltp-patches/0002-Introduce-LTP_QUIET-env-var.patch
 
 EXTRA_DIST = \
        docs/valgrind-listener-manpage.xml \
@@ -211,14 +212,14 @@ $(GSL_SRC_DIR)/gsl-patched: $(GSL_TAR)
         autoreconf -f -i -Wnone)
        touch $@
 
-$(LTP_SRC_DIR): $(LTP_TAR)
+$(LTP_SRC_DIR): $(LTP_TAR) ltp-apply-patches.sh $(LTP_PATCHES)
        echo "$(LTP_SHA256_SUM)  $(LTP_TAR)" | @SHA256SUM@ --check -
        (cd $(AUX_CHECK_DIR) && \
         tar Jxf $(LTP_TAR_NAME) && \
         $(abs_top_srcdir)/auxprogs/ltp-apply-patches.sh $(LTP_SRC_DIR) && \
         cd $(LTP_SRC_DIR) && \
         ./configure CC="${CC}" CXX="${CXX}" CFLAGS="$(LTP_CFLAGS)" && \
-        ${MAKE} -j $(nproc) -C testcases/kernel/syscalls)
+        ${MAKE} -B -j $(nproc) -C testcases/kernel/syscalls)
        touch $@
Comment 2 mcermak 2025-09-17 15:57:33 UTC
Created attachment 185011 [details]
updated patch

Thank you Mark for the review.  I'm attaching updated patch.  Please check.
Comment 3 mcermak 2025-09-17 19:12:18 UTC
Created attachment 185023 [details]
proposed patch

One more update re $(nproc) auxprogs/Makefile.am.
Comment 4 Mark Wielaard 2025-09-17 20:11:55 UTC
(In reply to mcermak from comment #3)
> Created attachment 185023 [details]
> proposed patch
> 
> One more update re $(nproc) auxprogs/Makefile.am.

I think the patch is OK. But I don't understand this change.

-	 ${MAKE} -j $(nproc) -C testcases/kernel/syscalls)
+	 ${MAKE} clean && \
+	 ${MAKE} -j $(shell nproc) -C testcases/kernel/syscalls)

Why and what does it do?

I understand the clean part, that is to make sure the next make invocation really builds the syscalls tests.

But why and what does the $(shell ...) part do?

And do really need/want that here?
I understand the -j $(nproc) part was there before.
But now that we are changing it should it really be there?
I though that a sub-make always communicated through the jobserver started by the parent make to control parallelism.
So can we just remove the whole -j ... part?
Comment 5 mcermak 2025-09-17 20:44:30 UTC
Created attachment 185025 [details]
updated patch

Right, I've verified that the value of -j is passed to child make invocations inherently.  It can be seen via $(MAKEFLAGS).

Updated patch attached.  Please check.
Comment 6 Mark Wielaard 2025-09-17 21:44:44 UTC
(In reply to mcermak from comment #5)
> Created attachment 185025 [details]
> updated patch
> 
> Right, I've verified that the value of -j is passed to child make
> invocations inherently.  It can be seen via $(MAKEFLAGS).
> 
> Updated patch attached.  Please check.

Looks good. Lets go with this variant. Then pick up what ends up in upstream ltp when they release their next version.

commit ccf065e7077459d902f6ad221c21b0771bedbb1a
Author: Martin Cermak <mcermak@redhat.com>
Date:   Wed Sep 17 16:27:09 2025 +0200

    Run the LTP tests with LTP_QUIET
    
    Introduce a new LTP_QUIET env var which suppresses certain
    types of LTP log messages, specifically TCONF, TWARN, TINFO,
    and TDEBUG.  This helps us keep the test logs briefer, while
    still keeping the important information in the logs.
    
    This update avoids several false positives, specifically with
    the following testcases: eventfd2_03, shmctl05, mlock03,
    poll02, prctl09, setsockopt10, and select02.
    
    This update also adds a brief summary for the LTP testsuite,
    something like the following:
    
    > ...
    > [6/7] Testing select02 ...
    > [7/7] Testing setsockopt10 ...
    >
    > Brief LTP test results summary
    > -----------------------------------------
    > PASS: 6
    > FAIL: 1
    > -----------------------------------------
    >
    > TESTING FINISHED, logs in ...
    
    Also fix the way -j param spec in auxprogs/Makefile.am.
    
    https://bugs.kde.org/show_bug.cgi?id=509590