Bug 390310

Created attachment 117477 [details]
Patch file

Updates and replaces the previous patch file.

To be more precise, the second patch file is an update to the previous one that applies cleanly on top of d7d82317 (the HEAD of master as of January the 12th, 2019). Fell free to comment or advise if this does not follow the coding standards of the project.

Created attachment 175624 [details]
Patch file

(In reply to renaultd from comment #3)
> Created attachment 175624 [details]
> Patch file

The third patch file is supposed to be up-to-date with the latest changes in valgrind. I know this bug report is not much, but I can assure that we are still using it.

Which tool is using this?

(In reply to Paul Floyd from comment #5)
> Which tool is using this?

We use valgrind at our university to provide information on their code. Basically, we execute their code through valgrind, and digest the results. The students are not fluent with valgrind at that point, so we need to have a simple summary. And we extract this summary from the XML export.

Sorry, cannot find a way to edit a comment. We use valgrind to test student code at our university.

Just out of curiosity, which university is that?

My comments so far
1. There's a FORK message which probably needs removing.
2. White space diffs.
3. There's a change to the non-xml  "possible leaks" indentation. Previously the output was aligned on the ":" of each row.
4. The existing xml looks to me to be almost all tag based rather than attribute based. So rather than

<error_summary errors="1" from="1" suppressed="0" suppressed_from="0"/>

the following would be more consistent

<errorsummary>
   <errors>1</errors>
   <suppressed>0</suppressed>
   <suppressedfrom>0</suppressedfrom>
</errorsummary>

(this isn't something I consider a requirement).

5. I get 18 regtest failures. The XML tests will need to have their expecteds updated.
6. I haven't looked at the manual but I guess that this will need documenting.

Created attachment 176131 [details]
Patch file

I have added a new patch that tries to address the different remarks from Paul (https://bugs.kde.org/show_bug.cgi?id=390310#c8). We are from the the University of Bordeaux (https://www.u-bordeaux.fr/en), in an engineering school belonging to the Bordeaux-INP group (https://www.bordeaux-inp.fr/en). 

Some things that should have been corrected :

1. The message should have been removed
2. The white space diffs should have been removed
3. The indentations should have been corrected
4. The XML output has been changed to match the existing style

Some things that are still a work-in-progress :

5. I have tried to change the regression tests, but still have
some strange behaviors on my machine (AMD64_LINUX). There are
still 2 regressions at the time :

  * memcheck/tests/mismatches_xml : there is a bunch of new errors
    appearing just by using some calls to VG_(printf_xml) I tried
    launching the test by adding and removing all my modification, but
    even putting only the modifications in
    memcheck/mc_malloc_wrappers.c induce these new errors.

  * tests/vg_regtest drd/tests/bar_bad_xml : same things, new errors
    that appear, this time only adding the modifications in
    coregrind/m_main.c, which, AFAIK, only *remove* calls to
    VG_(printf_xml)

6. I have not written anything in the documentation. I imagine one
should have a look at docs/internals/xml-output-protocol4.txt or even
docs/internals/xml-output-protocol5.txt since it looks more recent.

Thanks for the updated patch. I've started going though it, I'll get back to you shortly.

I've done some testing with CLion and Qt Creator (which both use XML via --xml-socket) and they seem OK.

There's one warning that I've fixed.

Some of the XML expected files seem system dependent. For instance this diff

paulf> cat memcheck/tests/sized_aligned_new_delete_misaligned1_xml*diff
--- sized_aligned_new_delete_misaligned1_xml.stderr.exp 2024-12-02 21:43:54.456599000 +0100
+++ sized_aligned_new_delete_misaligned1_xml.stderr.out 2024-12-02 21:57:55.929230000 +0100
@@ -465,9 +465,9 @@
     <in_blocks>0</in_blocks>
   </memory_in_use_at_exit>
   <total_heap_usage>
-    <allocs>6</allocs>
-    <frees>6</frees>
-    <bytes_allocated>72,864</bytes_allocated>
+    <allocs>5</allocs>
+    <frees>5</frees>
+    <bytes_allocated>160</bytes_allocated>
   </total_heap_usage>
 </heap_summary>

(produced on FreeBSD with clang++ and libc++. GCC allocates a memory block for exception handling which adds 1 to the alloc and free counts).

I'm now down to
== 899 tests, 7 stderr failures, 0 stdout failures, 0 stderrB failures, 0 stdoutB failures, 0 post failures ==
memcheck/tests/long_namespace_xml        (stderr)
memcheck/tests/mismatches_xml            (stderr)
memcheck/tests/sized_aligned_new_delete_misaligned1_xml (stderr)
memcheck/tests/sized_aligned_new_delete_misaligned2_xml (stderr)
memcheck/tests/sized_aligned_new_delete_misaligned3_xml (stderr)
memcheck/tests/threadname_xml            (stderr)
memcheck/tests/xml1                      (stderr)

(on FreeBSD). I'll need to update some expecteds on Linux and also look at Solaris and Darwin.

I'm fairly sure that this is a regression. On S11.3 I have a build that I did in Dec 2023 that seems OK for several of the test cases with signals.

Need to give git bisect a go.

Ignore my last comment - wrong window in Firefox. It was meant for 496370.

It's difficult to get the summaries to be stable across different platforms. What I'll do is filter them all out and then add a few tests that don't use filter_xml but use a filter to only look for the summary blocks.

As a note to myself, on Illumos I currently get the following extra fails:

> memcheck/tests/gone_abrt_xml             (stderr)
> memcheck/tests/mismatches_xml            (stderr)
> memcheck/tests/xml1                      (stderr)
> helgrind/tests/pth_mempcpy_false_races   (stderr)
> helgrind/tests/tc06_two_races_xml        (stderr)
> drd/tests/annotate_barrier_xml           (stderr)
> drd/tests/annotate_trace_memory_xml      (stderr)
> drd/tests/bar_bad_xml                    (stderr)
> drd/tests/fp_race_xml                    (stderr)
> drd/tests/thread_name_xml                (stderr)
> none/tests/solaris/stack_overflow        (stderr)

The last one looks spurious

And FreeBSD
> memcheck/tests/long_namespace_xml        (stderr)
> memcheck/tests/threadname_xml            (stderr)
> memcheck/tests/xml1                      (stderr)
> drd/tests/annotate_barrier_xml           (stderr)
> drd/tests/annotate_trace_memory_xml      (stderr)
> drd/tests/bar_bad_xml                    (stderr)
> drd/tests/fp_race_xml                    (stderr)

Created attachment 180193 [details]
With updated expecteds

Now clean on FreeBSD

Created attachment 181079 [details]
Patch with updated expecteds for illumos

Just need to update for Linux

Created attachment 181084 [details]
Add a FreeBSD expected

Created attachment 181085 [details]
With Linux amd64 expected updates

Created attachment 181164 [details]
With filter change for Helgrind suppression counts

Seeing some variation in suppressed_from fields.

Created attachment 181738 [details]
Update protocol verison and doc

Thanks for the contribution. I've just pushed the change.

I made a few xml changes. I used error_contexts and suppressed_contexts for the error summary. For example

<error_summary>
 <errors>1</errors>
 <error_contexts>1</error_contexts>
 <suppressed>3</suppressed>
 <suppressed_contexts>3</suppressed_contexts>
</error_summary>

Also I moved the heuristic kind info within the still_reachable element rather than following it, for instance

  <still_reachable>
    <bytes>159</bytes>
    <blocks>6</blocks>
    <reachable_heuristic>
      <kind>length64</kind>
      <bytes>31</bytes>
      <blocks>1</blocks>
    </reachable_heuristic>
  </still_reachable>

I think that regtest is clean.

I've added

docs/internals/xml-output-protocol6.txt

that describes everything, with an addendum for the memcheck error kinds that I've added and didn't document.

commit 7786b075abef51ca3d84b9717915f04b32950b32 (HEAD -> master, origin/master, origin/HEAD)
Author: Paul Floyd <pjfloyd@wanadoo.fr>
Date:   Sun May 11 15:41:10 2025 +0200

    Bug 390310 - Output summaries in XML files
    
    Original patch contributed by renaultd@free.fr