Bug 492382 - Nondeterministic results found for DHAT tool reports and Valgrind-produced DHAT log
Summary: Nondeterministic results found for DHAT tool reports and Valgrind-produced DH...
Status: REPORTED
Alias: None
Product: valgrind
Classification: Developer tools
Component: dhat (show other bugs)
Version: 3.18.1
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: Julian Seward
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-08-30 01:19 UTC by jo.alen1@outlook.com
Modified: 2024-09-01 08:59 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
ZIP file that contains two files -- the two individual DHAT reports and the file comparisons between the Valgrind-produced logs of two runs (663.71 KB, application/zip)
2024-08-30 01:19 UTC, jo.alen1@outlook.com
Details

Note You need to log in before you can comment on or make changes to this bug.
Description jo.alen1@outlook.com 2024-08-30 01:19:32 UTC
Created attachment 173108 [details]
ZIP file that contains two files -- the two individual DHAT reports and the file comparisons between the Valgrind-produced logs of two runs

Hi Valgrind Maintainers and Team!

I have been using Valgrind as a tool to check for nondeterministic behaviors and I know that there are multiple subtools that Valgrind's suite comes with. I would like to inquire about the DHAT subtool in analyzing some of the reports generated after each run against a binary. This nondeterminism concerns the tool generated report and the Valgrind generated report. We noted that 14/41 repositories showed nondeterministic results, where 4/14 repositories showed flakiness in DHAT's generated report and 10/14 occurring from Valgrind's generated report on DHAT (specifically, Valgrind's summary section printed for DHAT). 

Thank you very much!

STEPS TO REPRODUCE
1. Use a GitHub Actions runner with Ubuntu 22.04 + Valgrind 3.18.1 pulled from the Ubuntu package manager
2. Run your executable with DHAT enabled. 
3. Obtain the Valgrind report + view the DHAT results through the dh_view.html program

OBSERVED RESULT
When viewing the DHAT generated results, I noticed that one of the binaries had a significant difference in the number of children nodes that DHAT produced for two of the same runs under the same conditions. When viewing the Valgrind log for DHAT for another binary, I noticed that the summary sections contained in the Valgrind log for DHAT are different across runs.

ADDITIONAL INFORMATION
Within the ZIP file, you'll find two files: the two individual DHAT reports viewed through the DHAT Viewing tool + the file comparison between the Valgrind logs of two runs.