Bug 446103 - Memcheck: `--track-origins=yes` causes extreme slowdowns for large mmap/munmap
Summary: Memcheck: `--track-origins=yes` causes extreme slowdowns for large mmap/munmap
Status: RESOLVED FIXED
Alias: None
Product: valgrind
Classification: Developer tools
Component: memcheck (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Julian Seward
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-11-26 07:34 UTC by Julian Seward
Modified: 2021-12-08 06:55 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
C test case (1.63 KB, text/plain)
2021-11-26 07:34 UTC, Julian Seward
Details
Proposed patch (needs commit message) (26.21 KB, text/plain)
2021-12-03 13:08 UTC, Julian Seward
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Julian Seward 2021-11-26 07:34:44 UTC
Created attachment 143957 [details]
C test case

The attached test program repeatedly mmaps/munmaps a 2GB chunk of memory.
Without `--track-origins=yes`, it runs in about 0.7 seconds.  But with origin
tracking enabled, it takes 95.5 seconds, that is, it runs 137 times slower.

The problem occurs because the 2GB mmaps/munmaps cause Memcheck to perform a
set-address-range-perms operation on the area, changing it to "defined" and
then back to "noaccess".  Large-range SARPs are (highly) optimised for the
V/A-bit (normal) Memcheck shadow memory, but they are very inefficient for the
origin-tracking shadow area.  Origin-tracking shadows are kept in a 64MB 2-way
set associative cache, which is pretty fast.  But lines ejected from that are
kept in a OSet (AVL-tree) based backing store.  Because these SARP lengths
greatly exceed the size of the cache, a huge AVL tree is created, which is
very time- and space-inefficient.

This was initially noticed in
https://bugzilla.mozilla.org/show_bug.cgi?id=1742851.
Comment 1 Julian Seward 2021-12-03 13:08:05 UTC
Created attachment 144170 [details]
Proposed patch (needs commit message)

Attached is a proposed fix.  It makes gigabyte-sized SARPs about
3 x faster, when --track-origins=yes.

Note, from comment 0:
> a huge AVL tree is created

This was speculation and is totally wrong.  The slowdowns were for
other reasons.
Comment 2 Julian Seward 2021-12-08 06:55:55 UTC
Fixed, 8ee1656165902125c414d598cff788c7bb0b1556.