Bug 380037 - valgrind + AddressSanitizer: ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
Summary: valgrind + AddressSanitizer: ASan runtime does not come first in initial libr...
Status: RESOLVED NOT A BUG
Alias: None
Product: valgrind
Classification: Developer tools
Component: general (show other bugs)
Version: 3.12.0
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Julian Seward
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-05-20 18:44 UTC by Graham Leggett
Modified: 2017-05-22 23:36 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:

Attachments
Add an attachment

Note You need to log in before you can comment on or make changes to this bug.
Description Graham Leggett 2017-05-20 18:44:28 UTC 
When trying to run valgrind on a binary that has been built linked to AddressSanitizer, valgrind immediately crashes at startup with a SIGSEGV as below.

It appears whatever valgrind does to the LD_PRELOAD environment variable breaks AddressSanitizer, which then causes valgrind to crash.

minfrin@towerofpi9:/var/www/html/stream$ GST_LEAKS_TRACER_STACK_TRACE=1 GST_LEAKS_TRACER_SIG=1 ASAN_OPTIONS=atexit=1 GST_TRACERS="leaks(stack-traces-flags=full,filters=GstMemory,check-refs=true)" GST_DEBUG_DUMP_DOT_DIR=. G_DEBUG="fatal-warnings" LD_PRELOAD=/usr/local/gcc-6.2.0/lib/libasan.so.3 LD_LIBRARY_PATH=/usr/local/lib:/usr/local/gcc-6.2.0/lib /tmp/valgrind/bin/valgrind gst-launch-1.0 --gst-debug-no-color --gst-debug=1,GST_TRACER:7,GST_REFCOUNTING:0,GST_PADS:1,GST_STATES:5,tdttsparse:5,tsdemux:1,omx:1,omxvideo:1,omxvideodec:1,omxh264dec:1,omxvideoenc:1,omxh264enc:1,videoencoder:1,videorate:1,videoscale:1,mpegtsmux:4,decodebin:1,encodebin:1,h264parse:1,transcoder:5,hlssink:5 udpsrc multicast-iface=eth0 uri=udp://239.106.0.0:1234 "caps=application/x-rtp,media=(string)video,clock-rate=(int)90000" ! rtpbin ! rtpmp2tdepay ! progressreport update-freq=5 ! transcoder name=transcoder ! capsfilter caps=audio/x-raw ! fakesink   transcoder. ! capsfilter caps=video/mpegts ! hlssink target-duration=0    transcoder. ! capsfilter caps=audio/x-raw ! fakesink
==4810== Memcheck, a memory error detector
==4810== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==4810== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==4810== Command: gst-launch-1.0 --gst-debug-no-color --gst-debug=1,GST_TRACER:7,GST_REFCOUNTING:0,GST_PADS:1,GST_STATES:5,tdttsparse:5,tsdemux:1,omx:1,omxvideo:1,omxvideodec:1,omxh264dec:1,omxvideoenc:1,omxh264enc:1,videoencoder:1,videorate:1,videoscale:1,mpegtsmux:4,decodebin:1,encodebin:1,h264parse:1,transcoder:5,hlssink:5 udpsrc multicast-iface=eth0 uri=udp://239.106.0.0:1234 caps=application/x-rtp,media=(string)video,clock-rate=(int)90000 ! rtpbin ! rtpmp2tdepay ! progressreport update-freq=5 ! transcoder name=transcoder ! capsfilter caps=audio/x-raw ! fakesink transcoder. ! capsfilter caps=video/mpegts ! hlssink target-duration=0 transcoder. ! capsfilter caps=audio/x-raw ! fakesink
==4810== 
==4810==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
==4810== Jump to the invalid address stated on the next line
==4810==    at 0x0: ???
==4810==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==4810== 
==4810== 
==4810== Process terminating with default action of signal 11 (SIGSEGV)
==4810==  Bad permissions for mapped region at address 0x0
==4810==    at 0x0: ???
==4810== 
==4810== HEAP SUMMARY:
==4810==     in use at exit: 0 bytes in 0 blocks
==4810==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==4810== 
==4810== All heap blocks were freed -- no leaks are possible
==4810== 
==4810== For counts of detected and suppressed errors, rerun with: -v
==4810== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Comment 1 Tom Hughes 2017-05-20 19:25:27 UTC 
Using valgrind and asan together is never likely to work anyway. I haven't seen that error before but usually it falls over because the address space manipulation each tries to do is incompatible.
Comment 2 Tom Hughes 2017-05-20 19:27:13 UTC 
By the way the specific problem you're seeing here is that asan is trying to make sure it's intercepting malloc etc but of course valgrind also wants to do the same thing and they can't both do so.
Comment 3 Graham Leggett 2017-05-20 20:13:15 UTC 
Having completely rebuilt both gstreamer git-master and valgrind v3.12.0 from source using gcc v6.20 with AddressSanitizer disabled, I am still getting the same error:

==26049== 
==26049==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
==26049== Jump to the invalid address stated on the next line
==26049==    at 0x0: ???
==26049==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==26049== 
==26049== 
==26049== Process terminating with default action of signal 11 (SIGSEGV)
==26049==  Bad permissions for mapped region at address 0x0
==26049==    at 0x0: ???
==26049== 
==26049== HEAP SUMMARY:
==26049==     in use at exit: 0 bytes in 0 blocks
==26049==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==26049== 
==26049== All heap blocks were freed -- no leaks are possible
==26049== 
==26049== For counts of detected and suppressed errors, rerun with: -v
==26049== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Is there any mechanism to switch off AddressSanitizer, or at least get valgrind to coexist peacefully with AddressSanitizer on modern compilers?
Comment 4 Graham Leggett 2017-05-20 21:10:48 UTC 
Turned out the session I was using to run valgrind had libasan preloaded with LD_PRELOAD, and this turns AddressSanitizer on and triggers this crash.

Removing LD_PRELOAD produces the next crash, this time in valgrind:

vex: priv/guest_arm_toIR.c:13352 (decode_V8_instruction): Assertion `szBlg2 <= 3' failed.
vex storage: T total 243148328 bytes allocated
vex storage: P total 0 bytes allocated

valgrind: the 'impossible' happened:
   LibVEX called failure_exit().

Looks like this is this bug:

https://bugs.kde.org/show_bug.cgi?id=372794

Building bleeding edge valgrind to see if that will get further...
Comment 5 Julian Seward 2017-05-22 07:57:08 UTC 
Really, this isn't likely to work, per Tom's comment 1 and comment 2.
Don't do it.
Comment 6 Graham Leggett 2017-05-22 08:34:49 UTC 
For the record, bug that prevents valgrind running raised with the RPi people here:

https://bugs.kde.org/show_bug.cgi?id=380037
Comment 7 Graham Leggett 2017-05-22 23:36:41 UTC 
Let's try that again. For the record, the bug with the RPi people is here:

https://github.com/RPi-Distro/repo/issues/68

At their suggestion moving /etc/ld.so.preload out of the way to prevent the loading the optimised RPi specific code causes valgrind to run successfully on the RPi.