Bug 354883

Created attachment 95325 [details]
Patch to reset magic_delta value to 224 (0xE0) for OSX 10.11

Hello,
That is very interesting.
Could you please attach the full output of '$ ./valgrind -v -v true'. It will start with something similar to this:
=======================================
==41292== Memcheck, a memory error detector
==41292== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==41292== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==41292== Command: true
==41292== 
--41292-- Valgrind options:
--41292--    -v
--41292--    -v
--41292-- Output from sysctl({CTL_KERN,KERN_VERSION}):
--41292--   Darwin Kernel Version 15.0.0: Sat Sep 19 15:53:46 PDT 2015; root:xnu-3247.10.11~1/RELEASE_X86_64
--41292-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-sse3
--41292-- Page sizes: currently 4096, max supported 4096
=======================================

Can you provide details of a specific program, which uses pthreads, that triggers this assertion for you?

I presume that you can use valgrind on other simple programs on OS X 10.11 at present without the assertion?

Created attachment 95346 [details]
Output of command valgrind -v -v true

Added output of valgrind -v -v true as requested. This output is from my patched version (magic_delta==224).

Unfortunately I do not have a simple pthreads program to demonstrate the issue. The assertion happens (always and almost immediately) when running a large file system test suite that uses pthreads. It may be possible however to create a simple repro if I get some free time.

Created attachment 95351 [details]
Program to demonstrate the reported issue

The attached program uses Grand Central Dispatch and fails pretty consistently with unpatched valgrind:

billziss@macbook-pro:~/Projects/ext$ clang d.c
billziss@macbook-pro:~/Projects/ext$ valgrind-3.11.0/.install/bin/valgrind ./a.out
==49754== Memcheck, a memory error detector
==49754== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==49754== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==49754== Command: ./a.out
==49754== 
[[[SNIP]]]

valgrind: m_syswrap/syswrap-amd64-darwin.c:507 (void wqthread_hijack(Addr, Addr, Addr, Addr, Int, Addr)): Assertion 'tst->os_state.pthread - magic_delta == self' failed.

host stacktrace:
==49754==    at 0x23804F0DE: ???
==49754==    by 0x23804F4FC: ???
==49754==    by 0x23804F4DA: ???
==49754==    by 0x23810C762: ???

sched status:
  running_tid=0

Thread 1: status = VgTs_WaitSys (lwpid 2567)
==49754==    at 0x1002F27BA: write$NOCANCEL (in /usr/lib/system/libsystem_kernel.dylib)
==49754==    by 0x1001E8E95: _swrite (in /usr/lib/system/libsystem_c.dylib)
==49754==    by 0x1001E13BF: __sflush (in /usr/lib/system/libsystem_c.dylib)
==49754==    by 0x1001E3CD3: __sfvwrite (in /usr/lib/system/libsystem_c.dylib)
==49754==    by 0x1001EE658: __vfprintf (in /usr/lib/system/libsystem_c.dylib)
==49754==    by 0x10021435C: __v2printf (in /usr/lib/system/libsystem_c.dylib)
==49754==    by 0x1001EA01D: vfprintf_l (in /usr/lib/system/libsystem_c.dylib)
==49754==    by 0x1001E7EB7: printf (in /usr/lib/system/libsystem_c.dylib)
==49754==    by 0x100000F19: __main_block_invoke (in ./a.out)
==49754==    by 0x100104027: _dispatch_client_callout2 (in /usr/lib/system/libdispatch.dylib)
==49754==    by 0x100103F78: _dispatch_apply_invoke (in /usr/lib/system/libdispatch.dylib)
==49754==    by 0x100103E1D: dispatch_apply_f (in /usr/lib/system/libdispatch.dylib)
==49754==    by 0x100000EDB: main (in ./a.out)

Thread 2: status = VgTs_WaitSys (lwpid 4099)
==49754==    at 0x1002F278A: __workq_kernreturn (in /usr/lib/system/libsystem_kernel.dylib)
==49754==    by 0x23810C4F5: ???
==49754==    by 0x7000003979FF: ???
==49754==    by 0x200016F: ???
==49754==    by 0x16F: ???

Thread 3: status = VgTs_WaitSys (lwpid 4355)
==49754==    at 0x1002F278A: __workq_kernreturn (in /usr/lib/system/libsystem_kernel.dylib)
==49754==    by 0x23810C762: ???
==49754==    by 0x7000003979FF: ???
==49754==    by 0x200016F: ???
==49754==    by 0x227: ???

Thread 4: status = VgTs_WaitSys (lwpid 3843)
==49754==    at 0x1002F278A: __workq_kernreturn (in /usr/lib/system/libsystem_kernel.dylib)
==49754==    by 0x23810C4F5: ???
==49754==    by 0x7000003979FF: ???
==49754==    by 0x200016F: ???
==49754==    by 0x2DF: ???


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

I'm experiencing this issue on OSX 10.11 as well using Valgrind 3.11.0, trying to analyse a relatively simple Qt5 application.  Will checkout the valgrind sources and try the patch

I am also experiencing this error with a program I am writing. I am also on osx 10.11, and using valgrind version 3.11.  I am not sure how to try the patch since I am using homebrew. I will look at the ruby script and see if it is obvious how to patch from there. If not can someone tell me how to patch with homebrew?

ok I added the patch to the brew formula and it gives me:

can't find file to patch at input line 5
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|Index: coregrind/m_syswrap/syswrap-amd64-darwin.c
|===================================================================
|--- coregrind/m_syswrap/syswrap-amd64-darwin.c	(revision 15721)
|+++ coregrind/m_syswrap/syswrap-amd64-darwin.c	(working copy)
--------------------------
No file to patch.  Skipping patch.

Does homebrew want a different format of patch? or is downloading it directly from the attachment link adding extra stuff to the file?

ok I can confirm the patch works here too.

For others using homebrew that find this page here are the steps to add the patch.

brew edit valgrind
find the "stable do" section and before the "end" put "patch :DATA"

then paste this at the end of the file (NOTE: if there is already an __END__ DON"T put in another one!!)

__END__
diff --git a/coregrind/m_syswrap/syswrap-amd64-darwin.c b/coregrind/m_syswrap/syswrap-amd64-darwin.c
index 8f13e71..7fb8b2c 100644
--- a/coregrind/m_syswrap/syswrap-amd64-darwin.c
+++ b/coregrind/m_syswrap/syswrap-amd64-darwin.c
@@ -479,10 +479,8 @@ void wqthread_hijack(Addr self, Addr kport, Addr stackaddr, Addr workitem,
        UWord magic_delta = 0;
 #      elif DARWIN_VERS == DARWIN_10_7 || DARWIN_VERS == DARWIN_10_8
        UWord magic_delta = 0x60;
-#      elif DARWIN_VERS == DARWIN_10_9 || DARWIN_VERS == DARWIN_10_10
+#      elif DARWIN_VERS == DARWIN_10_9 || DARWIN_VERS == DARWIN_10_10 || DARWIN_VERS == DARWIN_10_11
        UWord magic_delta = 0xE0;
-#      elif DARWIN_VERS == DARWIN_10_11
-       UWord magic_delta = 0x100;
 #      else
 #        error "magic_delta: to be computed on new OS version"
          // magic_delta = tst->os_state.pthread - self

save and close then to force brew to compile instead of install binaries from "bottle"
brew install --build-from-source valgrind

Hi am trying to run valgrind on OSX and I think I am hitting the same problem.

I have this embarrassingly simple Qt 5 program:

cat main.cpp

#include <QCoreApplication>

int main(int argc, char** argv)
{
    QCoreApplication app(argc, argv);

    return app.exec();
}

and I build it with Qt 5.7 alpha which I built from source.

When I run valgrind on it, I get this:

valgrind qt.app/Contents/MacOS/qt
==75971== Memcheck, a memory error detector
==75971== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==75971== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==75971== Command: qt.app/Contents/MacOS/qt
==75971==
==75971== Conditional jump or move depends on uninitialised value(s)
==75971==    at 0x7FFF5FC24A27: bcmp (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC11914: ImageLoaderMachO::validateFirstPages(linkedit_data_command const*, int, unsigned char const*, unsigned long, long long, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC16B8A: ImageLoaderMachOCompressed::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, unsigned int, unsigned int, linkedit_data_command const*, encryption_info_command const*, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC10A7E: ImageLoaderMachO::instantiateFromFile(char const*, int, unsigned char const*, unsigned long long, unsigned long long, stat const&, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC038C2: dyld::loadPhase6(int, stat const&, char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0846D: dyld::loadPhase5(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0818D: dyld::loadPhase4(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC07EF2: dyld::loadPhase3(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC07647: dyld::loadPhase1(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0347A: dyld::loadPhase0(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0315E: dyld::load(char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0870A: dyld::libraryLocator(char const*, bool, char const*, ImageLoader::RPathChain const*) (in /usr/lib/dyld)
==75971==
==75971== Use of uninitialised value of size 8
==75971==    at 0x7FFF5FC24A3F: bcmp (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC11914: ImageLoaderMachO::validateFirstPages(linkedit_data_command const*, int, unsigned char const*, unsigned long, long long, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC16B8A: ImageLoaderMachOCompressed::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, unsigned int, unsigned int, linkedit_data_command const*, encryption_info_command const*, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC10A7E: ImageLoaderMachO::instantiateFromFile(char const*, int, unsigned char const*, unsigned long long, unsigned long long, stat const&, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC038C2: dyld::loadPhase6(int, stat const&, char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0846D: dyld::loadPhase5(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0818D: dyld::loadPhase4(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC07EF2: dyld::loadPhase3(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC07647: dyld::loadPhase1(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0347A: dyld::loadPhase0(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0315E: dyld::load(char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0870A: dyld::libraryLocator(char const*, bool, char const*, ImageLoader::RPathChain const*) (in /usr/lib/dyld)
==75971==
==75971== Use of uninitialised value of size 8
==75971==    at 0x7FFF5FC24A44: bcmp (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC11914: ImageLoaderMachO::validateFirstPages(linkedit_data_command const*, int, unsigned char const*, unsigned long, long long, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC16B8A: ImageLoaderMachOCompressed::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, unsigned int, unsigned int, linkedit_data_command const*, encryption_info_command const*, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC10A7E: ImageLoaderMachO::instantiateFromFile(char const*, int, unsigned char const*, unsigned long long, unsigned long long, stat const&, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC038C2: dyld::loadPhase6(int, stat const&, char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0846D: dyld::loadPhase5(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0818D: dyld::loadPhase4(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC07EF2: dyld::loadPhase3(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC07647: dyld::loadPhase1(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0347A: dyld::loadPhase0(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0315E: dyld::load(char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0870A: dyld::libraryLocator(char const*, bool, char const*, ImageLoader::RPathChain const*) (in /usr/lib/dyld)
==75971==
==75971== Conditional jump or move depends on uninitialised value(s)
==75971==    at 0x7FFF5FC11917: ImageLoaderMachO::validateFirstPages(linkedit_data_command const*, int, unsigned char const*, unsigned long, long long, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC16B8A: ImageLoaderMachOCompressed::instantiateFromFile(char const*, int, unsigned char const*, unsigned long, unsigned long long, unsigned long long, stat const&, unsigned int, unsigned int, linkedit_data_command const*, encryption_info_command const*, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC10A7E: ImageLoaderMachO::instantiateFromFile(char const*, int, unsigned char const*, unsigned long long, unsigned long long, stat const&, ImageLoader::LinkContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC038C2: dyld::loadPhase6(int, stat const&, char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0846D: dyld::loadPhase5(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0818D: dyld::loadPhase4(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC07EF2: dyld::loadPhase3(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC07647: dyld::loadPhase1(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0347A: dyld::loadPhase0(char const*, char const*, dyld::LoadContext const&, std::__1::vector<char const*, std::__1::allocator<char const*> >*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0315E: dyld::load(char const*, dyld::LoadContext const&) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0870A: dyld::libraryLocator(char const*, bool, char const*, ImageLoader::RPathChain const*) (in /usr/lib/dyld)
==75971==    by 0x7FFF5FC0E86D: ImageLoader::recursiveLoadLibraries(ImageLoader::LinkContext const&, bool, ImageLoader::RPathChain const&) (in /usr/lib/dyld)
==75971==
--75971-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option
--75971-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 2 times)
--75971-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 4 times)
--75971-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 8 times)

HANG!!!!!!!

So, valgrind just hangs.

This is before and after the patch in comment 9. That is, I patched valgrind and instructed homebrew to build from source.

Is there some other step to make valgrind usable on OSX?

(In reply to Stephen Kelly from comment #10)
> Hi am trying to run valgrind on OSX and I think I am hitting the same
> problem.
> 
> I have this embarrassingly simple Qt 5 program:
> 
> cat main.cpp
> 
> #include <QCoreApplication>
> 
> int main(int argc, char** argv)
> {
>     QCoreApplication app(argc, argv);
> 
>     return app.exec();
> }

Oops, I pasted a bad testcase. I get the hang even without the 

     return app.exec();

line. 

My testcase is 


#include <QCoreApplication>

int main(int argc, char** argv)
{
    QCoreApplication app(argc, argv);
}

Hello Stephen Kelly,

Thank you for the testing and report.

However, given you are seeing a hang with valgrind on your OS X environment regardless of the patch on this bug report I would believe you are hitting one of the other currently known or unknown bugs within valgrind on OS X.

As you can see from the bug tracker here, whilst support for OS X in valgrind has improved, a number of further refinements are required. This is particularly the case for more complex programs or those programs that use larger external libraries (greater surface area for valgrind to interact with).

Unfortunately I haven't had as much opportunity to work on OS X valgrind improvements in the last two quarters, although I hope to remedy that soon.

Hi Rhys,

Ok, thanks for the response. So in terms of realistic expections, I shouldn't expect valgrind with Qt5 to work on OSX for now.

Thanks for the update.

*** Bug 361351 has been marked as a duplicate of this bug. ***

Same problem here, same crash as reported, changing the magic number (i.e. applying the patch) works great, thank you.

valgrind: m_syswrap/syswrap-amd64-darwin.c:507 (void wqthread_hijack(Addr, Addr, Addr, Addr, Int, Addr)): Assertion 'tst->os_state.pthread - magic_delta == self' failed.

Applying the patch resolved the issue.

*** Bug 362920 has been marked as a duplicate of this bug. ***

Slightly modified patch applied in r15891.

Whilst I’ve seen different magic_delta values on one of my older development machines (Intel Nehalem-based), enough other users have reported success with this change.

If this causes regressions, please report your hardware details in our Bugzilla.

*** Bug 366222 has been marked as a duplicate of this bug. ***