Bug 72082

Version:           2.1.0 (using KDE Devel)
Installed from:    Compiled sources
Compiler:          gcc 3.2.2 
OS:          Linux

I'm trying to use valgrind to debug a program that uses an alternative
memory allocator (a garbage collector):

http://www.hpl.hp.com/personal/Hans_Boehm/gc/

I've added appropriate uses of VALGRIND_MALLOCLIKE_BLOCK and
VALGRIND_FREELIKE_BLOCK and written suppressions for the code inside
the memory allocator itself.  However now I've run into a problem
relating to the valgrind threading model and how this library uses
threads.

The symptom is this assertion failure:

valgrind: vg_scheduler.c:3236 (scheduler_sanity): Assertion `cv == ((void *)0)' failed.

That's when a thread is in state WaitMX.  There's usually just one
thread that this could refer to, and it looks like this:

Thread 3: status = WaitMX, associated_mx = 0x40235540, associated_cv = 0x817B7C4
==11440==    at 0x40222BDB: __pthread_mutex_lock (vg_libpthread.c:944)
==11440==    by 0x40224DEB: sem_post (vg_libpthread.c:2343)
==11440==    by 0x8127FA3: GC_suspend_handler (pthread_stop_world.c:121)
==11440==    by 0x401670CF: (within /usr/local/lib/valgrind/valgrind.so)

This is in a signal handler.  I believe that what's happened is that
this thread was waiting on a condition variable, then got signaled
with pthread_kill.  In the signal handler, it still has an
associated_mx and associated_cv.  The signal handler calls sem_post
(which is supposed to be async-signal safe).  In valgrind, this is
implemented using a mutex, which causes the thread to go into state
WaitMX.  I imagine that this may overwrite the associated_mx, which
would be a secondary problem.

The sem_post man page says:

  the sem_post function is async-signal safe and can therefore be
  called from signal handlers. This is the only thread synchronization
  function provided by POSIX threads that is async-signal safe.

The pthread_mutex_lock man page says:

  The mutex functions are not async-signal safe. What this means is
  that they should not be called from a signal handler. In particular,
  calling pthread_mutex_lock or pthread_mutex_unlock from a signal
  handler may deadlock the calling thread.

I take this to mean that sem_post shouldn't be implemented with a
mutex.  (Of course maybe that's not true in the context of valgrind's
threading model, I'm not really sure.)

Here's a little more background on what this code is doing: when a
garbage collection is performed, the collector needs to suspend all
other threads (so that it can safely walk all global data, thread
stacks, and heap blocks to perform the mark phase of garbage
collection).  To do this, it sends a signal to each thread which is
handled by GC_suspend_handler.  This calls sem_post to inform the
signaling thread that it has suspended itself, and then calls
sigsuspend(2) to wait for another signal to wake it back up.  After
the garbage collection is done, the threads are woken back up with
another pthread_kill.

I'd be willing to help fix this problem, but I've only just started
reading the valgrind source, and I haven't quite figured out how to
attack it yet.

I've tried this on both a RedHat 9 machine and a Debian stable machine
(with valgrind 2.0.0 and 2.1.0 locally built from source).