Bug 382515 - valgrind: "Assertion 'di->have_dinfo' failed." on wine's dlls/mscoree/tests/mscoree.c
Summary: valgrind: "Assertion 'di->have_dinfo' failed." on wine's dlls/mscoree/tests/m...
Status: RESOLVED FIXED
Alias: None
Product: valgrind
Classification: Developer tools
Component: general (show other bugs)
Version: 3.14 SVN
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Julian Seward
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-19 18:01 UTC by Austin English
Modified: 2017-08-01 04:27 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
debug log (433.63 KB, application/x-bzip)
2017-07-21 23:32 UTC, Austin English
Details
debug log 2 (422.41 KB, application/x-bzip)
2017-07-28 17:49 UTC, Austin English
Details
better report/handle and survive to invalid/unexpected pdb format (3.82 KB, text/plain)
2017-07-30 16:40 UTC, Philippe Waroquiers
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Austin English 2017-07-19 18:01:36 UTC
=8481== Warning: Missing or un-stat-able /home/austin/.wine/drive_c/windows/mono/mono-2.0/bin/libmono-2.0-x86.pdb

valgrind: m_debuginfo/debuginfo.c:1411 (vgPlain_di_notify_pdb_debuginfo): Assertion 'di->have_dinfo' failed.

host stacktrace:
==8481==    at 0x580398B9: show_sched_status_wrk (m_libcassert.c:355)
==8481==    by 0x580399C1: report_and_quit (m_libcassert.c:426)
==8481==    by 0x58039AA3: vgPlain_assert_fail (m_libcassert.c:492)
==8481==    by 0x580689A1: vgPlain_di_notify_pdb_debuginfo (debuginfo.c:1411)
==8481==    by 0x5808AA4D: do_client_request (scheduler.c:2029)
==8481==    by 0x5808AA4D: vgPlain_scheduler (scheduler.c:1433)
==8481==    by 0x58098963: thread_wrapper (syswrap-linux.c:103)
==8481==    by 0x58098963: run_a_thread_NORETURN (syswrap-linux.c:156)

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 8481)
==8481==    at 0x7BCAC22A: map_image (virtual.c:1363)
==8481==    by 0x7BCB03F2: NtMapViewOfSection (virtual.c:2747)
==8481==    by 0x7BC5C2B7: load_native_dll (loader.c:1824)
==8481==    by 0x7BC5DDA7: load_dll (loader.c:2334)
==8481==    by 0x7BC5E024: LdrLoadDll (loader.c:2367)
==8481==    by 0x7B4613D5: load_library (module.c:947)
==8481==    by 0x7B461582: LoadLibraryExW (module.c:1007)
==8481==    by 0x7B4616BE: LoadLibraryW (module.c:1049)
==8481==    by 0x6C663126: ???

Thread 2: status = VgTs_WaitSys (lwpid 8495)
==8481==    at 0x424CEEF: ??? (syscall-template.S:84)
==8481==    by 0x7BC89730: wait_select_reply (server.c:348)
==8481==    by 0x7BC8A49D: server_select (server.c:607)
==8481==    by 0x7BC979A8: NtWaitForKeyedEvent (sync.c:1193)
==8481==    by 0x7BC98D34: RtlSleepConditionVariableCS (sync.c:1854)
==8481==    by 0x7B484166: SleepConditionVariableCS (sync.c:2430)
==8481==    by 0x6C762A50: ???

valgrind-3.14.0.SVN-16459-vex-3398 / wine-2.12-205-g9118512135
Comment 1 Philippe Waroquiers 2017-07-21 18:47:28 UTC
I improved the indentation of the code handling missing or un-stat-able
pdb info (as the indentation is misleading).  Rev 16460

But whatever, in this 'missing case', the code should execute a 'goto out"
and not assert due to a failure to read a pdb.

Can you run with more traces and attach the result ?
(e.g; with -v -v -v -d -d -d, so that we can see which pdb file causes
the assertion failure)
Comment 2 Austin English 2017-07-21 23:32:41 UTC
Created attachment 106785 [details]
debug log
Comment 3 Philippe Waroquiers 2017-07-22 09:41:18 UTC
The dll and pdb being loaded are:
mscorlib.dll
mscorlib.pdb

Looking at the trace compared to the code, once we have seen the traces
   
==22926== LOAD_PDB_DEBUGINFO:   Scanning PE section .reloc at avma 0x7ac000 svma 0x3ac000
--22926-- LOAD_PDB_DEBUGINFO: rx_map: avma 0x402000 size 3831716 foff 512
--22926-- LOAD_PDB_DEBUGINFO: rw_map: avma 0x7aa000 size    1024 foff 3832320
--22926-- LOAD_PDB_DEBUGINFO:   text: avma 0x402000 svma 0x2000 size 3831716 bias 0x0

there is only one place where the readpdb code returns without dinfo, that is:
   hdr = find_pdb_header( pdbimage, &signature );
   if (0==hdr)
      return False; /* JRS: significance? no pdb header? */

This find_pdb_header is searching for a specific character '\032'
and the string "Microsoft C/C++"
Is the pdb file containing the expected pdb header ?
I guess wine and/or microsoft-windows have objdump like utilities ?
Or else just look with emacs this pdb and another working pdb, to
see if the expected data is there ?

Would be good also to redo the tracing above adding something like
--trace-symtab=yes --trace-symtab-patt=*mscorlib*
Comment 4 Austin English 2017-07-25 03:57:53 UTC
(In reply to Philippe Waroquiers from comment #3)
> This find_pdb_header is searching for a specific character '\032'
> and the string "Microsoft C/C++"
> Is the pdb file containing the expected pdb header ?

No, it does not. These files were produced by mingw64. File considers them data:
/home/austin/.wine/drive_c/windows/mono/mono-2.0/lib/mono/4.5/mscorlib.pdb:                                                                                                                      data

whereas real .pdb files from visual studio appear as:
./memory/mozalloc/mozalloc.pdb:                   MSVC program database ver 7.00, 4096*95 bytes

it's produced by Mingw, not Microsoft. Whereas my real pdb's first line is:
Microsoft C/C++ MSF 7.00
DS

The first hex character is 0x4D, however (maybe I'm misunderstanding what you're asking).

FYI, I uploaded the file here:
http://austinenglish.com/files/for_valgrind/mscorlib.pdb

the genuine pdb files I used are at https://phoenixnap.dl.sourceforge.net/project/wine/Wine%20Gecko/2.36/wine_gecko-2.36-x86-dbg-msvc-pdb.tar.bz2

> I guess wine and/or microsoft-windows have objdump like utilities ?
> Or else just look with emacs this pdb and another working pdb, to
> see if the expected data is there ?
> 
> Would be good also to redo the tracing above adding something like
> --trace-symtab=yes --trace-symtab-patt=*mscorlib*

I'll get this in a bit or tomorrow.
Comment 5 Austin English 2017-07-25 03:59:36 UTC
Forgot to say, first line of mingw64's pdb is:
BSJB PDB V1.0

first character is \042
Comment 6 Philippe Waroquiers 2017-07-25 20:24:20 UTC
mscorelib.pdb starts with:
0000000    5342    424a    0001    0001    0000    0000    000c    0000
          B   S   J   B 001  \0 001  \0  \0  \0  \0  \0  \f  \0  \0  \0
0000020    4450    2042    3156    302e    0000    0000    0000    0005
          P   D   B       V   1   .   0  \0  \0  \0  \0  \0  \0 005  \0
0000040    0070    0000    2154    000a    7e23    0000    21c4    000a
          p  \0  \0  \0   T   !  \n  \0   #   ~  \0  \0 304   !  \n  \0
0000060    8c14    0000    5323    7274    6e69    7367    0000    0000
        024 214  \0  \0   #   S   t   r   i   n   g   s  \0  \0  \0  \0
0000100    add8    000a    0040    0000    4723    4955    0044    0000
        330 255  \n  \0   @  \0  \0  \0   #   G   U   I   D  \0  \0  \0

while another wine pdb starts with:
od -xc wine_gecko-2.36-x86-dbg/gfx/angle/src/libEGL/libEGL.pdb | head -10
0000000    694d    7263    736f    666f    2074    2f43    2b43    202b
          M   i   c   r   o   s   o   f   t       C   /   C   +   +    
0000020    534d    2046    2e37    3030    0a0d    441a    0053    0000
          M   S   F       7   .   0   0  \r  \n 032   D   S  \0  \0  \0
0000040    1000    0000    0002    0000    0255    0000    09f4    0000
         \0 020  \0  \0 002  \0  \0  \0   U 002  \0  \0 364  \t  \0  \0
0000060    0000    0000    0253    0000    0000    0000    0000    0000
         \0  \0  \0  \0   S 002  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000100    0000    0000    0000    0000    0000    0000    0000    0000
         \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0

The valgrind code expects something very precise as a header.
See function static void* find_pdb_header( void* pdbimage,
                              unsigned* signature )
followed by one of 2 struct (either an old pdb or a new pdb struct).
It looks like the code is not ready at all to read the above. 
If only the header (slightly) differs, it might be easy to enhance the valgrind pdb header.
But if the mscorelib.pdb is very different, then that will be a significant work.
Note that the readpdb.c seems to be a forked copy of some wine code, as I understand
Comment 7 Austin English 2017-07-28 17:49:18 UTC
Created attachment 106926 [details]
debug log 2

with --trace-symtab=yes --trace-symtab-patt=*mscorlib*
Comment 8 Austin English 2017-07-28 17:53:06 UTC
(In reply to Philippe Waroquiers from comment #6)
> The valgrind code expects something very precise as a header.
> See function static void* find_pdb_header( void* pdbimage,
>                               unsigned* signature )
> followed by one of 2 struct (either an old pdb or a new pdb struct).
> It looks like the code is not ready at all to read the above. 
> If only the header (slightly) differs, it might be easy to enhance the
> valgrind pdb header.
> But if the mscorelib.pdb is very different, then that will be a significant
> work.
> Note that the readpdb.c seems to be a forked copy of some wine code, as I
> understand

It seems I misspoke, the pdb comes from mono, via csc/roslyn:
http://www.mono-project.com/docs/about-mono/releases/5.0.0/#csc
https://github.com/dotnet/roslyn

also, note that Microsoft released info about the PDB format a while back for LLVM/Clang, under the MIT license:
https://github.com/dotnet/roslyn

While getting these pdbs would be cool long term, my immediate concern is valgrind crashing. Printing an error/fixme would be enough for me, for this bug report.
Comment 9 Philippe Waroquiers 2017-07-30 16:39:28 UTC
Can you try the attached patch ? This produces more user messages when it cannot
read a pdb file.
Also, it should recover properly from an invalid/unsupported pdb file.

Note also that a quick search on the web seems to indicate that Mingw does
not generate pdb files, bur rather seems to generate dwarf debug info.
Strange however that mscorlib.pdb indicates a PDB 1.0 version
(which seems a very old format?). Maybe PDB 1.0 is just the container format ?
Even in the wine sources (pdb.c), I could not find anything about this
format, wine pdb.c seems to expect the same as valgrind readpdb.c expects.

So, at this time, unclear how to read this pdb file, so let's try to make
Valgrind resists to it ...
Comment 10 Philippe Waroquiers 2017-07-30 16:40:50 UTC
Created attachment 106973 [details]
better report/handle and survive to invalid/unexpected pdb format
Comment 11 Philippe Waroquiers 2017-07-30 16:42:44 UTC
Note that the patch is compiling on linux, but is completely untested.
So, expect fire, smoke and explosions ...
Comment 12 Austin English 2017-07-31 06:15:33 UTC
(In reply to Philippe Waroquiers from comment #11)
> Note that the patch is compiling on linux, but is completely untested.
> So, expect fire, smoke and explosions ...

Thanks Philippe, the patch makes things look much nicer! Here's the output (with wine specific output you don't care about removed):

../../../tools/runtest -q -P wine -T ../../.. -M mscoree.dll -p mscoree_test.exe.so mscoree && touch mscoree.ok
==15080== Warning: Missing or un-stat-able /home/austin/.wine/drive_c/windows/system32/shlwapi.pdb
==15194== Warning: Missing or un-stat-able /home/austin/.wine/drive_c/windows/system32/shlwapi.pdb
==15194== Warning: Missing or un-stat-able /home/austin/.wine/drive_c/windows/mono/mono-2.0/bin/libmono-2.0-x86.pdb
==15194== LOAD_PDB_DEBUGINFO: \032 header character not found.  possible invalid/unsupported pdb file format?
==15194== LOAD_PDB_DEBUGINFO: find_pdb_header found no hdr.  possible invalid/unsupported pdb file format?
==15194== LOAD_PDB_DEBUGINFO: failed loading info from /home/austin/.wine/drive_c/windows/mono/mono-2.0/lib/mono/4.5/mscorlib.pdb
==15080== Warning: Missing or un-stat-able /home/austin/.wine/drive_c/windows/mono/mono-2.0/bin/libmono-2.0-x86.pdb
==15080== LOAD_PDB_DEBUGINFO: \032 header character not found.  possible invalid/unsupported pdb file format?
==15080== LOAD_PDB_DEBUGINFO: find_pdb_header found no hdr.  possible invalid/unsupported pdb file format?
==15080== LOAD_PDB_DEBUGINFO: failed loading info from /home/austin/.wine/drive_c/windows/mono/mono-2.0/lib/mono/4.5/mscorlib.pdb
==15080== LOAD_PDB_DEBUGINFO: \032 header character not found.  possible invalid/unsupported pdb file format?
==15080== LOAD_PDB_DEBUGINFO: find_pdb_header found no hdr.  possible invalid/unsupported pdb file format?
==15080== LOAD_PDB_DEBUGINFO: failed loading info from /home/austin/.wine/drive_c/windows/mono/mono-2.0/lib/mono/gac/Mono.Security/4.0.0.0__0738eb9f132ed756/Mono.Security.pdb

the first (missing shlwapi), is valid, wine doesn't build .pdb files for its own dlls. 
libmono-2.0-x86.pdb is also not included in wine-mono, not sure why it didn't get one when others did, but that's how it is
the other two are C# dlls, so the unexpected pdb format makes sense

FYI, I've written a patch for this format for `file`, it's header is:
BSBJ\001\000\001\000\000\000\000\000\f\000\000\000PDB\ v1.0

though note that the 'v' is case insensitive.

As far as I'm concerned, this patch is good enough for now. I'll likely file a follow up bug once this is resolved so that it's known/documented, but I don't think the effort to implement is worth the gain, unless running c# code under valgrind gets popular..
Comment 13 Philippe Waroquiers 2017-07-31 20:45:25 UTC
Thanks for the testing
Patch committed as revision 16465.

As discussed in comment 12, really handling this kind of debug info
will be another bug (if needed)
Comment 14 Austin English 2017-08-01 04:27:49 UTC
(In reply to Philippe Waroquiers from comment #13)
> Thanks for the testing
> Patch committed as revision 16465.
> 
> As discussed in comment 12, really handling this kind of debug info
> will be another bug (if needed)

Fixed in r16465, thanks Philippe. Next bug is bug 382978.