When I run my application (which uses Qt) where the Qt libs were compiled with some newer compiler (>= clang++-9 or >= g++-9 but I'm not sure which version still did work), then valgrind terminates with vex amd64->IR: unhandled instruction bytes: 0x37 0x48 0x83 0xF8 0x2E 0xF 0x84 0x17 0xFF 0xFF vex amd64->IR: REX=0 REX.W=0 REX.R=0 REX.X=0 REX.B=0 vex amd64->IR: VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=NONE vex amd64->IR: PFX.66=0 PFX.F2=0 PFX.F3=0 ==19360== valgrind: Unrecognised instruction at address 0x2bf6ede1. When I run it with Qt libs compiled with older compiler (whatever was current at around 23.Nov 2020 on openSuse TW), valgrind works just fine. It seems newer compilers create some code valgrind can not handle yet. Maybe there is any workaround or compiler flag I could use to prevent the problem ?
Well 0x37 is the AAA (ASCII Adjust After Addition) instruction which seems pretty unlikely, not least because it's not valid in 64 bit mode. I'm using a 2019 edition of the Intel manual so maybe it has been reused in 64 bit mode now - are you using a very recent processor and did you compile with -march=native or similar?
Checked the December 2021 edition of the manual now and nothing seems to have changed so that is not a valid instruction. Either this is self modifying code that hasn't been detected (does --smc-check=all help?) or the program has taken a bogus branch somewhere - are there any warnings before that about bad memory accesses?
So it has to do with libQtCore. When I use the option --smc-check=all it does not crash any more! There are TONS of warnings (with or without this option), all of the types: Conditional jump or move depends on uninitialised value(s) or Use of uninitialised value of size I also get a huge number of the like ==22110== Conditional jump or move depends on uninitialised value(s) ==22110== at 0x2E482C35: ??? (in /memfd:sljit (deleted)) ==22110== by 0x1755BF17: ??? which does not help me in any way, sadly. Since the "smc" option helps, does it give any specific warning which I can search for ? (since there are so many, that I can not check all manually)
Well that's quite interesting - the default is --smc-check=all-non-file which means we check for self modifying code on heap, stack and in anonymous mappings but not in file backed mappings. So it sounds like it is modifying code in a file backed mapping which is quite an odd thing to do! Either that or there is some sort of bug in our detection of whether a mapping is file backed.
Ah the /memfd:sljit path is probably a big clue - it's using an anonymous file created with memfd_create so it is file backed but not with a real file.
Specifically https://github.com/zherczeg/sljit/blob/8d0f668fad91ca1d6c8afb3b72eeeeb34db41bee/sljit_src/sljitProtExecAllocator.c#L116 in sljit.
I found out that QtCore uses 3rdparty/pcre2 code, which itself contains the sljit code. So does this mean, valgrind should handle this (aka: a bug/limitation in valgrind) ?
Well in principle we should probably treat them as non-file backed but it's going to be tricky to identify them. This is what a mapping from such a file looks like in valgrind's address map: --3993090:1: aspacem 10: file 0004033000-0004033fff 4096 rwx-- d=0x001 i=445531 o=0 (8,345) So the device is 00:01 which is same as any other shared memory backed file, including files on tmpfs filesystems. I guess we could look at the filename but we'd probably want to do that when updating the map and store a flag because the point where actually want to check whether something is file backed is a hot path that needs to avoid unnecessary slowdowns.