Bug 450952

Summary: unhandled instruction bytes on a program which executes normally outside valgrind [amd64 avx]
Product: [Developer tools] valgrind Reporter: Rich <Rincebrain>
Component: vexAssignee: Julian Seward <jseward>
Status: RESOLVED DUPLICATE    
Severity: normal CC: gabravier, mark, pjfloyd
Priority: NOR    
Version: 3.18.1   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: the erroring function after being passed through gcc -E

Description Rich 2022-02-28 06:18:15 UTC
Created attachment 147181 [details]
the erroring function after being passed through gcc -E

SUMMARY
I was running a benchmark of some different hashes, and encountered a strange error at the very end.

So I fired up valgrind, and instead, about halfway through the benchmark, it dies with SIGILL on:

vex amd64->IR: unhandled instruction bytes: 0xC5 0x79 0xD6 0xEE 0xC5 0xA1 0x73 0xD1 0x2C 0xC5
vex amd64->IR:   REX=0 REX.W=0 REX.R=1 REX.X=0 REX.B=0
vex amd64->IR:   VEX=1 VEX.L=0 VEX.nVVVV=0x0 ESC=0F
vex amd64->IR:   PFX.66=1 PFX.F2=0 PFX.F3=0
==2654301== valgrind: Unrecognised instruction at address 0x4aec8b4.
==2654301==    at 0x4AEC8B4: _mm_set_epi64x (emmintrin.h:592)
==2654301==    by 0x4AEC8B4: _mm_set_epi64 (emmintrin.h:598)
==2654301==    by 0x4AEC8B4: _mm_loadl_epi64 (emmintrin.h:709)
==2654301==    by 0x4AEC8B4: sseu2KeccakPermutationOnWordsAfterXoring1344bits (KeccakF-1600-opt64.c:302)
==2654301==    by 0x4AE96B3: Keccak256_sseu2_Absorb (Keccak-sseu2-common.inc:204)
[...]

The binary was compiled with -march=native -O3 -ftree-vectorize on gcc 9.3, the code is from the sseu2 implementation of Keccak in SUPERCOP (https://bench.cr.yp.to/supercop.html).

The segment that it seems upset about appears to be in the middle here (there are no other c5 79 d6 ee in the function):
  29d867:       c4 c1 61 73 d3 3f       vpsrlq $0x3f,%xmm11,%xmm3
  29d86d:       c4 c1 01 73 f3 01       vpsllq $0x1,%xmm11,%xmm15
  29d873:       c5 b9 6c e1             vpunpcklqdq %xmm1,%xmm8,%xmm4
  29d877:       c5 01 eb fb             vpor   %xmm3,%xmm15,%xmm15
  29d87b:       c5 01 ef f9             vpxor  %xmm1,%xmm15,%xmm15
  29d87f:       c5 c9 73 d4 3f          vpsrlq $0x3f,%xmm4,%xmm6
  29d884:       c5 f9 70 c9 ee          vpshufd $0xee,%xmm1,%xmm1
  29d889:       c5 d9 73 f4 01          vpsllq $0x1,%xmm4,%xmm4
  29d88e:       c5 d9 eb e6             vpor   %xmm6,%xmm4,%xmm4
  29d892:       c5 c9 73 d1 3f          vpsrlq $0x3f,%xmm1,%xmm6
  29d897:       c5 f1 73 f1 01          vpsllq $0x1,%xmm1,%xmm1
  29d89c:       c4 c1 59 ef db          vpxor  %xmm11,%xmm4,%xmm3
  29d8a1:       c5 f1 eb ce             vpor   %xmm6,%xmm1,%xmm1
  29d8a5:       c4 c1 71 ef e0          vpxor  %xmm8,%xmm1,%xmm4
  29d8aa:       c5 79 70 c3 ee          vpshufd $0xee,%xmm3,%xmm8
  29d8af:       c5 b9 ef 4d a0          vpxor  -0x60(%rbp),%xmm8,%xmm1
  29d8b4:       c5 79 d6 ee             vmovq  %xmm13,%xmm6
  29d8b8:       c5 a1 73 d1 2c          vpsrlq $0x2c,%xmm1,%xmm11
  29d8bd:       c5 f1 73 f1 14          vpsllq $0x14,%xmm1,%xmm1
  29d8c2:       c4 c1 71 eb cb          vpor   %xmm11,%xmm1,%xmm1
  29d8c7:       c5 c9 ef f4             vpxor  %xmm4,%xmm6,%xmm6
  29d8cb:       c4 c1 21 73 dd 08       vpsrldq $0x8,%xmm13,%xmm11
  29d8d1:       c5 c9 6c f1             vpunpcklqdq %xmm1,%xmm6,%xmm6

The preprocessed version of this function (which I'm sharing because the code in question is a morass of twelve degrees of preprocessor macros) can be found attached.

OBSERVED RESULT
SIGILL

EXPECTED RESULT
Running application

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: 
Ubuntu 20.04 with a homebuilt 3.18.1 after the distro-provided 3.15.0 died the same way.

ADDITIONAL INFORMATION
I'm sorry if this is a dupe, it's difficult to sift through the unhandled opcodes bugs, for me, but I couldn't immediately turn up a similar one. Closest I found was https://bugs.kde.org/show_bug.cgi?id=417572.
Comment 1 Mark Wielaard 2023-04-20 11:50:59 UTC

*** This bug has been marked as a duplicate of bug 383010 ***