Bug 462135

Summary: vex amd64->IR: unhandled instruction bytes: 0x62 0xF2 0xFD 0x28 0x7C 0xE9 0xB1 0x2 0x62 0xF2
Product: [Developer tools] valgrind Reporter: Stephan Mueller <smueller>
Component: memcheckAssignee: Julian Seward <jseward>
Status: RESOLVED DUPLICATE    
Severity: normal CC: gabravier, tom
Priority: NOR    
Version: 3.19.0   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Stephan Mueller 2022-11-22 16:52:48 UTC 
SUMMARY

vex amd64->IR: unhandled instruction bytes: 0x62 0xF2 0xFD 0x28 0x7C 0xE9 0xB1 0x2 0x62 0xF2
vex amd64->IR:   REX=0 REX.W=0 REX.R=0 REX.X=0 REX.B=0
vex amd64->IR:   VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=NONE
vex amd64->IR:   PFX.66=0 PFX.F2=0 PFX.F3=0


STEPS TO REPRODUCE
1. git clone https://github.com/smuellerDD/leancrypto
2. meson setup build
3. meson compile -C build
4. valgrind build/signature/tests/dilithium_tester_avx2

SOFTWARE/OS VERSIONS
Linux Fedora 36

Unhandled operation is Intel's AVX2 _mm256_sub_epi8()
Comment 1 Tom Hughes 2022-11-22 17:02:28 UTC 
That's an AVX512 instruction, or at least is encoded with an EVEX prefix which means it needs the AVX512 support to work.

*** This bug has been marked as a duplicate of bug 383010 ***
Comment 2 Stephan Mueller 2022-11-22 17:18:50 UTC 
Am Dienstag, 22. November 2022, 18:02:28 CET schrieb Tom Hughes:

Hi Tom,

> That's an AVX512 instruction, or at least is encoded with an EVEX prefix
> which means it needs the AVX512 support to work.
> 

This code works on a system that has no AVX512 support (i.e. an Alder Lake 
system). Furthermore, the used registers are ymm (i.e. the AVX2 256 bit 
registers).

Ciao
Stephan
Comment 3 Tom Hughes 2022-11-22 18:00:15 UTC 
Sure, but something has chosen to output the instruction with an EVEX prefix, which was added as part of AVX512 so you'll need the AVX512 support to have any chance of decoding it.

It may well be possible to encode the instruction in other ways, but that specific example has been encoded with an EVEX prefix as you can tell from the 0x62 at the start.
Comment 4 Stephan Mueller 2022-11-22 20:02:29 UTC 
Am Dienstag, 22. November 2022, 19:00:15 CET schrieb Tom Hughes:

Hi Tom,

> https://bugs.kde.org/show_bug.cgi?id=462135
> 
> --- Comment #3 from Tom Hughes <tom@compton.nu> ---
> Sure, but something has chosen to output the instruction with an EVEX
> prefix, which was added as part of AVX512 so you'll need the AVX512 support
> to have any chance of decoding it.

Agreed, it is an AVX512 instruction because I accidentally compiled the code 
with -march=native on a Rocket Lake with AVX512 instruction set. Without it, 
AVX2 instructions are emitted and there is no segfault.


Ciao
Stephan