When using valgrind --tool=memcheck on an MPI program compiled and linked with IBM Parallel Environment, I get :

fthomas@hydra01:~> grep "^0:" errors.7751
0:==21416== Memcheck, a memory error detector
0:==21416== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
0:==21416== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
0:==21416== Command: ./a
0:==21416== 
0:vex amd64->IR: unhandled instruction bytes: 0xF 0x1 0xF9 0xBF 0x90 0xD0 0x3 0x0
0:vex amd64->IR:   REX=0 REX.W=0 REX.R=0 REX.X=0 REX.B=0
0:vex amd64->IR:   VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=0F
0:vex amd64->IR:   PFX.66=0 PFX.F2=0 PFX.F3=0
0:==21416== valgrind: Unrecognised instruction at address 0x6515d52.
0:==21416==    at 0x6515D52: PAMI::Global::Global() (Ptime.h:176)
0:==21416==    by 0x65133F8: global constructors keyed to Global.cc (Global.cc:25)
0:==21416==    by 0x65CDB95: ??? (in /opt/ibmhpc/pe1209/ppe.pami/gnu/lib64/pami64/libpami.so)
0:==21416==    by 0x63F547A: ??? (in /opt/ibmhpc/pe1209/ppe.pami/gnu/lib64/pami64/libpami.so)
0:==21416==    by 0x7FEFFEE07: ???
0:==21416==    by 0x400E1B7: call_init (in /lib64/ld-2.11.3.so)
0:==21416==    by 0x400E2E6: _dl_init (in /lib64/ld-2.11.3.so)
0:==21416==    by 0x4000B39: ??? (in /lib64/ld-2.11.3.so)

Using this little C code :
int main(void)
{
asm(".byte 0xF, 0x1, 0xF9, 0xBF, 0x90, 0xD0, 0x3, 0x0");
return 0;
}
shows that the problem is with instruction rdtscp.