| Summary: | Segfalut when attaching to binary (Ubuntu 20.04 docker) | ||
|---|---|---|---|
| Product: | [Applications] Heaptrack | Reporter: | Jacob <domagalke> |
| Component: | general | Assignee: | Milian Wolff <mail> |
| Status: | REPORTED --- | ||
| Severity: | normal | ||
| Priority: | NOR | ||
| Version First Reported In: | 1.1.0 | ||
| Target Milestone: | --- | ||
| Platform: | Compiled Sources | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed/Implemented In: | ||
| Sentry Crash Report: | |||
can you try to run `heaptrack -d -p ...` and then get a backtrace from the gdb shell that should be kept open when it crashes? can you share a minimal docker file to reproduce the issue? Hello Milian, thank you for replying.
It seems to be reproducable with following docker image:
------------------------------------------------------------------
FROM ubuntu:latest
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y heaptrack gdb clang-11
------------------------------------------------------------------
1. docker run --cap-add=SYS_PTRACE --security-opt seccomp=unconfined
2. Create file test.cc with following content:
-------------------------------
int main(int argc, char** argv)
{
while(1){
int* a = new int(10);
delete a;
}
return 0;
}
-------------------------------
3. Build it 'clang++-11 test.cc -o test'
4. Start it and get its PID
5. Attach heaptrack using the PID 'heaptrack -d -p PID'
6. Heaptrack output:
heaptrack output will be written to "//heaptrack.test.1481.gz"
injecting heaptrack into application via GDB, this might take some time...
Attaching to process 1162
Reading symbols from /test...
(No debugging symbols found in /test)
Reading symbols from /lib/x86_64-linux-gnu/libstdc++.so.6...
(No debugging symbols found in /lib/x86_64-linux-gnu/libstdc++.so.6)
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...
Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libm-2.31.so...
Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libgcc_s.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...
Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.31.so...
Reading symbols from /lib64/ld-linux-x86-64.so.2...
(No debugging symbols found in /lib64/ld-linux-x86-64.so.2)
__GI___libc_malloc (bytes=4) at malloc.c:2940
2940 malloc.c: No such file or directory.
Symbols already loaded for /lib/x86_64-linux-gnu/libc.so.6
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Symbols already loaded for /usr/lib/heaptrack/libheaptrack_inject.so
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on".
Evaluation of the expression containing the function
(heaptrack_inject) will be abandoned.
When the function is done executing, GDB will silently stop.
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x00007f50eb626049 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#2 0x00007f50eb1e66ff in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so
#3 0x00007f50eb1e7ffe in heaptrack_init () from /usr/lib/heaptrack/libheaptrack_inject.so
#4 0x00007f50eb1e53f2 in heaptrack_inject () from /usr/lib/heaptrack/libheaptrack_inject.so
#5 <function called from gdb>
#6 __GI___libc_malloc (bytes=4) at malloc.c:2940
#7 0x00007f50eb5f9c29 in operator new(unsigned long) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#8 0x0000000000401160 in main ()
(gdb)
Oh ok, it seems that for this example adding -pthread solves the issue. I will see whether it was also the case in my previous testings. Sadly it doesn't help with my initial test. I will try to isolate the issue (my guess is that some libs that I'm linking with, could be somehow causing the issue) heaptrack output will be written to "/vt/heaptrack.memory_checker.31540.gz" injecting heaptrack into application via GDB, this might take some time... Attaching to process 31464 Reading symbols from /build/vt/tests/memory_checker... (No debugging symbols found in /build/vt/tests/memory_checker) Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2... Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libdl-2.31.so... Reading symbols from /lib/x86_64-linux-gnu/libz.so.1... (No debugging symbols found in /lib/x86_64-linux-gnu/libz.so.1) Reading symbols from /usr/local/lib/libmpicxx.so.12... (No debugging symbols found in /usr/local/lib/libmpicxx.so.12) Reading symbols from /usr/local/lib/libmpi.so.12... (No debugging symbols found in /usr/local/lib/libmpi.so.12) Reading symbols from /lib/x86_64-linux-gnu/libstdc++.so.6... (No debugging symbols found in /lib/x86_64-linux-gnu/libstdc++.so.6) Reading symbols from /lib/x86_64-linux-gnu/libm.so.6... Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libm-2.31.so... Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1... (No debugging symbols found in /lib/x86_64-linux-gnu/libgcc_s.so.1) Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0... --Type <RET> for more, q to quit, c to continue without paging-- Reading symbols from /usr/lib/debug/.build-id/e5/4761f7b554d0fcc1562959665d93dffbebdaf0.debug... [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Reading symbols from /lib/x86_64-linux-gnu/libc.so.6... Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.31.so... Reading symbols from /lib64/ld-linux-x86-64.so.2... (No debugging symbols found in /lib64/ld-linux-x86-64.so.2) Reading symbols from /lib/x86_64-linux-gnu/librt.so.1... Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/librt-2.31.so... 0x00000000005fddd0 in main () Symbols already loaded for /lib/x86_64-linux-gnu/libc.so.6 Symbols already loaded for /usr/lib/heaptrack/libheaptrack_inject.so Program received signal SIGSEGV, Segmentation fault. 0x00007f423d9bfe76 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so The program being debugged was signaled while in a function called from GDB. GDB remains in the frame where the signal was received. To change this behavior use "set unwindonsignal on". Evaluation of the expression containing the function (heaptrack_inject) will be abandoned. When the function is done executing, GDB will silently stop. (gdb) bt #0 0x00007f423d9bfe76 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #1 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #2 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #3 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #4 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #5 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #6 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #7 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #8 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #9 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #10 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #11 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #12 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #13 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #14 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #15 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #16 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #17 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #18 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #19 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #20 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #21 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #22 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #23 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #24 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #25 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #26 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #27 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #28 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #29 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #30 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #31 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #32 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #33 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #34 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #35 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #36 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #37 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #38 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #39 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #40 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #41 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #42 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #43 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #44 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #45 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #46 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so #47 0x00007f423d9bfe83 in ?? () from /usr/lib/heaptrack/libheaptrack_inject.so --Type <RET> for more, q to quit, c to continue without paging--q Quit (gdb) q |
I'm getting the Segfault when running heaptrack with attach mode (heaptrack -p) on programs compiled inside docker container (ubuntu 20.04 clang-10, heaptrack 1.1.80 and 1.2.80). I've tested it on pre-compiled programs, such as nano, and it works fine, so I'm guessing I'm doing something wrong. Even as simple program as this will fail: int main(int argc, char** argv) { while(1){ int* a = new int(10); delete a; } return 0; } I've tried to debug the issue (see ADDITIONAL INFORMATION below) but wasn't able to fix the issue. STEPS TO REPRODUCE Not sure whether it's related to my docker container or how I compile things, but anything that I compile and try to run heaptrack on it, will fail. OBSERVED RESULT heaptrack output will be written to "//heaptrack.a.out.17470.gz" injecting heaptrack into application via GDB, this might take some time... The program being debugged was signaled while in a function called from GDB. GDB remains in the frame where the signal was received. To change this behavior use "set unwindonsignal on". Evaluation of the expression containing the function (heaptrack_inject) will be abandoned. When the function is done executing, GDB will silently stop. heaptrack stats: allocations: 0 leaked allocations: 0 temporary allocations: 0 injection finished removing heaptrack injection via GDB, this might take some time... ptrace: No such process. No symbol table is loaded. Use the "file" command. The program is not being run. Heaptrack finished! Now run the following to investigate the data: heaptrack --analyze "//heaptrack.a.out.17470.gz" EXPECTED RESULT SOFTWARE/OS VERSIONS Docker image: Ubuntu 20.04 Compiler: clang-10 or gcc-9.3 heaptrack 1.1.80 and 1.2.80 ADDITIONAL INFORMATION I've tried to debug where exactly it segfaults and it seems that it fails during call to heaptrack_inject, on this line: libheaptrack.cpp:317 s_data = new LockedData(out, stopCallback); and then inside: heaptrack_inject.cpp:94 static void* hook(size_t size) noexcept Let me know if you need any other information.