Bug 477705 - A program being executed under callgrind terminates as soon as using fmt::format() in case of being compiled as RelWithDebInfo.
Summary: A program being executed under callgrind terminates as soon as using fmt::for...
Status: REPORTED
Alias: None
Product: valgrind
Classification: Developer tools
Component: callgrind (show other bugs)
Version: 3.22.0
Platform: Debian stable Linux
: NOR normal
Target Milestone: ---
Assignee: Paul Floyd
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-11-29 07:51 UTC by kde3201
Modified: 2024-02-06 14:07 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
demo program to reproduce the issue (1.92 KB, application/x-zip-compressed)
2023-11-29 07:51 UTC, kde3201
Details
CMake project showing the issue on clang++ 32 bit (1.28 MB, application/x-zip-compressed)
2024-02-06 07:26 UTC, kde3201
Details
vgdb output (1.45 KB, text/plain)
2024-02-06 07:27 UTC, kde3201
Details
gdb output (1.56 KB, text/plain)
2024-02-06 07:27 UTC, kde3201
Details
console output showing the issue (24.61 KB, text/plain)
2024-02-06 07:36 UTC, kde3201
Details

Note You need to log in before you can comment on or make changes to this bug.
Description kde3201 2023-11-29 07:51:06 UTC
Created attachment 163604 [details]
demo program to reproduce the issue

SUMMARY
A program being executed under callgrind terminates as soon as using fmt::format() in case of being compiled as RelWithDebInfo. 

STEPS TO REPRODUCE
1) unzip the attachment callgrind_demo.zip to ~
2) cd ~/callgrind_demo; rm -rf build/;  mkdir build; cd build/; conan install --profile=../linux_x86_release .. hello-world/0.0.1@siemens-energy/stable; cmake ..; cmake --build . --verbose; /usr/local/bin/valgrind --tool=callgrind --callgrind-out-file=callgrind.$RANDOM.log ~/callgrind_demo/build/bin/callgrind_demo

OBSERVED RESULT
The RelWithDebInfo version of "callgrind_demo" program terminates unexpectedly when being executed under callgrind:

==17794== Callgrind, a call-graph generating cache profiler
==17794== Copyright (C) 2002-2017, and GNU GPL'd, by Josef Weidendorfer et al.
==17794== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==17794== Command: /root/callgrind_demo/build/bin/callgrind_demo
==17794==
==17794== For interactive control, run 'callgrind_control -h'.
terminate called after throwing an instance of 'fmt::v9::format_error'
  what():  string pointer is null
==17794==
==17794== Process terminating with default action of signal 6 (SIGABRT)
==17794==    at 0x401B3E2: _dl_sysinfo_int80 (in /usr/lib/i386-linux-gnu/ld-linux.so.2)
==17794==    by 0x4C1F296: __pthread_kill_implementation (pthread_kill.c:43)
==17794==    by 0x4BCE100: raise (raise.c:26)
==17794==    by 0x4BB7269: abort (abort.c:79)
==17794==    by 0x48C59D5: ??? (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==17794==    by 0x48D3143: ??? (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==17794==    by 0x48D31BC: std::terminate() (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==17794==    by 0x48D34BB: __cxa_throw (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==17794==    by 0x10A52B: fmt::v9::detail::throw_format_error(char const*) (src/include/fmt/format-inl.h:44)
==17794==    by 0x10BEEF: void fmt::v9::detail::vformat_to<char>(fmt::v9::detail::buffer<char>&, fmt::v9::basic_string_view<char>, fmt::v9::basic_format_args<fmt::v9::basic_format_context<std::conditional<std::is_same<fmt::v9::type_identity<char>::type, char>::value, fmt::v9::appender, std::back_insert_iterator<fmt::v9::detail::buffer<fmt::v9::type_identity<char>::type> > >::type, fmt::v9::type_identity<char>::type> >, fmt::v9::detail::locale_ref) (src/include/fmt/format.h:3322)
==17794==    by 0x10AA79: fmt::v9::vformat[abi:cxx11](fmt::v9::basic_string_view<char>, fmt::v9::basic_format_args<fmt::v9::basic_format_context<fmt::v9::appender, char> >) (src/include/fmt/format-inl.h:1472)
==17794==    by 0x10A33B: main (core.h:3206)
==17794==
==17794== Events    : Ir
==17794== Collected : 1985341
==17794==
==17794== I   refs:      1,985,341
Aborted

EXPECTED RESULT
Successful execution (as well as it is successful for the debug version)

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Debian GNU/Linux 12 (bookworm) in a WSL

ADDITIONAL INFORMATION
If callgrind_demo is compiled as debug version everything works fine.

The simple program to reproduce the issue can be found in the attachment as well:

#include <fmt/core.h>
int main() {
  auto test = fmt::format("{}", "Hello world.");
  return 0;
}
Comment 1 Paul Floyd 2023-11-29 11:17:20 UTC
Is this only calgrind?

Does it run with --tool=none?
Does it run without errors with memcheck?
Comment 2 kde3201 2024-01-11 08:24:19 UTC
(In reply to Paul Floyd from comment #1)
> Is this only calgrind?
> 
> Does it run with --tool=none?
> Does it run without errors with memcheck?

Regarding your three questions the answers are no, no and no.

Nulgrind shows similar behavior:

/usr/local/bin/valgrind --tool=none ~/callgrind_demo/build/bin/callgrind_demo
==24052== Nulgrind, the minimal Valgrind tool
==24052== Copyright (C) 2002-2017, and GNU GPL'd, by Nicholas Nethercote.
==24052== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==24052== Command: /root/callgrind_demo/build/bin/callgrind_demo
==24052==
terminate called after throwing an instance of 'fmt::v9::format_error'
  what():  string pointer is null
==24052==
==24052== Process terminating with default action of signal 6 (SIGABRT)
==24052==    at 0x401B3E2: ??? (in /usr/lib/i386-linux-gnu/ld-linux.so.2)
==24052==    by 0x4C1F296: ??? (in /usr/lib/i386-linux-gnu/libc.so.6)
==24052==    by 0x4BCE100: raise (in /usr/lib/i386-linux-gnu/libc.so.6)
==24052==    by 0x4BB7269: abort (in /usr/lib/i386-linux-gnu/libc.so.6)
==24052==    by 0x48C59D5: ??? (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==24052==    by 0x48D3143: ??? (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==24052==    by 0x48D31BC: std::terminate() (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==24052==    by 0x48D34BB: __cxa_throw (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==24052==    by 0x10A52B: fmt::v9::detail::throw_format_error(char const*) (src/include/fmt/format-inl.h:44)
==24052==    by 0x10BEEF: void fmt::v9::detail::vformat_to<char>(fmt::v9::detail::buffer<char>&, fmt::v9::basic_string_view<char>, fmt::v9::basic_format_args<fmt::v9::basic_format_context<std::conditional<std::is_same<fmt::v9::type_identity<char>::type, char>::value, fmt::v9::appender, std::back_insert_iterator<fmt::v9::detail::buffer<fmt::v9::type_identity<char>::type> > >::type, fmt::v9::type_identity<char>::type> >, fmt::v9::detail::locale_ref) (src/include/fmt/format.h:3322)
==24052==    by 0x10AA79: fmt::v9::vformat[abi:cxx11](fmt::v9::basic_string_view<char>, fmt::v9::basic_format_args<fmt::v9::basic_format_context<fmt::v9::appender, char> >) (src/include/fmt/format-inl.h:1472)
==24052==    by 0x10A33B: main (core.h:3206)
==24052==
Aborted

And also memcheck brings up related output:

/usr/local/bin/valgrind --tool=memcheck ~/callgrind_demo/build/bin/callgrind_demo
==24103== Memcheck, a memory error detector
==24103== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==24103== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==24103== Command: /root/callgrind_demo/build/bin/callgrind_demo
==24103==
==24103== Conditional jump or move depends on uninitialised value(s)
==24103==    at 0x10BDFD: write<char, fmt::v9::appender> (src/include/fmt/format.h:3321)
==24103==    by 0x10BDFD: operator()<const char *> (src/include/fmt/format.h:3374)
==24103==    by 0x10BDFD: visit_format_arg<fmt::v9::detail::default_arg_formatter<char>, fmt::v9::basic_format_context<fmt::v9::appender, char> > (src/include/fmt/core.h:1651)
==24103==    by 0x10BDFD: void fmt::v9::detail::vformat_to<char>(fmt::v9::detail::buffer<char>&, fmt::v9::basic_string_view<char>, fmt::v9::basic_format_args<fmt::v9::basic_format_context<std::conditional<std::is_same<fmt::v9::type_identity<char>::type, char>::value, fmt::v9::appender, std::back_insert_iterator<fmt::v9::detail::buffer<fmt::v9::type_identity<char>::type> > >::type, fmt::v9::type_identity<char>::type> >, fmt::v9::detail::locale_ref) (src/include/fmt/format.h:4078)
==24103==    by 0x11F59D: ??? (in /root/callgrind_demo/build/bin/callgrind_demo)
==24103==
terminate called after throwing an instance of 'fmt::v9::format_error'
  what():  string pointer is null
==24103==
==24103== Process terminating with default action of signal 6 (SIGABRT)
==24103==    at 0x401B3E2: _dl_sysinfo_int80 (in /usr/lib/i386-linux-gnu/ld-linux.so.2)
==24103==    by 0x4C37296: __pthread_kill_implementation (pthread_kill.c:43)
==24103==    by 0x4BE6100: raise (raise.c:26)
==24103==    by 0x4BCF269: abort (abort.c:79)
==24103==    by 0x48DD9D5: ??? (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==24103==    by 0x48EB143: ??? (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==24103==    by 0x48EB1BC: std::terminate() (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==24103==    by 0x48EB4BB: __cxa_throw (in /usr/lib/i386-linux-gnu/libstdc++.so.6.0.30)
==24103==    by 0x10A52B: fmt::v9::detail::throw_format_error(char const*) (src/include/fmt/format-inl.h:44)
==24103==    by 0x10BEEF: write<char, fmt::v9::appender> (src/include/fmt/format.h:3322)
==24103==    by 0x10BEEF: operator()<const char *> (src/include/fmt/format.h:3374)
==24103==    by 0x10BEEF: visit_format_arg<fmt::v9::detail::default_arg_formatter<char>, fmt::v9::basic_format_context<fmt::v9::appender, char> > (src/include/fmt/core.h:1651)
==24103==    by 0x10BEEF: void fmt::v9::detail::vformat_to<char>(fmt::v9::detail::buffer<char>&, fmt::v9::basic_string_view<char>, fmt::v9::basic_format_args<fmt::v9::basic_format_context<std::conditional<std::is_same<fmt::v9::type_identity<char>::type, char>::value, fmt::v9::appender, std::back_insert_iterator<fmt::v9::detail::buffer<fmt::v9::type_identity<char>::type> > >::type, fmt::v9::type_identity<char>::type> >, fmt::v9::detail::locale_ref) (src/include/fmt/format.h:4078)
==24103==    by 0x10AA79: fmt::v9::vformat[abi:cxx11](fmt::v9::basic_string_view<char>, fmt::v9::basic_format_args<fmt::v9::basic_format_context<fmt::v9::appender, char> >) (src/include/fmt/format-inl.h:1472)
==24103==    by 0x10A33B: format<char const (&)[13]> (core.h:3206)
==24103==    by 0x10A33B: main (main.cc:5)
==24103==
==24103== HEAP SUMMARY:
==24103==     in use at exit: 19,083 bytes in 3 blocks
==24103==   total heap usage: 4 allocs, 1 frees, 19,115 bytes allocated
==24103==
==24103== LEAK SUMMARY:
==24103==    definitely lost: 0 bytes in 0 blocks
==24103==    indirectly lost: 0 bytes in 0 blocks
==24103==      possibly lost: 104 bytes in 1 blocks
==24103==    still reachable: 18,979 bytes in 2 blocks
==24103==                       of which reachable via heuristic:
==24103==                         stdstring          : 35 bytes in 1 blocks
==24103==         suppressed: 0 bytes in 0 blocks
==24103== Rerun with --leak-check=full to see details of leaked memory
==24103==
==24103== Use --track-origins=yes to see where uninitialised values come from
==24103== For lists of detected and suppressed errors, rerun with: -s
==24103== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Aborted
Comment 3 Paul Floyd 2024-01-31 08:22:52 UTC
Which version of GCC and libstdc++ is this with?
Comment 4 Paul Floyd 2024-01-31 20:50:47 UTC
I see from the project file, clang++ 16. I couldn't get Conan to work - I'm not familiar with setting it up.

I tried with Fedora 39, and amd64 host with clang++ 17 and the system lib fmt (built with GCC I presume). And I didn't have any errors.
Comment 5 Paul Floyd 2024-01-31 21:32:25 UTC
Last go. I tried building fmt from source with clang++, still no problem.

Can you used vgdb to see what is causing the exception? valgrind --vgdb-error=0 ./main and then start gdb in another terminal, copy and paste the command from the first valgrind terminal.
Comment 6 kde3201 2024-02-06 07:26:26 UTC
Created attachment 165592 [details]
CMake project showing the issue on clang++ 32 bit

CMake project showing the issue on clang++ 32 bit without conan dependency
Comment 7 kde3201 2024-02-06 07:27:16 UTC
Created attachment 165593 [details]
vgdb output

vgdb output as requested
Comment 8 kde3201 2024-02-06 07:27:56 UTC
Created attachment 165595 [details]
gdb output

gdb output as requested
Comment 9 kde3201 2024-02-06 07:36:28 UTC
Created attachment 165599 [details]
console output showing the issue

output of repro.sh on my machine
Comment 10 kde3201 2024-02-06 07:51:24 UTC
Maybe the following information helps reproducing the issue:

I checked several toolchain / architecture combinations, and it turned out that only number 4 shows the reported issue:
(1) g++ 64bit: OK
(2) g++ 32bit: OK
(3) clang++ 64bit: OK
(4) clang++ 32bit: FAILURE

Regarding the conan dependency:
I created another CMake project that builds the callgrind_demo app showing the issue. It does not have any conan dependency, but builds fmt from source as part of the project.
The contained 'repro.sh' script can be used to trigger the build and callgrind execution of all 4 combinations. (the log from my machine is attached as repro.log)

STEPS TO REPRODUCE
1) unzip the attachment bug_477705.zip to ~
2) ~/bug_477705/repro.sh

Regarding the requested gdb output I have attached 2 log files.

Regarding the used libstdc++ version:
~/bug_477705/build# ldd ~/bug_477705/build/callgrind_demo/callgrind_demo
        linux-gate.so.1 (0xf7f4a000)
        libstdc++.so.6 => /lib/i386-linux-gnu/libstdc++.so.6 (0xf7cf3000)
        libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf7bee000)
        libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf7bc7000)
        libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf799f000)
        /lib/ld-linux.so.2 (0xf7f4c000)

My system:
- Debian GNU/Linux 12 (bookworm) (in WSL 2)
- g++ (Debian 12.2.0-14) 12.2.0
- Debian clang version 14.0.6 (Target: x86_64-pc-linux-gnu)
Comment 11 Paul Floyd 2024-02-06 14:07:30 UTC
I can reproduce the problem now. It's not obvious what is going on as everything is inlined and in registers.

First thing, running to the error I can't get a callstack all the way back to main.

Need to debug some more.