Created attachment 183620 [details] Whole DrKonqi log with the problem SUMMARY Encountered a rather bad situation, where a crash reporter project gets an exception itself, failing to report the actual crash it was supposed to handle. STEPS TO REPRODUCE 1. Use Fedora Kinoite -> Universal Blue -> Aurora (getaurora.dev) derivative atomic distro, rather regular KDE installation 2. Have DrKonqi configured to auto-send reports and grab symbols (both checkboxes checked after first random app crash, in drkonqirc DownloadSymbols=true and Sentry=true) 3. Have a random app crash 4. Have a DrKonqi notification displayed that a crash will be auto-send, and you can either add a comment, restart the app or do nothing 5. Now everything should be okay, but you can also click Add a Comment which I do OBSERVED RESULT If you open the DrKonqi window in step #5, it will report with a message at the top that it's failed reporting the error, giving you an option to view log or retry. The progress bar keeps moving. You have the option to enter crash comment, but doing so and confirming **WILL NOT HAVE IT SENT ANYWHERE** critically from UX-standpoint (which you could consider a separate UX bug on its own, it sucks to pour some heart into descriptive bug report, but that comment gets discarded, yikes). Besides said window also crashed on one attempt as I finished writing the bug comment, which is also not nice. Anyway, the real problem is the log has a bunch of lines like: warning: Can't open file /65/2cadaf974cb4dae6fd6815eb3913aaad7ae7dccf6bd922db9df17a10d4222a.file during file-backed mapping note processing Then DrKonqi ends up with `add symbol table from file "None"` trying to grab a symbol from non-existent file, this throws a Python error (in /usr/share/drkonqi/gdb/python/gdb_preamble/preamble.py, see whole log attachment), which in turn shows `gdb.error: None: No such file or directory.` and the whole script reports exit code 1 at the end. This causes Sentry crash not to be sent. Neither of the Sentry envelopes dirs exist in ~/.cache/drkonqi at all! Note that if I view the actual crash dump's info in DrKonqi's browser, I do generally see the crash info and symbols just fine. These suspicious files are just at the bottom in what seems to be the way the actual executables were launched in the stack: [...] #35 0x00007fe6677d7419 _ZN16QCoreApplication4execEv (libQt6Core.so.6 + 0x103419) #36 0x00005605367877ad n/a (/29/5c45f94ee5681696fb209cb4d505ca67b966d2dab16859fe5ff361ed48e991.file + 0xf7ad) #37 0x00007fe66726e5f5 __libc_start_call_main (libc.so.6 + 0x35f5) #38 0x00007fe66726e6a8 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x36a8) #39 0x0000560536788a45 n/a (/29/5c45f94ee5681696fb209cb4d505ca67b966d2dab16859fe5ff361ed48e991.file + 0x10a45) EXPECTED RESULT The crash report should go through, even if not all symbols could be loaded. SOFTWARE/OS VERSIONS Windows: macOS: (available in the Info Center app, or by running `kinfo` in a terminal window) Linux/KDE Plasma: 6.14.11-300.fc42.x86_64 KDE Plasma Version: 6.4.3 KDE Frameworks Version: 6.16.0 Qt Version: 6.9.1 ADDITIONAL INFORMATION Glimpse of the DrKonqi bug (I suspect "None" is the suspicious files without path): Core was generated by `/usr/bin/dolphin'. Program terminated with signal SIGSEGV, Segmentation fault. #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=11, no_tid=no_tid@entry=0) at pthread_kill.c:44 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0; [Current thread is 1 (Thread 0x7fbb28b37080 (LWP 22679))] Cannot QML trace cores :( add symbol table from file "/lib64/libc.so.6" add symbol table from file "/lib64/libKF6Crash.so.6" add symbol table from file "/lib64/libQt6Quick.so.6" add symbol table from file "/lib64/libQt6QuickWidgets.so.6" add symbol table from file "/lib64/libQt6Core.so.6" add symbol table from file "/lib64/libQt6Widgets.so.6" add symbol table from file "/lib64/libglib-2.0.so.0" add symbol table from file "None" Traceback (most recent call last): File "/usr/share/drkonqi/gdb/python/gdb_preamble/preamble.py", line 812, in print_preamble print_preamble_internal() ~~~~~~~~~~~~~~~~~~~~~~~^^ File "/usr/share/drkonqi/gdb/python/gdb_preamble/preamble.py", line 803, in print_preamble_internal print_sentry_payload(thread) ~~~~~~~~~~~~~~~~~~~~^^^^^^^^ File "/usr/share/drkonqi/gdb/python/gdb_preamble/preamble.py", line 727, in print_sentry_payload payload = SentryEvent().make(program, thread) File "/usr/share/drkonqi/gdb/python/gdb_preamble/preamble.py", line 520, in make stacktrace = SentryTrace(crash_thread, True).to_dict() File "/usr/share/drkonqi/gdb/python/gdb_preamble/preamble.py", line 367, in to_dict SentryTrace.load_solib(self.thread, cramped_memory) ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/share/drkonqi/gdb/python/gdb_preamble/preamble.py", line 349, in load_solib gdb.execute(f'add-symbol-file {solib}') ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^ gdb.error: None: No such file or directory. None: No such file or directory. Sentry is attempting to send 2 pending events Waiting up to 30 seconds Press Ctrl-C to quit Debugging ended with exit code '1' and exit status 'NormalExit'
Created attachment 183621 [details] Same issue even without debuginfod Note that even before I enabled DownloadSymbols=true which enabled debuginfod in GDB, the exact same error was happening (I thought enabling symbols might fix it, but nope, no change)
Created attachment 183622 [details] Meanwhile the whole crashdump info is pretty okay Meanwhile the whole crashdump info is pretty okay, it's just these suspicious files at the beginning of the stack that break the day, but they shouldn't impact the usefulness of the actual trace of the actual crash and bugs (in this case it's a Dolphin crash that I otherwise couldn't auto-report with a comment, about the first-time SMB password dialog crashing like crazy)
What's the output of cd /tmp coredumpctl dump -o core 19071 eu-unstrip --list-only --core core
Created attachment 183625 [details] Requested output (In reply to Harald Sitter from comment #3) > What's the output of > > cd /tmp > coredumpctl dump -o core 19071 > eu-unstrip --list-only --core core Putting it in attachment, since too long to paste
Probably needs reporting to your distribution as this indeed only happens with the mentioned distributions, according to our crash database. The problem is with these frames #36 0x00005605367877ad n/a (/29/5c45f94ee5681696fb209cb4d505ca67b966d2dab16859fe5ff361ed48e991.file + 0xf7ad) #37 0x00007fe66726e5f5 __libc_start_call_main (libc.so.6 + 0x35f5) #38 0x00007fe66726e6a8 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x36a8) #39 0x0000560536788a45 n/a (/29/5c45f94ee5681696fb209cb4d505ca67b966d2dab16859fe5ff361ed48e991.file + 0x10a45) These likely fail to map correctly and so we can't trace things. Indeed we can see that coredumpd also failed to trace them and it generally has more information than we do. The way things work is that when there isn't enough RAM to load all symbols (which would generally be more reliable) we instead walk all frames manually and for each frame resolve the library it belongs to based on the memory addresses involved and then run `sharedlibrary` and `add-symbol-file` in gdb to load whatever data is available such that the next frame may be resolved. That's where things fall over, one of the frames fails to find a suitable mapping in your eu-unstrip data and so things explode. That said, there's also some slight problem with the solib=file mapping dance inside drkonqi. At a glance it can happen that the file is None and still gets loaded, that's the bogus `add symbol table from file "None"` in your output.
A possibly relevant merge request was started @ https://invent.kde.org/plasma/drkonqi/-/merge_requests/350
> That said, there's also some slight problem with the solib=file mapping > dance inside drkonqi. At a glance it can happen that the file is None and > still gets loaded, that's the bogus `add symbol table from file "None"` in > your output. There definitely is, I could tell that from my general GDB shenanigans beforehand. The path from app's perspective can be wholly invalid from crash reporter's/debugger's external perspective, it might be gone, be bogus to begin with, be mapped into a sandbox with a different path (such as with stuff ran in Steam runtime for example). And even if it's there, there's no guarantee it's the same file (say on regular distro, if you ran a system upgrade, but had a program running with old zombie-file version of the library, your script would load the new version and probably screw with dump analysis within Sentry, or have it be completely discarded as invalid (in case that last possibility isn't accounted for by something like a checksum check, I have no idea if it is or not)). Also in my experience setting up some crash reporting stuff (on a different OS), conceptually inspecting the original memory of crashed app as much as possible rather than re-loading anything can go a long way, I once saw a crash where the code section with actual processor instructions was just corrupted (and even had the opportunity to contact the user, and they said they indeed were tinkering with unstable memory overclocking to have caused that). > The way things work is that when there isn't enough RAM to load all symbols > (which would generally be more reliable) we instead walk all frames manually > and for each frame resolve the library it belongs to based on the memory > addresses involved and then run `sharedlibrary` and `add-symbol-file` in gdb > to load whatever data is available such that the next frame may be resolved. > That's where things fall over, one of the frames fails to find a suitable > mapping in your eu-unstrip data and so things explode. So it is conditional then? I have 16 GB of memory on that machine (possibly no swap? I don't remember). I guess ideally there'd be some method to both avoid memory exhaustion and avoid trying to load the libraries anew from the disk. > Probably needs reporting to your distribution as this indeed only happens > with the mentioned distributions, according to our crash database. I can actually see more-less what the issue is. These .file files would actually reside at /ostree/repo/objects/ (or physically rather /sysroot/ostree/repo/objects/, as /sysroot is mounted on /). So the final path would be something like /ostree/repo/objects/29/5c45f94ee5681696fb209cb4d505ca67b966d2dab16859fe5ff361ed48e991.file Perhaps they set that library path somewhere for some early loader in similar way to how NixOS does its shenanigans? Well thankfully these very first few frames responsible for startup shouldn't matter much at all, other than just ensuring they don't cause DrKonqi to bail out and lose all potentially all crash reports from these distros. But I'm not sure if it can be considered an upstream bug frankly, these reported paths as I mentioned above cannot be guaranteed to correspond to physical paths from the perspective of all processes (as happens with sandboxes and stuff), it's downstream app that decided to rely on them in this way. And with that I said I hope there is indeed some better way to handle cases like this (I'd be glad to know actually if there is, since I was screwn over by similar issue before in general), in case you decide it's worth it to pursue/implement some better method for probably rare/minuscule cases like this. Is there maybe a chance it could be as simple as pointing GDB or that other tool at an extra library directory to try resolving? (at least to cover OSTree's case?) (inb4 fakesysroot with those two-letter dir names symlinked into its root)
Git commit 60e7d2cb651c30d5c36b3e296e3ad369f4001468 by Harald Sitter. Committed on 30/07/2025 at 06:45. Pushed by sitter into branch 'master'. prevent CoreImage from being invalid by having no file. this notably happens if we fall through the have_elf condition on kinoite M +1 -1 src/data/gdb_preamble/preamble.py https://invent.kde.org/plasma/drkonqi/-/commit/60e7d2cb651c30d5c36b3e296e3ad369f4001468
Git commit 1b4694ea1bce74905d4d12d3c2ae629f798fa024 by Harald Sitter. Committed on 30/07/2025 at 07:01. Pushed by sitter into branch 'Plasma/6.4'. prevent CoreImage from being invalid by having no file. this notably happens if we fall through the have_elf condition on kinoite (cherry picked from commit 60e7d2cb651c30d5c36b3e296e3ad369f4001468) Co-authored-by: Harald Sitter <sitter@kde.org> M +1 -1 src/data/gdb_preamble/preamble.py https://invent.kde.org/plasma/drkonqi/-/commit/1b4694ea1bce74905d4d12d3c2ae629f798fa024