Bug 489315 - With automatic crash reporting and tracing, something crashing in a loop can freeze and OOM the system
Summary: With automatic crash reporting and tracing, something crashing in a loop can ...
Status: REPORTED
Alias: None
Product: drkonqi
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR crash
Target Milestone: ---
Assignee: Plasma Bugs List
URL:
Keywords:
: 490527 495366 495953 (view as bug list)
Depends on:
Blocks:
 
Reported: 2024-06-27 15:03 UTC by Nate Graham
Modified: 2024-11-08 21:47 UTC (History)
8 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nate Graham 2024-06-27 15:03:25 UTC
When something crashes in a loop, multiple DrKonqi instances will be created, one per crash. Each one will try to trace and report the crash, spinning up a new instance of GDB. GDB can be heavy, especially when it's allowed to download debug symbols.

When this happen, you end up with multiple GDB processes competing for CPU and memory resources that are becoming more scarce. The system slows to a crawl and you either run out of memory or the OOM killer starts killings things.

It's not ideal, and worsens the already bad situation of something crashing a loop.

Some kind of throttling or repeated crash detection would be a good idea.
Comment 1 Harald Sitter 2024-07-10 11:56:34 UTC
Kind of dupe of bug #420443?
Comment 2 Nate Graham 2024-07-10 21:08:53 UTC
Sort of... this is about the performance impact more so than the ugly system tray full of sad face icons.
Comment 3 Harald Sitter 2024-07-11 07:39:55 UTC
I wonder if this is really a bug in drkonqi... in practice even a single trace can OOM. OOM in of itself isn't a defect.
Comment 4 Nate Graham 2024-07-22 20:53:59 UTC
It certainly is from a user's point of view. :) 

The reason I opened this on DrKonqi is because the issue gets triggered by automatic client-side gdb tracing. This wasn't a thing that would happen automatically until recently.

The window where you opt into it warns about unbounded networks downloads, but doesn't mention anything about local performance degradation while tracing happens. At the minimum this should be mentioned IMO, and ideally we should investigate why the tracing process consumes so much memory and CPU resources.
Comment 5 Nate Graham 2024-07-27 03:41:33 UTC
*** Bug 490527 has been marked as a duplicate of this bug. ***
Comment 6 Akseli Lahtinen 2024-08-07 17:03:28 UTC
I have same issue

- If i use automatic crash reporting, gdb manages to somehow eat all my 16gb of RAM
- If i use coredumpctl debug for same crash, it doesnt use nearly as much memory

Unsure what is going on here, I managed to get it with plasmashell crash today.

Operating System: Fedora Linux 40
KDE Plasma Version: 6.1.80
KDE Frameworks Version: 6.5.0
Qt Version: 6.7.2
Kernel Version: 6.9.12-200.fc40.x86_64 (64-bit)
Graphics Platform: Wayland
Processors: 12 × AMD Ryzen 5 3600 6-Core Processor
Memory: 15.5 GiB of RAM
Graphics Processor: AMD Radeon RX 6600
Comment 7 Bug Janitor Service 2024-08-28 16:42:56 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/drkonqi/-/merge_requests/268
Comment 8 Nate Graham 2024-10-28 03:09:52 UTC
*** Bug 495366 has been marked as a duplicate of this bug. ***
Comment 9 Harald Sitter 2024-11-06 15:32:29 UTC
Git commit c842292f5ac24babdd8c1e1fe4355b10181f8f2f by Harald Sitter.
Committed on 06/11/2024 at 15:29.
Pushed by sitter into branch 'master'.

backtracegenerator: implement a serialization lock

to prevent systems from drowning in gdb instances, we now establish a
serialization lock in the form of a file lock. only a single debugger
may be active at a time

this only partially fixes the problem since a single debugger may still
OOM the system. we still want a better solution, such as resource
constraining via systemd, in the long run.

M  +78   -3    src/backtracegenerator.cpp
M  +6    -1    src/backtracegenerator.h

https://invent.kde.org/plasma/drkonqi/-/commit/c842292f5ac24babdd8c1e1fe4355b10181f8f2f
Comment 10 Nate Graham 2024-11-08 21:47:18 UTC
*** Bug 495953 has been marked as a duplicate of this bug. ***