Bug 412461 - Crash or extreme memory usage (over 20 gigabytes, OOM) on large attachment, over a gigabyte
Summary: Crash or extreme memory usage (over 20 gigabytes, OOM) on large attachment, o...
Status: REPORTED
Alias: None
Product: kontact
Classification: Applications
Component: mail (show other bugs)
Version: 5.9.2
Platform: Debian stable Linux
: NOR major
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-09-29 23:37 UTC by Melvin Vermeeren
Modified: 2019-09-29 23:44 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
htop prior to freeze and OOM (43.37 KB, image/jpeg)
2019-09-29 23:37 UTC, Melvin Vermeeren
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Melvin Vermeeren 2019-09-29 23:37:33 UTC
Created attachment 122939 [details]
htop prior to freeze and OOM

(Note: perhaps this bug should be filed under akonadi and kmail too.)

These are really two distinct bugs, both related to absurdly large attachments.

1. When attaching a really large attachment (try one of a few GB), Kontact crashes. This one is not so bad, you restart Kontact and it still works.

2. When attaching a large-but-not-large-enough-to-crash attachment (a 779,5 MiB file), it gets accepted but akonadi then completely chokes and uses 20GB+ RAM spread over its agents until kernel OOM killer nukes it. The message is stuck in outbox, any access to local folders will trigger message processing again leading to the 20GB+ RAM usage again.

STEPS TO REPRODUCE
1. Attach a 779,5 MiB attachment to a mail in the composer.
2. Attempt to send the mail.

OBSERVED RESULT

The mail gets stuck in the local folders outbox and OOM will quickly follow. (Unless you have enough RAM, perhaps 100GB or so?) Every access of local folders will cause akonadi to re-read this mail, because it's stuck in the outbox, triggering the issue again and again.

EXPECTED RESULT

One of:
a. It works without crashing or consuming a ridiculous amount of memory.
b. It refuses to attach ridiculously oversized attachments.

WORKAROUND

If a mail with an extremely large attachment is stuck in the local folders, use "ncdu" or similar tool to locate the mail file on disk. For example "ncdu ~/.local/share/akonadi_maildir_resource_0".

Then, stop akonadi, delete the problematic file and start akonadi again.

RECOMMENDED FIX

Set the composer setting "Maximum Attachment Size" somewhere between 50MB and 100MB or so by default. Then also either disallow unlimited attachment size completely or add a gigantic warning regarding the memory usage if set to unlimited.

SOFTWARE/OS VERSIONS
Kontact 5.9.3 (18.08.3, Debian Buster/Stable)
Operating System: Debian GNU/Linux 10
KDE Plasma Version: 5.14.5
Qt Version: 5.11.3
KDE Frameworks Version: 5.54.0
Kernel Version: 4.19.0-6-amd64
OS Type: 64-bit

ADDITIONAL INFORMATION

See the attachment for htop output just before system unresponsiveness due to swapping and OOM killer invocation.

Personal note: Although it may seem silly trying to try and attach a gigantic file to an email, I've seen this bug in production. Dragging & dropping a large LibreOffice presentation from Dolphin into Kontact is easily done, and the user's Kontact/akonadi is completely crippled afterwards if the file size happens to be in the "sweet spot".