Bug 384359

Summary: Regression in 5.6.0 / 17.08 compared to 17.04.3: MySQL load always 100%, constant queries, extremely sluggisch behaviour
Product: [Frameworks and Libraries] Akonadi Reporter: Gunter Ohrner <kdebugs>
Component: IMAP resourceAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED WORKSFORME    
Severity: normal CC: danielroschka, egdfree, kdepim-bugs, matish, ogoffart
Priority: NOR    
Version: 5.6.0   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: Screenshot Akonadiconsole Query Sequence Debugger
akonadiconsole showing the queries to the db

Description Gunter Ohrner 2017-09-04 17:07:41 UTC
Since todays update, which pulled in KDE PIM 17.08, kMail really became unusable.

I'm using a normal IMAP resource with a locally running Dovecot IMAP server process.

Akonadi permanently executes quries as the following:

SELECT count(DISTINCT PimItemTable.id), sum(PimItemTable.size), sum(CASE WHEN ( SeenFlags.Flag_id IS NOT NULL OR IgnoredFlags.Flag_id IS NOT NULL ) THEN 1 ELSE 0 END) FROM PimItemTable LEFT JOIN PimItemFlagRelation AS SeenFlags ON ( PimItemTable.id = SeenFlags.PimItem_id AND SeenFlags.Flag_id = ? ) LEFT JOIN PimItemFlagRelation AS IgnoredFlags ON ( PimItemTable.id = IgnoredFlags.PimItem_id AND IgnoredFlags.Flag_id = ? ) INNER JOIN CollectionPimItemRelation ON ( CollectionPimItemRelation.PimItem_id = PimItemTable.id ) WHERE ( CollectionPimItemRelation.Collection_id = ? )

constantly loads the disk (Samsung 840 EVO SSD), lets the mysqld process eat 100% CPU (with the corresponding power consumption and fan noises...) and reacts very, very slowly in general. (E.g. moving messages takes several 10 minutes, opening folders often also does.)

kMail/Aknoadi already had severe performance problems before, which made working with it a trial of patience (See bug 338571, bug 352604), but this is an all-new quality now... (And also a new bug, as contrary to the other ones the background activity is not directly visible/represented in the UI - except by it not being responsive.)
Comment 1 Gunter Ohrner 2017-09-05 16:59:09 UTC
Update: Akonadi is really keeping mysqld at 100%, all the time, all day, independently of if I quit kmail or not. I need to stop akonadiserver (akonadictl stop) to make it stop after a while.

So maybe this needs to be reassigned to akonadi? It renders all akonadi using software more or less unusable, including kaddressbook...
Comment 2 Gunter Ohrner 2017-09-07 12:06:53 UTC
Created attachment 107732 [details]
Screenshot Akonadiconsole Query Sequence Debugger

The query debugger query sequence view "nicely" illustrates the problem:
The second query is very slow with 650 ms execution time per call, and uses most of the processing time.

The sum shown as "TOTAL" basically resembles wall clock time, as Akonadi loads the CPU 100% (actually mysqld).

This never seems to finish, at least it ran for lots of hours several times, before I finally terminated it using "akonadictl stop".
Comment 3 Gunter Ohrner 2017-09-07 12:08:21 UTC
Not quite sure if really IMAP ressource is the culprit, but I think so.

How can I find out who or what is triggering all those (slow) queries?
Comment 4 Gunter Ohrner 2017-09-24 17:25:13 UTC
For whatever reason, the query storm stopped on my system.

Now kMail is still as slow as ever, but more or less usable again.

Meanwhile I also upgraded to 5.6.1 - no idea if it's related to this improvement or if both are independent events.
Comment 5 Gunter Ohrner 2017-09-24 17:26:12 UTC
Is anyone else seeing this problem? Otherwise I'll probably close the bug.

Or does one of the developers fear / see a problem here and want to keep this issue open to have a reminder for having a look at it?
Comment 6 matish 2017-10-06 21:53:07 UTC
I get mysqld, kontact, akonadiserver, korgac, and akonadi_ical_resource that run like crazy and whose memory footprint grows indefinitely.

kmail seems to be fine, the madness starts when I run korganizer, or open the calendar within kontact.

By looking at akonadiconsole the db is queried pretty much continuously, I'm attaching a screen shot after a few seconds. I am running 17.04.2, Kde Frameworks 5.32, qt 5.6.2, on OpenSUSE 42.3.

Let me know if there's anything I can do to get more info. Thanks.
Comment 7 matish 2017-10-06 21:53:51 UTC
Created attachment 108214 [details]
akonadiconsole showing the queries to the db
Comment 8 matish 2017-10-07 10:56:06 UTC
ok, I found out that my problem is with google calendars. I had a google calendar displayed in korganizer and korgc was using 3.5 GB of memory, and running queries like there was no tomorrow. I removed the calendar and it seems to be ok. So maybe these are two different issues... not sure.
Comment 9 matish 2017-10-07 11:03:48 UTC
apologies for the many comments... I made a mistake, it was not a google calendar but a remote ics file, imported from office365.
Comment 10 Daniel Roschka 2018-01-01 07:34:55 UTC
I got the exactly the same problem after updating to KDE PIM 17.08 and haven't been able to solve it yet.
Comment 11 Daniel Roschka 2018-01-05 18:14:32 UTC
I might have figured out what caused the issue.

The search folder in KMail showed 100000 results and when I opened the search window it showed the search folder itself as destination for the search. I then did a search in a different folder and the problem immediately went away.

So my guess is that it's somehow an odd default which results in a recursive search, as the search searches in the search results.

I haven't been able to reproduce it once it was gone, but luckily the problem is gone for me now.
Comment 12 Justin Zobel 2022-11-10 22:33:14 UTC
Thank you for reporting this issue in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the issue with a recent software version?

If you can reproduce the issue, please change the status to "REPORTED" when replying. Thank you!
Comment 13 Bug Janitor Service 2022-11-25 05:17:18 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 14 Bug Janitor Service 2022-12-10 05:14:27 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!