Bug 277770 - KMail2 is slow when accessing messages in folder
Summary: KMail2 is slow when accessing messages in folder
Status: RESOLVED UNMAINTAINED
Alias: None
Product: kmail2
Classification: Applications
Component: message list (show other bugs)
Version: 4.9.0
Platform: Arch Linux Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-07-14 16:45 UTC by Michał
Modified: 2015-09-10 10:55 UTC (History)
14 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michał 2011-07-14 16:45:40 UTC
Version:           2.1.1 (using Devel) 
OS:                Linux

KMail2 is slow when accessing folders. When I click my inbox folder it takes about 3..5 seconds before contents show up. This isn't cached, so every time I click it - it takes several seconds. There are about 7000 mails in my main folder.
It worked fast with previous KMail, at least that fast that I never actually caught my eye. Now it's irritating.


Reproducible: Always

Steps to Reproduce:
Run KMail2. Get a folder with about 7000 messages. Click another folder, then click the first one.

Actual Results:  
There's considerable delay before messages actually show up.

Expected Results:  
Messages show instantly.
Comment 1 Jonathan 2011-08-15 16:06:07 UTC
*** This bug has been confirmed by popular vote. ***
Comment 2 Jonathan 2011-08-15 16:53:34 UTC
I have the same problem, but in my case it seems that the data is cached. Only the first access takes a very long time.

My biggest folder with more than 3000 mails (467 MB) takes 20 seconds to open on the first access, consecutive accesses to this folder are instant.

The system load is used by the process akonadi_mixedmaildir_resource and a lot of io-wait.
Comment 3 Martin Bednar 2011-09-11 16:14:27 UTC
I also have this issue, but have minimalized its impact. Go to Kmail appearance settings, select messagelist. configure aggregation, go to the advanced tab, change settings there, see which works best.
Comment 4 Joachim 2012-01-19 13:54:20 UTC
Tried several combinations of the minimalize impact hints. Nothing helped so far.

Restarting KMail sometimes helps. 

I have openSuse 12.1's kdepim 4.7.2, so not the newest - but since the bug is still new (since last september? Not important??) I don't see the point of going to self compile/update hell.

I use kmail since ages (since it is part of suse distro) and am a big fan, but like this, it is unusable :( 

I am aware that by the information given so far there is not much one can do, so if I can help by providing more details (like logs), I will. Just let me know which/how (I can use sh).
Comment 5 Joachim 2012-01-19 14:33:53 UTC
some generic infos:
- wait time differs from seconds to hours/forever.
- system is a lenovo 510i (i5, 8GB RAM, SSD HD)
- system is idle (cpu, hdd, net) during the wait
- 4 mailboxes (several 1000 mails): 2x google imap, 2x unknown imap, 1x local  
- mailboxes are not migrated but created from scratch after migration failed

If I start kmail from the konsole, I see lots of messages like these:

"/usr/bin/kmail(8490)" Soprano: "Invalid iterator."
"/usr/bin/kmail(8490)" Soprano: "org.freedesktop.DBus.Error.ServiceUnknown - The name org.kde.nepomuk.services.nepomukstorage was not provided by any .service files"
"/usr/bin/kmail(8490)" Soprano: "QLocalSocket::connectToServer: Invalid name"
"/usr/bin/kmail(8490)" Soprano: "org.freedesktop.DBus.Error.ServiceUnknown - The name org.kde.nepomuk.services.nepomukstorage was not provided by any .service files"
"/usr/bin/kmail(8490)" Soprano: "Unsupported operation (2)": "Invalid model"
"/usr/bin/kmail(8490)" Soprano: "org.freedesktop.DBus.Error.ServiceUnknown - The name org.kde.nepomuk.services.nepomukstorage was not provided by any .service files"
Comment 6 Mauricio Vergara Ereche 2012-01-24 18:57:10 UTC
I confirm the same behaviour with Fedora 16 and KDE 4.7.4

I have the same issue with account migration or with a fresh new started over configuration.
At least for me, switching from mysql to sqlite in akonadi helped a little bit, but not too much to notice.
Comment 7 Joachim 2012-02-02 14:27:06 UTC
(In reply to comment #5)

I fixed the soprano errors by allowing nepomuk to use much more ram.
But KMail is still not usable. Selected mails do not show content but "please wait" forever. Also replying to a mail results in endless wait messages.

Sorry, but I can't wait that long for some reaction. I fixed the problem by upgrading to Thunderbird 9.0 after over a decade of using KMail. 

Access to thousands of mails in multiple accounts is instant again, even during retrieval of new mails. Signing, encrypting, offline caching of selected subfolders: everything works now and was only 1/2 hour of migration effort.

:(
Comment 8 Peter Lewis 2012-02-02 15:08:56 UTC
This has definitely improved a lot in KDE 4.8, so thanks for that :-)
Comment 9 Joachim 2012-02-02 16:37:49 UTC
Encouraged by comment #8 i followed this thread (post #2 and #7)

http://forums.opensuse.org/english/get-technical-help-here/applications/471887-proper-way-upgrade-opensuse-12-1-kde-4-7-kde-4-8-release.html

Still no joy, but after recreating the akonadi resources it started working again.
I recreated these resources multiple times with 4.7 and it did not help, so this is progress!

Back into kmail business :)
Comment 10 Alexandre Bonneau 2012-02-02 23:25:21 UTC
Joachim, can you confirm that upgrading to 4.8, and recreating maildir resources in akonadi made kmail fast again ? Even after a reboot ?
Comment 11 Joachim 2012-02-03 10:22:57 UTC
(In reply to comment #10)

Yes I can confirm this.
I rebooted right after upgrading and again this morning. Still working fine.

I just wouldn't say "fast again", because it wasn't slow.
After selecting some specific mails it completely stopped working and now it works again.

This is the exact version I have now:

Name        : kdepim4
Version     : 4.8.0
Release     : 325.4
Architecture: x86_64
Install Date: Do 02 Feb 2012 17:04:26 CET
Group       : System/GUI/KDE
Size        : 17324700
License     : GPL-2.0+ ; LGPL-2.1+
Signature   : DSA/SHA1, So 29 Jan 2012 01:17:04 CET, Key ID 27c070176f88bb2f
Source RPM  : kdepim4-4.8.0-325.4.src.rpm
Build Date  : So 29 Jan 2012 01:08:06 CET
Build Host  : build15
Relocations : (not relocatable)
Vendor      : obs://build.opensuse.org/KDE
URL         : http://www.kde.org
Summary     : Base package of kdepim
Description :
This package contains the core files of the kdepim module.
Distribution: KDE:Release:48 / openSUSE_12.1

And these are the repositories I added to get it:
download.opensuse.org/repositories/KDE:/Extra/KDE_Release_48_openSUSE_12.1
download.opensuse.org/repositories/KDE:/Release:/48/openSUSE_12.1         

HTH
Comment 12 Alexandre Bonneau 2012-02-06 13:41:14 UTC
I upgraded to 4.8 following comment #11 (even though I'm using kubuntu).
Could you describe exactly how you recreated the akonadi resources Joachim ?

I guess you have to go in the akonadi tray > Configure, then delete the imap and maildir ressources ? In doing so, wouldn't you lose all your tags (important) on your email ?

Rigth now, without having recreated the ressources, kde 4.8 kmail is a bit faster, but still unusable in the long run.
Comment 13 Joachim 2012-02-06 14:32:02 UTC
Sorry, not sure about the exact english terms, but I used kickoff menu System Settings/General Appearance and Behaviour/Personal Information/Setup of Akonadi Resources to delete the IMAP resource and the Kontact menu Settings/KMail Setup/Accounts to recreate it.

I don't use tags much, so I didn't notice, but what I know of IMAP (not much) it depends on the server which attributes are stored local (you would probably loose them) or on the IMAP server (should be preserved).
You could check, by setting up your account in another mailer. Attributes you can see there will be preserved by the recreation.

But from my point of view anything was better than what I had. After being slow in the beginning it didn't work at all later on, so tags weren't useful anyways.
Comment 14 Alexandre Bonneau 2012-02-10 22:45:52 UTC
Joachim, you say you recreated your akonadi IMAP ressource and accounts from kmail.
Are you saying that this action also boosted the 'maildir' synchronisation hell too ? (because for me when I take a look in akonadiconsole, akonadi never stops to synchronise my local maildir folders, not the IMAP ones)
Comment 15 Alexandre Bonneau 2012-02-10 23:28:34 UTC
It seems disabling in "Configure kmail > Accounts > Maildir" the 'include in manual mail check' and 'check mail on startup' options speeds kmail like never before !

Not sure though if 'not checking local folders' was activated before akonadi existed to achieve the same speed..
Comment 16 Joachim 2012-02-11 00:59:38 UTC
(In reply to comment #14)

Didn't notice a problem with local maildir.
But I completely stopped nepomuk services because virtuoso permanently occupied my cpu at 90+% (on nice level, ok, but battery drained and noise terror anyways).
Comment 17 Alexandre Bonneau 2012-03-13 00:17:37 UTC
Ok, the bug is back with version 4.8.1. T_T
The 'workaround' I mentioned earlier isn't working anymore.

I guess this very annoying bug is _very_ simple to fix : priorization.
Just tell kmail2 to fetch the selected mail BEFORE finishing scanning my 20k+ maildir folder for the 5th time in on hour.

Although it's not constructive, I understand why some kmail2 users' comments are full of anger !
Comment 18 Joachim 2012-03-13 16:16:57 UTC
Some update from my system:

I am on KDE: 4.8.00 (4.8.0 "release 462" now.
I followed this advice: http://lists.kde.org/?l=kdepim-users&m=132216525522932&w=2
-> removed ~/.kde4/share/apps/nepomuk
Before that I could not activate desktop search (cpu load was 100% for 2 of 4 cores)
Now I can activate it (cpu load ~10%, disk ~1MB/s) _and_ kmail works much better.
I.e. while no mails are fetched I usually get a preview in less than a second.
Concurrency is still bad but also better now: While fetching mail and read at the same time preview typically within 5s.

BUT: Thunderbird at the same time (i.e. same load, also fetching new mails in background) can show mail previews instantly -> room for improvement.
Comment 19 Alexandre Bonneau 2012-04-12 10:29:18 UTC
For those not having this slow retrieval problem, when you right click on your maildir "Local folder" and select "Folder properties", what are your synchronization settings (for the root folder and the sub-folders) ?

When selecting "Synchronize when selecting this folder", when entering a big mail folder containing from 3000 to 20.000 mails, kmail2 synchronize it before showing the content of the selected mail, resulting in a 5 seconds to 3-4 minutes delay.

When deselecting "Synchronize when selecting this folder" :
- If I specify the delay to "Never", Kmail2 still synchronize the (local) folder in which you try to view a mail...resulting in annoyingly long waiting time.
- If I specify a XXX delay between each automated synchronization (option "automatically synchronize after"), it's even worse because it'll force Kmail2 to resynchronize _all_ the sub-folders as they all have the "same configuration as parent" option checked, _each XXX minutes_ !
Moreover, as soon as you select that option, you're good to go for a total resynchronization of your mails. Not good.
Comment 20 Alexandre Bonneau 2012-04-17 15:06:11 UTC
I don't know exactly what I did, but after fiddling with the folder options I talked about in the previous comment, the slowness has disappeared (again).

It now takes between instant and ~2 seconds to display any mail content, even in the big folders.

Perhaps a configuration file hasn't been correctly written during update ? My options are the same as before, but without any problem now.

I hope that answer could help people experiencing this kind of bug.
Comment 21 Thomas Arend 2012-08-28 20:05:10 UTC
Can confirm this for 4.9.

It takes 20 to 30 seconds and more to display a simple plain text message in the preview window.
Comment 22 Allan Sandfeld 2013-02-03 10:59:34 UTC
I just tried KMail2 from KDE 4.10, it takes 5-10 minutes to load a folder after clicking on it. This is independant of whether the folder has has 40 or 30000 emails. Though it may depend on the total number of emails I imported from KMail1 (around 200.000).
Comment 23 Alexandre Bonneau 2013-02-04 10:09:14 UTC
On my end, I can confirm that this bug doesn't affect Kmail (4.9.5) anymore.
On my main local maildir folder, I use the following configuration :
- synchronize when selecting this folder
- automatically synchronize : never
- receive message body on demand
- keep message body locally for 1 minute

Oddly, all my mails folder shows a 'last indexing date' the 2012-11-26, although Nepomuk mail indexation is checked..

And finally, to be completely honest, selecting a folder doesn't display the mail instantly (as it should imho), but with a small 0,5 second delay (thanks to my SSD perhaps ?).
Comment 24 Emre 2013-06-09 18:10:33 UTC
I can confirm this. I have a mailbox with 10+ thousand mails inside. It took some time to open the mailbox when the account is first setup, but when this is done, I got stuck into this bug.

When I select a message (ie 10 meg message) to display contents, the preview pane stays displaying "Retrieving Folder Contents... Please wait". if you wait for like 1-4 minutes, the message displays and consequent displays of this message is now fast. 

Note that this is an IMAP (Dovecot) server installed locally. It's super fast in Thunderbird. So no idea why it takes minutes to open a message.
Comment 25 Martin Steigerwald 2015-09-10 10:55:12 UTC
Thank you  Michał for your report and commenters. It is about a version of KMail which uses Nepomuk and is unmaintained. Thus closing. If you still see performance issues please open new reports. But please follow the following guide lines to avoid unnecessary work for the developers:

- Ideally test with KDEPIM and Akonadi 15.08. It contains some performance
improvements like the binary protocol.

- Otherwise at least use KDEPIM 4.14.10 and newest Akonadi 1.13 you can get as
it already contains some performance improvements.

- If you can wait, please retest with KDEPIM and Akonadi 15.12 once they become
available for you. Akonadi 15.12 will contain *massive* performance
improvements implemented by Dan due to new database indexes, optimized queries
and leveled file_db directory. All of these are in master already, so if you
dare use kdesrc-build to compile KF5, kdepim and kdepim-runtime. I am using
this currently and it basically moves the bottleneck to KMail (slow threading
for example). It is a *huge* improvement. Also Volker and others work on
performance improvements in KMail as well.

- Also always just report one issue per report. If you have several different issues open a report for each issue.

- Only open new reports after having researched whether someone reported the issue already.

Thank you and greetings from KDE Randa Meetings,
Martin