Bug 285809

Summary: Kmail dead slow while stuck synchronizing two folders in KMail maildirs
Product: [Applications] kmail2 Reporter: Richard Van Den Boom <rvdb>
Component: foldersAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED UNMAINTAINED    
Severity: normal CC: amantia, aspotashev, holy, julien, kde2, kde, markotahal, Martin, rvdb
Priority: NOR    
Version: 4.7   
Target Milestone: ---   
Platform: Slackware   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Richard Van Den Boom 2011-11-05 11:37:56 UTC
Version:           4.7 (using KDE 4.7.3) 
OS:                Linux

I just upgraded my slackware system with the latest packages of KDE 4.7.3 and akonadi 1.6.2 from Alien Bob, which are usually very reliable.
KMail and Kontact worked roughly well with the 4.7.2 packages, provided I disabled Nepomuk which brought Kontact to a crawl.
After the upgrade, Kontact is stuck when starting at 0% synchronizing 2 subdirs, one with about 7000 mails and the other with about 3500, both being the target to new mails for two different accounts. The two folders on which Kontact gets stuck are always the same, called "Nouveaux" which means "New" in french.

I created initially three maildirs akonadi ressources, as Kmail maildirs to be able to choose their name, as I manage several different POP account and I like to keep my mails from different accounts if different places. I also created a fourth KMail maildir but for some reason it never showed up in Kmail, I'll file another bug report when I'll have time.
 
The issue is that I cannot see any mail anymore as long as the two folders are not synchronized. I can read new mails and the ones in small folders, that probably were synchronized before Kontact gets to my two "Nouveaux" maildirs.

I can still use Kontact, opening menus and config.
Curiously, Kmail in the same situation does not show the maildir being synchronized but is still very slow to display some mails. I get the following errors in the console :


"/usr/bin/kmail(5244)" Soprano: "QLocalSocket::connectToServer: Invalid name"
"/usr/bin/kmail(5244)" Soprano: "org.freedesktop.DBus.Error.ServiceUnknown - The name org.kde.nepomuk.services.nepomukstorage was not provided by any .service files"
"/usr/bin/kmail(5244)" Soprano: "Unsupported operation (2)": "Invalid model"
"/usr/bin/kmail(5244)" Soprano: "org.freedesktop.DBus.Error.ServiceUnknown - The name org.kde.nepomuk.services.nepomukstorage was not provided by any .service files"
"/usr/bin/kmail(5244)" Soprano: "Unsupported operation (2)": "Invalid model"
"/usr/bin/kmail(5244)" Soprano: "Unsupported operation (2)": "Invalid model"
"/usr/bin/kmail(5244)" Soprano: "Invalid iterator."
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 
kmail2(5244): Error while fetching items.  103 "Erreur inconnue. (Impossible de lister la collection racine.)" 



Reproducible: Always

Steps to Reproduce:
This is very much related to my settings, I suppose.
I don't know how you could reproduce this, probably create a Kmail maildir from a maildir with a lot of mails, under KDE 4.7.2. Then upgrade.
I still use Soprano 4.7.2

Actual Results:  
When I click on a mail, I get the soprano errors and the Fetching mail screen. Then suddenly, I get the "Error when fetching item" message repeated several times and then the mail content is displayed. This occurs about 10 minutes after clicking on the mail. After that, I seem not to be able to view any other mail from that folder.

Expected Results:  
Viewing my mail instantly.
Comment 1 András Manţia 2012-02-11 23:31:24 UTC
Are you using mysql or sqlite backend (check if a mysqld process is running or not)?

What is you folder type? Run akonadiconsole, on the Agents tab click on the account and see the Type in the Detail field below.
Comment 2 Tim Holy 2012-04-26 23:51:44 UTC
I'm having this problem too, under 4.8.2 (Kubuntu 12.04). 

Details:
My inbox has about 25000 messages in it.

This process is running:
usr/sbin/mysqld --defaults-file=/home/tim/.local/share/akonadi/mysql.conf --datadir=/home/tim/.local/share/akonadi/db_data/ --socket=/home/tim/.local/share/akonadi/socket-diva/mysql.socket

Here are the agents under akonadiconsole:
akonadi_pop3_resource_0 (POP3 E-Mail Server)
akonadi_akonotes_resource_0 (AkoNotes)
akonadi_pop3_resource_1 (POP3 E-Mail Server)
akonadi_maildispatcher_agent (Mail Dispatcher Agent)
akonadi_ical_resource_0 (ICal Calendar File)
akonadi_maildir_resource_1 (Maildir)
akonadi_mailfilter_agent (Mail Filter Agent)
akonadi_nepomuk_feeder (Akonadi Nepomuk Feeder)
akonadi_contacts_resource_0 (Personal Contacts)
Comment 3 Mark 2013-03-24 23:22:41 UTC
Guys, not sure if related, but for the record. I always had kmail slow reading IMAP folder list (many mails ~20k), even on a decent machine (i7,16GB ram, ssd). Today we switched on Archlinux to MariaDB MySQL implementation, while doing that, I read config file 

/etc/mysql/my.cnf 
allow InnoDB and change line to
innodb_buffer_pool_size = 2G
..to any reasonable value from too small buffer (16M), it works Wonders now! 

(raise related log files sizes too, few lines below)

kmail and whole KDE works blazingly swift now. 

merry Eastern! :)
Comment 4 julien 2013-07-11 07:44:43 UTC
Hi,
Same problem here.
Kmail 1.13.7 with Debian Wheezy.

Everything was working fine until 2 days ago, Kmail starts but hangs on the "reception" of new messages (Progress window on right down corner shows "Synchronising" for two email accounts, without progression).

I can have access to my local emails, and my two offline-IMAP accounts, but I can't fetch new messages.

What information should I provide ?
Comment 5 András Manţia 2013-07-11 08:17:09 UTC
Julien, the report is for KMail2, completely different architecture. KMail 1.x is unmaintained, so I dount there will be any fixes there.
Comment 6 Diggory Hardy 2014-10-20 08:28:24 UTC
@Marek, thanks for the advice, increasing the MySQL buffer size helps tremendously!

@kmail/akonadi team: please get this fixed by default!
Comment 7 Martin Steigerwald 2015-09-09 20:59:15 UTC
Thank you Richard for the report and all the commenters for the comments. It is about a version of KMail which uses Nepomuk and is unmaintained. Thus closing. If you still see performance issues please open new reports. But please follow the following guide lines to avoid unnecessary work for the developers:

- Ideally test with KDEPIM and Akonadi 15.08. It contains some performance improvements like the binary protocol.

- Otherwise at least use KDEPIM 4.14.10 and newest Akonadi 1.13 you can get as it already contains some performance improvements.

- If you can wait, please retest with KDEPIM and Akonadi 15.12 once they become available for you. Akonadi 15.12 will contain *massive* performance improvements implemented by Dan due to new database indexes, optimized queries and leveled file_db directory. All of these are in master already, so if you dare use kdesrc-build to compile KF5, kdepim and kdepim-runtime. I am using this currently and it basically moves the bottleneck to KMail (slow threading for example). It is a *huge* improvement.

Thank you and greetings from KDE Randa Meetings,
Martin
Comment 8 Janek Bevendorff 2015-09-24 10:27:18 UTC
I experience similar issues when syncing large folders for the first time using KMail 5.0.1 and Akonadi 15.08.1.
KMail just stops loading folders while showing "Syning folder…" without any visible progress. Looks like some kind of database deadlock. The only way to revive KMail is to run akonadictl restart.

Should I reopen this issue or file a new report for Akonadi?