| Summary: | Exponential growth of CollectionFetchJobs (infinite loop) | ||
|---|---|---|---|
| Product: | [Applications] kmail2 | Reporter: | Thiago Macieira <thiago> |
| Component: | folders | Assignee: | kdepim bugs <kdepim-bugs> |
| Status: | RESOLVED FIXED | ||
| Severity: | grave | CC: | anmeldungen, bugs, cfeck, dvratil, fabian, greve, kerub, laurent.rineau, martin.ruessler, Martin, mollekopf, rdieter, vito.detullio |
| Priority: | NOR | ||
| Version: | Git (master) | ||
| Target Milestone: | --- | ||
| Platform: | Compiled Sources | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | 4.14.6 | |
| Sentry Crash Report: | |||
| Attachments: |
Kmail, freshly started up at 10:44
Kmail, after idling for 6 hours |
||
Hmm... note how there's a listing for *both* a localised and a non-localised name of the outbox: * 6 4 (NAME "caixa de saída" MIMETYPE (message/rfc822 inode/directory) REMOTEID "caixa de saída" REMOTEREVISION "0" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) SpecialCollectionAttribute "outbox" ENTITYDISPLAY "(\"caixa de saída\" \"mail-folder-outbox\" \"\" ())") * 5 4 (NAME "outbox" MIMETYPE (message/rfc822 inode/directory) REMOTEID "outbox" REMOTEREVISION "0" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) SpecialCollectionAttribute "outbox" ENTITYDISPLAY "(\"caixa de saída\" \"mail-folder-outbox\" \"\" ())") Both folders exist on disk. There is nothing in that "outbox" folder, though. It shouldn't exist, but does. After removing the unwanted folder from disk (rm -rf) and restarting the maildir agent, the problem seems to go away. +1 After upgrading to KDE 4.11 (in Arch Linux) Kmail (or Kontact) stops displaying email content very soon after starting. With akonadiconsole I discovered that Kmail/Kontact drops at least two (or up to 7) Akonadi::CollectionFetchJob per timestamp and they keep on waiting for minutes. As far as I can tell local folders are set up correctly. Same problem here: after 4.11 (on slackware-current + alienBob kde 4.11) reading email content needs 5+ minutes... Confirm with: Kubuntu 13.04 KDE 4.11.00 Akonadi Console Version 0.99 I can confirm this (Kubuntu 13.10, KDE 4.11.2). In the akonadi console I could see up to 20 new jobs every second. Right before I shut down akonadi, kmail was consuming more than 1GB of RAM. Seeing this on two folders on Fedora, installed KDE Red Hat repos: kdepim-4.11.3-1.fc19.x86_64 kdepim-debuginfo-4.11.3-1.fc19.x86_64 kdepim-libs-4.11.3-1.fc19.x86_64 kdepim-runtime-4.11.3-1.fc19.x86_64 kdepim-runtime-debuginfo-4.11.3-1.fc19.x86_64 kdepim-runtime-libs-4.11.3-1.fc19.x86_64 kdepimlibs-4.11.3-2.fc19.x86_64 kdepimlibs-akonadi-4.11.3-2.fc19.x86_64 kdepimlibs-debuginfo-4.11.3-2.fc19.x86_64 kdepimlibs-kxmlrpcclient-4.11.3-2.fc19.x86_64 Let me know if there is anything I can do to provide debug info. Georg, can you check in Akonadi Console in Browser tab for duplicated folders in "Local Folders"? If there are any duplicates, remove them. The duplicates can be like "Inbox" and "whatever inbox is in your language" I've looked into akonadi console, but there are no duplicate folders, also the folder affected is not inbox, it's a subfolder of inbox, and subfolder of that subfolder, so something folder "c" in "j" in the following structure: inbox/ ├── a ├── b ├── c │ ├── h │ ├── i │ ├── j │ ├── k │ ├── l │ ├── m │ ├── n │ ├── o │ └── p ├── d ├── e ├── f └── g FWIW, some IMAP folders are marked red in akonadiconsole, including (but not only) the folders affected. Christian Mollekopf has some debug output, although he cannot share it (contains data from my system) but he might be able to look for something suspicious. If there is anything I can do to help track this down, please shout. Created attachment 83599 [details]
Kmail, freshly started up at 10:44
Created attachment 83600 [details]
Kmail, after idling for 6 hours
Just to get an idea of the memory consumption/leakage when doing absolutely nothing.
Note this keeps one core of the CPU at 100% (four core system, hence 25%) and since it is a laptop it keeps it from scaling down, ever, which means it burns through battery like nothing and the laptop is becoming quite warm, including all the side effects that brings.
FWIW, another data point: Switching Kmail to offline and visiting the affected folder can stop the CPU churn. When i then keep it open in one tab and go back online, the CPU churn seems gone for the moment. (In reply to comment #7) > Georg, can you check in Akonadi Console in Browser tab for duplicated > folders in "Local Folders"? If there are any duplicates, remove them. The > duplicates can be like "Inbox" and "whatever inbox is in your language" For me this seems to have solved the problem. I had two local trash folders. So far akonadi has stopped spamming processes. (In reply to comment #11) > FWIW, another data point: > > Switching Kmail to offline and visiting the affected folder can stop the CPU > churn. > > When i then keep it open in one tab and go back online, the CPU churn seems > gone for the moment. This problem is unrelated and handled separately in https://bugs.kde.org/show_bug.cgi?id=327865. (In reply to comment #13) > > This problem is unrelated and handled separately in > https://bugs.kde.org/show_bug.cgi?id=327865. I meant to say; Georg's problem is unrelated, so you can ignore his debug information for this issue =) I had a trash and a wastebin folder. I then let Akonadi replace the local folders with a fresh set and now kmail works again. I am using KDE 4.11.3. Thiago, thank you for reporting the bugs and for all your comments and research. Does this still happen with KDEPIM 4.14.6/7? (In reply to Martin Steigerwald from comment #16) > Thiago, thank you for reporting the bugs and for all your comments and > research. > > Does this still happen with KDEPIM 4.14.6/7? I cannot reproduce this anymore. |
I don't know what caused this. Downgrading to a previous build of akonadi + kdepim* did not help. This problem happens now with the build from April 11 as well as the latest sources (as of right now). Whenever Kontact / KMail is running, it is constantly requesting a full folder listing, for whatever reason. Here's what I can tell from debugging and from Akonadi Console: Job tracker shows that KMail is creating *two* Akonadi::CollectionFetchJobs at a time, two or three times per second. Since each job takes about a quarter second to complete running, there's a build up of jobs. That is, every second, KMail is requesting 1.25 to 1.5 second of work from the Akonadi Server. This does not stop. The Debugger in Akonadi Console shows an endless stream of: [red] 24 LSUB 0 INF (MIMETYPE (message/rfc822)) () [blue] * 420 419 (NAME "Common Folders" MIMETYPE (message/rfc822 inode/directory) REMOTEID ".Common Folders" REMOTEREVISION "" RESOURCE "akonadi_imap_resource_3" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 60 SYNCONDEMAND true LOCALPARTS (ENVELOPE HEAD)) AccessRights "wcdCD" imapacl "anyone lrswipcda %% " collectionannotations "/vendor/cmu/cyrus-imapd/partition default" timestamp "1375301134") [a lot more blue] [blue] 24 OK List completed In stderr, kontact is printing these two messages over and over again: kontact(8751)/libakonadi Akonadi::SpecialCollectionsPrivate::emitChanged: Emitting changed for "akonadi_maildir_resource_0" kontact(8751)/libakonadi Akonadi::SpecialCollectionsPrivate::emitChanged: Emitting defaultFoldersChanged. (see below for the lines in LSUB that contained "akonadi_maildir_resource_0") The backtrace of that function is: #0 Akonadi::SpecialCollectionsPrivate::emitChanged (this=0xda54f0, resourceId=...) at /home/thiago/src/kde4/kde/kdepimlibs/akonadi/specialcollections.cpp:76 #1 0x00007ffff686f5ce in Akonadi::SpecialCollections::registerCollection (this=0xddf780, type=..., collection=...) at /home/thiago/src/kde4/kde/kdepimlibs/akonadi/specialcollections.cpp:266 #2 0x00007ffff6880046 in Akonadi::SpecialCollectionsDiscoveryJob::slotResult (this=0x9c7190, job=<optimized out>) at /home/thiago/src/kde4/kde/kdepimlibs/akonadi/specialcollectionsdiscoveryjob.cpp:68 #3 0x00007ffff6c7bb2a in QMetaObject::activate (sender=0x16bf8f0, m=0x7ffff711f460, local_signal_index=3, argv= 0x7fffffffc8d0) at /home/thiago/src/qt/qt-4.8/src/corelib/kernel/qobject.cpp:3547 #4 0x00007ffff6f62442 in KJob::result (this=this@entry=0x16bf8f0, _t1=_t1@entry=0x16bf8f0) at /home/thiago/obj/kde4/kdelibs/kdecore/kjob.moc:207 #5 0x00007ffff6f62480 in KJob::emitResult (this=0x16bf8f0) at /home/thiago/src/kde4/kdelibs/kdecore/jobs/kjob.cpp:318 I should note that mDefaultResourceId is "akonadi_maildir_resource_0" (a match) in frame 0. The next CollectionFetchJob is created from: #0 Akonadi::CollectionFetchJob::CollectionFetchJob (this=0x18c85d0, collection=..., type= Akonadi::CollectionFetchJob::Recursive, parent=0x1951090) at /home/thiago/src/kde4/kde/kdepimlibs/akonadi/collectionfetchjob.cpp:103 #1 0x00007ffff687fbce in Akonadi::SpecialCollectionsDiscoveryJob::start (this=0x1951090) at /home/thiago/src/kde4/kde/kdepimlibs/akonadi/specialcollectionsdiscoveryjob.cpp:54 #2 0x00007fffa8122ead in MailCommon::Kernel::initFolders (this=0xda3bb0) at /home/thiago/src/kde4/kde/kdepim/mailcommon/kernel/mailkernel.cpp:155 #3 0x00007ffff6c7bb2a in QMetaObject::activate (sender=0xdb29c0, m=0x7ffff69581a0, local_signal_index=1, argv=0x0) at /home/thiago/src/qt/qt-4.8/src/corelib/kernel/qobject.cpp:3547 #4 0x00007ffff686e188 in Akonadi::SpecialCollectionsPrivate::emitChanged (this=0xdb9250, resourceId=...) at /home/thiago/src/kde4/kde/kdepimlibs/akonadi/specialcollections.cpp:86 (note: this is a release build; the frame for slot MailCommon::Kernel::slotDefaultCollectionsChanged is omitted is not present above, but it calls initFolders()). Conclusion: 1) when KMail starts, I suppose MailCommon::Kernel::initFolders is called to initialise the folder listing 2) MailCommon::Kernel::initFolders creates a Akonadi::SpecialMailCollectionsDiscoveryJob job (derived from Akonadi::SpecialCollectionsDiscoveryJob) 3) when Akonadi::SpecialCollectionsDiscoveryJob finishes, Akonadi::SpecialCollectionsDiscoveryJob::slotResult is called 4) Akonadi::SpecialCollectionsDiscoveryJob::slotResult iterates over the result (201 folders). For two folders, it calls: 68 d->mSpecialCollections->registerCollection(collection.attribute<SpecialCollectionAttribute>()->collectionType(), collection); The two folders have mRemoteId = "outbox" and mRemoteId = "caixa de saída" (don't ask me why one is localised and the other isn't). 5) because of those calls, Akonadi::SpecialCollectionsPrivate::emitChanged gets called 6) since those two folders have resourceId == mDefaultResourceId, the defaultCollectionsChanged() signal gets called, which ends up calling MailCommon::Kernel::initFolders 7) repeat ad nauseam Reproducible: Always Steps to Reproduce: (for me, at least) 1. Start KMail Actual Results: KMail's UI is responsive, but reading any email is impossible after a while, since the message lists do not update and the mail contents do not show. Expected Results: Reading email is possible. The lines in the LSUB result that contain "akonadi_maildir_resource_0" are: * 9 4 (NAME "caixa de entrada" MIMETYPE (message/rfc822 inode/directory) REMOTEID "caixa de entrada" REMOTEREVISION "1375310411000" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) SpecialCollectionAttribute "inbox" ENTITYDISPLAY "(\"caixa de entrada\" \"mail-folder-inbox\" \"\" ())" newmailnotifierattribute "") * 6 4 (NAME "caixa de saída" MIMETYPE (message/rfc822 inode/directory) REMOTEID "caixa de saída" REMOTEREVISION "0" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) SpecialCollectionAttribute "outbox" ENTITYDISPLAY "(\"caixa de saída\" \"mail-folder-outbox\" \"\" ())") * 7 4 (NAME "enviado" MIMETYPE (message/rfc822 inode/directory) REMOTEID "enviado" REMOTEREVISION "1375309940000" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) SpecialCollectionAttribute "sent-mail" ENTITYDISPLAY "(\"enviados\" \"mail-folder-sent\" \"\" ())") * 11 4 (NAME "lixo" MIMETYPE (message/rfc822 inode/directory) REMOTEID "lixo" REMOTEREVISION "1368747529000" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) SpecialCollectionAttribute "trash" ENTITYDISPLAY "(\"lixo\" \"user-trash\" \"\" ())") * 12 4 (NAME "modelos" MIMETYPE (message/rfc822 inode/directory) REMOTEID "modelos" REMOTEREVISION "" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) SpecialCollectionAttribute "templates" ENTITYDISPLAY "(\"modelos\" \"document-new\" \"\" ())") * 5 4 (NAME "outbox" MIMETYPE (message/rfc822 inode/directory) REMOTEID "outbox" REMOTEREVISION "0" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) SpecialCollectionAttribute "outbox" ENTITYDISPLAY "(\"caixa de saída\" \"mail-folder-outbox\" \"\" ())") * 10 4 (NAME "rascunhos" MIMETYPE (message/rfc822 inode/directory) REMOTEID "rascunhos" REMOTEREVISION "0" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT true INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) SpecialCollectionAttribute "drafts" ENTITYDISPLAY "(\"rascunhos\" \"document-properties\" \"\" ())") * 4 0 (NAME "Pastas locais" MIMETYPE (message/rfc822 inode/directory) REMOTEID "/home/thiago/.local/share/local-mail" REMOTEREVISION "" RESOURCE "akonadi_maildir_resource_0" VIRTUAL 0 CACHEPOLICY (INHERIT false INTERVAL -1 CACHETIMEOUT 1 SYNCONDEMAND true LOCALPARTS (ENVELOPE)) AccessRights "wcdC" SpecialCollectionAttribute "local-mail" ENTITYDISPLAY "(\"Pastas locais\" \"folder\" \"\" ())")