Bug 328601

Summary: Callbacks to ResourceTasks with an invalid state can lead to crashes
Product: [Frameworks and Libraries] Akonadi Reporter: Christian Mollekopf <mollekopf>
Component: IMAP resourceAssignee: Christian Mollekopf <chrigi_1>
Status: RESOLVED FIXED    
Severity: normal CC: aaronw, dozab2000, honkir, jon, kdepim-bugs, knut.hildebrandt, lacsilva, lozol, pa_collins, rjvbertin, rod, vkrause
Priority: NOR Flags: mollekopf: Backport+
Version: GIT (master)   
Target Milestone: ---   
Platform: unspecified   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: New crash information added by DrKonqi
New crash information added by DrKonqi
New crash information added by DrKonqi
New crash information added by DrKonqi

Description Christian Mollekopf 2013-12-09 21:02:37 UTC 
Currently ResourceTasks are prone to crashes because it can quite easily happen that a ResourceTask receives a callback, after the state of the ResourceTask already became invalid.
Typical situations are:
* the resource is shutdown
* the resource goes offline/online
* the server goes offline/online

All these situations can lead to i.e. the SessionPool reconnecting resulting in all sessions becoming invalid. ResourceTasks can then return to a state where they:
* access an invalid session
* other assumptions about the resource are no longer true
* the task is already canceled

This bugreport is used to collect the various related crashreports (although the backtraces can vary, since this can happen pretty much to every ResourceTask).

As countermeasures:

* tasks should be always killed first, in order to avoid them getting signaled by the dropped session resulting form the sessionpool disconnect.
* always delete the tasks immediately, otherwise callbacks can occur to tasks that are already killed and have an invalid state.
* always stop the idlemanager first, and only delete it if it is also available

One particular case that I observed, was a RetrieveItemTask during which the
SessionPool was disconnected while the KIMAP::ItemFetchJob was running,
resulting in the RetrieveItemTask trying to deliver an item while it was already canceled.
This triggered: ASSERT: "d->scheduler->currentTask().type == ResourceScheduler::FetchItem"
Of course we're dealing with the more general case that a ResourceTask must not be called again,
once it's state becomes invalid.


Reproducible: Always
Comment 1 Christian Mollekopf 2013-12-09 21:03:21 UTC 
Some debuginfo from the above described crash:
processing retrieval request for item 1659  parts: ("RFC822")  of resource: "akonadi_imap_resource_0"                                [9026/14489]akonadi_imap_resource_0(2425) ImapIdleManager::onSessionRequestDone:
akonadi_imap_resource_0(2425)/kdepimlibs (kimap) RetrieveItemTask::onMessagesReceived: MESSAGE from Imap server "264"
akonadi_imap_resource_0(2425)/kdepimlibs (kimap) RetrieveItemTask::onMessagesReceived: Has Payload:  true
akonadi_imap_resource_0(2425) SessionPool::releaseSession:
continuing 
request for item 1659 succeeded 
posting retrieval request for item 1660  there are  1  queues and  0  items in mine 
request for item 1660 still pending - waiting 
processing retrieval request for item 1660  parts: ("RFC822")  of resource: "akonadi_imap_resource_0" 
akonadi_imap_resource_0(2425) ImapIdleManager::onSessionRequestDone:
akonadi_imap_resource_0(2425) getCurrentAuthMode: current auth mode:  "PLAIN"
akonadi_imap_resource_0(2425) SetupServer::applySettings: saving IMAP auth mode:  "PLAIN"
akonadi_imap_resource_0(2425) Settings::setPassword: "mykolab@PlsiaCT!0"
akonadi_imap_resource_0(2425) Settings::setPassword: Wallet save:  0
akonadi_imap_resource_0(2425)/kdecore (KConfigSkeleton) KCoreConfigSkeleton::writeConfig:
akonadi_imap_resource_0(2425) SetupServer::applySettings: wrote "imap.mykolab.com" "christian@mailqueue.ch" 1
akonadi_imap_resource_0(2425)/kdecore (KConfigSkeleton) KCoreConfigSkeleton::writeConfig:
akonadi_imap_resource_0(2425)/libakonadi Akonadi::AgentBase::setOnlineInternal: false
akonadi_imap_resource_0(2425) ImapResource::doSetOnline: online= false
akonadi_imap_resource_0(2425) ImapIdleManager::onPoolDisconnect:
akonadi_imap_resource_0(2425) ResourceTask::kill:
akonadi_imap_resource_0(2425)/kdepimlibs (kimap) KIMAP::IdleJob::stop:
akonadi_imap_resource_0(2425) Settings::clearCachedPassword:
akonadi_imap_resource_0(2425)/libakonadi Akonadi::AgentBase::setOnlineInternal: true
continuing 
request for item 1660 "265" failed: "Unable to retrieve item from resource: <html>Invalid item retrieved</html>" 
akonadi_imap_resource_0(2425) ImapResource::doSetOnline: online= true
ItemRetrieverException :  Unable to retrieve item from resource: <html>Invalid item retrieved</html>
akonadi_imap_resource_0(2425)/kdepimlibs (kimap) RetrieveItemTask::onMessagesReceived: MESSAGE from Imap server "265"
akonadi_imap_resource_0(2425)/kdepimlibs (kimap) RetrieveItemTask::onMessagesReceived: Has Payload:  true
ASSERT: "d->scheduler->currentTask().type == ResourceScheduler::FetchItem" in file /home/chrigi/devel/kde/kdepimlibs/akonadi/resourcebase.cpp, li
ne 520
akonadi_kolabproxy_resource(2426) ImapItemAddedJob::onItemFetchJobDone: creating new item
KCrash: Application 'akonadi_imap_resource' crashing...
KCrash: Attempting to start /usr/libexec/kde4/drkonqi from kdeinit



#9  0x0000003a5de71294 in qFatal(char const*, ...) () from /lib64/libQtCore.so.4
#10 0x00007f82281630e3 in Akonadi::ResourceBase::itemRetrieved (this=0x2200aa0, item=...) at /home/chrigi/devel/kde/kdepimlibs/akonadi/resourcebase.cpp:520
#11 0x0000000000425537 in ResourceState::itemRetrieved (this=0x23d7810, item=...) at /home/chrigi/devel/kde/kdepim-runtime/resources/imap/resourcestate.cpp:337
#12 0x0000000000455d2d in ResourceTask::itemRetrieved (this=0x236cef0, item=...) at /home/chrigi/devel/kde/kdepim-runtime/resources/imap/resourcetask.cpp:250
#13 0x000000000045fdb7 in RetrieveItemTask::onMessagesReceived (this=0x236cef0, mailBox=..., uids=..., messages=...) at /home/chrigi/devel/kde/kdepim-runtime/resources/imap/retrieveitemtask.cpp:128
#14 0x000000000046f0a0 in RetrieveItemTask::qt_static_metacall (_o=0x236cef0, _c=QMetaObject::InvokeMetaMethod, _id=1, _a=0x7fff71f103d0) at /home/chrigi/devel/kde/build/master/kdepim-runtime/resources/imap/moc_retrieveitemtask.cpp:55
#15 0x0000003a5df8df78 in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) () from /lib64/libQtCore.so.4
#16 0x00007f8227d5a5bf in KIMAP::FetchJob::messagesReceived (this=0x2893f30, _t1=..., _t2=..., _t3=...) at /home/chrigi/devel/kde/build/master/kdepimlibs/kimap/moc_fetchjob.cpp:121
#17 0x00007f8227d5b22b in KIMAP::FetchJobPrivate::emitPendings (this=0x2322c50) at /home/chrigi/devel/kde/kdepimlibs/kimap/fetchjob.cpp:61
#18 0x00007f8227d58233 in KIMAP::FetchJob::handleResponse (this=0x2893f30, response=...) at /home/chrigi/devel/kde/kdepimlibs/kimap/fetchjob.cpp:249
#19 0x00007f8227d6cb7d in KIMAP::SessionPrivate::responseReceived (this=0x24a80b0, response=...) at /home/chrigi/devel/kde/kdepimlibs/kimap/session.cpp:285
Comment 2 Christian Mollekopf 2013-12-09 21:12:47 UTC 
*** Bug 326437 has been marked as a duplicate of this bug. ***
Comment 3 Christian Mollekopf 2013-12-09 21:14:12 UTC 
*** Bug 326242 has been marked as a duplicate of this bug. ***
Comment 4 Christian Mollekopf 2013-12-09 21:14:30 UTC 
*** Bug 326391 has been marked as a duplicate of this bug. ***
Comment 5 Christian Mollekopf 2013-12-09 21:23:45 UTC 
*** Bug 326969 has been marked as a duplicate of this bug. ***
Comment 6 Christian Mollekopf 2013-12-09 21:24:42 UTC 
*** Bug 328030 has been marked as a duplicate of this bug. ***
Comment 7 Christian Mollekopf 2013-12-09 21:49:20 UTC 
I fixed at least some of the crashes in master:
c69d27a
ca1c508
63dbe90
4b6f801

It's entirely possible that I missed some cases, so just reopen if you find something new.
Comment 8 Jon Skanes 2013-12-20 21:53:32 UTC 
Created attachment 84203 [details]
New crash information added by DrKonqi

akonadi_imap_resource (4.12) on KDE Platform 4.12.0 using Qt 4.8.4

- What I was doing when the application crashed:

I moved multiple items between folders in kmail.

I noticed this was fixed in trunk.  Today I installed KDE 4.12.0 from the Kubuntu PPA.  If the fix made it into this release, please reopen this.

-- Backtrace (Reduced):
#7  0x0808d2ce in MoveItemsTask::recordNewUid (this=this@entry=0x8c5b8b8) at ../../../resources/imap/moveitemstask.cpp:290
#8  0x0808d903 in MoveItemsTask::onSearchDone (this=0x8c5b8b8, job=0x8c5bc18) at ../../../resources/imap/moveitemstask.cpp:242
[...]
#10 0xb5a971e5 in KJob::result (this=this@entry=0x8c5bc18, _t1=_t1@entry=0x8c5bc18) at ./kjob.moc:208
#11 0xb5a97232 in KJob::emitResult (this=0x8c5bc18) at ../../kdecore/jobs/kjob.cpp:318
#12 0xb655863e in KIMAP::Job::handleErrorReplies (this=this@entry=0x8c5bc18, response=...) at ../../kimap/job.cpp:84
Comment 9 Jekyll Wu 2014-01-06 03:18:24 UTC 
*** Bug 329162 has been marked as a duplicate of this bug. ***
Comment 10 honkir 2014-01-13 20:04:38 UTC 
Created attachment 84629 [details]
New crash information added by DrKonqi

akonadi_imap_resource (4.12) on KDE Platform 4.12.0 using Qt 4.8.4

- What I was doing when the application crashed:

Deleted an e-mail in spam folder in KMail. Since then the imap resource keeps crashing with message that it can not move the message from folder "spam" to "trash".

-- Backtrace (Reduced):
#7  0x0808d2ce in MoveItemsTask::recordNewUid (this=this@entry=0x9d989c8) at ../../../resources/imap/moveitemstask.cpp:290
#8  0x0808d903 in MoveItemsTask::onSearchDone (this=0x9d989c8, job=0x9cf71f8) at ../../../resources/imap/moveitemstask.cpp:242
[...]
#10 0xb5ace1e5 in KJob::result (this=this@entry=0x9cf71f8, _t1=_t1@entry=0x9cf71f8) at ./kjob.moc:208
#11 0xb5ace232 in KJob::emitResult (this=0x9cf71f8) at ../../kdecore/jobs/kjob.cpp:318
#12 0xb658f63e in KIMAP::Job::handleErrorReplies(KIMAP::Message const&) () from /usr/lib/libkimap.so.4
Comment 11 honkir 2014-01-13 20:12:27 UTC 
The bug seems to still exist but this one is marked as resolved. Shoud I fill a new one? I think I'm not allowed to reopen this one.
Comment 12 Christian Mollekopf 2014-01-13 21:34:38 UTC 
(In reply to comment #11)
> The bug seems to still exist but this one is marked as resolved. Shoud I
> fill a new one? I think I'm not allowed to reopen this one.

Since the fixes are only in master (4.13) and not in 4.12 I suspect this is still the same crash.
Comment 13 honkir 2014-01-14 07:48:59 UTC 
Than thanks for info and sorry for duplicate report. The 4.13 is scheduled for April, this is a long time. Does the Backport flag mean that the fix will be available also in 4.12.2?
Comment 14 Christian Mollekopf 2014-01-15 11:46:00 UTC 
(In reply to comment #13)
> Than thanks for info and sorry for duplicate report. The 4.13 is scheduled
> for April, this is a long time. Does the Backport flag mean that the fix
> will be available also in 4.12.2?

Please close the duplicate if you didn't already.

The backport flag means that I should backport the fix if I get to it, I don't know yet when that will be.
Comment 15 lozol 2014-04-06 20:52:39 UTC 
Created attachment 85979 [details]
New crash information added by DrKonqi

akonadi_imap_resource (4.12) on KDE Platform 4.12.3 using Qt 4.8.6

- What I was doing when the application crashed:

I was readin messages from imap resource атв deleting some of them

-- Backtrace (Reduced):
#7  0x0808d5fe in MoveItemsTask::recordNewUid (this=this@entry=0x96fe260) at ../../../resources/imap/moveitemstask.cpp:290
#8  0x0808dc33 in MoveItemsTask::onSearchDone (this=0x96fe260, job=0x96fdc60) at ../../../resources/imap/moveitemstask.cpp:242
[...]
#10 0xb5a091e5 in KJob::result (this=this@entry=0x96fdc60, _t1=_t1@entry=0x96fdc60) at ./kjob.moc:207
#11 0xb5a09232 in KJob::emitResult (this=0x96fdc60) at ../../kdecore/jobs/kjob.cpp:318
#12 0xb64ca63e in KIMAP::Job::handleErrorReplies (this=this@entry=0x96fdc60, response=...) at ../../kimap/job.cpp:84
Comment 16 RJVB 2015-09-08 07:54:11 UTC 
Created attachment 94467 [details]
New crash information added by DrKonqi

akonadi_imap_resource (4.13) on KDE Platform 4.14.11 using Qt 4.8.7

- What I was doing when the application crashed:

Kontact had been waiting for me to enter my wallet password for an unknown time (cats woke the computer from sleep); after I unlocked the situation I got this crash.

-- Backtrace (Reduced):
#6  0x00007efd5df59524 in QObject::thread() const () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#7  0x00007efd5df6012b in QObject::QObject(QObject*) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#8  0x00007efd5aa348fd in KJob::KJob (this=0x2a75a70, parent=0x40807) at ../../kdecore/jobs/kjob.cpp:51
#9  0x00007efd5cb66d43 in KIMAP::Job::Job (this=0x2a75a70, dd=...) at ../../kimap/job.cpp:37
#10 0x00007efd5cb7e4d7 in KIMAP::SelectJob::SelectJob (this=0x2a75a70, session=<optimized out>) at ../../kimap/selectjob.cpp:60