Bug 257722 - IMAP resource can't handle temporary network connection losses
Summary: IMAP resource can't handle temporary network connection losses
Status: RESOLVED FIXED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: IMAP resource (show other bugs)
Version: unspecified
Platform: OpenSUSE Linux
: NOR normal with 120 votes (vote)
Target Milestone: ---
Assignee: Christian Mollekopf
URL:
Keywords:
: 247061 272660 282486 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-11-23 18:36 UTC by Michi
Modified: 2014-04-25 11:05 UTC (History)
17 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michi 2010-11-23 18:36:22 UTC
Version:           unspecified (using Devel) 
OS:                Linux

When I change my network connection, e.g. switching to VPN, or even after suspend I have to stop akonadi in order to be able to work with IMAP resources again. If I don't restart akonadi the IMAP resources can't synchronize/download anything.

Reproducible: Always
Comment 1 Fest 2010-12-05 14:11:15 UTC
Confirm that bug.
On network loss, akonadi imap stuck on mail check. Kmail can't check mail anymore even after network back agaoin. Restarting akonadi solves the issue.
Comment 2 Franz Trischberger 2011-05-16 21:04:15 UTC
Still present in akonadi-server-1.5.3/kde-4.6.3/kdepim-4.6-rc1.
Here we have a quite unstable connection, several times per day, the connection crashes. Every time this happens, akonadi needs to be restarted.
PLEASE fix this. This is for me one of the most annoying bugs I have to live with. The other two are the buggy addressbook-completion in the kmail-composer and the dvd-drive spinning up always, when the places-sidebar gets shown.
Comment 3 Georg 2011-05-31 09:28:24 UTC
Confirm. Very annoying. 
Cheers,
  Georg
Comment 4 Kevin Ottens 2011-09-17 09:38:28 UTC
*** Bug 247061 has been marked as a duplicate of this bug. ***
Comment 5 Médéric Boquien 2011-09-17 09:48:56 UTC
*** This bug has been confirmed by popular vote. ***
Comment 6 Andreas Schneider 2011-09-17 12:49:46 UTC
This happened to me once and since then the resource is always stuck in connection established. coolo run into the same issue last week.

I've traced the traffic with wireshark. I can see that it is authenticating and it exchanges a lot of information and then get stuck. The resource lacks of debug message. I'm a C developer and the source code was bit confusing when I've looked at it. But if someone can help me debugging we can do it in a IRC or mumble session.
Comment 7 Kevin Ottens 2011-09-17 12:52:47 UTC
*** Bug 272660 has been marked as a duplicate of this bug. ***
Comment 8 Kevin Ottens 2011-09-22 05:31:11 UTC
*** Bug 282486 has been marked as a duplicate of this bug. ***
Comment 9 Franz Trischberger 2011-10-09 19:47:28 UTC
I noticed, that if i leave akonadi enough time, it somehow reestablishes a working connection, again. I can't say, when and how, but i need to wait several minutes after the connection loss.
BUT: In akonadi-1.6.1 (I think) must have been a change, that somehow reacts on the changes the connection loss leaves, and tries to roll back or something like that. The effect: some - not all - already read and fully downloaded mails get marked as unread and removed from cache. This means a) confusion about the current state in correspondence (re-reading mails until the obvious gets realised) and re-downloading mails. The last one is the really bad thing here: We have 45kb/s, and larger mails (which arrive quite often) really take a lot of time to download! Re-downloading means loss of time.

This problem is really so annoying. I don't know, what to do! Up to more then 30 (!) internet-reconnects a day the last weeks, and every time akonadi-imap refuses to work. akonadictl restart took 20-30 seconds, after that it worked again. But now I need to redownload Megabytes again and again. If there won't be a fix, I need to change email-client - sorry :\ Akonadi looked and looks quite promising, kmail integrates so nice into the desktop. But this problem makes daily-email a gambling.
Comment 10 S. Burmeister 2011-11-12 10:29:14 UTC
Maybe http://git.reviewboard.kde.org/r/103112/ will help.
Comment 11 Franz Trischberger 2011-11-14 09:34:53 UTC
(In reply to comment #10)
> Maybe http://git.reviewboard.kde.org/r/103112/ will help.
Did not fix anything here, sorry :(
setNeedsNetwork(needsNetwork()) is done already in reconnect(), which gets triggered in quite some places...
Comment 12 Will Stephenson 2011-11-15 09:31:47 UTC
May be resolved by Sergio's commits to 4.7 branch in kdepim-runtime: b62d1bf7e5e856a5845140aaf2e740bb71e50956
and master:
5d343b97144dbe2a371837111f950dfbdf1fe9cb

I'm adding this patch to openSUSE KDE:Distro:Stable kdepim4-runtime to test.
Comment 13 David Faure 2012-01-06 12:25:20 UTC
Git commit 679a05cb8ee30c81ea207b6cfc0e4131fd1a0c78 by David Faure.
Committed on 06/01/2012 at 13:21.
Pushed by dfaure into branch 'master'.

Restore the default timeout to 30s, after the kdelibs+kdepimlibs fixes.

The timeout doesn't trigger by mistake anymore on large emails,
so it can be restored to 30s, in order to fix the issues with
network disconnections, suspend/resume etc.
Related: bug 258378, bug 258271, bug 286047

M  +1    -1    resources/imap/imapresource.kcfg

http://commits.kde.org/kdepim-runtime/679a05cb8ee30c81ea207b6cfc0e4131fd1a0c78
Comment 14 Kevin Ottens 2012-02-11 08:38:43 UTC
Does it stil happen? If yes please reopen but David latest patches should have done the trick.
Comment 15 Franz Trischberger 2012-02-12 15:00:23 UTC
(In reply to comment #14)
> Does it stil happen? If yes please reopen but David latest patches should have
> done the trick.
Still the same here :/ When the network goes down while kmail is running, imap-accounts don't work anymore, even if network reestablishes (If I leave kmail, it seems to fix itself when I don't touch it for quite some time, half an hour or so). Also, large Mails are not shown when download finished (the other bug I have with my IMAP-Accounts).
I ensured I have a fixed version: kdepim-runtime-4.8.0 ships with "30" as default for the session-timeout. I am running kde-4.8.0 since it was released, Qt-4.8.0, linux-3.2.5.

I did a new installation some months ago, and there it happend exactly once, that I got a system-notification saying something like "Network connection lost, IMAP-account will go offline". No message, when the connection reestablished again. Since then I never ever got this notification again. I did NOT install networkmanager (wired connection, static IP -> no need for something like a networkmanager).
I really would like to help fix this. Just tell me, what I should do!
Comment 16 Kevin Ottens 2012-02-14 21:19:43 UTC
Sorry, my bad, I was unclear. I meant latest patches in upcoming 4.8.1. It should be out soonish, please test with that one when it is available.
Comment 17 Franz Trischberger 2012-03-07 20:52:05 UTC
Installed 4.8.1 today - nothing changed. Started kontact, let it idle, run into disconnect, wait for reconnection, click an unread mail - nothing happens, just get the "Retrieving Folder Contents"-page.
If there are any special steps to help fixing this - I am willing to help!
Comment 18 Franz Trischberger 2012-03-08 09:56:10 UTC
I thought about this again. In the early days of kmail2 this problem was not present. I think it was introduced together with Push-IMAP. Could that be?
Comment 19 Kevin Ottens 2012-03-08 17:07:32 UTC
Could you please provide us the task list and the change notification when your resource is stuck?

It now can be done from akonadiconsole, just right click on the resource and use "Show task list" and "Show notification log".
Comment 20 S. Burmeister 2012-03-09 08:45:21 UTC
Resource 1:
ResourceScheduler: Offline
 current task: 2379 Invalid (no task) 
 queue 0 is empty
 queue 1 is empty
 queue 2 is empty
 queue 3 is empty
 queue 4 10 tasks:
  2376 Custom startConnect 
  2373 Custom startConnect 
  2317 SyncCollectionTree 
  2318 SyncCollection collection 33 
  2319 SyncCollection collection 34 
  2320 SyncCollection collection 35 
  2321 SyncCollection collection 36 
  2322 SyncCollection collection 37 
  2323 SyncCollection collection 38 
  2324 SyncCollection collection 39 

notification log is empty.

Resource 2:
ResourceScheduler: Offline
 current task: 407 Invalid (no task) 
 queue 0 is empty
 queue 1 is empty
 queue 2 is empty
 queue 3 is empty
 queue 4 5 tasks:
  401 Custom startConnect 
  398 Custom startConnect 
  387 SyncCollectionTree 
  388 SyncCollection collection 40 
  389 SyncCollection collection 42 

notification log is empty.
Comment 21 Franz Trischberger 2012-03-09 10:35:32 UTC
resource_1:
ResourceScheduler: Online
 current task: 137 FetchItem item 16204 
 queue 0 is empty
 queue 1 is empty
 queue 2 is empty
 queue 3 is empty
 queue 4 is empty

resource_0:
ResourceScheduler: Online
 current task: 96 SyncCollection collection 98 
 queue 0 is empty
 queue 1 is empty
 queue 2 is empty
 queue 3 1 tasks:
  98 FetchItem item 16228 
 queue 4 1 tasks:
  97 Custom triggerCollectionExtraInfoJobs 

For both resources the notification log was empty.
Comment 22 Kevin Ottens 2012-03-09 16:56:17 UTC
OK, indeed a startConnect which doesn't complete, and for some reason the LoginJob not timing out probably... reopening then.

David, any idea for guidance to debug further? All I have in mind is to attach gdb to the resource and check if/where the KIMAP::LoginJob is stuck. Definitely not a trivial task, and at the same time I cannot reproduce here.
Comment 23 Franz Trischberger 2012-03-14 16:14:49 UTC
OK, for me it seems to be fixed mostly. The resources in my last post were not "stuck", It was waiting the 30 seconds timeout (I think). Kmail just didn't show the message (probably related to https://bugs.kde.org/show_bug.cgi?id=266429).
But I still can make at least my web.de-account stuck:
When connection ist lost, go to the inbox, select some messages. When connection comes up again, the resource stays Offline. There were several possibilities after that: 

1) 30 seconds timeout -> Connection established -> Ready ->not working:
ResourceScheduler: Offline
 current task: 20 Invalid (no task) 
 queue 0 is empty
 queue 1 is empty
 queue 2 is empty
 queue 3 is empty
 queue 4 2 tasks:
  15 SyncCollection collection 69 
  16 Custom triggerCollectionExtraInfoJobs

AgentBase(akonadi_imap_resource_1): Cannot fetch item in offline mode.
AgentBase(akonadi_imap_resource_1): Cannot fetch item in offline mode.
AgentBase(akonadi_imap_resource_1): Cannot fetch item in offline mode.
AgentBase(akonadi_imap_resource_1): Cannot fetch item in offline mode.
AgentBase(akonadi_imap_resource_1): Cannot fetch item in offline mode.
AgentBase(akonadi_imap_resource_1): Cannot fetch item in offline mode.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_0): Connection lost
AgentBase(akonadi_imap_resource_0): Connection to server lost.
NotificationManager::notify ( Collection (98, /INBOX) in collection 94 modified parts (timestamp) )
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
NotificationManager::notify ( Collection (69, /INBOX) in collection 12 modified parts (timestamp) )
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.
AgentBase(akonadi_imap_resource_1): There is currently no session to the IMAP server available.

(resource_0 was working, as I did not touch it while connection was down)

Then I have to restart akonadi (akonadictl restart), it does not help to just turn the resource offline and online again.

2) It just stays "Offline", even after waiting 30 seconds. Manually turning it Online:
2.1) nothing happens (trying to connect without any success)
2.2) It connects, says "Connection established" but does not turn "Ready"
2.3) -> see 1)

For 2) I don't have a task list, as it happend during testing, what I can do, how to reproduce certain behaviour. In any case, the web.de-resource stays unusable (reproducable!), when I start using it while there is no connection. If I don't touch kmail while connection is down, everything works great after connection was reestablished.

I will further look, if I can get tasklists for 2). But it is more difficult ATM. Some weeks ago I had dozens of connection losses a day, somtimes 10 within half an hour (in the evening). But since I try to find out how it behaves, things got better connection-wise, get 3-5 connection losses per day. :( I know, I should be happy :D
Comment 24 Franz Trischberger 2012-03-15 08:12:56 UTC
Ok, on the other PC I got this:

ResourceScheduler: Offline
 current task: 7 Invalid (no task) 
 queue 0 1 tasks:
  4 Custom startConnect 
 queue 1 1 tasks:
  2 ChangeReplay 
 queue 2 is empty
 queue 3 is empty
 queue 4 2 tasks:
  3 Custom startConnect 
  5 SyncCollection collection 87

Notification list:
session=plasma-desktop-495252308 type=Item operation=Add uid=17677 remoteId= resource=akonadi_imap_resource_7 parentCollection=103 parentDestCollection=-1 mimeType=text/x-vnd.akonadi.note itemParts=
session=plasma-desktop-495252308 type=Item operation=Add uid=17678 remoteId= resource=akonadi_imap_resource_7 parentCollection=103 parentDestCollection=-1 mimeType=text/x-vnd.akonadi.note itemParts=

gdb attached to the resource-process
(gdb) bt
#0  0x00007f17f76d51d3 in poll () from /lib64/libc.so.6
#1  0x00007f17f64165b8 in g_main_context_iterate.isra.21 () from /usr/lib64/libglib-2.0.so.0
#2  0x00007f17f6416a3b in g_main_context_iteration () from /usr/lib64/libglib-2.0.so.0
#3  0x00007f17fb1fb636 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/qt4/libQtCore.so.4
#4  0x00007f17fa5f4326 in QGuiEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/qt4/libQtGui.so.4
#5  0x00007f17fb1cbb12 in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/qt4/libQtCore.so.4
#6  0x00007f17fb1cbd97 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib64/qt4/libQtCore.so.4
#7  0x00007f17fb1d08a5 in QCoreApplication::exec() () from /usr/lib64/qt4/libQtCore.so.4
#8  0x00007f17fb90ce03 in Akonadi::ResourceBase::init(Akonadi::ResourceBase*) () from /usr/lib64/libakonadi-kde.so.4
#9  0x000000000041cb36 in int Akonadi::ResourceBase::init<ImapResource>(int, char**) ()
#10 0x00007f17f762742d in __libc_start_main () from /lib64/libc.so.6
#11 0x00000000004173f5 in _start ()

This happens to ALL IMAP-Resources. There also is used the same Web.de-Account, as I described in the last post. But here, triggering "Toggle Online/Offline" immediately starts the resource and makes it usable!

What I can see: There are two "Custom startConnect"-tasks. S.Burmeister also had two. Does "startConnect" connect to the server? Probably the server just can't/won't handle two simultaneous connects?
Comment 25 Kevin Ottens 2012-03-15 08:21:06 UTC
Hm, I might have a theory... Could you tell me which auth scheme you're using? If that's not the clear text authentication, could you try it for a while and tell me if you can reproduce?

At that point I'm thinking about a LoginJob getting stuck, and since here we're not in full control for some of the authentication modes (that it all of them except clear text) could be that it needs extra care... The 30 seconds timeout we set on the session might not be enough to detect the inactivity.
Comment 26 Franz Trischberger 2012-03-15 08:43:59 UTC
(In reply to comment #25)
> Hm, I might have a theory... Could you tell me which auth scheme you're
> using? If that's not the clear text authentication, could you try it for a
> while and tell me if you can reproduce?
OK, was set to PLAIN. Set them to Clear Text now.
BTW.: Changing the Athentication scheme for one Account will change it for all others, too. Is that intended?
Comment 27 Kevin Ottens 2012-03-15 08:49:45 UTC
Nope, sounds wrong to me. I don't remember seeing that happening though... definitely strange.
Comment 28 Franz Trischberger 2012-03-15 10:04:38 UTC
(In reply to comment #27)
> Nope, sounds wrong to me. I don't remember seeing that happening though...
> definitely strange.
Probably an issue with my akonadi-configs from kde-4.7/4.6? Because now I manually changed settings, saved them and it seems I can't reproduce it.
If I can, I will open a reperate bugreport.
Comment 29 Franz Trischberger 2012-03-16 21:18:03 UTC
I have turned all resources to Clear Text Authentication. Now I got this after a connection loss:

ResourceScheduler: Offline
 current task: 71 Invalid (no task) 
 queue 0 is empty
 queue 1 is empty
 queue 2 is empty
 queue 3 is empty
 queue 4 1 tasks:
  64 Custom triggerCollectionExtraInfoJobs

I had to manually toggle Online through akonadiconsole, then it became usable, again.
Comment 30 Kevin Ottens 2012-03-18 19:48:26 UTC
OK, thanks for trying. Kind of confirm my theory... Now the difficult point will be to find out why the others auth scheme create such an issue and how to work around it...
Comment 31 Franz Trischberger 2012-03-21 06:28:08 UTC
BTW.: Resources still hang sometimes with this message printed out every time I trigger an action that requires a connection to the server:
AgentBase(akonadi_imap_resource_0): There is currently no session to the IMAP server available.
although the resource says "Ready" and the icon shows "connected". I need to restart akonadi in order to make it work again.

With Authentication set to Clear Text I had a funny issue sometimes. akonadiconsole prints "Ready", but the icons says "disconnected". How could that be?
Comment 32 Franz Trischberger 2012-03-21 17:42:20 UTC
Argh, got this now with all IMAP accounts, still all with Clear text:
ResourceScheduler: Offline
 current task: 144 Invalid (no task) 
 queue 0 1 tasks:
  141 Custom startConnect 
 queue 1 is empty
 queue 2 is empty
 queue 3 is empty
 queue 4 2 tasks:
  140 Custom startConnect 
  142 SyncCollection collection 69

I started kontact while connection was down, probably lost seconds before. Manually turning each resource online makes them usale again. I viewed the task list while connection was down, and the Custom startConnects already were there. I waited several minutes for a possible timeout, but nothing happend, so really stuck.
Comment 33 S. Burmeister 2012-04-07 09:55:44 UTC
KDE SC 4.8.2 with auth scheme PLAIN:

ResourceScheduler: Offline
 current task: 2667 Invalid (no task) 
 queue 0 is empty
 queue 1 is empty
 queue 2 is empty
 queue 3 is empty
 queue 4 9 tasks:
  2664 Custom startConnect 
  2656 SyncCollectionTree 
  2657 SyncCollection collection 8 
  2658 SyncCollection collection 9 
  2659 SyncCollection collection 10 
  2660 SyncCollection collection 11 
  2661 SyncCollection collection 12 
  2662 SyncCollection collection 13 
  2663 SyncCollection collection 14
Comment 34 S. Burmeister 2012-08-28 18:23:23 UTC
Still valid for 1.8.0.

Another culprit could be that it has to "re-connect" as well if e.g. a new vpn connection was established.
Comment 35 Heinz Ulrich Stille 2012-12-12 15:14:59 UTC
Still there in 1.8.1 (in fact, this never has worked for me since the introduction of akonadi). If there is a network problem that doesn't affect the local connection, akonadi IMAP ressources go permanently offline. Status usually is "Offline, Running (0%)". Dis- and reconnecting the local network interface restarts all IMAP connections at once, as does clicking "Check Mail" in KMail. A closed wallet is not a problem. Setting authentication to "Clear text" and encryption to "None" doesn't seem to make any difference. Perhaps the "running" part of the status keeps the ressource from restarting?
Comment 36 Andreas Sturmlechner 2013-03-16 10:53:59 UTC
Just hit the same bug with 1.9.1 it seems. First, it was stuck forever on fetching a folder, now it is stuck forever trying to abort.
Comment 37 Kevin Ottens 2013-11-16 07:29:06 UTC
The IMAP resource has a new maintainer, reassigning to him.
Comment 38 Christian Mollekopf 2013-12-12 10:08:10 UTC
I think this is fixed in master. I fixed the bug where kimap got into a reconnect frenzy on connection loss in 4.11.4, and switching to a vpn and back works for me. Let me know if there are other issues.
Comment 39 itsef-admin 2013-12-12 14:40:06 UTC
I wonder whether the fix you mentions also has impact on https://bugs.kde.org/show_bug.cgi?id=327513...

If you think that is the case: Which version of akonadi should this be fixed in, then? On the download page for 4.11.4, there seems no source for akonadi present? I'd be willing to build my own packages for kontact (what else? akonadi, I presume?) so I can test this, as it will probably be a while before Kubuntu offers a full update to 4.11.4 for Kubuntu 13.10 (they haven't even released 4.11.3 yet, as far as I can see). I'm already at akonadi 1.10.80.
Comment 40 Christian Mollekopf 2013-12-12 14:49:55 UTC
The reconnect frenzy was kdepimlibs 4b8adabf. I also fixed a bunch of crashes in the imap resource in the master branch of kdepim-runtime that typically occurred during disconnects, so that might fix you problems regarding 327513, but since I don't know what's happening there I can't tell for sure.