Bug 77862

Summary: kio_imap processes hang when connection status changes
Product: [Applications] kmail Reporter: Mike Culbertson <mike>
Component: IMAPAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED FIXED    
Severity: normal CC: aaronw, andreas-stangl, arne.schmitz, bonaventur, bugs, bve, claudio.f.gil, dev-kde, dev-kde, diego.ml, dion, dom, dr.scient, faure, fetzer.ch, forum08, freeman3, fungs, greg, hakan, ismail, j.cook, John, jstrunk, julian, kde-bugs, kde, kde, kde_bugs, kedadi, krux, langer_mann, martin.nowack, mcguire, mirza.dervisevic, mrgrim, niburu1, nicolas, null, pfeiffer, ppurka, renda.krell, rezbit.hex, roth, ry, sebastian, silje, stst, sven.burmeister, tallboy258, tampakrap, thenktor, tiposchi, tuju, vesa.muhonen
Priority: NOR    
Version: 1.10.1   
Target Milestone: ---   
Platform: Gentoo Packages   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: the definition of KIpAddressInUse
the implementation of KIpAddressInUse
test for KIpAddressInUse
the diff for kdepim
the diff for kdepimlibs

Description Mike Culbertson 2004-03-17 20:15:21 UTC
Version:           1.6.1 (using KDE KDE 3.2.1)
Installed from:    Gentoo Packages
Compiler:          gcc version 3.3.3 20040217 (Gentoo Linux 3.3.3, propolice-3.3-7) 
OS:          Linux

A brief description of my network:

I have several Linux workstations sitting behind a Linux iptables-based firewall. I run a PPTP VPN client on the firewall itself, to connect to a remote network.  There are is no routing or any other specialized configuration on any of the client systems, all is handled on the Firewall/VPN Client box.

The remote IMAP server is acessible both via the VPN and via the internet (unfortunately at the same IP in both cases).  If the VPN connection is up, the client would connect over the VPN, if the VPN is down, the connection would go directly over the internet.


The issue occurs when there is a change in VPN connection status on the firewall. For example, say the VPN is up and kmail connects.  Then for some reason, the VPN tunnel goes away. Kmail then does not seem to drop or reconnect it's IMAP connection, it just stops working, so I have to restart. Upon closing kmail, the kio_imap4 processes do not exit, and when I restart kmail, I do not get login prompts for the IMAP server, and IMAP does not otherwise work.  I usually then have to manually kill the kio_imap4 processes, and there is about a 50% chance of kmail crashing upon the next start.

My mostly ignorant assessment of the problem is that kio_imap4 does not manage to see that it's connection is no longer working and/or kmail does not properly identify a "dead" kio_imap4 process when it starts.  I have not determined yet how long the "dead" kio_imap4 processes hang around.  I think my main concern is just that kmail is not resilient enough against network topology changes. I will attempt to steadily recreate this issue so that I can glean more information.

Please feel free to ask me for any additional info that might be helpful.
Comment 1 Till Adam 2004-12-13 23:29:01 UTC
Mike, any chance to try this with a more current KMail, maybe?
Comment 2 Robin Rosenberg 2004-12-18 11:33:44 UTC
No crash, but imap hangs with 1.7.1 if the connection is lost for some reason.m typically when the vpn connetion is lost. One such case is after suspend/resmue. This happens with bothe the normal imap and cached imap.
Comment 3 Till Adam 2004-12-18 11:53:26 UTC
Yes, the hangs I'm seeing as well. Glad the crash is gone, though. Re-assigning to me and adjusting summary and severity.
Comment 4 Carsten Burghardt 2005-01-22 15:54:41 UTC
When your vpn goes down, can you still ping the host? I mean, the kioslave uses the normal network interfaces to connect so I don't really understand why it should hang.
Comment 5 Kevin Goeser 2005-03-29 16:11:34 UTC
No crashes or dead processes here, but kmail loses its connection quite often when suspending/resuming, kmail restart helps. Probably the same like in bug 94018 (I'm using 3.4.0/1.8).
Comment 6 Matthias Wieser 2005-06-25 11:38:33 UTC
What I am thinking about that problem (hopefully, I made no false assumption)
Does this setup produce an error?
server <-> firewall <-> NATed-Computer
Connect to server from nated Computer . after that change the IP-Adress of the firewall.
The NATed computer has still the same IP, but to the outside it has changed.
(after the change the connection should hang, if my theory is right)

It is possible to ping the host but the tcp-connection is not anymore working because the source address changed (the packets have the adress of the firewall as it is nated).
Comment 7 Petr Burian 2005-10-09 20:45:28 UTC
I gues the same or similar problem is when the computer "lost" it's connection (e.g. notebook susspend and resume after while). The KMail hangs and the only quick solution to solve this is to kill all kio_imap processes and start KMail again. Second probably the same problem is when I shut down the KMail (CTRL+Q), the kio_imap processes are not closed. They have to be killed manually to start KMail again. Unfortunately it happens also in last beta1 of KMail (1.8.91).
Comment 8 Andras Georgy Bekes 2006-05-03 09:54:52 UTC
Isn't this bug the same as #118638 AND #104743?
Comment 9 Ismail Onur Filiz 2006-05-05 09:42:40 UTC
*** Bug 104743 has been marked as a duplicate of this bug. ***
Comment 10 Ismail Onur Filiz 2006-05-05 09:46:48 UTC
*** Bug 118638 has been marked as a duplicate of this bug. ***
Comment 11 Ismail Onur Filiz 2006-06-20 19:13:33 UTC
*** Bug 129473 has been marked as a duplicate of this bug. ***
Comment 12 Isaac Kriegman 2006-10-03 03:25:27 UTC
I have the same problem whenever I suspend my computer and restart it, I must also restart kmail in order to get new IMAP mail.
Comment 13 Michael Kreitzer 2006-11-21 19:42:02 UTC
KMail exhibits the same behavior for me whenever the connection to the imap server is lost for whatever reason. I can no longer check mail, open messages, or do pretty much anything on the account until kmail is closed and the kio_imap processes are killed. I've been experiencing this behavior through many major kde revisions (3.3, 3.4, 3.5, maybe earlier I can't remember). I've found many related bug reports, and many marked as resolved with svn commits, but the problem never goes away. This problem has shown itself through 3 different imap servers I have had accounts on. The only step needed to reproduce is to lose the connection or for the mail server to simply stop responding without dropping the connection (it would be nice to be able to set a low timeout value).

Often trying to do many things at a time will also cause this (quickly selecting every folder in my account in succession before loading the folder is complete or moving all messages in a folder to trash, emptying the trash, and moving to a new folder without waiting for any operation to finish, for example, will either cause kmail to pretty much freak out and stop communicating with the server at all or just crash.)
Comment 14 Andras Georgy Bekes 2006-11-26 20:49:03 UTC
Hey, this bug has been filed more than two years ago, and still no progress...? :-(

Why don't you check how other apps handle the situation of a lost connection?
Like kopete. It does not have this problem.
Comment 15 John Andersen 2007-02-24 23:34:20 UTC
Version 1.9.6 Same Problem.  Starting on the THIRD YEAR this bug has been outstanding!!!

FIX THIS PEOPLE!!!

Suspend laptop, restart, and kmail just shows "refreshing folder" when you check mail on any imap folder.  It will NEVER reconnect, it ALWAYS requires a restart.

This bug is not hard to reproduce.  Just simply unplug from the net (by what ever means), and attempt to check mail.   You are then stuck and must shutdown kmail and restart it to get service again. 
Comment 16 Apollon Oikonomopoulos 2007-03-31 16:42:46 UTC
IMHO a good solution would be to have the stopNetworkJobs() DCOP call (and the equivalent "Work Offline" menu action) actually destroy all kio_imap4 child processes as this doesn't seem to happen currently. Personnaly, I'm using ifplugd on my laptop to help me automatically roam between networks, and have automated some tasks (like Kopete's connect and disconnect and mail checking) using DCOP. Unfortunatelly, the aforementioned bug doesn't let kmail work properly: everytime I resume my laptop or I change networks, the IMAP processes hang waiting on a dead or timed-out connection and I have to kill all kio_imap4 instances manually to get things back to work.
Comment 17 Martin Zdila 2007-04-30 14:28:06 UTC
Kubuntu, KMail 1.9.6 and the same problem. After resume from suspend I must allways run this script:
killall kmail; killall kio_imap4; sleep 3; kmail
Comment 18 Thomas McGuire 2007-05-12 14:34:12 UTC
*** Bug 100478 has been marked as a duplicate of this bug. ***
Comment 19 Thomas McGuire 2007-05-12 14:34:12 UTC
*** Bug 145308 has been marked as a duplicate of this bug. ***
Comment 20 Thomas McGuire 2007-05-22 21:40:16 UTC
*** Bug 145766 has been marked as a duplicate of this bug. ***
Comment 21 Thomas McGuire 2007-06-07 16:56:55 UTC
*** Bug 146497 has been marked as a duplicate of this bug. ***
Comment 22 Greg Rundlett 2007-07-02 16:57:57 UTC
I have the same issue with KMail 1.9.6 on Kubuntu.  Whenever I change networks, I can't get past the blue screen.  Even if I try to manually killall kio_imap4.  I've tried to completely restart all my network interfaces with sudo /etc/init.d/networking restart and although that will kill my wired, VPN and wireless connections, that doesn't correct KMail.  I've tried restarting KMail and/or Kontact, but the only solution that works for me is to completely restart the computer.
Comment 23 Ed Swierk 2007-07-02 17:07:01 UTC
In response to comment #22: I find that I have to killall kio_file as well as kio_imap4 to get KMail unstuck after reconnecting the network.
Comment 24 Thomas McGuire 2007-09-01 16:08:06 UTC
*** Bug 149439 has been marked as a duplicate of this bug. ***
Comment 25 Gary Krueger 2007-09-06 19:51:36 UTC
This seems related to Bug 126182 and Bug 135376.  I get kio_imap4 and kio_file processes that hang around after kmail crashes in Bug 135376.  And, Bug 126182 is probably why kio_imap4 processes are hanging around.  Just my 2 cents.
Comment 26 Gary Krueger 2007-10-05 16:52:46 UTC
I find that the kio_imap4 and kio_file processes hang regardless of connection status.  You can run KMail, and exit KMail after viewing the first message.  The processes will still be there.
Comment 27 Hans Meine 2007-11-18 19:36:53 UTC
I am not 100% sure that I suffer from this exact bug; for me the situation is *a little* better: Usually, after suspending/resuming my notebook with KMail left open, KMail is unusable in the sense that I cannot read mail (using IMAP/ssl) anymore.  However, I often get the desired "connection lost" msgbox when I quit KMail.  Still, I am dreaming of KMail seamlessly detecting lost connections and reconnections - ideally, the above-mentioned message box would be replaced with a passive popup, since the internet connection going away/coming back is a very frequent situation with mobile computers.

BTW: When I just tried reproducing the error without a suspend/resume cycle (simply by switching off the WLAN for 5 minutes, and in the meantime changing the IMAP folder), KMail would not show the error, but silently quit, leaving a kio_imap behind.  Maybe the timeout is larger than some minutes.  At least after re-starting KMail I got a new (mostly working) IMAP slave.
Comment 28 Thomas McGuire 2008-04-14 00:40:08 UTC
*** Bug 160636 has been marked as a duplicate of this bug. ***
Comment 29 Thomas McGuire 2008-06-17 19:03:05 UTC
*** Bug 163940 has been marked as a duplicate of this bug. ***
Comment 30 Jesse 2008-06-17 20:21:27 UTC
Note: for me this is no longer very important, because the new cachedimap account type ("Disconnected IMAP") does not exhibit this behavior.  On my Mac OS X 10.5 MacBook Pro, running KDE 3.5.7 (under X11), with a cachedimap account, I can receive mail, put the laptop to sleep, wake it up, and receive more mail.  The need to quit and re-start KMail went away when I switched to the new cachedimap account type.
Comment 31 Philipp 2008-08-05 11:54:54 UTC
I'm not sure if it's the same bug that's bothering me for quite a while, but at least it looks very similar, so I'll try it without opening another bug report.

Description:  About once a day, when either (a) having recently (re-)booted or (b) having resumed from standby/hibernate the system gets totally locked up, colourful horizontal stripes are drawn on screen and the only way to fix this is a reboot.  Rebooting from this state always results in some filesystem errors (I've come to literally love journaling FS) but the lockup doesn't show up again until the next day or so (not reproducible).

All the time I've kmail running with several IMAP accounts open; they're not automatically checking for new mail.  The lockup may appear (mostly after resume) when I explicitly ask kmail to check one of the accounts.  But after a fresh boot, when kmail renews the connection to the IMAP-servers on its own, it may also appear without me interacting.

This is not a hardware-caused problem as I thought earlier because it continues to appear even after I switched to another laptop.

It is, too, not version dependent: I had it with kdepim 3.5.5 that came with my distro (Debian 4.0) and with 3.5.9 that I compiled from source.

And I frequently check /var/log but *never* ever found any traces of this.

Thanks for any help, if you need logs or want me to use a debug build just contact me via email.
Comment 32 Greg Rundlett 2008-09-15 16:41:36 UTC
Still happens with KMail 1.9.9
Comment 33 Cláudio F. Gil 2008-11-05 16:31:00 UTC
I believe I'm also experiencing this in kmail 1.10.1 (KDE 4.1.2 (KDE 4.1.2) "release 48.1", KDE:KDE4:Factory:Desktop / openSUSE_11.0).

My connection if somewhat unstable and often I loose the ability to obtain messages from one imap folder. I can refresh and read mails from other imap folders but the job of "Checking folder-foo" never ends and nothing else on that folder ends. Reselecting the affected folder gets me the system page "Retrieving Folder Contents" that does not disappear.

Exiting kmail does not solve the problem. I have to kill kio_imap4 processes to recover functionality.
Comment 34 George Kiagiadakis 2008-11-23 18:33:33 UTC
*** Bug 173407 has been marked as a duplicate of this bug. ***
Comment 35 George Kiagiadakis 2008-11-23 18:35:38 UTC
Still happens with kmail 1.10.1
Comment 36 Martin 2009-01-10 11:56:29 UTC
Still happens with 1.11.90
Comment 37 Krenar Qehaja 2009-01-28 17:16:22 UTC
I can confirm this, using KMail 1.11.0 (KDE 4.2.0)
Comment 38 Sebastian Voitzsch 2009-02-21 22:10:05 UTC
Same here with latest KMail (KDE 4.2 / Kubuntu 8.10 version). This bug together with #184977 makes KMail unusable for me. What's going on here? This bug is present since 2004, and nobody seems to really care about?
Comment 39 Thomas McGuire 2009-05-27 11:29:36 UTC
*** Bug 194135 has been marked as a duplicate of this bug. ***
Comment 40 Jeff Strunk 2009-07-20 17:19:12 UTC
This bug is in the top 5 hated bugs on the kde bugzilla. Has Till Adam abandoned this bug? Should it be reassigned?

This issue is increasingly important as more people are suspending their computers, and this bug make kmail unusable for those people.
Comment 41 Stephan Diestelhorst 2009-08-04 18:58:57 UTC
Ingo Klöcker <kloecker@kde.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|kmail-devel@kde.org         |

WTF?
Seems like this thing really is abandoned. Time to *really* look for another mail client :-(
Comment 42 Jaime Torres 2009-08-04 20:28:48 UTC
This bug is not abandoned. In fact there are two approaches for it:
* Akonadi imap implementation, that hopefully will solve all of the imap problems.
* fixing the kio_imap (until the akonadi implementation will be ready).

Ingo just removed one mailing list that is used by developers, but not to receive updates about bugs, which is kdepim-bugs@kde.org.
Comment 43 Felix Tiede 2009-08-28 10:25:47 UTC
I don't know, if my problem is exactly the same, but I'm sure it's related.

I'm using IMAP/ssl.

Usually before suspend I put kmail (1.12.0) into offline mode. But connections are obviously not always closed, as sometimes after resume (and changing kmail back to online) I get the problem that mail checking in inbox never completes, often not.

But I sure get this problem, if I change from (Open)VPN to WLAN/LAN or back. I have to restart kmail in those cases.
Comment 44 ZJ Laczik 2009-09-11 17:21:12 UTC
I can confirm that this bug is affecting my version (1.11.2) of kmail (used with kontact 1.4.2 and kde 4.2.2). When combined with a laptop and a wireless network connection this bug is very irritating. The only workaround I have been able to find is the 'killall' approach, as also mentioned by others. Is there anything I can do to help find a proper fix?
John
Comment 45 P Purkayastha 2009-09-22 04:46:04 UTC
The behaviour mentioned in comment #43 is pretty easy to trigger. Simply put kmail in offline mode, wait for 5min and then put kmail back into online mode (using 3 different IMAP accounts here). It is impossible to resume checking of mails. The progress bar at the bottom right of the kmail window never finishes, even if you leave kmail like that for an hour. 

This can be confirmed in kmail-1.12.1, using kde-4.3.1 and kontact-4.3.1.
Comment 46 Thorsten Mühlfelder 2009-10-06 19:36:02 UTC
*** Bug 206580 has been marked as a duplicate of this bug. ***
Comment 47 Thorsten Mühlfelder 2009-10-06 19:41:46 UTC
Kmail 1.12.1 and I'm using 2 IMAP accounts (gmx, gmail):

After reconnecting my internet connection I always get this error message:

Unexpected Program Termination
The program on your computer which provides access to the
imaps://imap.googlemail.com protocol has unexpectedly terminated.
Details of the request:
URL: (unknown)
Date and time: Monday, 7. September 2009 04:10
Additional information: imaps://imap.googlemail.com

After that I have to restart Kmail. Otherwise imap won't work anymore.
For now I had to switch to claws-mail again because this bug really was annoying.
Comment 48 Robert Huitl 2009-10-25 18:42:47 UTC
I had this problem too and found a workaround that works fine for me. Open the Control center -> Network -> Connection options and set the "Server response" timeout from 600 to something like 20 seconds.

Now, resuming the notebook in a different network makes the IMAP and POP3 connections timeout and reestablish properly. Didn't have to restart KMail ever since to work around the "inbox not updated" bug.
Comment 49 P Purkayastha 2009-11-11 01:20:41 UTC
(In reply to comment #48)
> I had this problem too and found a workaround that works fine for me. Open the
> Control center -> Network -> Connection options and set the "Server response"
> timeout from 600 to something like 20 seconds.

Thank you! This is a decent workaround. I had to decrease the timeout to 40sec. It seems to work fine most of the time.
Comment 50 Thorsten Mühlfelder 2010-02-19 12:14:15 UTC
After using claws-mail for a long time again because of this bug, I'm trying Kmail 1.13.0 in KDE 4.4 now. But this bug still exists, I can't believe it :-/ The only change so far: No message about a terminated imap process anymore, only a message that connection was lost and it will be reconnected. But this never happens. The process seems to be stuck. I still have to restart kmail to get it working again.
Comment 51 niburu1 2010-02-24 16:35:44 UTC
It's quite surprising to find this bug still present in KDE 4.4. Both priority and severity ought to be raised, if you were to ask me.
Comment 52 Thorsten Mühlfelder 2010-02-24 16:41:53 UTC
I agree. I'm always forced to use claws-mail again because of this bug. BTW: This bug always appears after reconneting to the internet with a new IP.
Comment 53 Björn Ruberg 2010-02-28 23:55:09 UTC
*** Bug 196155 has been marked as a duplicate of this bug. ***
Comment 54 Mirza 2010-03-01 11:44:03 UTC
This bug is really really annoying, and what is worse, it seems like an easy problem to fix.
It is a shame that is has become a routine for me to close and start kmail again every time my laptop comes out from 'suspend to ram'

Also IMPORTANT note, i see that some of you imply that new IP has something to do with this problem, but that may not be the case. My laptop gets internet right after connecting to wireless, and connection is maintained by router so my IP is always the same!
Comment 55 Petr Burian 2010-03-01 11:56:23 UTC
More 16 days and this bug celebrates 6th anniversary :) It has been almost 5 years I commented this bug. Making note not to forget to add another comment in 2015 ;)

Now more seriously. This bug prevents KMail working and it is 6 years old. Still has Status: NEW and Severity: Normal. I understand you create an Open Source and many of us don't pay for your work. But guys don't you ever used notebook and suspend to RAM/disk?

We appreciate your work and love to be happy Kmail users. But this bug prevents us to be ;(
Comment 56 Andras Georgy Bekes 2010-03-01 12:05:20 UTC
> It is a shame that is has become a routine for me to close and start kmail
> again every time my laptop comes out from 'suspend to ram'
My suspend script closes kmail before suspending and starts it thereafter.

	Georgy
Comment 57 Hans Meine 2010-03-01 12:26:49 UTC
Recall comment #42 - AFAICS from that, this bug will eventually be solved using Akonadi's IMAP implementation.
But you're right, it's strange that such an annoying bug can last so long.  Nevertheless, it happens, not only with KDE.
Comment 58 Hakan Bayindir 2010-03-01 14:18:23 UTC
I don't think that adding an additional connection method is a valid method to leave another existing method buggy. 

If KMail will give up its native connection and only use akonadi services, that might be OK but personally it's very wrong to leave a simple bug because of the existence of alternate method.

About the IP address: I'm experiencing this bug consistently at my home desktop computer which runs on an infinite-lease DHCP assigned IP address.
Comment 59 Juha Tuomala 2010-03-01 14:34:08 UTC
(In reply to comment #58)
> I don't think that adding an additional connection method is 
> a valid method to leave another existing method buggy. 
> 
> If KMail will give up its native connection and only 
> use akonadi services, that might be OK but personally 
> it's very wrong to leave a simple bug because of the
> existence of alternate method.

I second that. Regardless of all Akonadi hype, I have never got it to work myself, nor heard anyone actually using it even I've asked quite a few KDE users about it.

Relaying heavily on server software that is now doomed to die and scatter to many competing projects, its future looks even more confusing.

I'd rather see an working implementation and proof of people adopting that, than driving people to something that doesn't exist yet. That's how it was seen in freedesktop.org which somehow came as a surprise for people pushing it.

Personally I'd like to see native IMAP and Directory resources supported as everything else until there is enough proof of concept from new radical approaches.
Comment 60 Mirza 2010-03-01 14:52:05 UTC
Is there anything else that we, as users can do to help fix this problem in near future. It is standing idle for 5 years!
Comment 61 Björn Ruberg 2010-03-26 10:22:14 UTC
*** Bug 232105 has been marked as a duplicate of this bug. ***
Comment 62 Andreas Stangl 2010-03-26 10:42:47 UTC
I am currently using kmail 1.13.1 with kde 4.4.1, and I must say IMAP now works pretty well :-)
Comment 63 Thorsten Mühlfelder 2010-03-26 12:43:54 UTC
No, 1.13.1 does not work well for me... switched to Claws-mail again :-(
Comment 64 niburu1 2010-03-26 12:55:48 UTC
(In reply to comment #62)
> I am currently using kmail 1.13.1 with kde 4.4.1, and I must say IMAP now works
> pretty well :-)

Not here, unfortunately. Still with Thunderbird.
Comment 65 Frank Steinmetzger 2010-05-30 21:24:17 UTC
Wanting to report this issue, I found this bug. My setup is quite similar to the OP’s: I’m in my uni’s WLAN over vpnc. Occasionally it so happens that the VPN connection kicks the bucket. Or I suspend the laptop and later switch it on again, which makes the VPN connection invalid.

Now I reinstate the connection, but when I select my uni’s IMAP account, I only get the blueish "downloading folder contents" screen, but no message list. So I decide to restart Kontact, but most times it doesn’t even start again if I don’t kill the kio_imap process(es) beforehand.

KDE 4.4.3, KMail 1.13.3, Gentoo.
Comment 66 Jesse 2010-06-01 08:21:25 UTC
Has everyone who is still having this problem (Frank, Thorsten, Mirza, Juha, ...) tried setting up KMail with the Cached IMAP account type?  Or is that not a good option for some reason?  I found that switching to the Cached IMAP account type solved this problem for me, and I suspect this is a pretty good workaround for most people.  I'm afraid I don't have KMail in front of me at the moment, but the process is something like this: create a new email account, and for the type select "cached IMAP".

To the KDE developers: maybe a solution to this bug would simply be to hide the old, non-cached IMAP account type, and steer people to cached IMAP instead.
Comment 67 Hakan Bayindir 2010-06-01 13:45:37 UTC
I'm currently having this problem with my KMail setup (2 IMAP + 1 POP3) pretty regularly if not always after a suspend/resume cycle. My connections arbitrarily broke and re-establish (generally oafter resume but sometimes connections break again) and my internet connection is not shaky by any means (with an uptime comparable to a top iron Linux server).

OTOH, the aim is not to workaround the bug. The aim is to make KMail work as expected. I expect if a feature is included in a software, it should work. It's there to work, not to be worked around. I'm a computer programmer and I strongly object the perspective of dodging bugs.
Comment 68 Robert Huitl 2010-06-01 15:25:24 UTC
I think no KDE developer cares about this bug because a lot of functionality offered by KMail is being moved to Akonadi. AFAIK, this will also be the case for the POP3, IMAP, etc. protocol support, so any work on the current implementation will probably be obsolete in a couple of weeks.
Comment 69 Hakan Bayindir 2010-06-01 16:18:54 UTC
In that case I think it should be stated clearly so users can migrate. Neither in this ticket nor elsewhere I cannot see any announcement or help about this issue. The only thing I object is leaving a feature in orphan state. If it's obsolete, please tell us and remove the feature completely (don't hide) so we (at least I) can move on and test that implementation.
Comment 70 niburu1 2010-06-01 16:29:00 UTC
Should we get our hopes up? The feature hasn't worked properly for years and there's no signs of a bug fix. Now we're finally promised that, with the move to Akonadi, everything's going to be hunky dory? I wonder how long that will take to properly implement.
Comment 71 Robert Huitl 2010-06-01 16:39:17 UTC
I'm not in the position to promise anything here, so don't get your hopes up too much ;-) I do have some strong non-KDE-development background, though, and I think that the whole PIM/email/sync(!) story might turn out quite nicely after all.

The KDE 4.5 Beta announcement (http://kde.org/announcements/announce-4.5-beta1.php) states: "Initially planned for KDE SC 4.5.0, the KDE PIM team have decided to delay the release of the Akonadi-based KMail for one month. [...] Akonadi will centralize syncing and caching of PIM data, deliver wider support for groupware servers and makes handling PIM data, such as contacts, calendaring and email more efficient by sharing it across applications."
Comment 72 Unknown 2010-07-30 18:22:40 UTC
Created attachment 49689 [details]
the definition of KIpAddressInUse
Comment 73 Unknown 2010-07-30 18:25:19 UTC
Created attachment 49690 [details]
the implementation of KIpAddressInUse
Comment 74 Unknown 2010-07-30 18:27:05 UTC
Created attachment 49691 [details]
test for KIpAddressInUse
Comment 75 Unknown 2010-07-30 18:28:07 UTC
Created attachment 49692 [details]
the diff for kdepim
Comment 76 Unknown 2010-07-30 18:28:42 UTC
Created attachment 49693 [details]
the diff for kdepimlibs
Comment 77 Unknown 2010-07-30 18:30:21 UTC
unfortunatly it might be too late to integrate the patch to the next release.
Comment 78 Unknown 2010-07-30 18:34:03 UTC
As written at comment #42, the bug isn't abandoned. I whish to let you test my
solution for it, until the migration to akonadi is done.

Now, kio_imap process doesn't hang anymore on IP-Address change.

To solve it I propose to introduce the class KIpAddressInUse.
With this class, we can:
- save the actually used IP-address by the connection
- check if the saved IP-address is still available

kmail only need to:
- check the availibity of the IP-address each time
    _before_ using the socket to write to the IMAP-server and
    _before_ using the socket to read from the IMAP-server
If the socket is not available kmail:
- marks the error as ERR_CONNECTION_BROKEN and close the connection

Then kmail has to:
- check the state of the connection after each use of the socket
  and return from the actual method/function ie.:
    if (getState () == ISTATE_NO) return ;

The rest of the work is already present at release 4.4:
- SlaveBase::error
    sends a signal to the application.
- The application sends a DISCONNECT to the slave
    and cleans some data to be ready for the next connection.

As defined at imapaccountbase.h:533
    /** used to send a noop to the slave in regular intervals to keep it from
disonnecting */
    QTimer mNoopTimer;
and used at kmacctimap.cpp:63
    mNoopTimer.start( 60000 ); // // send a noop every minute
The timer sends a NOOP-command after this time and defines two periods:
One before the NOOP, one after.

Let us have a look to what appens when the IP-Address has changed.

If the user makes an action with network traffic, this leads to a
detection of the IP-Address and "killAllJobs" throws all the jobs and
clears the connection. The user must try again.

If the user action occurs after the NOOP, all is already clear.

The attachments are base on branches/KDE/4.4 files.
Comment 79 Samat Jain 2010-08-01 22:29:54 UTC
Maurel, I haven't tried your patch, but one question after about your approach and looking at the source: how well does this work behind NAT?

If an upstream router, the one with a public IP, changes that IP, then a client computer's connections behind that router's NAT (with a non-routable IP address) will all still be invalid. The client computer's non-routable IP address does not change so none of the code added by this patch will catch this situation.

Correct me if wrong, but this patch won't catch such a situation?
Comment 80 Unknown 2010-08-02 15:24:18 UTC
you are right. 
I have not test this situation, which is new.
Comment 81 Unknown 2010-08-02 17:47:56 UTC
I make the test, without my patch.
Kmail works already pretty well on this situation.
Comment 82 Carsten Pfeiffer 2010-08-19 21:57:11 UTC
Very annoying. I have to killall -9 kio_imap4 twice a day.
Comment 83 Diego 2010-09-07 10:47:30 UTC
(In reply to comment #77)
> unfortunatly it might be too late to integrate the patch to the next release.

There will be another kdepim 4.4 release, please consider getting the patch in (ask for the KDEPim team for help for testing):
http://www.kdedevelopers.org/node/4320
Comment 84 Aaron Lewis 2010-09-07 13:52:11 UTC
I'm very glad to say , i've verified this patch recently , and also i'm suffering from this problem , see the default timeout for 600 seconds , thus even internet connection is down , kmail never knows before it thought it's `timeout'.

It's very simple to check it , get offline and online again , kmail dies until 600 seconds later , and reconnects after , it's definitely a bug.
Should it be coded deadly here , what do you think ?
And everytime an action is triggered , we would check of availability of connection.
( i'm not criticize kmail anyway ;-P )

-    waitForResponse(600);

Guy had just provided a better way of checking timeout , see detailed in patch file , anyway , please confirm his work like what i've done if you thought it helps too ;-)

Thanks !
Comment 85 David Faure 2010-10-22 00:43:04 UTC
Dropping the connection as soon as the IP address goes away is not a good solution. In particular, it would make kmail lose the connection more often when using it in a flaky (unstable) network connection, for instance airport wifi. The way it works with TCP 
connections is that if you get the same IP address again later, the communication will resume. But if kmail drops the connection immediately when losing the IP, then it behaves 
worse than currently, in such a situation...
I talked to the Qt networking people today, and they agreed.

How about just
1) making the timeout shorter (60s instead of 600s or responseTimeout() which defaults to 600 anyway) (at least with responseTimeout() the users can adjust the timeout in a config file, but the default of 10 minutes makes this inpractical for this issue, I think), and:
2) keeping the error handling part of your patch

Does this already make things better?
Comment 86 Unknown 2010-10-22 18:32:22 UTC
1) the proposal of a shorter timeout was already made at comment#48 and #49.
   This can be done *now*, this will help lot of users.
2) the error handling has to be rewrite under this aspect, I'll work on this.
Comment 87 Unknown 2010-11-10 18:28:06 UTC
I have made an analyse of the problem, one can read at the 
kde-pim@ on 2010-10-31. The bug 77862 is:
- not an IMAP problem, all other protocols would have it too,
- not a kmail problem, all other front-end mail-tool also,
- not a KDE  problem.
This is an IP-v4 problem we cannot solve. All applications which have 
(long time) connections could have the problem at any time.

Let me quote a white paper from "www.isode.com".
    http://www.isode.com/whitepapers/imap-idle.html

Beside my analyse they publish the same :

"... Another practical problem is that current phone networking technology 
will lose IP network connectivity from time to time, and this will need to be automatically re-established, and the IMAP connection re-established if this is lost due to a long network failure. ..."

We can only propose some workarounds. 
Even my proposals were workarounds. The best one is to change the 
timeout at waitForRead, as proposed, a long time ago, by Robert Huitl on 2009-10-25
   https://bugs.kde.org/show_bug.cgi?id=77862#c48

and improve the errorhandling after that.

As David Faure notices at Comment #85, the situation at airport wifi is not very stable. The timeout should not be too short. As the connection comes again, we might get the same IP-address as before.
I propose a value of 20 seconds.
Comment 88 Michael Kreitzer 2010-11-11 17:57:30 UTC
There are two main issues when it comes to dead connection detection, and both place the burden of dealing with them on the client to some degree so this is a bug for the client if it fails to handle them gracefully.

The first issue is detecting dead daemons. The symptoms will be that tcp packets are sent and the proper ACKs returned indicating the connection is fine, but the server does not respond. This is resolved via timeouts as you mentioned. 

The second is detecting dead connections in protocols where traffic may not be sent or received for long periods of time. This is generally resolved by regular "pings" if the protocol supports it. IMAP has a NOOP command specifically for this. In some cases the connection may be immediately reset by a firewall, router, or the server tcp stack. In other cases a tcp timeout may be required waiting for the ACK in which case a timeout may be issues by the remote end firewall, router, server or the local tcp stack. Both error codes are ICMP so if an ultra paranoid server admin is blocking these on either end the client will may to rely on a client side timeout making this the same issue as the former.

I've seen kmail/kio_imap4 not timeout indefinitely hanging on a dead connection. This is one issue this bug has touched.

Another is kmail/kio_imap4 will not properly re-establish a connection even when a connection is reset or it otherwise detects a failed connection. I'll get the error dialog mentioning the connection was lost and it will try to reconnect, but I'm never able to browse my mail again after that until I close kmail and manually kill off the hung kio_imap4 processes.

Both related issues need to be addressed to correct the central issue this bug report is about.
Comment 89 Christophe Marin 2011-02-15 00:58:16 UTC
*** Bug 254254 has been marked as a duplicate of this bug. ***
Comment 90 J.D. 2011-06-10 09:47:54 UTC
Same problem here: FIX it, please. Many can't use this reliably in the office.

My workaround is offline mode, which works most of the time if I don't press check mail after connection loss and re-establishment.

You can do this automatically via dbus:

#go offline
qdbus org.kde.kmail /KMail org.kde.kmail.kmail.stopNetworkJobs
#wait for 30 seconds
sleep 30
#go online again
qdbus org.kde.kmail /KMail org.kde.kmail.kmail.resumeNetworkJobs
#check accounts for new mails
qdbus org.kde.kmail /KMail org.kde.kmail.kmail.checkMail
Comment 91 René Krell 2011-10-03 11:19:13 UTC
Similar to me:
KMail 4.7.1 doesn not show message contents, but just a
"Retrieving Folder Contents
Please wait ..."
Interesting: This happens on all kinds of Akonadi resources, not just on IMAP.
Akonadi uses an external MYSQL server here and does not report to have problems with it.
Messages seem to be retrieved and filters executed, just displaying messages doesn't work.
Comment 92 Christophe Marin 2011-10-03 11:46:28 UTC
> Similar to me:
> KMail 4.7.1 [...]

Unrelated to this report
Comment 93 Aaron Williams 2012-01-30 21:51:41 UTC
I am seeing this same behavior with KMail 4.8.0. It is extremely frustrating! It is as if the IMAP support breaks if there are parallel activities taking place or if something happens to the connection. I have several email filters applied to IMAP in order to keep several mailing lists from flooding my inbox.
Comment 94 Aaron Williams 2012-01-30 22:35:58 UTC
I have some additional debugging information that might be helpful.

In this case, I clicked on a never-opened email which basically hung.  Eventually the debug log showed "request for item 50965 "31165" failed..."

At this point I clicked on a previous email which loaded immediately and the next email which previously said it was waiting and it opened immediately.

It is as if the notification that the email eventually loaded is lost (or it timed out waiting for the notification).

[Clicked on unopened email]

kontact(1763)/kdewebkit KWebPage::acceptNavigationRequest: url:  QUrl( "file:///" )  , type: 5 , frame: QWebFrame(0x8ce2b0)
akonadi_imap_resource_1(1824)/kdepimlibs (kimap) KIMAP::StoreJob::handleResponse: We asked for UID but the server didn't give it back, resultingFlags not stored. 
void Nepomuk::Query::QueryServiceClient::close() 
posting retrieval request for item 50965  there are  1  queues and  0  items in mine 
request for item 50965 still pending - waiting 
processing retrieval request for item 50965  parts: ("RFC822")  of resource: "akonadi_imap_resource_1" 
31199 +FLAGS (\Deleted)
akonadi_imap_resource_1(1824)/kdepimlibs (kimap) KIMAP::StoreJob::handleResponse: We asked for UID but the server didn't give it back, resultingFlags not stored. 
kontact(1763)/kdewebkit KWebPage::acceptNavigationRequest: url:  QUrl( "file:///usr/share/kde4/apps/kmail2/about/main.html" )  , type: 5 , frame: QWebFrame(0x8ce2b0)
kontact(1763)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x3b2ba80)
kontact(1763)/kio (KIOJob) KIO::TransferJob::slotFinished: KUrl("file:///usr/share/kde4/apps/kmail2/about/kmail.css")
kontact(1763)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::TransferJob(0x3b2ba80) KIO::Slave(0xe22310)
kontact(1763)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x2498e80)
kontact(1763)/kio (KIOJob) KIO::TransferJob::slotFinished: KUrl("file:///usr/share/kde4/apps/kmail2/about/top-left-kmail.png")
kontact(1763)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::TransferJob(0x2498e80) KIO::Slave(0xe22310)
kontact(1763)/kio (Slave) KIO::Slave::kill: killing slave pid 1806 ( "file://" )
kontact(1763)/kio (Slave) KIO::Slave::kill: killing slave pid 1801 ( "file://" )
kontact(1763)/kio (Slave) KIO::Slave::kill: killing slave pid 1805 ( "file://" )
kontact(1763)/kio (Slave) KIO::Slave::kill: killing slave pid 1802 ( "file://" )
akonadi_kabc_resource_3(1828)/kio (Slave) KIO::Slave::kill: killing slave pid 1882 ( "ldap://xxxxx.xxxxx.com" )
akonadi_kabc_resource_2(1827)/kio (Slave) KIO::Slave::kill: killing slave pid 1885 ( "ldap://xxxxx.xxxxx.com" )
akonadi_kabc_resource_1(1826)/kio (Slave) KIO::Slave::kill: killing slave pid 1905 ( "ldap://xxxxx.xxxxx.com" )
akonadi_kabc_resource_6(1831)/kio (Slave) KIO::Slave::kill: killing slave pid 1906 ( "ldap://xxxxx.xxxxx.com" )
akonadi_kabc_resource_5(1830)/kio (Slave) KIO::Slave::kill: killing slave pid 1907 ( "ldap://xxxxx.xxxxx.com" )
akonadi_nepomuk_feeder(1837)/kdecore (trader) KMimeTypeTrader::query: query for mimeType  "inode/directory" ,  "AkonadiNepomukFeeder"  : returning  0  offers
akonadi_nepomuk_feeder(1837) FeederPluginloader::feederPluginsForMimeType: No feeder for type  "inode/directory"  found 
akonadi_nepomuk_feeder(1837) ItemQueue::fetchJobResult: Not all items were fetched:  8 17 
continuing 
request for item 50965 "31165" failed: "Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken." 
ItemRetrieverException :  Unable to retrieve item from resource: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
posting retrieval request for item 50965  there are  1  queues and  0  items in mine 
request for item 50965 still pending - waiting 
processing retrieval request for item 50965  parts: ("RFC822")  of resource: "akonadi_imap_resource_1" 
akonadi_imap_resource_1(1824)/kdepimlibs (kimap) RetrieveItemTask::onMessagesReceived: MESSAGE from Imap server "31165"
akonadi_imap_resource_1(1824)/kdepimlibs (kimap) RetrieveItemTask::onMessagesReceived: Has Payload:  true
continuing 
request for item 50965 succeeded 

[screen not updated, still shows "Retrieving Folder Contents". click previous email]

kontact(1763)/kdewebkit KWebPage::acceptNavigationRequest: url:  QUrl( "file:///" )  , type: 5 , frame: QWebFrame(0x8ce2b0)
kontact(1763)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x347e260)
kontact(1763)/kio (KIOJob) KIO::TransferJob::slotFinished: KUrl("file:////usr/share/kde4/apps/libmessageviewer/pics/quicklistOpened.png")
kontact(1763)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::TransferJob(0x347e260) KIO::Slave(0xe22310)
void Nepomuk::Query::QueryServiceClient::close()

[previous email shown, click next email]

kontact(1763)/kdewebkit KWebPage::acceptNavigationRequest: url:  QUrl( "file:///" )  , type: 5 , frame: QWebFrame(0x8ce2b0)
void Nepomuk::Query::QueryServiceClient::close() 
31165 FLAGS (\Seen)
akonadi_imap_resource_1(1824)/kdepimlibs (kimap) KIMAP::StoreJob::handleResponse: We asked for UID but the server didn't give it back, resultingFlags not stored.
Comment 95 Aaron Williams 2012-01-30 22:37:00 UTC
I forgot to mention that this is with the very latest OpenSUSE release which appears to include all of the latest GIT patches.
Comment 96 David Faure 2012-02-13 14:07:56 UTC
I fixed all the issues that I could find in akonadi_imap_resource (some for 4.8.0, some after 4.8.0, therefore for 4.8.1), when losing connection, or switching to another network, or after suspend/resume, etc. This makes the main report here fixed in 4.8.1.

Aaron Williams: you are either hitting one of the bugs I fixed, or your resource is busy for a very long time with something else, which needs to be investigated (e.g. using RMB / Show  task list and Show Notification Log, in the first tab of akonadiconsole). Please file a separate bug report for this issue, it doesn't seem related to "connection status change".

Everyone else: kio_imap is not used anymore in the akonadi version of kmail (i.e. KDEPIM >= 4.7, but please upgrade to 4.8), which is why I'm talking about akonadi_imap_resource, which doesn't have this issue (anymore).