Bug 63353 - Disconnected IMAP wipes email
Summary: Disconnected IMAP wipes email
Status: RESOLVED WORKSFORME
Alias: None
Product: kmail
Classification: Applications
Component: disconnected IMAP (show other bugs)
Version: 1.10.1
Platform: Compiled Sources Linux
: NOR grave
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords: triaged
: 68512 134292 173115 (view as bug list)
Depends on:
Blocks:
 
Reported: 2003-08-28 08:16 UTC by lakeland
Modified: 2009-12-26 21:22 UTC (History)
16 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description lakeland 2003-08-28 08:16:22 UTC
Version:            (using KDE Devel)
Installed from:    Compiled sources
Compiler:          gcc 2.95 
OS:          Linux

I'm running kmail 1.5.9 with disconnected imap.  Generally this works quite well, with the occasional crash.  Recently kmail crashed while I was editing my subscription options and shortly afterwards all my inbox archive was gone.  

Now, no complains -- I'd be running stable software if I wanted stability.  And yes, I do have backups.  But this seems pretty serious and really needs to be fixed before 3.2 is released.  I'm sorry I don't have more reproducable steps. PS: Disconnected imap is awesome!
Comment 1 lakeland 2003-10-07 21:17:45 UTC
Hi, just letting you know I tried to reproduce the bug today with some success.  Steps: 
 
Create a new disconnected imap account (I deleted my normal IMAP account).  Choose 
synchronise.  Have something go wrong, causing kmail to abort midway through synchronising. 
 
In my case I was trying to read mail but I had a large folder to synchronise (several hundred 
deletes), and so kmail didn't get a chance to finish synchronising. 
 
The IMAP server is currently bincimap, though when I first reported the bug it was courier imap.  
Last time I ended up with some folders wiped, but this time I appear to have just wiped the 
bodies of some messages. 
Comment 2 Ingo Klöcker 2003-10-07 23:57:22 UTC
Subject: Re:  Disconnected IMAP wipes email

Disconnected IMAP in KMail 1.5.9 is very buggy. We are working on 
porting all the bug fixes that went into the Kroupware version of KMail 
to KMail HEAD. Until these bug fixes are in you should better not use 
disc. IMAP (or you should take extra care so that you don't risk losing 
important mail).

Comment 3 lakeland 2003-10-08 05:32:20 UTC
Subject: Re:  Disconnected IMAP wipes email

Cool.  I'll let you know if I can still reproduce the bug once you've got the 
fixes ported.

Corrin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/g4Ufi5A0ZsG8x8cRAv2QAKCYimoyMI2+C2NRRgkAzWXGYiBTKQCeLFhP
/av9D9rEXbuLU4UDXnTPt4s=
=5vud
-----END PGP SIGNATURE-----

Comment 4 bo 2003-10-28 18:04:11 UTC
They have now been ported, and I can't trigger your bug currently (I could earlier). However, I only have access to cyrus imapd, so I would be grateful if you can confirm to me that we did indeed fix it.
Comment 5 lakeland 2003-10-28 19:55:24 UTC
Subject: Re:  Disconnected IMAP wipes email

>
> ------- Additional Comments From bo@sonofthor.dk  2003-10-28 18:04 -------
> They have now been ported, and I can't trigger your bug currently (I could
> earlier). However, I only have access to cyrus imapd, so I would be
> grateful if you can confirm to me that we did indeed fix it.

Cool, will try as soon as I can convince CVS to compile again -- it really
doesn't like playing with gcc 3.3.2

Corrin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/nrvji5A0ZsG8x8cRAmzKAJ4vz+P0dJ169wf7id07qNtPLNO1NQCfT4f0
o0LyeCFLmuSblUYVujyJ4Ak=
=FMq4
-----END PGP SIGNATURE-----

Comment 6 kevin geiss 2003-11-07 23:21:58 UTC
I just got half the messages in my inbox deleted! I had around 700 messages. thank goodness for backups. I'd say this problem still exists.

I've been using imap for quite a while, and tried the new disconnected imap today. (I left the regular imap account active too). 

I'm using gentoo's 3.2.0_beta1 ebuilds.
Comment 7 lakeland 2003-11-11 02:50:09 UTC
Ok, I just tried again and had little trouble wiping my email.  It seemed to be something to do with a failed synchronisation.  I guess the steps were:

Attempt to sync
Fail for some reason (too many messages, whatever) and segfault, so our cache says there are no messages in that folder
Restart kmail and attempt to sync again -- this time telling the server to delete all messages.

Corrin
Comment 8 Carsten Burghardt 2003-11-18 22:48:53 UTC
Not fixed according to Kevin Geiss (original report 68512)
Comment 9 Carsten Burghardt 2003-11-18 22:49:48 UTC
*** Bug 68512 has been marked as a duplicate of this bug. ***
Comment 10 Derick Swanepoel 2004-05-26 00:30:23 UTC
Today, using KMail 1.6.2 on KDE 3.2.2, compiled from source, KMail wiped all my mail. When I synced, KMail started trying to download mail in an infinite loop and eventually I had to cancel it. When I synced again it reported an error which unfortunately I can't remember. Either way, I chose "No" because what it seemed to want to sounded dangerous. The next time I synced, it deleted all 460 messages in my inbox.

I had thought that this particular issue with dimap was fixed after reading about it in the CVS digest of 26 Mar 2004, but obviously I was mistaken, or it has not been backported to 3.2 yet? Now I have lost tons of extremely important mail. Disconnected IMAP should not merely be marked "experimental" in KMail, it should have a big fat warning that says "BROKEN!!!" I'm sorry, but eating people's inboxes is not cool, and I am very unhappy about this.
Comment 11 Corrin Lakeland 2004-06-16 07:03:33 UTC
Hi, just letting you know that I had a good go at reproducing this bug today and was unable to.  I wonder if it has been fixed?
Comment 12 Corrin Lakeland 2004-06-17 00:48:23 UTC
Ok, I've just spent a couple hours trying to reproduce this bug.  I deliberately did lots of dangerous things, moving and deleting folders and messages while checks were occuring.  I also ran kmail on two machines at the same time and tried to get them out of sync with each other.  I did manage to get two error messages:

Error while uploading folder
Could not make the folder core on the server.
An attempt to create the requested folder failed.
Details of the request:
URL: (kaore i te mohio)
Date and time: Thu, 17 Jun 2004 10:28
Additional information: newimaps://lakeland@go.org.nz:993/INBOX.kde/core

However, the folder was still there (I managed to get this error message for a few other folders, but in no case was mail lost).

Some time later I got the message:

Error while deleting messages on the server: 
Could Not Delete Resource
An attempt to delete the specified resource newimaps://lakeland@go.org.nz:993/INBOX.archive.in.2003.10_(Oct)/;UID=0,0,0,0,0,0 failed.

With two options (continue and cancel/abort)

This is the error I remember getting shortly before kmail wipes the email.  I can see no reason why kmail would be trying to delete that folder.  However, it didn't delete that folder (as opposed to my other bug report when running normal IMAP) so no mail was lost. 

Since I was unable to reproduce the bug, I suspect it has been fixed.  Well done!
Comment 13 lakeland 2004-06-17 23:47:16 UTC
Ok, I have tried everything to trigger this bug with no success.  Since nobody else has piped up and said they can trigger it as well I am closing the bug. This comment applies to kmail as distributed in KDE 3.3.  I think kmail from 3.2 is vulnerable.

Of course, dimap seems to be somewhat slower than it used to be -- 4 mins to do a check compared to 20 seconds for imap, but reliability is worth the extra time.  

Derick, Kevin.  You're welcome to have another go and reopen if you have any problems.
Comment 14 Derick Swanepoel 2004-08-24 17:40:30 UTC
I have lost another 500+ mails with KMail 1.6.82 (KDE 3.3 beta2) using disconnected IMAP.

Here is what happened:
1. I had more than 500 *unread* messages in my inbox, and checked mail via my dial-up connection.
2. After receiving about 50 messages KMail's CPU usage went up to about 100% and became completely unresponsive. (This may be related to bug #81156)
3. When KMail did not return to normal after a few minutes, I killed it.
4. When I started KMail again, the new messages in my inbox were all empty, and although the message list displayed the subject, sender, etc., upon viewing the message these changed to "No Subject" or became empty.
5. When I checked mail again, KMail deleted all my new mail except for the few (empty) messages it had downloaded.

Please reopen this bug because it seems like KMail can still cause mail loss if it dies before completing mail sync. I feel like an idiot for trusting my mail with KMail again.
Comment 15 lakeland 2004-08-24 22:29:55 UTC
Very sorry to hear your story.  I should add that I've used it every day for 
months now, including simultaneous checking from different computers and 
folders with > 500 messages.  I have not seen any faults during all that 
time.

Corrin
Comment 16 Chris Browet 2004-08-25 01:03:11 UTC
It happened to me too no latter than yesterday, after upgrading kdepim to CVS. Somehow, kmail succeeded in emptying all my IMAP folders after I tried to do a complete refresh (not too sure of the complete sequence of event, though). As I also experienced an empty messages syndrome (only with the korganizer/kadressbook kolab folders, the mesages are there but completely empty), I resolved to go back to connected IMAP (looks like it works with kolab now) hoping for more stability.
I must say, though, that before the upgrade dimap, was quite stable since 3.3b2, besides the empty messages thingy.

I would say dimap probably needs in-depth crash testing.
Comment 17 Randall Wood 2004-09-21 16:31:02 UTC
I also lost some mail via a different path.  I was having trouble with dimap and decided to delete that account and reopen it using a regular IMAP account.  Configure Kmail --> Accounts --> Receiving.  Removed the DIMAP account and recreated it as IMAP.  When I logged back in to IMAP, all my messages had been deleted.

Thanks for looking into this.
Comment 18 Mathieu Jobin 2005-06-07 07:19:38 UTC
SOLVED ???

running KDE 3.4

I lost VERY IMPORTANT messages and I am very UPSET.

Comment 19 Mathieu Jobin 2005-06-07 07:22:07 UTC
forgot to add me to CC

Comment 20 Corrin Lakeland 2005-06-07 08:19:13 UTC
On 7/06/2005, at 5:19 PM, Mathieu Jobin wrote:

Odd.. I've been running dimap for _ages_ now, and haven't lost email 
for I guess a year or more.

Are you using Courier?

Corrin
Comment 21 Mathieu Jobin 2005-06-07 17:44:41 UTC
I'm using different servers

my main email is on dovecot
I have other account on courrier

basically, I just move emails from inbox and from search result to folders and as they were transfering the number of email shown next to the folder were going crazy, up and down, never saw that before.

i'm sad, but its been a week or two now, so i live a bit better with this lost.

Comment 22 Corrin Lakeland 2005-06-08 00:19:15 UTC
On Wed, 08 Jun 2005 03:44, Mathieu Jobin wrote:

Well, after your message I decided to have another go at reproducing it, and 
sadly I was able to :-(

Here is what I did:
	DDOSed my DSL connection, so that it randomly drops TCP connections.  This is 
not hard with my ADSL router, simply running a p2p app will do it.

	Kept checking mail through kmail (dimap) all of that time.

	By the next day my folders were no longer hierachical in kmail (the hierachy 
still existed on the server), and when I clicked on the inbox I found all 
mail older than 5PM yesterday was deleted from my inbox.  I did not notice 
any other folders affected.

I don't know what the problem is.  I managed to reproduce the bug, but I 
cannot reproduce it easily.  My best guess is that it has something to do 
with the socket being closed at a critical point, e.g. kmail clears its 
message list, requests a list from the server, gets a socket closed/EOF, is 
left with an empty message list, deletes all messages in the local cache, 
resynchronises in ten minutes -- telling the server to delete everything.  
But really, I'm just making that up -- for instance that doesn't explain why 
my inbox was affected rather than other folders :( 

Corrin
Comment 23 Till Adam 2005-06-08 09:38:40 UTC
On Wednesday 08 June 2005 00:19, Corrin Lakeland wrote:

> Well, after your message I decided to have another go at reproducing it,
> and sadly I was able to :-(


That's good. If you have a) a test user b) a way to reproduce it somewhat 
reliably c) a build with full debugging and d) some time, please, please, 
please contact me via mail or in #kontact on irc.kde.org, so I have a chance 
of tracking it down. I can't reproduce it, and I really need help with that, 
otherwise there is no way to fix it.

Thanks,

Till
Comment 24 Silas De Munck 2005-06-27 00:18:08 UTC
Kmail deleted the last 200 messages I received with dimap. I'll try to reproduce and contact you when it did it again.
Comment 25 Ismail Donmez 2006-05-05 19:25:21 UTC
I have one inbox with 20 or so sub folders. I have filtering enabled and mail is moved to subfolders based on filters.

Here is how to reproduce:

- Check mail
- Note that one of the subfolders have new mail
- Click to that folder and read new mail
- KMail wipes the whole thread ( including the new mail ) when syncing folders

using KMail 1.9.1 or 3.5 SVN reproducible always. Sometimes mail is wiped _without_ any user interaction. New mail get lost when syncing.
Comment 26 Ismail Donmez 2006-05-05 23:15:25 UTC
Looks like it leaves the message on the server but the message is removed from the listview until you check mail again.
Comment 27 Richard Chudoba 2006-05-07 16:26:59 UTC
*** This bug has been confirmed by popular vote. ***
Comment 28 Orhan 2006-06-24 04:56:05 UTC
I can recreate thsi bug also by changing the time backwards about a half hour.  I had my ntp client messed up, and when i fixed the time (2x) many messages were lost from my INBOX
Comment 29 Unknown 2006-09-21 14:28:50 UTC
I also still see this behavior.
There are two different scenarios when emails are lost:

1. kmail throws a lot of errors similar to this:
Mails on the server in folder ** were deleted. Do you want to delete them locally? UIDs: 198

sometimes these messages go away after some time, sometimes not, only if you hit "OK". If they disappear by itself, it cannot be assured whether the mail is really lost or not.

2. new emails that were put in the folder locally are never synchronized, so that kmail assumes they were deleted and deletes them locally, too.

If you try to solve this problem and try to use "refresh local IMAP cache", it gives you the following message: "Are you sure you want to refresh the IMAP cache? This will remove all changes that you have done locally to your IMAP folders." which indicates that there is one function missing: initiate a synchronization between cache and server instead of using the server as reference. Some collision resolution would be nice, too, how do I know which message is UID xyz when I can't read it and the numbering is not used by kmail?

Ciao
/dirk
Comment 30 Luka Renko 2006-10-17 13:34:34 UTC
I have seen the same problem as reported in comment #28: my time was wrong by two hours (due to having dual boot where one expects BIOS time in UTC and the other in local time). I have modified the time back for two hours and after that I have experienced that my Inbox (on MS Exchange Server 2003) was reduced from 3800+ e-mail to barely 50+ e-mails.

Funny thing is that e-mails were recovered by undelete feature of Exchange server, but Kmail keeps repoving them on sync. Also Refresh Cache feature crashes always - see bug 135804

I am running up-to-date Kubuntu Edgy (release candidate) with KDE 3.5.5
Comment 31 George Kiagiadakis 2008-08-31 15:43:38 UTC
I tried to reproduce this bug with kmail 1.10.0 svn r854573 and I didn't lose any mail. I tried the steps from comment 28 and also tried to kill kmail while it was syncing and reopen & resync. No mail lost, no error messages. Looks pretty stable to me.
Comment 32 George Kiagiadakis 2008-08-31 15:44:27 UTC
Err, sorry I meant comment 25, not 28
Comment 33 Wade Berrier 2008-09-23 00:52:50 UTC
I just lost my gmail 'Sent Mail' folder using kontact 1.9.10.

Was there something specifically that changed between that version and r854573?
Comment 34 Pascal d'Hermilly 2008-09-24 00:57:43 UTC
@wade: I think you need to state your kmail version, not kontact version
Comment 35 Wade Berrier 2008-09-24 02:16:18 UTC
@pascal: kmail version 1.9.10.
Comment 36 George Kiagiadakis 2008-10-18 21:47:09 UTC
*** Bug 173115 has been marked as a duplicate of this bug. ***
Comment 37 George Kiagiadakis 2008-10-18 21:48:30 UTC
According to bug 173115, this is still valid on kmail 1.10.1 :(
Comment 38 Axel Braun 2008-10-22 09:50:38 UTC
I noticed the same problem when synchronizing mails and the line (WLAN) drops. All content of an account gets lost locally, and in some cases on the server as well (KMail 1.9.10 on KDE 3.5.10)
Comment 39 Christoph Lange 2008-10-22 09:57:40 UTC
(In reply to comment #38)
> I noticed the same problem when synchronizing mails and the line (WLAN) drops.
> All content of an account gets lost locally, and in some cases on the server as
> well (KMail 1.9.10 on KDE 3.5.10)
I should point out that my mails were also lost on the server (bug 173115).  I think I had a continuous connection at that time.
Comment 40 Malte S. Stretz 2008-11-27 12:06:02 UTC
*** Bug 134292 has been marked as a duplicate of this bug. ***
Comment 41 Naveen Sivanandam 2009-01-22 10:40:12 UTC
I can confirm this on kmail 1.11 (kontact 1.3 kubuntu binaries)
Comment 42 Axel Braun 2009-04-22 11:54:49 UTC
As I just got an inbox wiped :-( it reminded me on that bug...is there any progress on this? Although it is really a *big* problem, no one seems to take it serious
Comment 43 Mathieu Jobin 2009-06-13 17:40:59 UTC
dIMAP should be disabled until fixed. this is a dangerous bug.
if not disabled, a warning to the user should be displayed saying it is broken and might kills emails.
Comment 44 Jaime Torres 2009-06-19 13:17:07 UTC
@Axel. If you look at the source code, you'll notice quite a lot of #ifdef LOST_MAIL (or something similar). So, yes, this bug is taken care of.

My guess, repeat, my guess, is that it will not be really solved until kmail totally uses akonadi, because solving the actual root cause of this problem probably will cause a lot more problems.
Comment 45 Thomas McGuire 2009-06-19 14:00:28 UTC
Disconnected IMAP is _not_ broken. The #ifdefs mentioned by Jaime are from debugging a problem that is solved already.
Saying something vague like "got my inbox wiped" also doesn't help here. If there really is a problem, we need exact steps to reproduce that. But really, dIMAP right now is very solid, I don't think that there is such a grave problem left.

In fact I would recommend disconnected IMAP over online IMAP, for stability reasons.
Comment 46 Thomas McGuire 2009-06-19 15:08:41 UTC
Note that you should at least use version 1.10.4 (from KDE 4.1.4) and no versions below that (including KDE 3 versions, which are unsupported).
Comment 47 Axel Braun 2009-06-19 15:20:59 UTC
One of the things that stops me from moving to KDE4 is a missing migration tool for all KDE and KMail settings. That was discussed with KDE and SUSE team, but as it looks, nothing has happened so far. Can you point me to a How-to or something?
Comment 48 codyregister 2009-11-04 02:11:48 UTC
I would like to know if anybody else has had good luck with dIMAP under KDE4? I am probably going to start using IMAP (switching from POP) when I upgrade to Kubuntu Karmic and would like to know if it is stable and safe. All the reports of it deleting mail under KDE3 scare me a little.

Ps. I think dIMAP is what I want... I mostly use a laptop and want the benefits of IMAP with out needing to be online at all times. Is disconnected IMAP what I'm looking for?
Comment 49 Luka Renko 2009-11-04 09:28:45 UTC
I am heavy Disconnected IMAP user (with MS Exchange server :-( ) since KDE 3.5.x times and have seen "wipe my inbox" bug often in 3.5 times (much less with kdepim enterprise branch that Kubuntu used at the end, but still). With KDE 4.1 (which is the time I fully switched to KDE4), I have not seen wipe-my-inbox bug at once, so from my side, I could say this bug is fixed (I would personally suggest to close this bug)

Note: my system crashes from time to time (intel driver bug) and this was also causing configuration losses before (lost accounts), but not with KDE4 anymore. The only problem I see with Disconnected IMAP is that if I move mail to some local folders and sync again with my mail servers, that on crash I might still get already moved messages again from imap server (as they were not removed from server yet). But this is bearable problem. ;-)
Overall, I am very satisfied with Kmail in KDE4 - good work!
Comment 50 Rolf Eike Beer 2009-11-04 09:33:38 UTC
That's probably what you are looking for. I have no mail loss since I switched to Kontact from KDE4 about half a year ago or such. And I did not have any mail losses the year before using dIMAP with Kontact from 3.5.10.
Comment 51 Björn Ruberg 2009-12-11 00:33:50 UTC
Looks as if this is a bug concerning unmaintained KDE 3 code. I'm heavily using dIMAP for half a year now and had no such problem.
As suggested I'm closing this bug now. Please says so if you think it needs to be reopened. But if you see a clearly reproduceable loose of your inbox, maybe better fill a new bug.