Version: (using KDE KDE 3.2.0) Installed from: Gentoo Packages Compiler: gcc version 3.2.3 OS: Linux I was reading email with kmail within kontact. I was reading my mails received on an imap account approx 1100 mails in inbox. I tried to apply a filter action to one message (pipe message through spamc) this action used to work, I've already done it some times on various messages. But that time I got an error from kmail something like "could not delete folder ...UID ...". And there you go : all mails in inbox lost ! I've checked the content of my inbox with other mail clients, all mails are really lost ! I've not touched kontact from that point, maybe traces of the error remain if they were logged...
I've setup a test imap account and could reproduce that problem. first I created a filter that matches any of "status contains unread" "status contains new". Filter action is pipe through spamc (from spamassassin). Now when reading messsages on an imap account, I can apply that filter action. And sometimes I see : "Could not remove folder imap://username@server:143/INBOX/;UID=" and all the emails are deleted... I'll attach a sample message so that you can try to reproduce all that.
Created attachment 4493 [details] sample email to reproduce the problem
Arnaud, thanks for informing us. Could you please provide us with the exact filter definition (as stated in your $KDEHOME/share/config/kmailrc) and the imap server you are connecting against? I cannot reproduce the behavior you describe, so I'll need to exactly match your setup.
Just ask for more if you need it ... Arnaud [Filter #0] ConfigureShortcut=true Icon= StopProcessingHere=true action-args-0=spamc action-name-0=filter app actions=1 apply-on=check-mail,manual-filtering contentsA=unread contentsB=new fieldA=<status> fieldB=<status> funcA=contains funcB=contains name=Spam Check operator=or rules=2 first time I saw the problem was with : telnet mailbox.epfl.ch 143 Trying 128.178.50.37... Connected to mailbox.epfl.ch. Escape character is '^]'. * OK Messaging Multiplexor (iPlanet Messaging Server 5.2 HotFix 1.18 (built Jul 28 2003)) C00000 CAPABILITY * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS CHILDREN LANGUAGE XSENDER X-NETSCAPE XSERVERINFO AUTH=PLAIN STARTTLS C00000 OK CAPABILITY completed and I could reproduce it many times on a kolab server : telnet kolab.ouvert.ch 143 Trying 212.74.171.57... Connected to kolab.ouvert.ch. Escape character is '^]'. * OK kolab.ouvert.ch Cyrus IMAP4 v2.1.14 server ready C00000 CAPABILITY * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND SORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES IDLE STARTTLS C00000 OK Completed
Thanks for the info. Can you still reproduce it if you disable load on demand for the account?
I disabled "load attachements on demand" and could reproduce that at my first try...
Arnaud, I have now sent that mail you sent (and a few others) through the filter you specified above quite a few times and am unable to trigger deletion of the inbox. This is against a UW server and a kolab. Current spamc from a suse 9.0 install. Is there any chance I could get access to your installation or could you send me the debug output of both kmail and kio_imap of a (minimal) execution that triggers it?
I just installed KDE 3.2.0 and had this exact thing happen to me. It deleted every email in my inbox and I have no way of recovering them. I'm really pissed off right now, but how could you release software with a known reproducable bug like this?
Apparently not all data was lost, ~/.kde/share/apps/kmail/imap/.accountname.directory/INBOX still contains the headers for the emails. I'd hope that KMail caches the entire message somewhere else, but I'm afraid I'm just out of luck and this is all that remains of the 600+ emails.
Jason, we are sorry about your loss of mail, but the bug was neither known to us nor can we reproduce it, so far. And no, the mails themselves are not cached anywhere. We are trying hard to resolve this.
Subject: kdepim/kmail CVS commit by tilladam: Don't do an expunge Folder, thereby removing all mails in an imap folder, when the deleteMessage method is called on a message that does not have a UID, for whatever reason. Ouch. Now we need to find out why the message has no uid. Ingo, backport, I assume? CCMAIL: 74017@bugs.kde.org M +15 -2 kmfolderimap.cpp 1.170 --- kdepim/kmail/kmfolderimap.cpp #1.169:1.170 @@ -1213,5 +1213,14 @@ void KMFolderImap::deleteMessage(KMMessa KURL url = mAccount->getUrl(); KMFolderImap *msg_parent = static_cast<KMFolderImap*>(msg->storage()); - url.setPath(msg_parent->imapPath() + ";UID=" + msg->headerField("X-UID")); + QString uid = msg->headerField("X-UID"); + /* If the uid is empty the delete job below will nuke all mail in the + folder, so we better safeguard against that. See ::expungeFolder, as + to why. :( */ + if ( uid.isEmpty() ) { + kdDebug( 5006 ) << "KMFolderImap::deleteMessage: Attempt to delete " + "an empty UID. Aborting." << endl; + return; + } + url.setPath(msg_parent->imapPath() + ";UID=" + uid ); if ( mAccount->makeConnection() != ImapAccountBase::Connected ) return; @@ -1234,5 +1243,9 @@ void KMFolderImap::deleteMessage(QPtrLis for ( QStringList::Iterator it = sets.begin(); it != sets.end(); ++it ) { - url.setPath(msg_parent->imapPath() + ";UID=" + *it); + QString uid = *it; + // Don't delete with no uid, that nukes the folder. Should not happen, but + // better safe than sorry. + if ( uid.isEmpty() ) continue; + url.setPath(msg_parent->imapPath() + ";UID=" + uid); if ( mAccount->makeConnection() != ImapAccountBase::Connected ) return;
Jason, couldn't your server admin restore your mails from backups ? Mine could and my 1200+ mails are back ...
I paste here a network dump of such an imap session, bad bad bad... * OK kolab.ouvert.ch Cyrus IMAP4 v2.1.14 server ready 0 CAPABILITY * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND SORT THREAD=ORDEREDSUBJECT THREAD=REFERENCES IDLE STARTTLS 0 OK Completed 1 LOGIN "kmail@ouvert.ch" "xxxxxxxx" 1 OK User logged in 2 LIST "" "%" * LIST (\HasNoChildren) "/" "INBOX" 2 OK Completed (0.000 secs 2 calls) 3 LIST "" "INBOX" * LIST (\HasNoChildren) "/" "INBOX" 3 OK Completed (0.000 secs 2 calls) 4 STATUS "INBOX" (UNSEEN) * STATUS INBOX (UNSEEN 0) 4 OK Completed 5 LIST "" "INBOX" * LIST (\HasNoChildren) "/" "INBOX" 5 OK Completed (0.000 secs 2 calls) 6 LIST "" "INBOX/%" 6 OK Completed (0.000 secs 1 calls) 7 LIST "" "INBOX" * LIST (\HasNoChildren) "/" "INBOX" 7 OK Completed (0.000 secs 2 calls) 8 SELECT "INBOX" * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] * 4 EXISTS * 0 RECENT * OK [UIDVALIDITY 1075767684] * OK [UIDNEXT 51] 8 OK [READ-WRITE] Completed 9 UID FETCH 47 (UID RFC822.SIZE INTERNALDATE FLAGS BODY.PEEK[]) * 1 FETCH (FLAGS (\Seen) UID 47 INTERNALDATE " 4-Feb-2004 13:45:42 +0100" RFC822.SIZE 3047 BODY[] {3047} Return-path: <uj4@bigfoot.com> ... more headers ... more body </body></html> ) 9 OK Completed 10 NOOP 10 OK Completed 11 NOOP 11 OK Completed 12 UID FETCH 1:* (UID FLAGS) * 1 FETCH (FLAGS (\Seen) UID 47) * 2 FETCH (FLAGS (\Seen) UID 48) * 3 FETCH (FLAGS (\Seen) UID 49) * 4 FETCH (FLAGS (\Seen) UID 50) 12 OK Completed 13 CLOSE 13 OK Completed 14 UNSUBSCRIBE "INBOX" 14 OK Completed 15 DELETE "INBOX" 15 NO Operation is not supported on mailbox 16 SELECT "INBOX" * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] * 4 EXISTS * 0 RECENT * OK [UIDVALIDITY 1075767684] * OK [UIDNEXT 51] 16 OK [READ-WRITE] Completed 17 UID STORE 1:* +FLAGS.SILENT (\DELETED) 17 OK Completed 18 CLOSE 18 OK Completed 19 DELETE "INBOX" 19 NO Operation is not supported on mailbox
kio (KSycoca): Trying to open ksycoca from /var/tmp/kdecache-ghort/ksycoca kio (KIOJob): stat imap://kmail%40ouvert.ch@kolab.ouvert.ch:143/INBOX/;SECTION=UNSEEN kio (KIOJob): error 38 imap://kmail@ouvert.ch@kolab.ouvert.ch:143/INBOX/;UID= kio (Slave): killing slave pid=9934 (imap://kolab.ouvert.ch) kio (Slave): slave died pid = 9934 kio (KIOJob): Job::kill this=0x832ee80 m_progressId=0 quietly=true these are the only kio outputs I get running kmail... I started kdebugdialog did "select all" apply then ok. And killed all kio I found. (I can't reboot nor close my kde session because of running tests...)
*** Bug 70429 has been marked as a duplicate of this bug. ***
Subject: kdepim/kmail CVS commit by tilladam: When an imap message is filtered through spamc and classified as spam, a new message is constructed and returned which has a new set of headers. Consequently there is no X-UID header anymore. If the message is then left in the same folder and not moved somewhere else, the original mail is removed from the folder on the server and the new one uploaded. To figure out which mail to remove, the (now missing) X-UID header is looked at. That is not there -> no uid -> message is not delete (used to be folder is expunged). To avoid that, restore the X-UID header after the pipe through to make sure it's there. Not really correct, technically, but I can't think of another reliable way to fix this. Thanks a lot to Arnaud Burlet for helping me track this bugger down and testing. Backport? CCMAIL: 74017-done@bugs.kde.org M +9 -1 kmfilteraction.cpp 1.115 --- kdepim/kmail/kmfilteraction.cpp #1.114:1.115 @@ -505,6 +505,14 @@ KMFilterAction::ReturnCode KMFilterActio QByteArray msgText = kmkernel->getCollectedStdOut( &shProc ); - if ( !msgText.isEmpty() ) + if ( !msgText.isEmpty() ) { + /* If the pipe through alters the message, it could very well + happen that it no longer has a X-UID header afterwards. That is + unfortunate, as we need to removed the original from the folder + using that, and look it up in the message. When the (new) message + is uploaded, the header is stripped anyhow. */ + QString uid = aMsg->headerField("X-UID"); aMsg->fromByteArray( msgText ); + aMsg->setHeaderField("X-UID",uid); + } else return ErrorButGoOn;
Subject: Re: grave imap bug : All messages of inbox lost On Friday 06 February 2004 08:35, Till Adam wrote: ... > When an imap message is filtered through spamc and classified as > spam, a new message is constructed and returned which has a new set > of headers. Consequently there is no X-UID header anymore. If the > message is then left in the same folder and not moved somewhere > else, the original mail is removed from the folder on the server > and the new one uploaded. To figure out which mail to remove, the > (now missing) X-UID header is looked at. That is not there -> no > uid -> message is not delete (used to be folder is expunged). To > avoid that, restore the X-UID header after the pipe through to make > sure it's there. Not really correct, technically, but I can't think > of another reliable way to fix this. > > Thanks a lot to Arnaud Burlet for helping me track this bugger down > and testing. > > Backport? Yes please. And please update the KMAIL_VERSION in kmversion.h to "1.6.1" so that users know whether they are using a version of KMail with these critical fixes. (Updating the version also helps with answering bugs). Don.
Subject: Re: grave imap bug : All messages of inbox lost On Friday 06 February 2004 03:39, Don Sanders wrote: > > Backport? > > Yes please. Done. > And please update the KMAIL_VERSION in kmversion.h to > "1.6.1" so that users know whether they are using a version of KMail > with these critical fixes. > > (Updating the version also helps with answering bugs). Yeah, makes sense. Done. Till
It would like to know where place I apply patche: kdepim-3.2.0-HOT_FIX_FOR_KMAIL.patch which is the directory? Remembering that I use Slackware 9.1
I installing the following packages: kdegraphics-3.2.0-i486-2.tgz - kdemultimedia-3.2.0-i486-2.tgz - kdepim-3.2.0-i486-2.tgz I solve this bug?
Hi all, Why don't I see this bug report before this morning ??? :-( I've tried the Jason's tip, so i've looked in the directory, but the inbox directory was disappeared :-((( back to thunderbird, and I will wait a little more time, before retried kontact ... too bad ! Neverless, congratulations to all the dev team for this great app !
I have encountered similar problem with kmail 1.6.2 and offline IMAP. After creating offline IMAP account kmail started to synchronize folders and their contents with server. Crashed upon that job. Started kmail again and ordered to check mail (Ctrl+L). All folders that had unread messages in showed error "Could not create folder ...." (contrary to delete in previous posts here). The result was the same, though: nothing left in folders. The folder with Sent Items was OK, as there were no unread messages inside. No more problems after Folder tree had been correctly created. Crashes while creating IMAP (usual also) folder trees (Exchange Public Folders) are quite usual. After folder tree has been created, kmail is stable.
I had exactly the same behavior with KMail 1.9.5 as Toomas Mõttus. disconnected IMAP, started to synchronize folders, crashed. when I restarted kmail, it correctly sychronized all the folders, but I've confirmed that the inbox is indeed empty.