Bug 289428

Summary: changing message flags (in KMail2) can lose message permanently
Product: Akonadi Reporter: Jonathan Marten <jjm>
Component: Maildir ResourceAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED FIXED    
Severity: critical CC: alvanx, faure, jajaxor, prozac, robby.engelmann, rulatir, sombragris, spam_bugs_kde, sven.burmeister, wstephenson, zerg2000-bugs.kde.org
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: Version Fixed In: 4.8.1
Attachments: Output from strace

Description Jonathan Marten 2011-12-20 13:36:35 UTC
Version:           unspecified (using Devel) 
OS:                Linux

Changing some message flags in KMail (e.g. "Important", "Action Item") sometimes causes the message to disappear and be permanently lost.


Reproducible: Sometimes

Steps to Reproduce:
In KMail2, select a message which is in a Maildir folder.
Do "Menu bar - Message - Mark Message - Mark Mail as Important"
or "Menu bar - Message - Mark Message - Mark Mail as Action Item"
a few times to turn the flags on or off.

Actual Results:  
The email sometimes disappears.  It cannot be found in the trash folder or anywhere else.


Expected Results:  
The message flags are just changed as requested.


A message appears in the debug log when this happens:

Maildir: Failed to add entry:  "/home/jjm/Mail.kmail2/.LocalFolders.directory/wastebin/cur/1324056139.R363.hostname:2,S" ! Error:  "Unknown error"                                        
kmail2(19249) KMKernel::instanceError:  instance : "akonadi_maildir_resource_4"  received error : "Failed to change the flags for the mail."                                              

The problem may be in Maildir::changeEntryFlags() in kdepimruntime/resources/maildir/libmaildir/maildir.cpp.  I have not yet examined this in detail, but a strace of the akonadi_maildir_resource process shows the following system calls:

open("/home/jjm/Mail.kmail2/.LocalFolders.directory/wastebin/cur/1324056139.R363.hostname:2,S", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 14
open("/home/jjm/Mail.kmail2/.LocalFolders.directory/wastebin/cur/1324056139.R363.hostname:2,S", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 14
unlink("/home/jjm/Mail.kmail2/.LocalFolders.directory/wastebin/cur/1324056139.R363.hostname:2,S") = 0
lstat64("/home/jjm/Mail.kmail2/.LocalFolders.directory/wastebin/cur/1324056139.R363.hostname:2,S", 0xbf9e04a8) = -1 ENOENT (No such file or directory)
rename("/home/jjm/Mail.kmail2/.LocalFolders.directory/wastebin/cur/1324056139.R363.hostname:2,S", "/home/jjm/Mail.kmail2/.LocalFolders.directory/wastebin/cur/1324056139.R363.hostname:2,S") = -1 ENOENT (No such file or directory)

This sequence of calls is consistent with realKey being equal to finalKey just before the

    QFile f( realKey );
    if (QFile::exists(finalKey)) {
        QFile destFile(finalKey);

sequence in this function.

The result is that the original message file is removed and the message disappears.

This may be a consequence of commit 8e09b79d1dd388ed9ad966638c29c483be6eb90a.
Comment 1 Jonathan Marten 2011-12-20 14:34:45 UTC
Created attachment 66935 [details]
Output from strace

The strace output in the report body may be truncated, so it is attached here.
Comment 2 eric 2012-01-04 15:52:35 UTC
This happens here as well. I'm using kde-4.8rc1 (4.7.95), but this also happened with kde-4.8beta2.

What I've noticed is that e-mails will disappear (it's also not in the trash folder) when I turn on (manually, not by a filter rule) more than one mark (I use the marks 'important', 'action item' and 'ham/spam').

When the messages will disappear is not clear to me, sometimes after quitting and immediatly restarting kmail, sometimes I have to wait longer between quitting and restarting.
Comment 3 jajaX 2012-01-26 22:25:26 UTC
Hi !

same problem with kmail2 on kde sc 4.8.0 !

when I clic to the spam filter, message disappear :(
Comment 4 eric 2012-01-29 03:26:28 UTC
Yes, still happens with kde-4.8

This bugreport may be about the same:
https://bugs.kde.org/show_bug.cgi?id=291899
Comment 5 Christophe Giboudeaux 2012-01-29 15:51:26 UTC
(In reply to comment #3)
> Hi !
> 
> same problem with kmail2 on kde sc 4.8.0 !
> 
> when I clic to the spam filter, message disappear :(

Check your filters settings. (settings/configure filters)
Look at the action for the "mark as spam" filter
Comment 6 jajaX 2012-01-29 16:52:09 UTC
Hi ! (sorry for my bad english !)

I have got this problem since 4.7.4 update.

I have got bogofilter since a long time. it worked very well.

since this problem, I try some thing :

- remove bogofilter => same thing
- install spamassassin => same thing

now, I have got nothing like spam filter. I removed bogofilter and spamassassin. two filter button are disapear too, of course.

all my new message are a flag "spam" ! if I clic for marker it as "available" message disapear.

I removed all spam filter. I have got just my manual filter that's all.
Comment 7 Szczepan Hołyszewski 2012-01-30 09:05:21 UTC
Please everyone affected duly add your 20 votes for this bug!

The severity should be *critical* because by the very nature of the bug, it "targets" the messages that are likely to be very important to the user - precisely those they'll want to flag! I just lost a message with a job offer, and I am currently unemployed with bills to pay and debt collectors at my door!
Comment 8 Jonathan Marten 2012-01-30 11:39:56 UTC
Setting severity to critical because of data loss.
Comment 9 Frits Spieker 2012-02-06 14:52:25 UTC
I can confirm this bug. Started happening since upgrade to 4.8.0. Am on OpenSuse 12.1, 64bit. Repo RPM's. Thankfully I have set up akonadi to not delete messages from the pop3 server, as I too would have lost some important mail otherwise. Now can still read online.
Comment 10 Benjamin M 2012-02-06 19:07:53 UTC
I can confirm too on 4.8 stable, with a vanilla configuration (except I use an internal postgresql server). Akonadi and nepomuk databases had just been freshly created. I lose mail about 1 in 5 times that I try to tag a message (e.g. as a to do with the preset tag). 
I get a notification saying (paraphrase from German): "Local Folders: Changing the tag of the email has failed." See also this thread I had opened: http://forum.kde.org/viewtopic.php?f=215&t=98978
Comment 11 Bartosz Kwitniewski 2012-02-07 18:24:47 UTC
Phew, only option to leave messages on the server saved me from losing important e-mail. I was unable to find lost message in local maildir or temporary directories.

Bug appeared after updating KMail to 4.8.0, but it is the first time in many months I have marked e-mail as action item on POP3 account, I usually work on IMAP accounts. When I marked message as action item in KMail 4.7.4 (IMAP) then that message have lost "action item" color and appeared as normal, so something was going on in 4.7.4 already - perhaps I did not lose any mail because it was IMAP.

Arch Linux 64bit
Kmail 4.8.0
Akonadi 1.7.0
Akonadi resource: POP3 E-mail Server
Nepomuk is disabled
Comment 12 David Faure 2012-02-11 10:37:08 UTC
Git commit 9101a5614f0a109602e085cfe904c6524af1679d by David Faure.
Committed on 11/02/2012 at 11:34.
Pushed by dfaure into branch 'KDE/4.8'.

Fix mail loss when the file on disk has an inconsistent name.

E.g. status in akonadi was "unread" but the file on disk had the "Seen"
flag, so marking it as read doesn't need to rename the file, it already
has the desired final name.
FIXED-IN: 4.8.1

M  +6    -1    resources/maildir/libmaildir/maildir.cpp

http://commits.kde.org/kdepim-runtime/9101a5614f0a109602e085cfe904c6524af1679d
Comment 13 David Faure 2012-02-11 10:37:35 UTC
Git commit 13a5b94bf6a95e84675de540415877ab0ab212e9 by David Faure.
Committed on 11/02/2012 at 11:34.
Pushed by dfaure into branch 'master'.

Fix mail loss when the file on disk has an inconsistent name.

E.g. status in akonadi was "unread" but the file on disk had the "Seen"
flag, so marking it as read doesn't need to rename the file, it already
has the desired final name.
FIXED-IN: 4.8.1
(cherry picked from commit 9101a5614f0a109602e085cfe904c6524af1679d)

M  +6    -1    resources/maildir/libmaildir/maildir.cpp

http://commits.kde.org/kdepim-runtime/13a5b94bf6a95e84675de540415877ab0ab212e9
Comment 14 Kevin Krammer 2012-02-11 10:44:16 UTC
Git commit 979dddef125dfde91a4e060bf6f51a9916e55a66 by Kevin Krammer, on behalf of David Faure.
Committed on 11/02/2012 at 11:34.
Pushed by krake into branch 'KDE/4.7'.

Fix mail loss when the file on disk has an inconsistent name.

E.g. status in akonadi was "unread" but the file on disk had the "Seen"
flag, so marking it as read doesn't need to rename the file, it already
has the desired final name.
FIXED-IN: 4.8.1

M  +6    -1    resources/maildir/libmaildir/maildir.cpp

http://commits.kde.org/kdepim-runtime/979dddef125dfde91a4e060bf6f51a9916e55a66
Comment 15 S. Burmeister 2012-02-11 10:52:22 UTC
It's KDE PIM meeting weekend and developers are on fire! :)

Thanks for fixing!
Comment 16 Tobias Koenig 2012-02-11 23:28:27 UTC
*** Bug 291042 has been marked as a duplicate of this bug. ***
Comment 17 sombragris 2012-03-04 01:14:30 UTC
got bitten by this bug too. I lost a very important email this way. Thanks for fixing it.