Bug 233069 - Kmail loses mail bodies
Summary: Kmail loses mail bodies
Status: RESOLVED UNMAINTAINED
Alias: None
Product: kmail2
Classification: Applications
Component: general (show other bugs)
Version: 4.11.4
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-04-02 14:56 UTC by xor
Modified: 2016-06-09 12:26 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Empty mails in mailfolder view (16.83 KB, image/png)
2010-04-02 14:56 UTC, xor
Details
Source of empty mails (17.14 KB, image/png)
2010-04-02 14:57 UTC, xor
Details
An example of an on-disk maildir file of an affected mail (178 bytes, application/octet-stream)
2010-12-15 00:41 UTC, xor
Details

Note You need to log in before you can comment on or make changes to this bug.
Description xor 2010-04-02 14:56:00 UTC
Version:           1.12.2 (using KDE 4.3.2)
OS:                Linux
Installed from:    Ubuntu Packages

The following happened to me:
- I was reading one of my devl mailing lists so I knew that certain messages existed there
- I used the spam filter and anti-virus filter wizards to create filters 
- I selected all mails in the mailing list folder and did "Apply all filters"
- Today I noticed that one of the mails I was reading is missing
- I grabbed content of the mail from a mailing list archive, searching for that content in Kmail showed that the mail is really gone
- I re-build the indexes for all mailfolders, still could not find the mail
- Then I looked manually and found empty mails with no subject or date in the mailing list folder where the message is missing
- Screenshots of the messages and the source of the messages are attached

Kmail configuration:
- Mails are stored in maildir format
- I just saw that there were 2 clamAV filters, one is probably legacy from my old Kubuntu 8 KMail configuration - I had copied over the Kmail dir and config from my old Kubuntu 8 homefolder to my new Kubuntu 9: One of the filters used kmail_clamav.sh and one klammail. Klammail is the old filter - the current wizard only creates the kmail_clamav.sh filter. Klammail is also installed on my system though
- bogofilter --help: bogofilter version 1.2.0       
- Klammail --help: KlamAV Mail Processing Client 0.46
Comment 1 xor 2010-04-02 14:56:38 UTC
Created attachment 42441 [details]
Empty mails in mailfolder view
Comment 2 xor 2010-04-02 14:57:02 UTC
Created attachment 42442 [details]
Source of empty mails
Comment 3 Björn Ruberg 2010-04-05 23:21:38 UTC
Think this is a duplicate - as long as you are using IMAP, of course

*** This bug has been marked as a duplicate of bug 104073 ***
Comment 4 xor 2010-04-06 02:25:44 UTC
I'm using POP3, not IMAP :|
Comment 5 xor 2010-12-14 18:57:04 UTC
Hi,
I've dist-upgraded to Kubuntu 10.04 now.
- This means that I am on KMail 1.13.2 now
- I have disabled the bogofilter filter and the clamav filter. Effectively now I only have filters now which move mails within the KMail mail folders, there are no filters which pipe through external applications.
- I am using POP3, so it is not a duplicate of the IMAP bug

And the issue STILL happens :(

IMHO it is an in-acceptable behavior for a mail program to lose mails :(
Therefore I would like to ask you whether you can raise the severity of this issue, given the fact that it has not been fixed for over 8 months
-  I would really appreciate if this issue could be fixed, because it typically causes mails from my development mailing lists to be dropped, which effectively results in MORE bugs in different applications because I might lose bug reports. So this bug breeds children :)

I will try to help with the bug hunting as much as I can. As I'm a developer myself I should be able to follow very technical instructions, however I do not have a setup for compiling KMail and I would prefer not doing that - I want my Kubuntu to stay in a virgin state. Reversible configuration changes can happen however.


Thank you, xor
Comment 6 xor 2010-12-14 19:18:32 UTC
I've tried the "Recreate index" feature on an affected folder and it didn't restore the emtpy mails.
So the index is not the problem I guess.
Comment 7 Torgny Nyblom 2010-12-14 19:49:33 UTC
Does it happen with the same mail on every view? If so could you post the entire message here (as it is on disk not as seen in the source viewer, or even better as it is/was on the server before the filters)?
Comment 8 xor 2010-12-15 00:39:34 UTC
It happens in the preview pane and also when I open the mail in its own window by double-clicking...

I've located the folder where the affected mails are as ~/.kde/share/apps/kmail/mail/.Freenet.directory/devl/cur/  ... sorted them by size and found that there are 6 files which are 178 byte only... this matches the count of broken mails in kmail which is also 6...

"UNFORTUNATELY", the date of all those files is April. That was when I still had bogofilter+clamav.

So I am unable to find the mail which recently showed up as empty in KMail ... either it is completely gone or it was a display error only :|

I am uploading one of the 6 broken mails anyway as attachment here...
Comment 9 xor 2010-12-15 00:41:47 UTC
Created attachment 54562 [details]
An example of an on-disk maildir file of an affected mail
Comment 10 xor 2011-04-19 18:48:34 UTC
I've seen it happen again with KMail 1.13.5
What can I do to make you guys finally attempt to fix this? Donation? How many EUR?
Mail software which deletes random mails is unusable and I don't feel like there are any suitable alternatives on KDE. I'm really sad about this. Please help.
Comment 11 mkkot 2012-02-20 19:07:43 UTC
xor maybe you should report it against kmail2 and not kmail. Maybe devs didn't received your bugreport...
Comment 12 xor 2012-02-21 22:15:54 UTC
@Comment #11: Why? Version says1.12.2
Comment 13 Halla Rempt 2014-01-21 08:53:14 UTC
I can confirm that this happens to me, too, with KMail 4.11.4. It's very treacherous since it seems to be mostly random. I download mail with pop3, read it, everything is fine.

Then a month later, I have to refer to that mail again, find it in the list and the body is gone. Really, really gone. The file on disk is empty or just has a header.

If this were a bug in Krita, I'd confirm it and bump it up to critical...
Comment 14 Martin Steigerwald 2016-06-08 20:07:38 UTC
Thank you for your report.Does this bug still happen to you with KMail and Akonadi 16.04? If so, please reopen as a new bug as KDEPIM SC 4 and Akonadi 1 is unmaintained. Also always use one bug report for one problem. Thank you, Martin
Comment 15 Halla Rempt 2016-06-08 20:17:31 UTC
Well, I stopped using KMail because of this and similar issues -- like endless waits and the "mime conflict resolution diff" dialog -- about a year or so ago. And given how hard it's to migrate between mail clients if you need a few hundred filter rules and get thousands of mails a day, I'm not even contemplating going back and trying kmail2 again.I just cannot afford risking my mail again just to verify whether this bug is still present. It's too valuable for little experiments like that.

Whether or not this bug might be magically fixed because there's now a new version and the old one isn't maintained anymore, the ship has sailed: I will never, ever, try kmail and akonadi again. I'm using _pine_ now.
Comment 16 Martin Steigerwald 2016-06-08 20:39:34 UTC
Sad, but understandable Boudewijn. I had data losses myself, but the last has been a very long time ago. I think Akonadi got much more stable related to data safety, but I can understand you.
Comment 17 mkkot 2016-06-09 12:26:49 UTC
I migrated to Thunderbird when Kmail started loosing data. There is a difference between 2 formats Kmail and Thunderbird use but they are all open, so there should be no problem with migration. However, I encountered problems until they were solved by simple python script:
http://www.ulduzsoft.com/2012/01/from-kmail-to-thunderbird/

It works really fast. You can have thousands of e-mails converted in a matter of just few minutes. And it doesn't malform your data while doing it.