Bug 121272

Summary: E-mail messages become blank when I open them
Product: [Applications] kmail Reporter: Lincoln Peters <sampln>
Component: indexAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED UNMAINTAINED    
Severity: grave CC: baeckham, bugzilla, c.vinante, david, jim, kde-2011.08, lophiomys, lukas.kasprowicz, m.michelbrink, mmodem00, omasenkel, redhen898-online, tsurd2, yez
Priority: NOR    
Version: 1.9.6   
Target Milestone: ---   
Platform: unspecified   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: E-mail that can't be displayed in KMail without turning blank

Description Lincoln Peters 2006-02-03 09:01:28 UTC
Version:           1.9 (using KDE 3.5.0, Debian Package 4:3.5.0-1 (testing/unstable))
Compiler:          Target: i486-linux-gnu
OS:                Linux (i686) release 2.6.14.4lincoln

The bug I'm experiencing seems to be similar to Bug 84896 (supposedly fixed), but I'm using KMail 1.9 (a newer version than the one that exhibited the bug), and what I'm seeing doesn't quite match the descriptions of that bug.  So I'm posting this as a new bug.

Here's what I observed: I use the "Search" bar to track down a certain e-mail that I previously read without any problems.  I find the e-mail, and then I click on it.  The message suddenly turns blank (unknown sender, no subject, unknown date, no body).  Furthermore, if I try to view the source of the message, I get a widow displaying absolutely nothing.

Unlike the reports in Bug 84896, I have not observed any messages exhibit this behavior prior to my clicking on them.  I can only trigger the bug when I've narrowed the view to certain messages using the "Search" bar, and even then it's only consistent with messages from some people (I can only consistently reproduce the bug using messages from *one* sender).


I can't make sense of the e-mail headers, but if I had to guess where to look for the problem, I'd guess that there's something odd about the headers produced by SquirrelMail that's triggering the bug.  Why SquirrelMail?  Because the only sender I can reproduce this bug with uses SquirrelMail, and I don't think that anyone else who sends me e-mail uses SquirrelMail.  I'm willing to investigate further if I can get a better idea of what to look for.


(NOTE: Even if this does prove to be at least partially related to a bug in SquirrelMail, I would still consider this a major bug in KMail because KMail should NEVER do something like this to a message, no matter how screwed up its headers are.  Furthermore, even if this somehow leads to a major big-fix of some kind in SquirrelMail, I suspect that it would be a bureaucratic nightmare to upgrade this particular SquirrelMail server.)
Comment 1 Thiago Macieira 2006-02-04 15:50:27 UTC
You probably have a corrupt index file. Have you experienced any crashes in KMail or power failures recently?

Can you also find the message that is causing this problem in your mailbox/maildir, extract it, zip and attach here?
Comment 2 Lincoln Peters 2006-02-04 19:17:24 UTC
Created attachment 14545 [details]
E-mail that can't be displayed in KMail without turning blank
Comment 3 Lincoln Peters 2006-02-04 19:18:09 UTC
No power failures, but I have noticed occasional crashes in KMail when I close it.  I don't know if they could have caused crashes, and I couldn't get any sort of debugging symbols out of it.  I've also occasionally logged out of KDE while KMail was still running; could that cause this kind of problem (I haven't noticed it cause problems with most other KDE applications)?

I've attached a ZIP file containing one of the messages that demonstrates the problem (and I'm relieved to find that it wasn't destroyed by this bug!).
Comment 4 Thomas McGuire 2007-05-07 17:17:38 UTC
*** Bug 122045 has been marked as a duplicate of this bug. ***
Comment 5 Thomas McGuire 2007-05-07 17:17:50 UTC
*** Bug 129720 has been marked as a duplicate of this bug. ***
Comment 6 Thomas McGuire 2007-05-07 17:17:57 UTC
*** Bug 145132 has been marked as a duplicate of this bug. ***
Comment 7 Thomas McGuire 2007-05-07 17:18:06 UTC
*** Bug 106287 has been marked as a duplicate of this bug. ***
Comment 8 Thomas McGuire 2007-05-07 17:18:15 UTC
*** Bug 102946 has been marked as a duplicate of this bug. ***
Comment 9 Thomas McGuire 2007-05-07 17:18:23 UTC
*** Bug 92006 has been marked as a duplicate of this bug. ***
Comment 10 Thomas McGuire 2007-05-07 17:19:57 UTC
Raising severity due to mail loss in some (all?) of those cases.
Comment 11 Thomas McGuire 2007-07-17 14:55:33 UTC
*** Bug 99460 has been marked as a duplicate of this bug. ***
Comment 12 Thomas McGuire 2007-07-17 15:07:11 UTC
*** Bug 130139 has been marked as a duplicate of this bug. ***
Comment 13 Bram Schoenmakers 2007-08-25 14:40:44 UTC
*** Bug 149180 has been marked as a duplicate of this bug. ***
Comment 14 2007-08-25 17:17:53 UTC
Why this bug is marked as unconfirmed? 
Comment 15 Lophiomys 2007-08-27 11:22:40 UTC
I have reproduced this bug/DATA LOSS under 
Kubuntu 7.04 Feisty Fawn, all updates with 
KDE 3.5.7, Kontact 1.2.5, KMail 1.9.7 using POP3
!!!
Comment 16 Thomas McGuire 2007-08-27 14:47:39 UTC
>Why this bug is marked as unconfirmed? 
Because no developer has confirmed this bug yet.
For example, I tried the mail from comment #2, but it works perfectly fine for me.
Comment 17 YeHaa 2007-12-12 15:37:21 UTC
I observed similar problems with KMail from Kubuntu 7.10 (identifies with "KMail 1.9.6 (enterprise 0.20070907.709405)").

I had however an even more irritating behavior:
In the thread-pane (message-list) I see a message with this (fictional) content:
Some Message-Title         sender@name.de           11.12.2007

When I click on it, it changes to some other content. But(!) the message displayed has the header-data that was previously displayed in the thread-pane. 

This happened with mails send by me as well as with mails received by me. But not with all such mails. 

I backuped the complete contents of 
  ~/.kde/share/apps/kmail/imap/.775800671.directory
and then deleted everything in that directory and these Problems were gone. So the first Thiago in #1 probably had the correct idea. It's really annoying though, that KMail displays the _correct_ info in the list, but displays _wrong_ info in the list while displaying the (correct) contents of the mail in the message-pane.
Comment 18 A. Lofaro 2008-01-04 14:08:24 UTC
I experienced recently the same problem as described in bug 99460 (which was closed before as "duplicate" of this one), and even before having moved or copied large number of messages. I lost the messages as well, including some important ones. 
  I have the rights on the folders , I deleted the indices without improvements, recreated the folders (even with a different format, and changing name) from inside KMail, still the same - I verified also that it is not a problem of communication with the server, the server is sending the message contents correctly.
   At least in my case, it is not a single e-mail that has the problem, some old e-mails were lost for this, but also any new e-mail received on a certain account and thus automatically routed to a folder gets the same problem: this even after I deleted the messages with "unknown" situation (which using the maildir format I could see were effectively empty, 130 or so bytes) and created a new folder.
  It looks to me like the problem is when Kmail creates the temporary file in which he puts the mails while downloading them as well as in parsing the contents of the e-mails (both when in maildir folders, indivually registered, and in mbox format, since I had the same problem in both cases). I even created completely from scratch an empty new folder, without copying any mail from previous folders, and directed there the mails from the account with the problem - same thing, the messages are directly "Unknown" and empty even before I try to see the contents (i.e. in the messages list pane).
   I have now a mix of maildir and mbox directories, but when I started experiencing the problem I had only mbox folders.
    I had tried to create a IMAP connection to an extra account just before the problem started, but I had already deleted that connection when the problem started. I also tried deleting the index file that was still in the directory indicated in comment number 17, but with no luck.
     This is a big bug, if you cannot reproduce it, ask for collaboration in testing by the people that are experiencing it. After many people have signalled it with different bug numbers, "solving" the other cases with a "duplicate" label and then saying it is "uncornfirmed", so that there is almost a guarantee the problem will not be dealt with, is in line with the worst practices of some companies often criticised by supporters of open source.
Comment 19 A. Lofaro 2008-01-04 15:52:53 UTC
   Further to my previous posting, I have found something I think useful.
   First, I am using kmail 1.9.5 on OpenSuse 10.2, KDE 3.5.5. Looking in the tmp directory, I found that the temporary files created by clamav (klammailxxxxxx is the format of the name) actually contains the "lost messages", it seems even with the full header. I use clamav 0.91.2 and Klamav 0.41.
  When the folder was in the maildir format, some of the half destroyed (i.e. "Uknown") messages contained text referring to I/O errors or command that were not understood. If kmail filters the messages through clamav even when you are just trying to see a message that has already been received or copy/move it to another folder, this could explain the problem at least in some cases: kmail passes the message throug clamav, clamav creates the temporary file which contains the full message, but for whatever reason when clamav tries to "pipe" the message back to kmail, something doesn't work, so kmail ends up with partial garbage.
   This could also explain why the developer did not get to reproduce the problem, if he does not use clamav with kmail - the problem is not with the message "per sé" (some of the messages with the problem in my case are five lines of pure gibberish casual text I type on the keyboard, nothing big or complicated).
   I will try later to disable temporarily the filtering of the messages through clamav, and see if this makes the problem go away for new messages.
  It may be worth checking and trying this even with kde 4, before it is released - the problem was not being corrected because it was "unconfirmed", but it could happen again.
  Unfortunately I would need to think how (if it's possible) to pipe the contents of temporary clamav files to kmail tricking it into adding the messages correctly to the folders. Also, since the clammail files are temporary files, it is not said that this trick can be used to recover the messages in all cases.
Comment 20 A. Lofaro 2008-01-04 16:18:02 UTC
Hello, I found a workaround and source of the problem described up to comment 16, workaround which worked in at least one case (mine). 
  It is the passage of the e-mails through clamav, when they are given back to kmail, kmail cannot really understand what's inside and creates crap. 
  Disabling the antivirus check with clamav avoids the risk that the problem will happen again in the future, at least when using kmail 1.9.5 with kde 3.5.5 and clamav 0.91.2 on OpenSuse 10.2 (but people that had the problem with other versions and distributions could try as well this).
  Unfortunately, the only way to recover the messages that have been destroyed is being lucky enough to still have the temporary files created by clamav in the /tmp directory, and finding a way (help here would be welcome) to pipe their contents back in such a way that kmail will accept them (would be sufficient to know how kmail decides the name of a file when using the maildir format, to just copy at file system level the mails in the right place).
Comment 21 Thomas McGuire 2008-01-12 16:12:44 UTC
*** Bug 155479 has been marked as a duplicate of this bug. ***
Comment 22 Attila GOLONCSER 2008-02-11 07:11:52 UTC
See my Comment #26 on Bug 80988, I also went to DATA LOSS with the same problem, after a crash. 
Comment 23 Tsur Doron 2008-03-02 17:02:22 UTC
I am reproducing this bug with KMail 1.9.6 (enterprise 0.20070907.709405),

On My Kubuntu 7.10 system.

I love my Kubuntu, but this bug forbids me to check out the "Keep messages in server" box. I use my other computer to clean my server, it has proprietry Email...

I know this bug does not reproduced often. So I find "Can't reproduce" claim weak, since the seldom bugs are the most annoying ones.

So I said I have a workaround (leaving messages on server), but since I have this bug in my net for now (I can repro it for now till I restart my comp), any help on debugging Kmail will be appreciated.

Even if not for this time, then for next time ....

:)



Comment 24 Jim Cortez 2008-04-07 23:17:54 UTC
I am also having this error on kUbuntu 8.04. I recently switched from thunderbird to Kmail and imported 5000+ messages. I have watched as very important emails have gone blank in the message list. I can confirm that this message did come from someone who might have used squirrelmail. I am available to help debug as I really like kmail and don't much want to switch back to thunderbird. This sort of Data Loss is not acceptable for me!

System info:
ThinkPad T43 with kUbuntu 8.04 Beta
Kmail 1.9.9 (part of Kontact suite)

thanks,

Jim Cortez
jim {at) jimcortez |DOT| com
Comment 25 Tsur Doron 2008-04-08 08:40:49 UTC
Hi Jim,

1. Do you have KlamAV installed?
I have the following filter for size of message >2:
klammail -d /home/doron/.klamav/database

2. Do you get "no subject" in the subject form?

3. DO you reproduce this problem always or in some probability?

Anyway, adding a filter with gdb instead of / added to klammail -d /home/doron/.klamav/database
 may help us debug the problem and hunt this bug down ...
Comment 26 Xavier Brochard 2008-07-24 21:02:56 UTC
One of my customer have got the same problem on Kubuntu 7.04 (Feisty) with KDE 3.5.8
It is a multi-user system setup with LTSP5, all users have similar email accounts on the same pop server. We don't use Clamav nor Klamav. Bogofilter is setup.

Only one user got the problem, suddenly. Unfortunatly I wasn't there, and the user was a beginner who couldn't described what she has done on the computer before.

The problem occured with various emails, not repeatedly (I send one email twice, one of them became unreadable). Those emails were totaly deleted and unrecoverable.

As a workaround, I solve the problem by deleting kmail folder index.

Comment 27 redhen898-online 2008-07-24 22:21:42 UTC
I no longer use Kmail. I do use Kontact for the address book and calendar, which sync through Kpilot my Palm PDA (Zire22). But I gave up on KMail well over a year ago for Thunderbird, because no one would validate much less solve this problem, and also because Kmail at that time at least had no means of blocking senders from seeing whether I'd opened the mail or not. As a mail client, I regret to say, Thunderbird is far preferable.
Comment 28 Tsur Doron 2008-09-05 19:08:24 UTC
klammail filter seems to be the cause to this issue in my 8.04 hardy.

Once I update KlamAV virus database, The messages are no longer changed.

I got more information when running kontact from konsole, and reviewing the output .
Comment 29 Freda Saul 2008-09-15 02:03:27 UTC
I had this bug also; it was claimed it was "unconfirmed" and no one did a thing about it. So I quit using Kmail and switched to Thunderbird. Now that you have your confirmation, maybe someone will fix KMail, which stinks overall.
Comment 30 Jim Cortez 2008-10-15 03:34:35 UTC
Someone suggested that I re-generate the index. I did this by right-clicking the dIMAP folder, and clicking troubleshoot IMAP. Then there is an option to regenerate indexes.

The problem seems to occur when there is an error while filtering. Sometimes the message is indeed moved, but a ghost message is left behind in the old folder. It can also happen when read emails are filtered automatically. For example, you check your email through a webmail client that does not have filters. When kmail starts, it downloads mail and runs it though filters. SOMETIMES it does these ghost messages, but it seems like a higher probability at this time.
Comment 31 L_V 2009-10-16 19:58:09 UTC
From Lincoln Peters  2006-02-03

" I find the e-mail, and then I click on it.  The message suddenly turns blank (unknown sender, no subject, unknown date, no body).  Furthermore, if I try to view the source of the message, I get a widow displaying absolutely nothing."

Exactly same bug seen today, 3.5 years later with Kamil 4.3.2.
Comment 32 Björn Ruberg 2010-02-02 01:31:29 UTC
*** Bug 176214 has been marked as a duplicate of this bug. ***
Comment 33 Laurent Montel 2015-04-12 10:24:58 UTC
Thank you for taking the time to file a bug report.

KMail2 was released in 2011, and the entire code base went through significant changes. We are currently in the process of porting to Qt5 and KF5. It is unlikely that these bugs are still valid in KMail2.

We welcome you to try out KMail 2 with the KDE 4.14 release and give your feedback.