Version: (using KDE KDE 3.2.2) Installed from: Debian stable Packages OS: Linux This morning after running kmail from debian woody packages on download.kde.org for 3.2.2, I started getting a bunch of corrupted emails.. Most had no subject, etc.. eg. Delivered-To: michael@linuxmagic.com X-AntiVirus: scanned for viruses by AMaViS 0.2.1 (http://amavis.org/) Status: R X-Status: N X-KMail-EncryptionState: X-KMail-SignatureState: X-KMail-MDN-Sent: @itomig.de> Reply-To: stefan.munz@itomig.de Organization: ITOMIG To: wine-users@winehq.org Subject: Re: [Wine]regedit not working User-Agent: KMail/1.5.4 Have been getting random problems with this, but now it seems much more pronounced.. This is jsut a quick report, I need to get another message which hasn't been pulled down, so I can send in a comparison report.
I got the same problem on KDE 3.4.1 KMail added his own header at the wrong place, before the end of normal header (in that case, bad added headers was: Status: R X-Status: NC X-KMail-EncryptionState: X-KMail-SignatureState: X-KMail-MDN-Sent: ) Theses header was introduce in the middle of the To: header. splitting in two the mail of someone. So i saw the end of the header with the mail in the kmail view, and the html was not parsed. (It was a hotmail html message full of useless tag, very difficult to read as it) As it involve personal data, i'll not attach the mail here, but i can provide it in private to someone.
I have the same problem. kmail 1.9.1 on upgraded suse10.
*** Bug 129032 has been marked as a duplicate of this bug. ***
I have the same problem with kmail 1.9.1 , kde-3.5.7 on suse-10. My spam-filter-settings do not apply anymore, because everything from "Date: Wed..." is recognized as mail-body. Example-extract from the spam-mail-header with blank line: X-Status: NC X-KMail-EncryptionState: X-KMail-SignatureState: X-KMail-MDN-Sent: Date: Wed, 23 May 2007 14:41:04 -0500 Message-Id: <10186.downtown@incorporable> X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on
I observed that Kmail places these headers (Status, X-Status, X-KMail-EncryptionState, X-KMail-SignatureState, X-KMail-MDN-Sent) splitting the list of recipients (To: header) after 994 characters. This behaviour is still present in kmail 3.5.8 both in Debian and Opensuse 10.3 It is harmless in most cases, but if the list of recipients is very large all the headers after go corrupted.
*** Bug 158284 has been marked as a duplicate of this bug. ***
I cannot reproduce this with kmail 1.10.1 (kde 4.1.1). What I did was that I sent a test email to myself, saved the source, removed the kmail headers, programmatically added 995 new recipients, added an mbox-style From line in the first line and did "cat testcase > /var/mail/gkiagia", then checked mail for my configured local account. Although the to list was far bigger than 994 characters (it was 996 lines!), the kmail headers were correctly added at the end of the headers list and the mail appeared correctly. I will attach the testcase I used. It would be useful someone to double-check that this file causes problems in 3.5.8 but not in 4.1.1.
Created attachment 27415 [details] testcase with 996 recipients that does *not* reproduce the bug in 4.1.1 (mbox file)
Just tried your test, but it doesn't reproduce the bug neither in kmail 3.5.9...
Also works for me in kmail 1.10.1.
*** Bug 175965 has been marked as a duplicate of this bug. ***
Reopen due to bug 175965
same problem here as described in bug 175965... I also use yahoo mail, maybe that's the point?
I see this problem for quite a long time, so now I think it is a good time to get involved. The problem is really in split header fields. I have free mail and KMail reading from it. The source mail (on free mail) contains this: ... X-Mailer: Apple Mail (2.919.2) X-Mailman-Approved-At: Thu, 13 Nov 2008 16:45:38 +0100 X-Content-Filtered-By: Mailman/MimeDel 2.1.9 Subject: =?us-ascii?Q?=5BMPlayer=2Ddev=2Deng=5D=20development=20project=20=22mplayer=20encapsulation=20for =09Apple=20TV=22?= X-Beenthere: mplayer-dev-eng@mplayerhq.hu X-Mailman-Version: 2.1.9 ... But the KMail's message suddenly contains: ... X-Mailer: Apple Mail (2.919.2) X-Mailman-Approved-At: Thu, 13 Nov 2008 16:45:38 +0100 X-Content-Filtered-By: Mailman/MimeDel 2.1.9 Subject: =?us-ascii?Q?=5BMPlayer=2Ddev=2Deng=5D=20development=20project=20=22mplayer=20encapsulation=20for X-UID: Status: RO X-Status: RC X-KMail-EncryptionState: N X-KMail-SignatureState: N X-KMail-MDN-Sent: =09Apple=20TV=22?= X-Beenthere: mplayer-dev-eng@mplayerhq.hu X-Mailman-Version: 2.1.9 ... So the problem is KMail adding it's headers in the middle of other header field content.
Note: the "Subject:" line is split onto two lines only, not as you can see here - the "=?us-ascii?Q?=..." string is on the same line.
Created attachment 28877 [details] Original message from free mail
I confirm the problem with the current svn snapshot.
I experience the same problem with the KMail version installed with Kubuntu (Intrepid Ibex).
I see exactly the same behaviour with Kubuntu Intrepid Ibex packages, KDE 4.1.3, KMail 1.10.3. I would suggest to change the title of this bug since it has been proven that long To: lists are not the reason for corrupted emails.
It would be interresting if all the original messages (before KMail breaks them) have the same problem: multi-line international header in this form: <Header>: =?<charset>?Q?<some characters><new-line> <some characters>?= The problem I see here is that the second line is not space/tab indented (it should be for header continuation). Anyway, the end can be found by searching for "?=". As far as I can tell the mailman web interface doesn't have this problem - compare the message at http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2008-November/058997.html with the "Original message from free mail" attachment and the KMail's one in comment #14.
Oldrich: I don't think that this is the common denominator of concerned emails. If you look at the example email in bug 175965, there is no multi-line international header.
(In reply to comment #21) > Oldrich: I don't think that this is the common denominator of concerned emails. > If you look at the example email in bug 175965, there is no multi-line > international header. It looks like there is also problem with header continuation - the second line (the line comment) doesn't start with space/tab character. But for this the original message (before KMail processed it) would be needed for final check. I've also tried to look at source files, but I'm unable to make head or tail of it.
Marking as a wish since the example emails are malformed. The wish is to have KMail make an intelligent guess when a header should end and not rely on the RFC (http://tools.ietf.org/html/rfc5322#section-2.2.3).
Created attachment 39599 [details] Patch for malformed wrapping of encoded-words (international headers)
If anybody here has problem with kmail messages, I think I can have a look at your particular parsing problem (I had a look at version 4.3.85 - beta2). But I will need the message (original one) before it got corrupted.
If I'm right, kmail uses mimelib from kdepim package. This is where my fix goes. There exists also a kmime package that is without a fix, but that one is not used by kmail anyway. Maybe it is not needed to get the original message, it is quite clear which headers were added by kmail. Just attach your message (save it to mbox format from kmail first) and I will have a look at it.
I went through my corrupted emails and the same line wrapping problem can be seen in CC and To fields, so it might be fixed by my fix too. But it depends on your particular message. What I fix is the line starting with "=09", which is apparently a tab character wrongly encoded into a printable ASCII string - that should be a header continuation character.
Created attachment 39601 [details] [mimelib-4.3.85] Patch for malformed wrapping of encoded-words (international headers) Better patch that finds both =09 (tab) and =20 (space) as continuation chracters.
I've found that the message list still has some wrong subjects, so the patch needs to be improved.
Created attachment 39602 [details] [kmail-4.3.85] Patch for malformed wrapping of encoded-words (international headers) Here is the patch for kmail for correct subject parsing.
> There exists also a kmime package that is without a fix, but that one is > not used by kmail anyway. KMail 1 uses mimelib, but KMail 2 and KNode use KMime. We're currently working on KMail 2 in the akonadi-ports branch, which will shortly be merged back to trunk. It would be good to have this fix, including a unit test, for KMime as well. Can you please post your patch to reviewboard.kde.org, usually more people check patches there, since a bugs.kde.org notification mail can drown in the noise. See http://techbase.kde.org/Contribute/Send_Patches. Thanks for the patch :)
(In reply to comment #31) > > There exists also a kmime package that is without a fix, but that one is > > not used by kmail anyway. > > KMail 1 uses mimelib, but KMail 2 and KNode use KMime. We're currently working > on KMail 2 in the akonadi-ports branch, which will shortly be merged back to > trunk. > > It would be good to have this fix, including a unit test, for KMime as well. I will have a look at that during next week. I've already checked-out branches/work/akonadi-ports for this purpose. I've updated KMime first, but kmail didn't use it, so I was little bit upset afterwards :-) The KMime patch needs to be split into two pieces anyway, I will do my best to do this with Subversion (thanks God KDE is moving to Git). > Can you please post your patch to reviewboard.kde.org, usually more people > check patches there, since a bugs.kde.org notification mail can drown in the > noise. See http://techbase.kde.org/Contribute/Send_Patches. Ok, I will send it (next week). > Thanks for the patch :) Thanks for your thanks :-)
I've sent the patch for kmime on work/akonadi-ports, let's see what is the opinion there.
Does it make sense to send the patches for 4.4 too?
I have commented on the review request for the KMime patch. About the mimelib + KMail 1 patches: How sure are you those don't introduce additional regressions, like crashes because the array bounds are not checked? If you think the patches are stable enough, please post them to reviewboard as well, so we can commit them to the KDE 4.4 branch.
(In reply to comment #35) > I have commented on the review request for the KMime patch. I've updated the patch. > About the mimelib + KMail 1 patches: How sure are you those don't introduce > additional regressions, like crashes because the array bounds are not checked? > If you think the patches are stable enough, please post them to reviewboard as > well, so we can commit them to the KDE 4.4 branch. The patches should not cause any regression AFAICT, I can imagine only some assert coming when comparing value at the end of the string. I will have a look at it during the week and prepare the patch for 4.4 branch.
SVN commit 1076273 by tmcguire: Handle malformed multi-line headers in a better way, some clients seem to send those. Patch by Oldrich Jedlicka <oldium.pro at seznam dot cz>, thank you! reviewboard.kde.org/r/2561 BUG: 86302 M +2 -32 kmime_header_parsing.cpp M +71 -24 kmime_util.cpp M +8 -0 kmime_util_p.h M +20 -1 tests/kmime_util_test.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=1076273
I've added review request for patches for kmail-4.4. My opinion is that there should be no regression and no additional assert().
SVN commit 1077597 by tmcguire: Handle malformed multiline headers correctly in this branch as well. http://reviewboard.kde.org/r/2644/ CCBUG: 86302 M +8 -0 kmail/kmfoldermaildir.cpp M +8 -0 kmail/kmfoldermbox.cpp M +9 -1 mimelib/entity.cpp M +31 -4 mimelib/headers.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=1077597