Bug 53015

Subject: Re:  New: Signature generation/check fails for Mails forwarded as attachement

The problem is that some MTAs change the forwarded message which of 
course results in a bad signature. This is due to conflicting RFCs. 
According to some RFC messages which are forwarded as attachment must 
be forwarded as is (i.e. with cte 8bit). OTOH the PGP/MIME specs tell 
us that signed message parts must not be 8bit. Therefore KMail encodes 
forwarded messages with the quoted-printable encoding. Unfortunately 
some MTAs seem to freak out when the header of a forwarded message 
isn't a valid header. Those MTAs then try to fix the headers and this 
breaks the signature. IMO this problem is at least partly caused by 
overly correct MTAs. Why does an MTA bother to fix the headers of an 
attached messages?

This even happens for simple, plain ASCII mails written by KMail itself, so neither 8bit 
issues nor invalid headers can be the cause of this problem.  
mutt does not have any similar issues, either. 
 

Forgot to add: I'm using KMail 1.5.1 (KDE 3.1.1).

I can confirm that this problem still exists
and that mutt (Mutt 1.5.0-gpgme-030408) does not have this problem.
(KMail --version
Qt: 3.1.2
KDE: 3.1.1a
KMail: kroupware-RC2
)

It seem to only occur with S/MIME in my last tests so far.

Adding one line qualified the signature again in mutt.
I'm attaching two mails and a diff.
Note that the forwarded message validated already verified okay.

Created attachment 1562 [details]
s/mime mail forwarded and signed w broken sig

An signed S/MIME email forwarded with KMail kroupware-RC2.
Does not verify okay (in mutts and KMail).
Inner email verifies okay.

Created attachment 1563 [details]
s/mime mail forwarde and signed w reparied sig

This one has one lined changed and verifies okay.

Created attachment 1564 [details]
context diff of last two mails

The diff between the working and non-working mail.
The line was added within mutt, thus the Content-Length value changes.

Might it be that we erroreously let the message be 'assembled' (before sending it) 
after we have singed everything? 
 
We had such a bug previously: this was due to us changing some header fields - 
which resulted in the Mimelib running a complete assemble() over the whole message  
->  which in turn resulted in us having one line-break less than we hade in our 
manually composed message before. 
 
see kmmessage.cpp, my old comment: 
  // We better do the assembling ourselves now to prevent the 
  // mimelib from changing the message *body*.  (khz, 10.8.2002) 
 

Created attachment 1576 [details]
MIME mail as mangled by Mailman 2.1.1 w/o signature but with problem!

I think I've cornercased the bug further
with a couple of tests.

It is triggered by a mime structure where an empty line exists.
This empy lines gets deleted before sending, the mails still verify
okay in the "sended mail" folder. I've verified that the mails handed
over to sendmail miss this empty line, though. I can exclude the MTA though.

The problematic mime structure is created by newer mailman versions
(from 2.1 on) under certain conditions. First condition: It must be a 
mime mail. Second condition it is not easy to just add an additional
mime part. So with application/signature Mailman now wraps
and multipart/mixed around to add its list signature part.
So it happends with S/MIME signed emails and probably OpenPGP signed
emails, too. Because I don't know how to trigger this behavious I've
crippeled a mail to look similiar, but not having a signature.
See the attached email.

KMail cannot properly forward as attachement 
and sign it with S/MIME nor OpenPGP.
Thus we probably need to change the topic of the bug
to clear up that it is a special mime structure triggering it
and that it is not specific to S/MIME.

Just to make double sure: 
Mailman is not the problem, as any email with this structure
would tickle the bug in KMail.

It still does not seem to be clear enough.

The bug is:
Forwarding and signing an email with S/MIME or OpenPGP
does not work in KMail if the email we want to forward
has a certain mime structure. (See attachement 1576 for an example).


Some  emails which have been coming through Mailman
just happen to have such a structure that they tickle the bug
this is why and how I noticed it.
I'm not sure if that mine structure is fine according to all
standards, still it can easily be created by other applications.
The bug therefore needs to be fixed within kmail.
 

Subject: Re:  Signature generation/check fails for Mails forwarded as attachement

This bug is caused by mimelib because it seems to normalize the message 
(i.e. it removes superfluous empty lines between message parts). So 
someone (not me!) has to go through mimelib hell and fix the bug in 
mimelib.

It's great that you found this! 
 
Please tell me how exactly you found out that this is the reason: can you name a place 
in the code where the mial is still OK, then mimelib is called and then the mail is bad? 
 
Or did you find it by some other way...? 
 
I am asking because I could have a look at the mimelib: but please give me a hint 
where in KMail I can see that mimelib is making this mistake we are looking for here! 
 

The failure is even easier to show than previously reported (kmail-1,5, kde-3.1).  
1. Send youself an email signed in GPG. 
2. When you receive it, it's painted green and recognized...good up to now. 
3. Double click on the mail received (or sent...doesn't matter). It comes up red. 
 
Or maybe this is a different bug. 
 
Mark 
makis(no-spam)@genasys.com 

Mark: I believe this to be a different bug, because I cannot reproduce it
       with the KMail I have here (kroupware-1.0.1) and it seems that the
message     send itself
       is okay. 

Karl-Heinz: You should be able to reproduce the bug, trying to forward the
        bad email I've submitted earlier. Then you should be able to track
        where exactely the bug occurs in mimelib.

Subject: Re:  Signature generation/check fails for Mails forwarded as attachement

mimelib is no longer the problem. Now the problem is simply that we 
qp-encode forwarded messages if they contain 8 bit characters. But this 
leads to line breaks in Received headers (because they are usually too 
long). This in turn lead to an invalid message (because the second part 
of a wrapped line is no valid header). Therefore, probably mimelib, 
probably some mail servers insert an empty line before the "invalid" 
header line. And this obviously breaks the signature.

The real fix for this problem is to go over the to-be-attached message 
and recode all message parts that aren't 7-bit clean (so all message 
parts that contain 8-bit characters need to be qp- or base64-encoded). 
After we made the message 7-bit clean we can attach it and sign it 
together with the rest of the message without problem.

Actually this shouldn't be very difficult. But someone has to implement 
it. Rough idea:
1.) create the partNode tree,
2.) traverse the tree children first and recode children that are not 
7-bit clean, 
3.) re-assemble the message.

Encoding a submime part should be okay,
the problem then seems to be that those header lines
in the attached messages must not be reformatted or touched at all.

Inserting that line should also not happen, though.
Not by MTAs nor by MUAs. 

(Just guessing on what I've understood from the problem so far.)

I'll give it a shot...

Fixed. Can't commit due to deep freeze, though. Attaching patch.

Created attachment 4182 [details]
Fix

is this bug really fixed? my signature check fails always when the mail contains an attachment. mails without an attachment are ok.
using kde 3.3.1 with kmail 1.7.1.

regards,
dominik

I have problem too with signed attachements.
I forward myself a mail and sign it, -> the signature is invalid.
(The headers are completely broken)
When forwarding withut signing, the mail forwarded is OK.

Problem in headers:
headers from the forwarded message, unsigned:

Return-Path: <netadmin-owner@u-strasbg.fr>
Received: from ns2.u-strasbg.fr (ns2.u-strasbg.fr [130.79.200.3])
          by urs.u-strasbg.fr (8.12.8/jtpda-5.4) with ESMTP id iBEBSsjU010967
          ; Tue, 14 Dec 2004 12:28:54 +0100


headers from the forwarded message, signed:

Return-Path: <netadmin-owner@u-strasbg.fr>
Received: from ns2.u-strasbg.fr (ns2.u-strasbg.fr [130.79.200.3])
          by urs.u-strasbg.fr (8.12.8/jtpda-5.4) with ESMTP id iBEBSsjU0109=

67
          ; Tue, 14 Dec 2004 12:28:54 +0100



Note the "=" and the newline present in the modified headers.

Reopening this bug after the two problem reports.
It would be nice if the reporters could try again and confirm
the bug or the fix.

Bug still outstanding in KMail 1.9.1 (KDE 3.5.2).

Note that this bug blocks Bug 38767 when signing is enabled by default.

Just checked that following the steps discribed in comment #3 and could not reproduce. So this seems to be fixed in 4.1

Dominik, thanks for testing this issue!
Can you be more precise about which version you've tested and how?
(I am asking because you've mentioned Comment #3 which does not contain
instructions how to recreate the issue.)

Sorry for this confusing comment. I've no idea what I meant with comment #3.
So to be clear, here are my steps:
1. Loaded your testcase: http://bugs.kde.org/attachment.cgi?id=1576
2. Opened the mail with kmail 1.10.0
3. choose "Forward as attachement"
4. choose "sign e-mail"
5. and sent it to another one of my mail addresses
The received mail was signed properly and drawn with a green box around it.
Did I miss anything?

Okay, I did an addition test with 
Kontact Version 1.2.9 (enterprise35 20080826.852422)
together with Dominiks test, I consider the issue resolved now.