Bug 88702 - view source doesn't show whole source
Summary: view source doesn't show whole source
Status: RESOLVED UNMAINTAINED
Alias: None
Product: kmail
Classification: Applications
Component: IMAP (show other bugs)
Version: 1.10.0
Platform: Debian testing Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords: triaged
: 94831 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-09-02 17:27 UTC by Andy Parkins
Modified: 2015-04-12 10:22 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
A trivial example, kmail replaces a tab by two spaces in the header (222 bytes, text/plain)
2006-05-16 19:34 UTC, Mark Martinec
Details
example 2, Content-Type folded into two lines (245 bytes, text/plain)
2006-05-16 19:41 UTC, Mark Martinec
Details
fixes bad MIME structure (779 bytes, text/plain)
2006-05-16 22:13 UTC, Mark Martinec
Details
example referred to in my previous comment, illustrates both header changes (685 bytes, text/plain)
2006-05-17 01:58 UTC, Mark Martinec
Details
Illustrates vanished strings from view source window (2.33 KB, text/plain)
2006-10-24 17:22 UTC, Mark Martinec
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andy Parkins 2004-09-02 17:27:42 UTC
Version:            (using KDE KDE 3.3.0)
Installed from:    Debian testing/unstable Packages
OS:                Linux

View source doesn't always show the true message source.  For example:

View source in kmail:
---- BEGIN EXTRACT[1] ----
Return-path: <andyp@ironbrew.leaseline-systems.co.uk>
Envelope-to: andyp@leaseline.plus.com
Delivery-date: Thu, 02 Sep 2004 16:01:44 +0100
Received: from [192.168.3.21] (helo=ironbrew.leaseline-systems.co.uk)
 by eggnog.leaseline-systems.co.uk with esmtp (Exim 3.35 #1 (Debian))
 id 1C2t5q-0007nE-00
 for <andyp@leaseline.plus.com>; Thu, 02 Sep 2004 16:01:42 +0100
Received: from andyp by ironbrew.leaseline-systems.co.uk with local (Exim 3.36 #1 (Debian))
 id 1C2t5l-00033Q-00
 for <andyp@leaseline.plus.com>; Thu, 02 Sep 2004 16:01:37 +0100
Content-Transfer-Encoding: binary
Content-Type: multipart/related;
  boundary="_----------=_1094137297117420"
MIME-Version: 1.0
X-Mailer: MIME::Lite 2.117  (F2.72; A1.62; B3.01; Q3.01)
Date: Thu, 2 Sep 2004 15:01:37 UT
To: andyp@leaseline.plus.com
Subject: Test of multipart/related email message
Message-Id: <E1C2t5l-00033Q-00@ironbrew.leaseline-systems.co.uk>
From: Andy Parkins <andyp@ironbrew.leaseline-systems.co.uk>
X-UID: 115
X-Length: 54689

--_----------=_1094137297117420


<html>
<body>
<p>Hello,
---- END EXTRACT[1] ----

The true source can be seen by pulling a section out of the mbox file on disk:

---- BEGIN EXTRACT[2] ----
From andyp@ironbrew.leaseline-systems.co.uk Thu Sep 02 16:01:44 2004
Return-path: <andyp@ironbrew.leaseline-systems.co.uk>
Envelope-to: andyp@leaseline.plus.com
Delivery-date: Thu, 02 Sep 2004 16:01:44 +0100
Received: from [192.168.3.21] (helo=ironbrew.leaseline-systems.co.uk)
    by eggnog.leaseline-systems.co.uk with esmtp (Exim 3.35 #1 (Debian))
    id 1C2t5q-0007nE-00
    for <andyp@leaseline.plus.com>; Thu, 02 Sep 2004 16:01:42 +0100
Received: from andyp by ironbrew.leaseline-systems.co.uk with local (Exim 3.36 #1 (Debian))
    id 1C2t5l-00033Q-00
    for <andyp@leaseline.plus.com>; Thu, 02 Sep 2004 16:01:37 +0100
Content-Transfer-Encoding: binary
Content-Type: multipart/related; boundary="_----------=_1094137297117420"
MIME-Version: 1.0
X-Mailer: MIME::Lite 2.117  (F2.72; A1.62; B3.01; Q3.01)
Date: Thu, 2 Sep 2004 15:01:37 UT
To: andyp@leaseline.plus.com
Subject: Test of multipart/related email message
Message-Id: <E1C2t5l-00033Q-00@ironbrew.leaseline-systems.co.uk>
From: Andy Parkins <andyp@ironbrew.leaseline-systems.co.uk>

This is a multi-part message in MIME format.

--_----------=_1094137297117420
Content-Disposition: inline
Content-Length: 596
Content-Transfer-Encoding: binary
Content-Type: text/html


<html>
<body>
<p>Hello,
---- END EXTRACT[2] ----

As far as I can see the following have been changed in the KMail "View Source"

* The top "From:" header has been removed
* Space has been removed in the lines following the "Received:" headers
* The "Content-Type:" header has been broken into two lines
* An "X-UID:" header has been added
* An "X-Length:" header has been added
* The non-visible body text "This is a multi-part message in MIME format." has been removed
* The "Content-Disposition:", "Content-Length:", "Content-Transfer-Encoding:" and "Content-Type:" headers have been removed from after the boundary

Some of these are obviously livable with, but it's a bit confusing to have a "view source" option that doesn't really view the source.
Comment 1 runevi 2004-11-12 09:42:57 UTC
Also, in SuSE 9.2 (Kmail 3.3.0) -- if there are attachments, the source of the attachments is not shown.  
Comment 2 Mark Martinec 2006-05-15 19:26:59 UTC
> * The top "From:" header has been removed

This line is not part of a mail header, the same address is present
in the Return-path header field, which *is* part of a mail header.

The X-UID: and X-Length: may have been added by the IMAP server
on the fly, I'm not sure.

Remaining changes are more troublesome: reformatting is entirely
unnecessary, undesirable and unexpected when 'viewing source';
Modified or removed subheaders likewise.

I have seen similar cases:

- removed recipient's email address in the 'for' subfield of all
  Received: header fields, perhaps because it contained a plus character
  (confirmed their presence by looking directly in the imap folder or
  using another IMAP client);

- completely screwed up mail (missing most of the body, broken MIME structure),
  resulting in chopped-off mail when 'saved as' - while the same message
  was perfectly legitimate when checked in the imap server's file.

Seems like kmail is taking liberty at changing mail according to its
beliefs, which sometimes are mistaken. Even when they are not mistaken,
it is unacceptable that viewing message source or saving an entire message
results in a message that is different from the original on the IMAP server.

  Mark
Comment 3 Mark Martinec 2006-05-15 19:29:35 UTC
*** This bug has been confirmed by popular vote. ***
Comment 4 Ismail Onur Filiz 2006-05-16 18:58:46 UTC
If possible, please attach emails that exhibit the behaviour you described.
Comment 5 Mark Martinec 2006-05-16 19:34:55 UTC
Created attachment 16126 [details]
A trivial example, kmail replaces a tab by two spaces in the header

Here is a trivial example, kmail replaces a tab by two spaces in the
Content-Type header field in the mail header. Attached diff is between
a file snatched from imap server (cyrus) and a file produced by
kmail with 'Save as' from the rightclick menu. I'll try to provide
more examples when I get hold of them.
Comment 6 Mark Martinec 2006-05-16 19:41:12 UTC
Created attachment 16127 [details]
example 2, Content-Type folded into two lines

Here is another one, as it happens the example comes from the
bugs.kde.org notification minutes ago. In this example
the Content-Type header field has been folded into two lines.
Comment 7 Mark Martinec 2006-05-16 22:13:25 UTC
Created attachment 16128 [details]
fixes bad MIME structure

I found an example where kmail modified mail body in an attempt
to fix a broken MIME structure of the original mail. I'm sorry
for having to strip off most material from the attached example
to protect the innocent, but I kept all MIME headers and subheaders
(in the attached message), and I'll describe what happened.

The original mail (still intact in IMAP folder) contains an attachment
of type message/rfc822, which has a missing terminating boundary
because the attached message was truncated. This makes the MIME structure
broken. Kmail seems to try to repair things, and inserts a terminating
inner boundary:

@@ -24,4 +24,7 @@
 ------=_NextPart_000_0009_565D7669.6C725D12

+
+------=_NextPart_000_0009_565D7669.6C725D12--
+
 --------------050803020100010000060502

in a well-intended attempt to fix the message.

Well, this makes it unsuitable to rely on kmail's viewing/saving
of mail sources for diagnostic purposes, as it covers errors
in the original mail. One needs to fetch file from IMAP
folder or use another mail reader to be able to reliably
determine contents of a mail. Not to mention that it breaks
signatures and DomainKeys.

Btw, To and Cc lists are also routinely reformatted,
which again breaks DomainKeys if one would want to check
the mail late in the food chain.
Comment 8 Ingo Klöcker 2006-05-17 00:56:22 UTC
View source is not meant to show the message exactly as it resides on the IMAP server (which might not even be as RFC 2822 message) or as the IMAP server has sent it to KMail. It's meant to show the message as KMail sees it (possibly after re-folding some headers, but are you sure this hasn't been done by the IMAP server before transfering the message to KMail?). The changes in the MIME structure are probably related to loading-of-demand of attachments. For the same reason the source of attachments might be missing. KMail definitely doesn't contain any code changing Received header lines (like in comment #2). This must have been done by someone else.

Please provide examples which prove that KMail breaks OpenPGP/MIME or S/MIME signatures before you claim that KMail *might* break signatures. I'm extensively using email signatures and am not aware of any bugs in KMail breaking signatures.

Did you actually fetch the messages with another email client from the IMAP server? If not, then why do you think KMail has made those changes and not the IMAP server itself?
Comment 9 Mark Martinec 2006-05-17 01:56:03 UTC
> View source is not meant to show the message exactly as it resides on the
> IMAP server (which might not even be as RFC 2822 message) or as the IMAP
> server has sent it to KMail. It's meant to show the message as KMail
> sees it (possibly after re-folding some headers

Well, yes, so it seems. I guess we'll have to live with it, although
it would be nice to at least clearly document it, so that one does not
attempt to diagnose mail problems by looking at mail source as KMail
shows it.

> but are you sure this hasn't been done by the IMAP server before
> transfering the message to KMail?
> Did you actually fetch the messages with another email client from
> the IMAP server? If not, then why do you think KMail has made those
> changes and not the IMAP server itself? 

Yes, I'm sure. I just verified it by capturing TCP IMAP traffic
(tcpdump, ethereal), and double checked by accessing the same message
on the IMAP server from thunderbird. KMail reformats Content-Type
and Cc list (and To list in other examples), thunderbird does not,
and shows it as it is on the IMAP server's file. The diff of this
particular message is again attached next (it illustrates both changes
in one example).

> KMail definitely doesn't contain any code changing Received header
> lines (like in comment #2). This must have been done by someone else.

I stared at such example ten days ago, but since then upgraded
KDE to the next minor release, so I can't tell if the problem was
fixed by now or not. It was certainly very much present on one
particular mail (not generally), and only showed (repeatedly) on kmail,
not on thunderbird and not on IMAP file (which I went checking to,
after disbelieveing that Received header field could have been so
broken). I'll keep watching if it re-occurs, I can't reproduce it now.

> Please provide examples which prove that KMail breaks OpenPGP/MIME or
> S/MIME signatures before you claim that KMail *might* break signatures.

I doubt anyone does DomainKeys checking after the messages is on MUA,
so my comment was more theoretical - reformatted To header would
likely affect DomainKeys according to its specs, assuming the sender
includes the To header in the list of header fields to be checked.
I believe OpenPGP/MIME or S/MIME do not care for header, so these would
probably not be affected.
Comment 10 Mark Martinec 2006-05-17 01:58:27 UTC
Created attachment 16131 [details]
example referred to in my previous comment, illustrates both header changes
Comment 11 Mark Martinec 2006-10-24 17:15:37 UTC
>> KMail definitely doesn't contain any code changing Received header 
>> lines (like in comment #2). This must have been done by someone else. 
> 
> I stared at such example ten days ago, but since then upgraded 
> KDE to the next minor release, so I can't tell if the problem was 
> fixed by now or not. It was certainly very much present on one 
> particular mail (not generally), and only showed (repeatedly) on kmail, 
> not on thunderbird and not on IMAP file

I have more information on this one, and it seems like a different
kind of a bug the rest in this topic, just corrupting the display of
'view source' window. It seems to happen when viewing long messages
(e.g. 1 MB) with attachments (the last time I've seen it today
it was a base64-encoded PDF file, but is not specific to that).
It only shows in a separate window as opened by 'view source',
but if message is save to a file (or viewed by another browser)
the header is intact.

What happens is that several header fields just lose header field
body, either entirely (Return-Path,Message-ID,...), or just
part of it (Received loses 'for' subfield, From retains comment
but looses address). On a second look, it seems like all strings
enclosed in <angle-brackets> just vanish.

Attached is a diff between a mail header as saved to a file by kmail,
and a screen cut/paste taken from 'view source' window when viewing
the same message. This is 1.9.4 (KDE 3.5.4) from FreeBSD ports,
but the problem has been there for a couple of versions.



Comment 12 Mark Martinec 2006-10-24 17:22:34 UTC
Created attachment 18248 [details]
Illustrates vanished strings from view source window

Promised diff
Comment 13 Mark Martinec 2006-10-24 17:24:37 UTC
(I should add that to protect the innocent, email and host addresses
and IP addresses in the attached diff were changed, but the rest is intact)
Comment 14 Thomas McGuire 2007-03-21 02:31:38 UTC
*** Bug 94831 has been marked as a duplicate of this bug. ***
Comment 15 George Kiagiadakis 2008-08-31 17:58:23 UTC
This bug seems to be still present in kmail 1.10.0 svn r854573. I saw some minor reformating issues with some emails.

For example, I had an email containing these on the server side:
X-UID: 7
Status: RO
Content-Length: 2017

and kmail shows:
X-Length: 3262
X-UID: 7

I also tried with the email that is attached on bug 94831 and I get some minor reformations like the Content-type header split into two lines and an empty line removed. However, mime part boundaries did not seem to be changed like they are in the attachment on bug 94831 comment 1.
Comment 16 bruce.lilly 2010-09-09 22:07:59 UTC
switched to GNOME
Comment 17 Laurent Montel 2015-04-12 10:22:41 UTC
Thank you for taking the time to file a bug report.

KMail2 was released in 2011, and the entire code base went through significant changes. We are currently in the process of porting to Qt5 and KF5. It is unlikely that these bugs are still valid in KMail2.

We welcome you to try out KMail 2 with the KDE 4.14 release and give your feedback.