Bug 392239

Summary: CR linebreaks in message view are not shown
Product: [Applications] kmail2 Reporter: kzi <kde>
Component: generalAssignee: kdepim bugs <kdepim-bugs>
Status: CONFIRMED ---    
Severity: normal CC: montel, null
Priority: NOR    
Version: 5.7.2   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: mbox where plain text has 0A (LF) line breaks
mbox where plain text has 0D (CR) line breaks
mbox where plain text has 0D0A (CRLF) line breaks

Description kzi 2018-03-23 16:15:07 UTC
Messages or parts of multipart messages with
    Content-Type: text/plain;charset="utf-8"
    Content-Transfer-Encoding: base64
are displayed without line breaks (but spaces are kept).

I apologize to not include an example, but the mails in question contain information that I'd rather not have disclosed here.

I verified by base64-decoding the blocks in question, and the results show line breaks and thus appear as "well-formatted plain text".

Mails from the same source but with
    Content-Type: text/plain;charset="utf-8"
    Content-Transfer-Encoding: quoted-printable
are shown alright ,so it's likely due to base64 post-processing.

Encountered on Tumbleweed.
Comment 1 null 2020-05-05 22:47:00 UTC
I tried reproducing with the following mbox file using git master:

MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: text/plain; charset="utf-8"

YWFhIGJiYgpjY2MgZGRkCmVlZSBmZmYK

According to "base64 -d" this should be shown as:

aaa bbb
ccc ddd
eee fff

..which is exactly what KMail showed, including line breaks, i.e. I'm unable to reproduce. (Let me know in case I misunderstood the steps to reproduce.)

Please check whether the issue still happens with a recent version of KMail, and add a test case. You can use a text editor to redact the mail after saving as mbox, of course.
Comment 2 Bug Janitor Service 2020-05-20 04:33:11 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 3 kzi 2020-05-21 16:18:29 UTC
Created attachment 128667 [details]
mbox where plain text has 0A (LF) line breaks
Comment 4 kzi 2020-05-21 16:19:03 UTC
Created attachment 128668 [details]
mbox where plain text has 0D (CR) line breaks
Comment 5 kzi 2020-05-21 16:19:22 UTC
Created attachment 128669 [details]
mbox where plain text has 0D0A (CRLF) line breaks
Comment 6 kzi 2020-05-21 16:31:48 UTC
Thx for hinting mbox redaction.

While trying to come up with a disclosable example I found the cause is the line break character used in the base64-decoded plain text message: Apparently 0D (CR) characters are stripped from the resulting message.

Please find attached made-up mbox files with different line break encodings of the source plain text. You'll find that 0D.mbox doesn't break correctly while the other two do.

I created the files after the mail example where I first encountered the issue. To me it is not unlikely that the issue may be completely independent of Content-Type and Content-Transfer-Encoding. However I do not have the time now to do more thorough and complete testing. Feel free to create a more suitable bug ticket if sensible.

The issue continues to exist in KMail 5.14.0.
Comment 7 kzi 2020-05-21 16:39:53 UTC
(In reply to kzi from comment #6)
> Apparently 0D (CR) characters are stripped from the resulting message.
...or just not rendered as line break.
Comment 8 null 2020-05-24 18:45:00 UTC
Thanks, test cases are always helpful, as they often allow to better pinpoint the core issue. Glad you found the cause!

Can confirm that CR linebreaks in base64-encoded messages are not displayed using current git master.

In fact, this is not really specific to base64 encoding: Converting any mbox file (e.g. with "Content-Transfer-Encoding: 7Bit") with "unix2mac" to use CR linebreaks yields an empty message view already, and converting only the mail body shows the header at least, but misses linebreaks in the message text.

Note that only the message view is affected. When opening the "Message as Plain Text" window through "View Source", the linebreaks are displayed just fine.
Comment 9 Laurent Montel 2020-05-24 19:45:37 UTC
indeed I confirm it.
But which mailer generates it ?

It seems that we can't load it from thunderbird.
Comment 10 kzi 2020-05-24 20:35:24 UTC
Salut Laurent,

> But which mailer generates it ?
While I think KMail should be capable do deal with any kind of line break, no matter the source, the answer (taken from the header) is:

esmtps (Exim 4.92.3)

I received the mail from the accounting of a web shop as an invoice carrier (hence the nondisclosure in the first place), so it's likely to originate from some kind of ERP system.
Comment 11 Laurent Montel 2020-05-25 05:07:33 UTC
Git commit 65372b5304a504446259ec1504aa5e2856105e46 by Laurent Montel.
Committed on 25/05/2020 at 05:06.
Pushed by mlaurent into branch 'release/20.04'.

Add autotest for bug 392239

So we extract correctly \r Now we need to know why it doesn't display
them

A  +19   -0    autotests/data/mails/bug392239.mbox
M  +9    -0    autotests/messagetest.cpp
M  +1    -0    autotests/messagetest.h

https://invent.kde.org/pim/kmime/commit/65372b5304a504446259ec1504aa5e2856105e46