Bug 173503 - Attachments are totally broken (and sometimes the messages also) sent from KMail
Summary: Attachments are totally broken (and sometimes the messages also) sent from KMail
Status: CLOSED FIXED
Alias: None
Product: kmail
Classification: Applications
Component: sending (show other bugs)
Version: 1.10.90
Platform: unspecified Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords: triaged
Depends on:
Blocks:
 
Reported: 2008-10-25 11:58 UTC by Unknown
Modified: 2009-01-29 18:00 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
The whole message with attachments became Chinese... (368.28 KB, image/jpeg)
2008-10-25 11:59 UTC, Unknown
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Unknown 2008-10-25 11:58:27 UTC
Version:           1.10.90 (using 4.1.70 (KDE 4.1.70 (KDE 4.2 >= 20081016)) "release 1.6", KDE:KDE4:UNSTABLE:Desktop / openSUSE_Factory)
Compiler:          gcc
OS:                Linux (i686) release 2.6.25.18-0.1-pae

I have 2 CSV files and I wanted to send them with KMail. But the names became totally bad after sending and the letter was unreadable.

Here are some example:

1st CSV:
Original name: 2nd_inn.csv
Name appears in GMail: ㄶ䉅✧╆䔥
Name appears in KMail: unnamed
Header:
--Boundary-00=_CnuAJ5xsxJdnfof
Content-Type: text/csv;
  charset="UTF-16BE";
  name*=UTF-16BE''%FE%FF
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename*=UTF-16BE''%FE%FF

2nd CSV:
Original name: text21.csv
Name appears in GMail: 呕ⵆ㘱䕌✧䘥╆䕆�
Name appears in KMail: unnamed
Header:
--Boundary-00=_CnuAJ5xsxJdnfof
Content-Type: text/csv;
  charset="UTF-16LE";
  name*=UTF-16LE''%FF%FEt
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename*=UTF-16LE''%FF%FEt

etc...

I can read the message text in GMail, but not in Thunderbird. I'll attach a TB screenshot (which is horrible, to tell the truth).
Comment 1 Unknown 2008-10-25 11:59:53 UTC
Created attachment 28130 [details]
The whole message with attachments became Chinese...
Comment 2 Jaime Torres 2008-10-25 13:49:32 UTC
Just to be sure this is not a problem with Thunderbird not understanding the utf-16 encoding, how is configured the "message property" for sending in this account: Mime-compliant or Allow 8-bit. 

If it is the second, please try again changing it to Mime-compliant.
Comment 3 Unknown 2008-10-25 13:57:52 UTC
I didn't change the default, now it is: "MIME-compliant (Quoted printable)".

The message is not only bad in Thunderbird but also in GMail and in KMail (when I received it from myself). In the latter two at least I could read the message text.
Comment 4 Unknown 2008-11-06 20:36:49 UTC
The bug still exists in 4.1.72-2.1 either with 7bit, base64 or quoted printable encoding.

Any file which was sent with KMail will arrive in corrupted state...

Some examples:

Original filename: 5. mérés.odt
Name appears in GMail: iso-8859-2''5%2E%20m%E9r%E9s%2Eodt
Header:
--Boundary-00=_3S0EJftFV+qUeyk
Content-Type: application/vnd.oasis.opendocument.text;
  name*=iso-8859-2''5%2E%20m%E9r%E9s%2Eodt
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename*=UTF-16LE''%FF%FE5

Original filename: baudrate_generátor.v
Name appears in GMail: UTF-8''baudrate_gener%C3%A1tor%2Ev
Header:
--Boundary-00=_3S0EJftFV+qUeyk
Content-Type: text/plain;
  charset="UTF-8";
  name*=UTF-8''baudrate_gener%C3%A1tor%2Ev
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename*=UTF-8''baudrate_gener%C3%A1tor%2Ev

Original filename: bekötés.ucf
Name appears in GMail: UTF-8''bek%C3%B6t%C3%A9s%2Eucf
Header:
--Boundary-00=_3S0EJftFV+qUeyk
Content-Type: text/plain;
  charset="UTF-8";
  name*=UTF-8''bek%C3%B6t%C3%A9s%2Eucf
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename*=UTF-8''bek%C3%B6t%C3%A9s%2Eucf

Original name: main.v
Name appears in GMail: 呕ⵆ㘱䕌✧䘥╆䕆�
Header:
--Boundary-00=_3S0EJftFV+qUeyk
Content-Type: text/plain;
  charset="UTF-16LE";
  name*=UTF-16LE''%FF%FEm
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename*=UTF-16LE''%FF%FEm
Comment 5 George Kiagiadakis 2008-11-12 10:31:08 UTC
I think this information is enough to forward the bug to the kmail developers, although I have no way to try this myself.
Comment 6 Thomas McGuire 2008-11-15 01:10:42 UTC
SVN commit 884446 by tmcguire:

Work around a problem in KEncodingProber which would detect UTF-16LE for
most binary files (for example PNGs) with confidence 99%, which is obviously
wrong. Sometimes it is also UTF-16BE.

This caused KMail to encode the filename and sometimes the whole message in
UTF-16 (even for CSV files!), which most clients don't seem to understand.

The workaround is to blacklist UTF-16 and rely on our own charset detection
in this case instead.

BUG: 173503
CCMAIL: zealot.hoi@gmail.com


 M  +7 -2      kmcomposewin.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=884446
Comment 7 Thomas McGuire 2008-11-15 01:25:10 UTC
The examples you added here seem to be valid MIME messages in all cases (I might be wrong, but they seem correct), see RFC 2231 and RFC 2278. Both UTF-16 charsets are also valid charsets, see http://www.iana.org/assignments/character-sets. But I can see why other clients wouldn't understand UTF-16, it is uncommon, so I fixed this.

What I don't understand is that in your bekötés.ucf example, GMail can't decode the name, UTF-8 really is a common charset. Maybe GMail doesn't understand RFC 2231, that would be a bug in GMail.
Comment 8 Thomas McGuire 2008-11-15 01:31:07 UTC
Ah, indeed a bug in GMail, see http://weblog.timaltman.com/archive/2006/03/22/reporting-bugs-microsoft
Comment 9 Unknown 2009-01-21 03:48:31 UTC
This problem still occures in KMail 1.11.0.

I sent a letter, containing 3 files; the names are similar like this: "Árvíztűrő tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!%&".odt/txt/txt~

KMail sent this way (I'll show only the first one, because the rest 2 files are the same):
--Boundary-00=_HlodJAp2c2LBGwL
Content-Type: application/vnd.oasis.opendocument.text;
  name*=utf-8''%C3%81rv%C3%ADzt%C5%B1r%C5%91%20t%C3%BCk%C3%B6rf%C3%BAr%C3%B3g%C3%A9p%3A%3A%3F%3F%C3%81RV%C3%8DZT%C5%B0R%C5%90%20T%C3%9CK%C3%96RF%C3%9AR%C3%93G%C3%89P!!%&%2Eodt
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename*=utf-8''%C3%81rv%C3%ADzt%C5%B1r%C5%91%20t%C3%BCk%C3%B6rf%C3%BAr%C3%B3g%C3%A9p%3A%3A%3F%3F%C3%81RV%C3%8DZT%C5%B0R%C5%90%20T%C3%9CK%C3%96RF%C3%9AR%C3%93G%C3%89P!!%&%2Eodt

GMail sent this way:
--0016e6407e9c4b18170460f508ea
Content-Type: application/vnd.oasis.opendocument.text; 
	name="=?ISO-8859-2?B?wXJ27Xp0+3L1IHT8a/ZyZvpy82fpcDo6Pz/BUlbNWlTbUtUgVNxL1lJG?=
	=?ISO-8859-2?B?2lLTR8lQISElJi5vZHQ=?="
Content-Disposition: attachment; 
	filename="=?ISO-8859-2?B?wXJ27Xp0+3L1IHT8a/ZyZvpy82fpcDo6Pz/BUlbNWlTbUtUgVNxL1lJG?=
	=?ISO-8859-2?B?2lLTR8lQISElJi5vZHQ=?="

As you can see, there is no * after 'name' and 'filename'.

KMail shows the files this way:
- Sent from KMail:
 Árvíztűrő tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!U2Eodt
- Sent from GMail:
 Árvíztűrő tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!%&.odt

Gmail shows the files this way:
- Sent from KMail:
 utf-8''%C3%81rv%C3%ADzt%C5%B1r%C5%91%20t%C3%BCk%C3%B6rf%C3%BAr%C3%B3g%C3%A9p%3A%3A%3F%3F%C3%81RV%C3%8DZT%C5%B0R%C5%90%20T%C3%9CK%C3%96RF%C3%9AR%C3%93G%C3%89P!!%&%2Eodt
- Sent from Gmail:
 Árvíztűrő tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!%&.txt

Please, reopen this bug.
Comment 10 Thomas McGuire 2009-01-23 16:50:31 UTC
> As you can see, there is no * after 'name' and 'filename'.

The * is actually needed, see http://www.ietf.org/rfc/rfc2231.txt, section 4 ("Parameter Value Character Set and Language Information").

GMail actually uses an illegal encoding which is not allowed in that place, but since Outlook does the same, KMail has support for reading this, and also an option "Outlook-compatible attachment naming" in the composer settings for sending mails.

However, the filename "tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!%&.odt" indeed seems to get corrupted, I'll have a look.
Comment 11 Thomas McGuire 2009-01-23 17:56:52 UTC
SVN commit 915735 by tmcguire:

Make sure to quote % itself.
CCBUG: 173503


 M  +1 -1      kmmsgbase.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=915735
Comment 12 Thomas McGuire 2009-01-23 17:59:55 UTC
Ok, the last commit fixed that it showed !!U2Eodt instead of !!%&.odt.

Other than that, I don't see any remaining problems here.
Comment 13 Thomas McGuire 2009-01-23 18:04:03 UTC
>Gmail shows the files this way:
>- Sent from KMail:

>utf-8''%C3%81rv%C3%ADzt%C5%B1r%C5%91%20t%C3%BCk%C3%B6rf%C3%BAr%C3%B3g%C3%A9p%3A%3A%3F%3F%C3%81RV%C3%8DZT%C5%B0R%C5%90%20T%C3%9CK%C3%96RF%C3%9AR%C3%93G%C3%89P!!%&%2Eodt

Just to clarify again: This is a bug in GMail, which doesn't seem to understand RFC2231 encoding.
Comment 14 Thomas McGuire 2009-01-29 18:00:03 UTC
SVN commit 918212 by tmcguire:

Backport r915735 by tmcguire from trunk to the 4.2 branch:

Make sure to quote % itself.
CCBUG: 173503



 M  +1 -1      kmmsgbase.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=918212