Bug 173503

Created attachment 28130 [details]
The whole message with attachments became Chinese...

Just to be sure this is not a problem with Thunderbird not understanding the utf-16 encoding, how is configured the "message property" for sending in this account: Mime-compliant or Allow 8-bit. 

If it is the second, please try again changing it to Mime-compliant.

I didn't change the default, now it is: "MIME-compliant (Quoted printable)".

The message is not only bad in Thunderbird but also in GMail and in KMail (when I received it from myself). In the latter two at least I could read the message text.

The bug still exists in 4.1.72-2.1 either with 7bit, base64 or quoted printable encoding.

Any file which was sent with KMail will arrive in corrupted state...

Some examples:

Original filename: 5. mérés.odt
Name appears in GMail: iso-8859-2''5%2E%20m%E9r%E9s%2Eodt
Header:
--Boundary-00=_3S0EJftFV+qUeyk
Content-Type: application/vnd.oasis.opendocument.text;
  name*=iso-8859-2''5%2E%20m%E9r%E9s%2Eodt
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename*=UTF-16LE''%FF%FE5

Original filename: baudrate_generátor.v
Name appears in GMail: UTF-8''baudrate_gener%C3%A1tor%2Ev
Header:
--Boundary-00=_3S0EJftFV+qUeyk
Content-Type: text/plain;
  charset="UTF-8";
  name*=UTF-8''baudrate_gener%C3%A1tor%2Ev
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename*=UTF-8''baudrate_gener%C3%A1tor%2Ev

Original filename: bekötés.ucf
Name appears in GMail: UTF-8''bek%C3%B6t%C3%A9s%2Eucf
Header:
--Boundary-00=_3S0EJftFV+qUeyk
Content-Type: text/plain;
  charset="UTF-8";
  name*=UTF-8''bek%C3%B6t%C3%A9s%2Eucf
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename*=UTF-8''bek%C3%B6t%C3%A9s%2Eucf

Original name: main.v
Name appears in GMail: 呕ⵆ㘱䕌✧䘥╆䕆�
Header:
--Boundary-00=_3S0EJftFV+qUeyk
Content-Type: text/plain;
  charset="UTF-16LE";
  name*=UTF-16LE''%FF%FEm
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename*=UTF-16LE''%FF%FEm

I think this information is enough to forward the bug to the kmail developers, although I have no way to try this myself.

SVN commit 884446 by tmcguire:

Work around a problem in KEncodingProber which would detect UTF-16LE for
most binary files (for example PNGs) with confidence 99%, which is obviously
wrong. Sometimes it is also UTF-16BE.

This caused KMail to encode the filename and sometimes the whole message in
UTF-16 (even for CSV files!), which most clients don't seem to understand.

The workaround is to blacklist UTF-16 and rely on our own charset detection
in this case instead.

BUG: 173503
CCMAIL: zealot.hoi@gmail.com


 M  +7 -2      kmcomposewin.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=884446

The examples you added here seem to be valid MIME messages in all cases (I might be wrong, but they seem correct), see RFC 2231 and RFC 2278. Both UTF-16 charsets are also valid charsets, see http://www.iana.org/assignments/character-sets. But I can see why other clients wouldn't understand UTF-16, it is uncommon, so I fixed this.

What I don't understand is that in your bekötés.ucf example, GMail can't decode the name, UTF-8 really is a common charset. Maybe GMail doesn't understand RFC 2231, that would be a bug in GMail.

Ah, indeed a bug in GMail, see http://weblog.timaltman.com/archive/2006/03/22/reporting-bugs-microsoft

This problem still occures in KMail 1.11.0.

I sent a letter, containing 3 files; the names are similar like this: "Árvíztűrő tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!%&".odt/txt/txt~

KMail sent this way (I'll show only the first one, because the rest 2 files are the same):
--Boundary-00=_HlodJAp2c2LBGwL
Content-Type: application/vnd.oasis.opendocument.text;
  name*=utf-8''%C3%81rv%C3%ADzt%C5%B1r%C5%91%20t%C3%BCk%C3%B6rf%C3%BAr%C3%B3g%C3%A9p%3A%3A%3F%3F%C3%81RV%C3%8DZT%C5%B0R%C5%90%20T%C3%9CK%C3%96RF%C3%9AR%C3%93G%C3%89P!!%&%2Eodt
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename*=utf-8''%C3%81rv%C3%ADzt%C5%B1r%C5%91%20t%C3%BCk%C3%B6rf%C3%BAr%C3%B3g%C3%A9p%3A%3A%3F%3F%C3%81RV%C3%8DZT%C5%B0R%C5%90%20T%C3%9CK%C3%96RF%C3%9AR%C3%93G%C3%89P!!%&%2Eodt

GMail sent this way:
--0016e6407e9c4b18170460f508ea
Content-Type: application/vnd.oasis.opendocument.text; 
	name="=?ISO-8859-2?B?wXJ27Xp0+3L1IHT8a/ZyZvpy82fpcDo6Pz/BUlbNWlTbUtUgVNxL1lJG?=
	=?ISO-8859-2?B?2lLTR8lQISElJi5vZHQ=?="
Content-Disposition: attachment; 
	filename="=?ISO-8859-2?B?wXJ27Xp0+3L1IHT8a/ZyZvpy82fpcDo6Pz/BUlbNWlTbUtUgVNxL1lJG?=
	=?ISO-8859-2?B?2lLTR8lQISElJi5vZHQ=?="

As you can see, there is no * after 'name' and 'filename'.

KMail shows the files this way:
- Sent from KMail:
 Árvíztűrő tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!U2Eodt
- Sent from GMail:
 Árvíztűrő tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!%&.odt

Gmail shows the files this way:
- Sent from KMail:
 utf-8''%C3%81rv%C3%ADzt%C5%B1r%C5%91%20t%C3%BCk%C3%B6rf%C3%BAr%C3%B3g%C3%A9p%3A%3A%3F%3F%C3%81RV%C3%8DZT%C5%B0R%C5%90%20T%C3%9CK%C3%96RF%C3%9AR%C3%93G%C3%89P!!%&%2Eodt
- Sent from Gmail:
 Árvíztűrő tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!%&.txt

Please, reopen this bug.

> As you can see, there is no * after 'name' and 'filename'.

The * is actually needed, see http://www.ietf.org/rfc/rfc2231.txt, section 4 ("Parameter Value Character Set and Language Information").

GMail actually uses an illegal encoding which is not allowed in that place, but since Outlook does the same, KMail has support for reading this, and also an option "Outlook-compatible attachment naming" in the composer settings for sending mails.

However, the filename "tükörfúrógép::??ÁRVÍZTŰRŐ TÜKÖRFÚRÓGÉP!!%&.odt" indeed seems to get corrupted, I'll have a look.

SVN commit 915735 by tmcguire:

Make sure to quote % itself.
CCBUG: 173503


 M  +1 -1      kmmsgbase.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=915735

Ok, the last commit fixed that it showed !!U2Eodt instead of !!%&.odt.

Other than that, I don't see any remaining problems here.

>Gmail shows the files this way:
>- Sent from KMail:

>utf-8''%C3%81rv%C3%ADzt%C5%B1r%C5%91%20t%C3%BCk%C3%B6rf%C3%BAr%C3%B3g%C3%A9p%3A%3A%3F%3F%C3%81RV%C3%8DZT%C5%B0R%C5%90%20T%C3%9CK%C3%96RF%C3%9AR%C3%93G%C3%89P!!%&%2Eodt

Just to clarify again: This is a bug in GMail, which doesn't seem to understand RFC2231 encoding.

SVN commit 918212 by tmcguire:

Backport r915735 by tmcguire from trunk to the 4.2 branch:

Make sure to quote % itself.
CCBUG: 173503



 M  +1 -1      kmmsgbase.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=918212