Bug 173195

Summary: Pipes plugin doesn't support Unicode
Product: [Unmaintained] kopete Reporter: Christoph Burgmer <chrislb>
Component: Pipes PluginAssignee: Kopete Developers <kopete-bugs-null>
Status: RESOLVED FIXED    
Severity: normal    
Priority: NOR    
Version First Reported In: unspecified   
Target Milestone: ---   
Platform: Debian testing   
OS: Unspecified   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: Proposed fix for "pipes" plugin encoding issues

Description Christoph Burgmer 2008-10-20 17:09:47 UTC 
Version:            (using KDE 4.1.2)
Installed from:    Debian testing/unstable Packages

The Pipes plugin (see bug 99060) only supports plain ASCII and no Unicode encoded string.

Either use the system encoding  or do UTF-8 by default.

See an example in Python that uses this plugin: http://www.stud.uni-karlsruhe.de/~uyhc/en/content/write-upside-down-using-unicode-ipa-characters
Comment 1 Christoph Burgmer 2008-10-20 17:31:53 UTC 
See http://websvn.kde.org/trunk/playground/network/kopete/plugins/pipes/pipesplugin.cpp?revision=746167&view=markup

        // poop out appropriate data
        if ( pipeOptions.pipeContents == HtmlBody )
                pipe.write ( msg.escapedBody().toLocal8Bit() );
        else if ( pipeOptions.pipeContents == PlainBody )
                pipe.write ( msg.plainBody().toLocal8Bit() );
        else if ( pipeOptions.pipeContents == Xml )
                pipe.write ( createXml ( msg ).toLocal8Bit() );

Everything is piped through using 8 bit without any encoding conversion.
Comment 2 Christoph Burgmer 2008-12-14 06:58:52 UTC 
Can somebody confirm that this bug is or is not simply solved by replacing toLocal8Bit() with toUtf8()? Both return a QByteArray, but for me only toUtf8() actually does not delete non-ASCII characters:

$ python
Python 2.5.2 (r252:60911, Nov 14 2008, 19:46:32)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt4.QtCore import QString
>>> a = QString(u"你好")
>>> a
PyQt4.QtCore.QString(u'\u4f60\u597d')
>>> a.toUtf8()
PyQt4.QtCore.QByteArray('\xe4\xbd\xa0\xe5\xa5\xbd')
>>> a.toLocal8Bit()
PyQt4.QtCore.QByteArray('')
>>> 

I don't really understand Qt's behaviour/documentation here. On [1] is states, that the system's encoding is used for conversion to 8 bit. My system uses utf8, and in the case of using 2-/4-byte encodings, I can't guess the semantics of that method.

[1] http://doc.trolltech.com/4.4/qstring.html#toLocal8Bit
Comment 3 Sebastian Strand 2008-12-15 19:27:55 UTC 
Created attachment 29362 [details]
Proposed fix for "pipes" plugin encoding issues

I've attached a patch that seems to fix the problem for me. I've tested plain text and xml input/output. The main problem, as far as I can tell, was that data coming back from the QProcess was implicitly converted to ascii when the QByteArray output was assigned to a QString. The patch contains some other modifications as well to reduce the number of conversions back and forth. The patch applies inside the "pipes" directory.
Comment 4 Christoph Burgmer 2009-01-16 14:46:36 UTC 
This patch seems not to address my specific problem with plain text. Please take that into account when applying the patch.
Comment 5 Roman Jarosz 2009-01-16 14:56:20 UTC 
SVN commit 912005 by rjarosz:

Fix bug 173195: Pipes plugin doesn't support Unicode.
Patch by Sebastian Strand, thanks.

BUG: 173195


 M  +12 -12    pipesplugin.cpp  
 M  +3 -3      pipesplugin.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=912005
Comment 6 Roman Jarosz 2009-01-16 14:59:00 UTC 
SVN commit 912008 by rjarosz:

Backport fix for bug 173195: Pipes plugin doesn't support Unicode.
Patch by Sebastian Strand, thanks.

CCBUG: 173195



 M  +12 -12    pipesplugin.cpp  
 M  +3 -3      pipesplugin.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=912008