Bug 133613 - Playlist encoding problems with non-ascii characters
Summary: Playlist encoding problems with non-ascii characters
Status: RESOLVED FIXED
Alias: None
Product: amarok
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Amarok Developers
URL:
Keywords:
: 133761 133793 133798 133970 133996 134076 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-09-05 21:35 UTC by Thomas Lindroth
Modified: 2006-09-17 13:51 UTC (History)
9 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
small fix (688 bytes, patch)
2006-09-05 21:37 UTC, Thomas Lindroth
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Lindroth 2006-09-05 21:35:26 UTC
Version:           SVN revision 581239 (using KDE KDE 3.5.4)
Installed from:    Compiled From Sources
Compiler:          gcc (GCC) 4.1.1 (Gentoo 4.1.1) 
OS:                Linux

Undo and redo will turn non ascii tags into ???
See patch for more info.
Comment 1 Thomas Lindroth 2006-09-05 21:37:12 UTC
Created attachment 17645 [details]
small fix

The playlist history xml file would not get saved correctly without this.
Comment 2 Seb Ruiz 2006-09-06 12:23:40 UTC
SVN commit 581392 by seb:

Write xml files with utf8, else we would get junk
BUG: 133613


 M  +1 -1      metabundle.cpp  


--- trunk/extragear/multimedia/amarok/src/metabundle.cpp #581391:581392
@@ -1442,7 +1442,7 @@
     {
         QDomElement tag = QDomSucksItNeedsADocument.createElement( exactColumnName( i ) );
         //debug() << "exactColumName(i) = " << exactColumnName( i ) << endl;
-        QDomText text = QDomSucksItNeedsADocument.createTextNode( exactText( i, true ) );
+        QDomText text = QDomSucksItNeedsADocument.createTextNode( exactText( i, true ).utf8() );
         //debug() << "exactText(i) = " << exactText( i ) << endl;
         tag.appendChild( text );
 
Comment 3 richlv 2006-09-07 11:33:57 UTC
i blame this change for the problem that has surfaced recently :)
opening playlist that was saved with diacritic symbols in it now displays garbage. and with each restart of amarok the borked string increases in length, as if saving it was done in utf8, but opening was not.
Comment 4 Mark Kretschmann 2006-09-07 14:54:36 UTC
SVN commit 581762 by markey:

Reverting commit 581392. It's causing playlast borkage for some folks.

CCBUG: 133613


 M  +1 -1      metabundle.cpp  


--- trunk/extragear/multimedia/amarok/src/metabundle.cpp #581761:581762
@@ -1442,7 +1442,7 @@
     {
         QDomElement tag = QDomSucksItNeedsADocument.createElement( exactColumnName( i ) );
         //debug() << "exactColumName(i) = " << exactColumnName( i ) << endl;
-        QDomText text = QDomSucksItNeedsADocument.createTextNode( exactText( i, true ).utf8() );
+        QDomText text = QDomSucksItNeedsADocument.createTextNode( exactText( i, true ) );
         //debug() << "exactText(i) = " << exactText( i ) << endl;
         tag.appendChild( text );
 
Comment 5 Seb Ruiz 2006-09-09 14:29:35 UTC
*** Bug 133793 has been marked as a duplicate of this bug. ***
Comment 6 Seb Ruiz 2006-09-09 14:30:13 UTC
since the fix was reverted, this bug is still valid.
Comment 7 Alexandre Oliveira 2006-09-10 06:34:56 UTC
SVN commit 582700 by aoliveira:

Another attempt to fix the encoding problems when saving current playlist.
BUG: 133613


 M  +2 -2      metabundle.cpp  


--- trunk/extragear/multimedia/amarok/src/metabundle.cpp #582699:582700
@@ -1448,8 +1448,8 @@
 
         item.appendChild( tag );
     }
-
-    item.save( stream, indent );
+    QDomSucksItNeedsADocument.appendChild( item );
+    stream << QDomSucksItNeedsADocument.toCString();
     return true;
 }
 
Comment 8 Alexandre Oliveira 2006-09-10 06:41:30 UTC
*** Bug 133798 has been marked as a duplicate of this bug. ***
Comment 9 richlv 2006-09-11 11:53:10 UTC
revision 582974.
the problem is here again, playlist entries with diacritic symbols grow in size upon each close/open iteration.

i can provide before/after, if that could help in any way.
Comment 10 Alexandre Oliveira 2006-09-11 13:22:38 UTC
*** Bug 133761 has been marked as a duplicate of this bug. ***
Comment 11 Alexandre Oliveira 2006-09-12 18:57:23 UTC
*** Bug 133970 has been marked as a duplicate of this bug. ***
Comment 12 Alexandre Oliveira 2006-09-12 19:00:32 UTC
Rich, which enconding do you use (UTF8, I would guess)?
Can you send me one of these files?
Comment 13 Mark Kretschmann 2006-09-13 08:50:39 UTC
SVN commit 583729 by markey:

Revert Alex's encoding fix attempt. I don't think is needed any more, as the real problem was in Playlist::saveXML().

CCBUG: 133613


 M  +6 -6      metabundle.cpp  


--- trunk/extragear/multimedia/amarok/src/metabundle.cpp #583728:583729
@@ -1428,8 +1428,8 @@
 
 bool MetaBundle::save( QTextStream &stream, const QStringList &attributes ) const
 {
-    QDomDocument qDomSucksItNeedsADocumentAndDoesntUseUnicodeWhenItShould;
-    QDomElement item = qDomSucksItNeedsADocumentAndDoesntUseUnicodeWhenItShould.createElement( "item" );
+    QDomDocument qDomSucksItNeedsADocument;
+    QDomElement item = qDomSucksItNeedsADocument.createElement( "item" );
     item.setAttribute( "url", url().url() );
     item.setAttribute( "uniqueid", uniqueId() );
     if( m_isCompilation )
@@ -1440,16 +1440,16 @@
 
     for( int i = 0; i < NUM_COLUMNS; ++i )
     {
-        QDomElement tag = qDomSucksItNeedsADocumentAndDoesntUseUnicodeWhenItShould.createElement( exactColumnName( i ) );
+        QDomElement tag = qDomSucksItNeedsADocument.createElement( exactColumnName( i ) );
         //debug() << "exactColumName(i) = " << exactColumnName( i ) << endl;
-        QDomText text = qDomSucksItNeedsADocumentAndDoesntUseUnicodeWhenItShould.createTextNode( exactText( i, true ) );
+        QDomText text = qDomSucksItNeedsADocument.createTextNode( exactText( i, true ) );
         //debug() << "exactText(i) = " << exactText( i ) << endl;
         tag.appendChild( text );
 
         item.appendChild( tag );
     }
-    qDomSucksItNeedsADocumentAndDoesntUseUnicodeWhenItShould.appendChild( item );
-    stream << qDomSucksItNeedsADocumentAndDoesntUseUnicodeWhenItShould.toCString();
+
+    item.save( stream, 1 );
     return true;
 }
 
Comment 14 richlv 2006-09-13 09:32:56 UTC
alexandre, do you still need the file ?
if so, is it current.xml or one of music files containing diacritic chars ?
Comment 15 Mark Kretschmann 2006-09-13 09:41:26 UTC
*** Bug 133996 has been marked as a duplicate of this bug. ***
Comment 16 richlv 2006-09-13 10:26:58 UTC
fix confirmed in revision 583736
Comment 17 Mark Kretschmann 2006-09-14 08:51:02 UTC
Can anyone else please confirm whether the playlist /encoding/ problem is fixed in SVN trunk? I would like to be sure before creating a patch.
Comment 18 Kristjan Ugrin 2006-09-14 09:52:24 UTC
Yes, it works fine now.
Comment 19 Christie 2006-09-14 10:09:25 UTC
Seems fine here
Comment 20 Mark Kretschmann 2006-09-14 17:45:59 UTC
*** Bug 134076 has been marked as a duplicate of this bug. ***
Comment 21 Erik Hovland 2006-09-14 23:54:27 UTC
amarok version: 1.4.3
compiler: gcc (GCC) 4.0.1 (4.0.1-5mdk for Mandriva Linux release 2006.0)
distribution: Mandriva Linux 2006.0

I am not so sure that this problem is fixed. I just encountered the same issue:
amarok:           [PlaylistLoader] [ERROR!] [PLAYLISTLOADER]: Error in \
///home/ehovland/.kde/share/apps/amarok/current.xml: Error \
loading XML: "tag mismatch", at line 7970, column 8.

The file has this at that line:
<item url="file:///export/home/music/ogg/Beastie_Boys/Check_Your_Head/20_Namast%E9.ogg" uniqueid="5f8759b3cfc979a6acb753f0abae3eb5" >

I changed the tag to remove the e with the accent over it and it loads fine now. So the issue is with the parsing of the special character.
Comment 22 Mark Kretschmann 2006-09-15 00:24:21 UTC
Good morning, Erik. We've been talking about SVN TRUNK, instead you haven't noticed.
Comment 23 Mark Kretschmann 2006-09-15 09:11:16 UTC
For the sake of bookkeeping, here is the commit that actually fixed this bug (I had forgotten to CC this report back then):

SVN commit 583706 by markey:

Try to fix the playlist encoding problem which was probably a regression of my saveXML performance tweaks.


 M  +1 -0      playlist.cpp  


--- trunk/extragear/multimedia/amarok/src/playlist.cpp #583705:583706
@@ -3188,6 +3188,7 @@
     stream << "</playlist>\n";
 
     QTextStream fstream( &file );
+    fstream.setEncoding( QTextStream::UnicodeUTF8 );
     fstream << buffer;
 }
 
Comment 24 Erik Hovland 2006-09-15 19:48:55 UTC
Thanks for the completeness post, I have this fix backported into my 1.4.3 tree and I will give it a try.
Comment 25 Nicolas Pomarede 2006-09-17 13:51:18 UTC
Hello,
I'm using amarok 1.4.3-5 for mandriva which includes this latest patch,
and for me the problem is still there, I can't load the playlists that
were saved with Amarok 1.4.2.

These playlists contain some latin1 character and this seems to break
the xml parser when trying to restore the file.

Apart from that, creating new playlist is OK, chars are saved as utf8 and reloading is OK, so the regression is when trying to use playlists from
amarok 1.4.2.

(you can see http://qa.mandriva.com/show_bug.cgi?id=25493 for more
details)