Bug 202027 - Encoding problem with shoutcast streams
Summary: Encoding problem with shoutcast streams
Status: RESOLVED UNMAINTAINED
Alias: None
Product: Phonon
Classification: Frameworks and Libraries
Component: Xine backend (show other bugs)
Version: 4.4.0 (KDE 4.4.2)
Platform: unspecified Linux
: NOR normal
Target Milestone: ---
Assignee: Matthias Kretz
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-07-30 20:27 UTC by Ignacio Serantes
Modified: 2011-01-28 11:47 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ignacio Serantes 2009-07-30 20:27:59 UTC
Version:           2.1.1 (using 4.2.98 (KDE 4.2.98 (KDE 4.3 RC3)) "release 150", KDE:KDE4:Factory:Desktop / openSUSE_11.1)
Compiler:          gcc
OS:                Linux (x86_64) release 2.6.27.25-0.1-default

When I listen shoutcast radio stations with asian characters Amarok displays garbage characters because it's using windows-1212 and not UTF-8.

I test the same stations with mplayer and Exaile and both programs display asian characters and not garbage.
Comment 1 Myriam Schweingruber 2009-07-30 21:27:10 UTC
This is solved in current 2.2-git. Thanks for your report.
Comment 2 Ignacio Serantes 2009-07-30 22:31:55 UTC
Amazing, this was really quick. Great job Myriam! :D.
Comment 3 Myriam Schweingruber 2009-07-30 23:21:10 UTC
Well, it is solved since a few weeks already :)
Comment 4 Ignacio Serantes 2009-07-31 03:22:40 UTC
I just compile Version 2.2-GIT, 31/07/09 3:15 GMT+1, in openSUSE v11.1, and the problem is not fixed.

I verify again using mplayer and Exaile and both do the job well but Amarok don't.
Comment 5 Ignacio Serantes 2009-07-31 22:51:12 UTC
If somebody told me where is located that code I can try to fix the problem.
Comment 6 Myriam Schweingruber 2009-08-01 08:55:12 UTC
Could you get in touch with us on #amarok on irc.freenode.net, please? Thank you :)
Comment 7 Ignacio Serantes 2009-08-01 12:38:48 UTC
Well, I try #amarok but people was soy busy so, while I wait I make a simple change in EngineControler.cpp, EngineController::slotMetaDataChanged(), and the problem was fixed:

old code:
    QStringList title = m_media->metaData( "TITLE" );
    debug() << "Title      : " << title;
    if( !title.isEmpty() )
        meta.insert( Meta::valTitle, title.first() );

fixed code:
    QStringList title = m_media->metaData( "TITLE" );
    //PATCH
    title.replace(0, title.at(0).toLatin1());
    //PATCH
    debug() << "Title      : " << title;
    if( !title.isEmpty() )
        meta.insert( Meta::valTitle, title.first() );

but for me the problem was not here and must be in

meta.insert( Meta::valUrl, m_media->currentSource().url().toString() );

or in m_media->metaData() code who it's not doing a proper encoding conversion but, that was so advanced for me.
Comment 8 Myriam Schweingruber 2009-08-01 13:58:36 UTC
Well, it is Saturday, so you need to be patient :)
Could you please submit suggestions to the mailing list instead? I don't think that a bug report is the right place to discuss code changes. The list is amarok@kde.org
Comment 9 Ignacio Serantes 2009-08-01 14:15:08 UTC
No, no, I don't discuss code changes, I'm not a C++ programmer and I only read a few lines of Amarok's code.

Please, consider my post only as aditional information for a proper bug fix :).

As I told, I think that the bug was not here.
Comment 10 Peter ZHOU 2009-08-01 14:41:23 UTC
i only added the encoding detector code in meta/p_file.h. I have no idea where the stream code located.
Comment 11 Mark Kretschmann 2009-08-01 14:47:23 UTC
@Ignacio: I'm confused, first you wrote that your patch above fixed the problem, but then you wrote that it did not. Could you elaborate?

Anyway, I could look at the problem, but need to find a way to reproduce it. Do you know a specific stream that I could try? :)
Comment 12 Ignacio Serantes 2009-08-01 15:14:26 UTC
@Mark: Yes, my patch solves the problem because I'm force an  encoding in the first place I found reading amarok -debug output and works, but I think that bug was not here. The fix works but I'm not sure if this fix is a good fix becouse the proper place is where Amarok obtains stream information but, I don't know where this code is located.

You can try adding this stream http://78.129.146.161:18314.
Comment 13 Ignacio Serantes 2009-08-01 23:39:00 UTC
Well, I repeat again, I'm not a C++ programmer and only read a few lines of Amarok's code so next deduction could be wrong.

Doing several test with Mplayer, Exaile, VideoLan and Xine I think that the bug is not in Amarok code and the bug is in Xine engine.

Both Mplayer, VideoLan and Exaile (GStreamer engine) works fine with japanese characters but Xine displays nothing.

Testing streaming with Amarok using GStreamer crash program so, I can't do this test but, opening a local song with Dragon Player the same encoding bug is displayed using Phonon Xine engine. If I change to GStreamer engine Dragon Player displays song title with correct encoding.

But, If I open the same file with Amarok japanese characters are displayed so, Amarok is not using Phonon to obtain song information or I'm totally wrong. You are the experts :).

It is unfortunate that Phonon GStreamer engine has so many bugs and is unusable and VideoLan and MPlayer engines was not available :(.
Comment 14 Ignacio Serantes 2009-08-02 23:42:28 UTC
I found an old phonon xine engine code in kde-apps.org, I can't locate where is current version, and in XineStream::updateMetaData() method (xinestream.cpp) there is the next code:

metaDataMap.insert(QLatin1String("TITLE"),
            QString::fromUtf8(xine_get_meta_info(m_stream, XINE_META_INFO_TITLE)));

I think that the wrong encoding conversion is here so this is a Phonon bug and not an Amarok bug.
Comment 15 Ignacio Serantes 2009-08-21 11:31:45 UTC
Any news about this bug?

If you wan't I send my patch, I'm currently using it, but I think that this bug must be assigned to Phonon.
Comment 16 Mikko C. 2009-08-21 11:37:16 UTC
you can find the current version of phonon xine here: http://websvn.kde.org/trunk/kdesupport/phonon/

If you have a patch that applies to phonon and fixes amarok, go ahead and file a bug for phonon. Then make sure to update this bug.
Comment 17 Mikko C. 2009-08-21 11:43:08 UTC
btw, this seems to have been fixed 7 months ago:
http://websvn.kde.org/trunk/kdesupport/phonon/xine/xinestream.cpp?r1=902669&r2=903466&sortby=date
Comment 18 Ignacio Serantes 2009-08-21 12:02:59 UTC
Ok, thank you for the info, I will do a try this weekend.
Comment 19 Ignacio Serantes 2009-08-23 17:27:58 UTC
Well, I have done some research and this was I saw:

Phonon code is the next
----------------------------------------------------------------
    QTextCodec *codec = QTextCodec::codecForMib(106); // utf-8
    if (!isUtf8) {
        QTextCodec *localCodec = QTextCodec::codecForLocale();
        if (localCodec == codec) {
            // if the local codec also is UTF-8 our best guess is Latin-1
            codec = QTextCodec::codecForName("ISO 8859-1");
        } else {
            // hoping that the local codec is the same as the one used in the meta data
            codec = localCodec;
        }
    }
    QMultiMap<QString, QString> metaDataMap;
----------------------------------------------------------------

and I send streaming info in UTF-8, this was confirmed using Konqueror and forcing UTF-8 encoding in streaming servers pages, but I receive garbage in Amarok.

My locale is UTF-8 so, reading de code only two things must happen but I don't know wich one:
1) isUtf8 works -> QTextCodec::codecForName("UTF-8")
2) isUtf8 fails -> QTextCodec::codecForName("ISO 8859-1"), because localCodec = codec = UTF-8.

In first case conversion toUnicode() was doing using UTF-8 and in the second case conversion was doing using ISO 8859-1.

My workaround patch is

title.replace(0, title.at(0).toLatin1())

and using it Japanese and Korean characters was displayed correctly.

I hope this information would be useful to fix this bug.
Comment 20 Ignacio Serantes 2009-08-29 12:39:14 UTC
This bug is assigned to Amarok but probably must be assigned to Phonon because seems to be a problem with Phonon Xine Backend.
Comment 21 Mikko C. 2009-08-29 12:41:26 UTC
ok, done
Comment 22 Myriam Schweingruber 2009-11-08 22:29:13 UTC
Subscribing Martin
Comment 23 Martin Sandsmark 2010-01-14 00:50:48 UTC
I'm sorry, but what string is `title` in your workaround?

Otherwise, there isn't any reliable way to autodetect encoding of a given string, so this is kind of a hard nut.
Comment 24 Ignacio Serantes 2010-01-14 17:41:36 UTC
The problem is that when stream information was in UTF-8 a wrong conversion is doing.

If system was in UTF-8 and string was in UTF-8 then a conversion to ISO 8859-1 breaks encoding texts.

My Amarok's patch works because I undo the wrong encoding conversion.
Comment 25 Martin Sandsmark 2010-01-16 17:03:41 UTC
Phonon only converts to unicode (it calls codec->toUnicode() a bit further down from the code snippet you pasted above).

And do you have another test stream? The stream you provided above isn't working anymore, and I didn't manage to find any good ones in shoutcast.
Comment 26 Ignacio Serantes 2010-01-16 21:37:06 UTC
This was a personal shoutcast radio I used in the past but now I'm using ampache so the radio was disabled.

But if you need it to solve this problem I can do a shoutcast for you. We can schedule it by mail.
Comment 27 Ignacio Serantes 2010-01-16 22:50:30 UTC
Better, I found a stream that works well with Exaile and fails with Amarok. You
must wait for a song with non western characters.

http://shitdicknipples.org:8000/jpopsuki
Comment 28 Myriam Schweingruber 2010-04-05 13:47:59 UTC
Martin, any news on this? Ignacio, is this still valid with KDE SC 4.4.2?
Comment 29 Ignacio Serantes 2010-04-16 21:50:52 UTC
The problem persist.
Comment 30 Myriam Schweingruber 2011-01-28 11:47:00 UTC
The xine backend will be deprecated soon as it is currently unmaintained. Please use the vlc or gstreamer backend instead.