Bug 177853

Summary: Implement and use ID3V2: Popularimeter
Product: [Applications] amarok Reporter: S. Burmeister <sven.burmeister>
Component: generalAssignee: Amarok Developers <amarok-bugs-dist>
Status: RESOLVED INTENTIONAL    
Severity: wishlist CC: info, mitchell, psychonaut, ralf-engels, tuju, unnamedrambler
Priority: NOR    
Version: 2.3-GIT   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description S. Burmeister 2008-12-15 14:32:38 UTC
Version:           2.0-SVN (using Devel)
OS:                Linux
Installed from:    Compiled sources

For rating purpose ID3V2 promotes "Popularimeter" which allows the user to set a rating for a file.

http://www.id3.org/id3v2.3.0#head-2452ec9cf8b42c5c117b518b69e129ff67970852

It does not talk about presentation/interaction, so one might stick to the simple stars and assign a certain numerical value to them, which might also solve the bug (I'll have to search for it) that one star is used as a rating for "very bad".

Of course one could also offer a more detailed interface to this tag, yet I think the stars are a very common and well understood concept on the internet and even across KDE, i.e. via nepomuk.

Having this tag within the file makes it possible have that information on any device the file is copied to.

For those users that share music files among many users and do not want to use/read the rating from the file but from a DB, the current functionality could just stay in place.

A simple setting to either use the file or the DB to store ratings would satisfy both user groups.
Comment 1 Tristan Grimaux 2008-12-15 14:38:03 UTC
*** Bug 139597 has been marked as a duplicate of this bug. ***
Comment 2 Juha Tuomala 2008-12-15 21:18:39 UTC
I can already see when program crashes it corrupts the file while doing so.

Ouh yeah, there must also be dudes who think that amarok does not crash, 
this could never happen, regardless the mess of output hw-frameworks 
out there, regardless that other kde programs do crash and corrupt files.

Whatever, implement it as long as others can keep their files intact once
ripped. And yes, you can call me pessimist.... or not: bug 171957
Comment 3 S. Burmeister 2008-12-15 23:07:43 UTC
Oh come on! Let's not write any tags to files and let's forbid the user to change the files because if this tag can corrupt the file, any tag can. Please do not get polemic but stick to reasonable arguments.
Comment 4 Casey Link 2008-12-16 00:00:21 UTC
The implementation of this feature is directly dependent on whether
taglib supports this tag. I would get in contact with them and see if
this is a supported feature of the library.
Comment 5 Juha Tuomala 2008-12-16 08:41:57 UTC
> Please do not get polemic but stick to reasonable arguments.

What particular 'unreasonable arguments' did I provide, in commment #2?
We already *have* such a case.

All I ask, that don't step others toes and we don't care if you hose
your files. That is, keep the separate storage still available. Even
Akregator provides config ui for other storage backends.
Comment 6 S. Burmeister 2009-02-17 10:34:01 UTC
bug 184610 is the one this feature depends on.
Comment 7 S. Burmeister 2009-02-17 21:32:07 UTC
taglib from svn can already handle the popularimeter tag, so now it's up to amarok to make use of it.
Comment 8 Jeff Mitchell 2009-09-30 00:57:24 UTC
As a matter of principle I'm rather against POPT and PCNT for two reasons.

First: Both frames are ID3-specific, and POPT is player-specific. Granted if you're only ever using Amarok it's not such a big deal, but it does mean that people that use various players quickly have the values get meaningless or at least totally inaccurate). Technically the key for a POPT frame is email address, but as I know of nobody that wants their email address embedded in all their files, it ends up being player-specific. There is also no standard way, interoperable way to record this information in most (all?) other formats.

Second: I'm also concerned over possible data loss/file corruption issues if e.g. Amarok (or something it's linked to) crashes during the write, or if it's doing this over a network connection and the connection is lost during the write, or some such thing.
Comment 9 S. Burmeister 2009-10-07 11:47:27 UTC
(In reply to comment #8)
> As a matter of principle I'm rather against POPT and PCNT for two reasons.

Both reasons are true, yet invalid.

> First: Both frames are ID3-specific, and POPT is player-specific. Granted if
> you're only ever using Amarok it's not such a big deal, but it does mean that
> people that use various players quickly have the values get meaningless or at
> least totally inaccurate). Technically the key for a POPT frame is email
> address, but as I know of nobody that wants their email address embedded in all
> their files, it ends up being player-specific. There is also no standard way,
> interoperable way to record this information in most (all?) other formats.

Invalid because those that only use Amarok win some feature, while those that use others as well do not loose anything but just have no gain/win. So you want to withhold a feature from those users that only use Amarok because of preventing others from having no gain.

> Second: I'm also concerned over possible data loss/file corruption issues if
> e.g. Amarok (or something it's linked to) crashes during the write, or if it's
> doing this over a network connection and the connection is lost during the
> write, or some such thing.

This is true for every file operation. You may even loose power and get a crash because of that. Yet e.g. digikam (network collections are possible) adds meta-data to pictures. All apps that write to a file take the risk of corrupting it.

So if you claim was valid, a lot of apps would have to stop manipulating data, including amarok, since editing any tag could cause this.

If you are against amarok writing anything to the file, your claim would be consistent, yet as soon as writing tag a is ok for you, tag b has to be as well, regarding file corruption.
Comment 10 Mark Kretschmann 2009-10-07 12:13:07 UTC
> First: Both frames are ID3-specific

This is not irrelevant at all. We use TagLib, which offers a unified interface for all tagging standards. Format specific special tags is not something that Amarok should toy around with.
Comment 11 Jeff Mitchell 2009-10-07 13:41:34 UTC
(In reply to comment #9)
> Invalid because those that only use Amarok win some feature, while those that
> use others as well do not loose anything but just have no gain/win. So you want
> to withhold a feature from those users that only use Amarok because of
> preventing others from having no gain.

No, I want to spend my time working on fixing existing bugs rather than implementing a feature that introduces data that becomes invalid the moment someone plays a song on their portable device.

It also introduces ambiguity when scanning files. If the file has playcount data that doesn't match the playcount data in the collection database, which is right? Do I overwrite what's in the file (potentially unwanted) with what's in the database, or do I overwrite what's in the database with what's in the file?

> This is true for every file operation. You may even loose power and get a crash
> because of that. Yet e.g. digikam (network collections are possible) adds
> meta-data to pictures. All apps that write to a file take the risk of
> corrupting it.
> 
> So if you claim was valid, a lot of apps would have to stop manipulating data,
> including amarok, since editing any tag could cause this.
> 
> If you are against amarok writing anything to the file, your claim would be
> consistent, yet as soon as writing tag a is ok for you, tag b has to be as
> well, regarding file corruption.

No, because currently writing metadata with digikam and Amarok is a user-initiated process, so the user has some control over what is happening and more specifically when. What you want would be automatic and frequent.

What portable media devices actually support this frame?
Comment 12 S. Burmeister 2009-10-07 17:16:01 UTC
fair enough, I won't tell you what to do with your free time.

I'll reopen again when somebody who cares more about amarok only users is in charge. Who cares about portable devices if I can use it within amarok?

And you still did not get it. If the user changes the rating (stars) it is a user initiated process. I guess you did not read the docs but assumed this was about the scoring.
Comment 13 Jeff Mitchell 2009-10-07 17:28:23 UTC
Dear jackass,

I do get it. You need to reread the popularimeter frame specification:

The frame contains the email
   address to the user, one rating byte and a four byte play counter,
   intended to be increased with one for every time the file is played.
   The email is a terminated string. The rating is 1-255 where 1 is
   worst and 255 is best. 0 is unknown. If no personal counter is wanted
   it may be omitted.

It's intended to be used as a combination playcounter and user rating. A playcounter is not a user-initiated process. It can be omitted, but then what's the point?

P.S. Your assertion that I don't care about Amarok-only users is provably false. The Amarok-only users already have ratings and playcounts in the collection, and thus do not actually need this feature. The problem is actually that I *also* care about users that do not only use Amarok.
Comment 14 S. Burmeister 2009-10-07 19:55:39 UTC
(In reply to comment #13)
> Dear jackass,

Even if you were right, using that kind of language disqualifies you and confirms the impression I had regarding your attitude. Thanks for clarifying that!
Comment 15 S. Burmeister 2009-10-07 20:13:16 UTC
Ah, and for those still interested in the actual matter. This would have enabled users to share their ratings across different clients, i.e. all those that do not use the same amarok db or in case that db gets corrupted.

Rating (user initiated [stars]) without counting is useful and used by all kinds of apps even non-audio such as digikam or dolphin+nepomuk. But they get it all wrong, because rating without counting does not make sense...
Comment 16 Jeff Mitchell 2009-10-07 23:18:46 UTC
(In reply to comment #15)
> Ah, and for those still interested in the actual matter. This would have
> enabled users to share their ratings across different clients, i.e. all those
> that do not use the same amarok db or in case that db gets corrupted.

No, it wouldn't, because the popularimeter is keyed by email address. Which means either a user puts their email address in every file (never met a user who wanted to do that) and every app knows what that email address is and uses it (also highly unlikely), or every app uses their own fake email address (much more likely). Even if you could get apps to agree on what fake email address to use, they won't agree on whether 1-10, 1-100, or 1-255 should be used, because most apps want to use the same number they are using for ratings internally and don't want to map those numbers out to a spread of 255. I'm not just saying that -- we've had discussions with other app developers in the past about exactly that. This is a reason why nobody supports that frame. And, things only get worse for non-MP3, where you don't even have a specified frame to store that information where you have to agree on an email address and rating scale, but where you simply have "comments" with key:value pairs.

However, I'm sure you don't actually care about the troubles interoperating with other clients, since you berated me for not caring about Amarok-only users.

> Rating (user initiated [stars]) without counting is useful and used by all
> kinds of apps even non-audio such as digikam or dolphin+nepomuk. But they get
> it all wrong, because rating without counting does not make sense...

And counting is not a user-initiated process. Hence back to my original statement about being wary of modifying files without user initiation.

And, FYI, I only developed an attitude when you started being a, well, jackass. (See your multiple insults in #12). It's the golden rule: be nice, and people will be nice to you.
Comment 17 Tristan Grimaux 2009-10-07 23:55:17 UTC
(In reply to comment #13)
> Dear jackass,

PLEASE, THIS IS CRAZY!! Open Source is Community Based, by insulting Mr Burmeister you are discouraging everyone who wants to collaborate with a valid opinion. You must rethink your doings: you really work to insult people? 

> I do get it. You need to reread the popularimeter frame specification:

No he doesn't. He understands the framework. It's a valid framework invented by some people who worked thinking a good way to save the popularimeter in the file.

The real problem with Amarok is not writing the popularimeter in the file but HANDLING the popularimeter by indexing some data that's written in the file.

By handling this information in the database Amarok can filter the files pretty fast, and you can have your favorite files in a snap. Indexing on some data that changes often somewhere else is traumatic. And having the data on both places sometimes lives with data you cannot trust.

> P.S. Your assertion that I don't care about Amarok-only users is provably
> false. The Amarok-only users already have ratings and playcounts in the
> collection, and thus do not actually need this feature. The problem is actually
> that I *also* care about users that do not only use Amarok.

There are moments where Amarok can handle this issue with elegance: when exporting the music to some music player, or when it's importing the files. In those moments Amarok can be instructed to specifically do something with the ratings.

There is something else to be said about popularimeter: the fact is that when you think on backing up your music you never think on backing up Amarok database. And by that, all the ratings that you spend sometime on thinking about are lost when you move your files. Amarok didn't import my ratings from version 1 to version 2, for example.

So there is something to do about this thing other than losing your temper with the people who loves using Amarok so much that wants to collaborate with the project itself.
Comment 18 Jeff Mitchell 2009-10-08 00:14:54 UTC
(In reply to comment #17)
> (In reply to comment #13)
> > Dear jackass,
> 
> PLEASE, THIS IS CRAZY!! Open Source is Community Based, by insulting Mr
> Burmeister

You need to read comment #12 before getting on *my* case about pulling out the insults.

> you are discouraging everyone who wants to collaborate with a valid
> opinion.

Nothing of the sort -- he's not collaborating.

> You must rethink your doings: you really work to insult people? 

No, and I also don't work to be insulted.

> Blah blah blah blah blah blah blah blah

<snip>

> So there is something to do about this thing other than losing your temper

In no way have I lost my temper. Not even in the slightest. This bug report is nowhere near worth it.

> the people who loves using Amarok so much that wants to collaborate with the
> project itself.

Funny, I didn't see him collaborating. Collaborating suggests sending patches, or being constructive. I see only a demand for a particular user's pet feature, a demand which I currently don't have the time nor inclination to satisfy, not to mention there being technical and community and user issues with satisfying this particular feature request.
Comment 19 Tristan Grimaux 2009-10-08 00:46:56 UTC
> > You must rethink your doings: you really work to insult people? 
> 
> No, and I also don't work to be insulted.

Great! At least if you 
 
> > Blah blah blah blah blah blah blah blah
> 

Now you are insulting me by commenting with blahs the most important part of my message to get back to that soap opera you are mounting with this bug report.


> In no way have I lost my temper. Not even in the slightest. This bug report is
> nowhere near worth it.

Your attitude is horrible.

> Funny, I didn't see him collaborating. Collaborating suggests sending patches,
> or being constructive. I see only a demand for a particular user's pet feature,


> a demand which I currently don't have the time nor inclination to satisfy, not
> to mention there being technical and community and user issues with satisfying
> this particular feature request.

I am so off this topic right now.
Comment 20 Jeff Mitchell 2009-10-08 01:03:35 UTC
> Now you are insulting me by commenting with blahs the most important part of my
> message to get back to that soap opera you are mounting with this bug report.

Me? I seem to recall you being the one sticking your nose in (with a bunch of mostly baseless claims). If you can't handle the heat, stay out of the kitchen.

> > In no way have I lost my temper. Not even in the slightest. This bug report is
> > nowhere near worth it.
> 
> Your attitude is horrible.

By refusing to become angry over a bug report?

> I am so off this topic right now.

Good. See ya.
Comment 21 S. Burmeister 2009-10-08 10:58:26 UTC
(In reply to comment #17)
> There is something else to be said about popularimeter: the fact is that when
> you think on backing up your music you never think on backing up Amarok
> database. And by that, all the ratings that you spend sometime on thinking
> about are lost when you move your files. Amarok didn't import my ratings from
> version 1 to version 2, for example.

This is exactly what made me file this bug. All my ratings were gone.
Comment 22 S. Burmeister 2009-10-08 11:21:11 UTC
> (In reply to comment #15)
> > Rating (user initiated [stars]) without counting is useful and used by all
> > kinds of apps even non-audio such as digikam or dolphin+nepomuk. But they get
> > it all wrong, because rating without counting does not make sense...
> 
> And counting is not a user-initiated process. Hence back to my original
> statement about being wary of modifying files without user initiation.

I assume you got the irony and ignored it. You yourself quoted that counting is not needed. So you deny user-initiated rating because of something which is not needed. That does not make any sense. So just to clarify, rating without counting makes perfectly sense, as can be seen in dolphin+nepomuk, digikam and others.

https://bugzilla.gnome.org/show_bug.cgi?id=532650

There is a patch which will be included and handles everything needed for banshee. Regarding the email-adress, there is no need to actually use an email-address. Amarok could just use amarok.

"A unique string which identifies the creator of the tag Examples: "Banshee" (used by this patch), "Windows Media Player 9 Series" (used by WMP and Vista's Windows Explorer), "no@email" (used by MediaMonkey)"

"This patch implements support for the Popularimeter field, giving Banshee the following functionality: - When a song using ID3v2 tagging is added to the library, its embedded rating (if any) is also imported. - When an ID3v2 song is rated, the rating is written to the song file. The user must have "Write metadata to files" turned on for this to happen."

So even if players differ in terms of what value is one star and at what value the two stars start, the worst that can happen is that those using a secondary app will get no rating if amarok does not implement the functionality to write his own and the user chosen unique strings, i.e. those users that want to use wmp and amarok could enable an option that makes amarok write both. On the win side, they actually get some rating which is not bound to amarok's database.

Those that only use amarok do not even have the tiny issue described above.

All users could import their ratings from other players and if amarok 1.x did already support this, none would have lost their ratings when switching to amarok2.
Comment 23 Tristan Miller 2010-12-07 14:14:30 UTC
It appears that, contrary to the "WONTFIX" claims of developers here, this
feature has actually been implemented in Amarok 2.4-GIT.  The rating, score,
and play count are now written to Ogg Vorbis files in the fields FMPS_RATING,
FMPS_RATING_AMAROK_SCORE, and FMPS_PLAYCOUNT, respectively.  It doesn't appear
that this information is written to MP3 files.  (I'm not sure about other
formats, as my library is entirely Vorbis and MP3.)

However, storing the play count and score introduces an additional problem when
synchronizing music collections, for which I have opened Bug 259117.

For the sake of consistency, can this bug report (or Bug 139597) be reopened?  It makes no sense to store metadata for Vorbis but not MP3.
Comment 24 Myriam Schweingruber 2010-12-07 19:54:29 UTC
Reopened at user request.
Comment 25 Jeff Mitchell 2010-12-07 19:58:32 UTC
Hi,

You probably are seeing an artifact of a short period of testing. For a time in the Git version these values were being written to files automatically -- which is not a good thing, as users should have to explicitly enable features that modify their data.

AFAIK this *should* be implemented for ID3/MP3s; Ralf would know for sure.

However, this is, in fact, a WONTFIX because support for the Popularimeter tag is not being implemented due to various problems, which is why FMPS was created in the first place...

If you're having problems with the new feature, then the right thing to do is open a new bug, as you did in bug 259117.
Comment 26 Ralf Engels 2010-12-08 13:37:56 UTC
The following is the current state and will probably go to Amarok 2.4 unless new bugs appear:

Amarok supports reading and writing statistic information to the file via taglib.
It is using FMPS tags for all formats and additionally the POPM tag (without e-mail) for MP3s.

A full scan will use the file tags if existing over the information in the database.
An incremental scan will only import those tags if the information is not already set in the database.

With information I mean score, rating and playcount. First and last played is currently not supported.

There is also a new option that allows you to switch on saving of those tags back to the files.
Switching this on also means that every played file is modified since the playcount and the score will be written back.

An additional note regarding the id3v2 POPM tag.
This tag stores a set of email/rating/playcount.
Amarok will only read and write POPM tags with an empty email address which is allowed in the id3v2 spec.
Ratings like this are sometimes seen in mp3 files but I don't know any other music player who is actively supporting this.

A personal note: I am switching the write back of rating on because I hate loosing rating information every time I copy tracks around which happened a couple of time in the past.
I see the rating as just another meta information and it really belongs to the file.