Bug 315329 - New semantics of MusizBrainz recording id (stored in the same tag as previous track id) breaks AFT for 2 albums with the same recording
Summary: New semantics of MusizBrainz recording id (stored in the same tag as previous...
Status: RESOLVED FIXED
Alias: None
Product: amarok
Classification: Unclassified
Component: Collections/Local (show other bugs)
Version: 2.7.0
Platform: Other Linux
: NOR normal with 20 votes (vote)
Target Milestone: 2.8
Assignee: Amarok Developers
URL:
Keywords:
: 315331 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-02-17 16:33 UTC by Kurt Roeckx
Modified: 2013-05-15 12:54 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 2.7.1


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kurt Roeckx 2013-02-17 16:33:55 UTC
I have several tracks that I have on multiple albums.  They are all tagged using information from musicbrainz.  So you can see that it's the same recording, because that's in the metadata.

Since 2.7.0 it now seems to only want to import one of those tracks.  This means that if I want to play an album it doesn't have all the tracks for that album anymore.

It then points you to http://userbase.kde.org/Amarok/Manual/Various/TroubleshootingAndCommonProblems#Duplicate_Tracks, but I think that's not really useful.  I can perfectly understand if the files have identical information, like it's the same album/track.  But in this case the albums are different.

I was using 2.6.0 which perfectly imported those tracks. If I then do a full rescan of my collection, it gives me all thos errors.  After all the duplicate warnings it also gives a bunch of mysql errors about updates that fail, and I'm guessing that the tracks that are currently already in the database cause the problem.

Reproducible: Always
Comment 1 Matěj Laitl 2013-02-17 17:28:04 UTC
(In reply to comment #0)
> I have several tracks that I have on multiple albums.  They are all tagged
> using information from musicbrainz.

This may be the cause of the problem - perhaps the same musicbrainz id is assigned to those tracks.

> It then points you to
> http://userbase.kde.org/Amarok/Manual/Various/
> TroubleshootingAndCommonProblems#Duplicate_Tracks, but I think that's not
> really useful.  I can perfectly understand if the files have identical
> information, like it's the same album/track.  But in this case the albums
> are different.

I've significantly improved that answer, please have a look at it now and perform the suggested investigation/remedy.
Comment 2 Kurt Roeckx 2013-02-17 17:48:56 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > I have several tracks that I have on multiple albums.  They are all tagged
> > using information from musicbrainz.
> 
> This may be the cause of the problem - perhaps the same musicbrainz id is
> assigned to those tracks.

They have the same musicbrainz_trackid (and musicbrainz_artistid), but different musicbrainz_albumid

> > It then points you to
> > http://userbase.kde.org/Amarok/Manual/Various/
> > TroubleshootingAndCommonProblems#Duplicate_Tracks, but I think that's not
> > really useful.  I can perfectly understand if the files have identical
> > information, like it's the same album/track.  But in this case the albums
> > are different.
> 
> I've significantly improved that answer, please have a look at it now and
> perform the suggested investigation/remedy.

I think the "loog" there should be "look".

So I did the --newid on one of the tracks, and after it's done amarok shows me the same dialog box again, but it now only gives the mysql errors about duplicate entry when inserting into urls and tracks.
Comment 3 Kurt Roeckx 2013-02-17 18:57:02 UTC
Anyway, I don't think people should be manually adding ids to files for amarok to be able to deal with a recording being on multiple albums.

(I also find it unfortuanate that picard uses musicbrainz_trackid for this instead of something like musicbrainz_recordingid, because it's the uuid of the recording not the track.)
Comment 4 Kurt Roeckx 2013-02-17 19:18:41 UTC
(In reply to comment #3)
> (I also find it unfortuanate that picard uses musicbrainz_trackid for this
> instead of something like musicbrainz_recordingid, because it's the uuid of
> the recording not the track.)

I opened a bug with picard for that:
http://tickets.musicbrainz.org/browse/PICARD-398
Comment 5 Kurt Roeckx 2013-02-17 19:45:59 UTC
> I opened a bug with picard for that:
> http://tickets.musicbrainz.org/browse/PICARD-398

To clarify, as I currently understand things musicbrainz_trackid used to identify a track, so in that case amarok would be correct in saying that it's the same and that it should only add this once.

But some time ago musicbrainz introduced a new database schema (NGS, https://wiki.musicbrainz.org/Next_Generation_Schema) and now has a concept of a recording which can be on multiple albums, and it seems picard uses the same metadata tag to store the recording that used to contain the track, and I think they should have renamed the metadata tag at that point.

(It's unclear to me if you can still get the track id.)
Comment 6 Matěj Laitl 2013-02-17 20:08:23 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > (In reply to comment #0)
> > > http://userbase.kde.org/Amarok/Manual/Various/TroubleshootingAndCommonProblems#Duplicate_Tracks
> > 
> > I've significantly improved that answer, please have a look at it now and
> > perform the suggested investigation/remedy.
> 
> I think the "loog" there should be "look".

Yes. It's a wiki, please correct it! :-)

> So I did the --newid on one of the tracks, and after it's done amarok shows
> me the same dialog box again, but it now only gives the mysql errors about
> duplicate entry when inserting into urls and tracks.

Hmm, it definitely shouldn't do this once you do the --newid. I guess restarting Amaork followed by full rescan doesn't help?

Anyway, because the musicbrainz id now identifies a recording, I think we should drop support for considering it a "track id", as it only makes problems.
Comment 7 Kurt Roeckx 2013-02-17 21:10:42 UTC
(In reply to comment #6)
> Anyway, because the musicbrainz id now identifies a recording, I think we
> should drop support for considering it a "track id", as it only makes
> problems.

Yes, that looks like the best thing to do for now.

I think the mysql errors I get in case of (real) duplicates also need to be dealt with, I guess that belongs in a different bug?  Should I file a new bug about that?
Comment 8 Myriam Schweingruber 2013-02-19 12:30:54 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > Anyway, because the musicbrainz id now identifies a recording, I think we
> > should drop support for considering it a "track id", as it only makes
> > problems.
> 
> Yes, that looks like the best thing to do for now.
> 
> I think the mysql errors I get in case of (real) duplicates also need to be
> dealt with, I guess that belongs in a different bug?  Should I file a new
> bug about that?

No, that is known already.
Comment 9 Matěj Laitl 2013-02-19 12:32:52 UTC
(In reply to comment #8)
> (In reply to comment #7)
> > I think the mysql errors I get in case of (real) duplicates also need to be
> > dealt with, I guess that belongs in a different bug?  Should I file a new
> > bug about that?
> 
> No, that is known already.

Is there a bug report? If not, I'd prefer if Kurt filled one as he provides nice info and steps to reproduce.
Comment 10 Matěj Laitl 2013-02-19 21:21:32 UTC
Hi,
it has been brought to my attention on 
https://bugs.kde.org/show_bug.cgi?id=315329 that MusicBrainz changed semantics 
of the "MusicBrainz id" stored in various meta tags in a way that we IMO 
cannot use it as track unique identifier any longer.

In short, the "MusicBrainz id" is now a recording id, and per [1], one exact 
recording may easily end up in more albums, for example the original one and a 
compilation one. This breaks Amarok's assumption of uniqueness and leads to 
"Duplicate Tracks Found, only one imported" problems.

[1] https://wiki.musicbrainz.org/Server_Release_Notes/20110516

Is it okay if I just drop support for treating MusicBrainz id as uniqueIds? We 
may introduce MusicBrainz id as a separate tag in future. 
(Meta::valMusicBrainzId or similar)

	Matěj
Comment 11 Matěj Laitl 2013-02-21 12:10:09 UTC
Git commit 216c18bdaf18acf28e9ca98b115623934c9b4401 by Matěj Laitl.
Committed on 21/02/2013 at 01:30.
Pushed by laitl into branch 'master'.

Drop support for treating MusicBrainz ids as track unique ids

No comments on my mail inquiry, which I treat as a consent. Since 2.6
the SqlScanResultProcessor should cope with changed track ids just fine.

CHANGES:
 * Problematic support for treating MusicBrainz ids as track unique ids
   was dropped; should avoid surprising "Duplicate Tracks Found" errors.

@Alberto, you'll probably need to rebase your patch as I've touched
MusicBrainzFinder.cpp
GUI: Mentions of MusicBrainz ids being useful for AFT (Amarok File Tracking)
     in various documentation need to be removed
CCMAIL: amarok-devel@kde.org
CCMAIL: Alberto Villa <avilla@FreeBSD.org>

M  +2    -0    ChangeLog
M  +0    -6    shared/tag_helpers/APETagHelper.cpp
M  +0    -6    shared/tag_helpers/ASFTagHelper.cpp
M  +0    -6    shared/tag_helpers/ID3v2TagHelper.cpp
M  +0    -6    shared/tag_helpers/MP4TagHelper.cpp
M  +1    -9    shared/tag_helpers/TagHelper.cpp
M  +0    -1    shared/tag_helpers/TagHelper.h
M  +0    -7    shared/tag_helpers/VorbisCommentTagHelper.cpp
M  +5    -4    src/core/meta/Meta.h
M  +0    -2    src/musicbrainz/MusicBrainzFinder.cpp
M  +0    -5    src/services/lastfm/ScrobblerAdapter.cpp
M  +1    -1    tests/core-impl/collections/db/sql/TestSqlTrack.cpp

http://commits.kde.org/amarok/216c18bdaf18acf28e9ca98b115623934c9b4401
Comment 12 Matěj Laitl 2013-02-21 12:13:02 UTC
(In reply to comment #9)
> (In reply to comment #8)
> > (In reply to comment #7)
> > > I think the mysql errors I get in case of (real) duplicates also need to be
> > > dealt with, I guess that belongs in a different bug?  Should I file a new
> > > bug about that?
> > 
> > No, that is known already.
> 
> Is there a bug report? If not, I'd prefer if Kurt filled one as he provides
> nice info and steps to reproduce.

Myriam?
Comment 13 Myriam Schweingruber 2013-02-21 12:16:00 UTC
I can't find any indeed, maybe that was reports that came in through other means, so yes, a good report is very welcome.
Comment 14 Myriam Schweingruber 2013-04-12 07:07:13 UTC
*** Bug 315331 has been marked as a duplicate of this bug. ***
Comment 15 Matěj Laitl 2013-05-14 21:25:22 UTC
Git commit af8926a94268051124de8b16eef3b7eb95ac2772 by Matěj Laitl.
Committed on 21/02/2013 at 01:30.
Pushed by laitl into branch 'v2.7.x'.

Drop support for treating MusicBrainz ids as track unique ids

No comments on my mail inquiry, which I treat as a consent. Since 2.6
the SqlScanResultProcessor should cope with changed track ids just fine.

CHANGES:
 * Problematic support for treating MusicBrainz ids as track unique ids
   was dropped; should avoid surprising "Duplicate Tracks Found" errors.

M  +4    -0    ChangeLog
M  +0    -6    shared/tag_helpers/APETagHelper.cpp
M  +0    -6    shared/tag_helpers/ASFTagHelper.cpp
M  +0    -6    shared/tag_helpers/ID3v2TagHelper.cpp
M  +0    -6    shared/tag_helpers/MP4TagHelper.cpp
M  +1    -9    shared/tag_helpers/TagHelper.cpp
M  +0    -1    shared/tag_helpers/TagHelper.h
M  +0    -7    shared/tag_helpers/VorbisCommentTagHelper.cpp
M  +5    -4    src/core/meta/Meta.h
M  +0    -2    src/musicbrainz/MusicBrainzFinder.cpp
M  +0    -5    src/services/lastfm/ScrobblerAdapter.cpp
M  +1    -1    tests/core-impl/collections/db/sql/TestSqlTrack.cpp

http://commits.kde.org/amarok/af8926a94268051124de8b16eef3b7eb95ac2772