Bug 176154 (collectionsucks) - amarok 2 ignores all id3 tags except title and track for half of my collection
Summary: amarok 2 ignores all id3 tags except title and track for half of my collection
Status: RESOLVED FIXED
Alias: collectionsucks
Product: amarok
Classification: Applications
Component: Collections/Local (show other bugs)
Version: 2.0-rc
Platform: unspecified Linux
: HI normal
Target Milestone: ---
Assignee: Amarok Developers
URL:
Keywords:
: 176134 176563 176942 177602 177746 177784 177878 177918 178124 178576 178668 178782 178866 178969 179271 180248 180473 182067 182877 183770 188122 189803 (view as bug list)
Depends on:
Blocks:
 
Reported: 2008-11-26 18:20 UTC by piemonkey
Modified: 2011-06-24 06:58 UTC (History)
32 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Log of Amarok messing up the collection (271.87 KB, text/plain)
2008-12-29 13:45 UTC, Knut Johansson
Details

Note You need to log in before you can comment on or make changes to this bug.
Description piemonkey 2008-11-26 18:20:59 UTC
Version:           1.98 (using 4.1.3 (KDE 4.1.3), Kubuntu packages)
Compiler:          cc
OS:                Linux (x86_64) release 2.6.27-8-generic

I've got all of my collection (~9000 songs) organised into folders within a single folder. When Amarok scans my collection, for about half of these, it doesn't import anything but the title and track number for the song. This leads to amarok having a huge number of unsorted music.

I have checked that the id3 tags are still there, and they are, it just seems that Amarok is ignoring them, and continues to do so when I rescan my collection.

I can't see any logic behind which files it does this for, most of the time it's whole albums, but some are not.
Comment 1 piemonkey 2008-11-26 18:44:38 UTC
I have just noticed that Amarok has also placed some albums under the name of a different album (that is, an album that has been correctly recognised as an album by a certain artist, has the name of an album that Amarok has not identified)
Comment 2 Mark Kretschmann 2008-11-26 19:51:59 UTC
*** Bug 176134 has been marked as a duplicate of this bug. ***
Comment 3 Mark Kretschmann 2008-11-26 19:57:26 UTC
I've seen this problem too, however it happens with far less tracks here.

One suspicion I have is that it's got to do with bad MySQL escaping, which we're currently investigating. Does it happen especially with files that have non-ASCII characters in their tags or filenames?

Could you make one of the problematic files accessible to us, for reproducing?

Comment 4 piemonkey 2008-11-26 20:23:22 UTC
I don't think it happens especially with non-ASCII characters, but it's hard to get an idea of the trend with such a large number of songs to look through.

Here is one of the files whose album and artist is ignored:
http://www.megaupload.com/?d=TIEE2IO3

Also, I store the music on an NTFS partition (a relic from my windows days), just in case that is relevant.
Comment 5 piemonkey 2008-11-26 20:45:44 UTC
Also, here is a zip of 2 mp3 files, one of which (track 12) is put into the correct album and given the correct artist name, the other of which is not.

http://www.megaupload.com/?d=V6II05KK
Comment 6 Peter ZHOU 2008-11-27 04:31:34 UTC
~/.kde/share/apps/amarok/mysqle/mysql/db.opt
~/.kde/share/apps/amarok/mysqle/amarok/db.opt


default-character-set=latin1
default-collation=utf8_general_ci

try to change the default-collation to utf8 in the two files to see the difference( from latin_swedish_ci )
Comment 7 Peter ZHOU 2008-11-27 04:32:29 UTC
and btw, please don't forget to rescan your collection :)
Comment 8 Mark Kretschmann 2008-11-27 09:48:21 UTC
SVN commit 889579 by markey:

Use mysql_real_escape_string() for escaping, instead of our own
concoction of an escaping routine. This might fix escaping errors that
could affect the following bugs:

CCBUG: 176154
CCBUG: 174784

Please test.

 M  +10 -1     MySqlEmbeddedCollection.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=889579
Comment 9 piemonkey 2008-11-27 14:40:35 UTC
(In reply to comment #6)
> try to change the default-collation to utf8 in the two files to see the
> difference( from latin_swedish_ci )

After trying this, Amarok now reads the album and artist name for all of my files, but it does still mix up some of the albums, claiming that their tags do contain an album name (the wrong album name). I checked the tags in another program, and it disagrees with Amarok, and claims that the tags are as they should be.

A confusing thing is that changing the default-collation entries back to what they were does not cause the collection to revert to what it was previously.

My apologies, but I don't know what to do with the MySqlEmbeddedCollection.cpp file. I have only recently started learning how to program, but I only really know enough to know that it's a C++ source file... Do I download the Amarok source code, and replace one file with the one you posted, then compile?
Comment 10 Peter ZHOU 2008-11-27 14:55:41 UTC
You have to grab the code from svn and compile, or you have to wait till the next amarok release.
Comment 11 Patrick Jensen 2008-11-27 23:46:52 UTC
(In reply to comment #6)
> ~/.kde/share/apps/amarok/mysqle/mysql/db.opt
> ~/.kde/share/apps/amarok/mysqle/amarok/db.opt
> 
> 
> default-character-set=latin1
> default-collation=utf8_general_ci
> 
> try to change the default-collation to utf8 in the two files to see the
> difference( from latin_swedish_ci )
>

Doing this actually caused the collection to lose track of all tracks bar those that I had manually edited in Kid3. If it helps any, all of the tracks are original rips, extracted with K3B.

Comment 12 Mark Kretschmann 2008-11-28 12:00:52 UTC
Please try again with latest SVN trunk. We have a new patch in for an encoding problem:

SVN commit 890023 by woebbe:

escape():
- fix encoding (don't use implicit const char* -> QString cast)
- build with -pedantic

 M  +4 -3      MySqlEmbeddedCollection.cpp

Comment 13 Patrick Jensen 2008-11-28 15:17:04 UTC
(In reply to comment #12)
> Please try again with latest SVN trunk. We have a new patch in for an encoding
> problem:
> 
> SVN commit 890023 by woebbe:
> 
> escape():
> - fix encoding (don't use implicit const char* -> QString cast)
> - build with -pedantic
> 
>  M  +4 -3      MySqlEmbeddedCollection.cpp
> 

No can do. The svn version will not compile because of a too old version of MySQL and since my experiences in bypassing the package manager to install stuff from outside the repositories have usually ended up in severely botched installations, I will instead wait until this update finds its way into the next RC version.
Comment 14 Mark Kretschmann 2008-12-12 18:20:36 UTC
*** Bug 177602 has been marked as a duplicate of this bug. ***
Comment 15 Mark Kretschmann 2008-12-14 17:07:39 UTC
*** Bug 177784 has been marked as a duplicate of this bug. ***
Comment 16 Mark Kretschmann 2008-12-14 17:09:49 UTC
*** Bug 176563 has been marked as a duplicate of this bug. ***
Comment 17 Mark Kretschmann 2008-12-15 08:05:24 UTC
*** Bug 177746 has been marked as a duplicate of this bug. ***
Comment 18 Rishad Bharucha 2008-12-15 11:48:11 UTC
Hi. My bug report was marked as a duplicate of this bug. Is there a solution to this problem?
Comment 19 Mark Kretschmann 2008-12-15 11:51:51 UTC
@Rishad: Not yet, otherwise this would have been marked as FIXED.
Comment 20 Mark Kretschmann 2008-12-15 22:26:24 UTC
*** Bug 177878 has been marked as a duplicate of this bug. ***
Comment 21 Mark Kretschmann 2008-12-16 19:15:26 UTC
*** Bug 177918 has been marked as a duplicate of this bug. ***
Comment 22 Rafael Tesser 2008-12-16 19:27:33 UTC
Ok! My bug was marked as duplicate of this one! 

What I've done is to create a new folder in the collection directory and put some new music files in there and also moved some from another folder into it. Amarok not only doesn't seem to find the new files as it think the ones I moved are still in the place they where before.

How can I get more useful information to discover where the error is happening. Here I am using the SVN version of Amarok.
Comment 23 Rafael Tesser 2008-12-16 19:44:55 UTC
WOAH! Something is really wrong. I just tried to rescan the collection using the button in the collection part of the configuration window. Now the collection is all messed up. Albums are showing in the wrong artists and the years displayed for the albums in the tree are wrong and musics are being shown in the wrong album.
I'll try to wipe my profile to see if the things get corrected.
Comment 24 Rafael Tesser 2008-12-16 20:06:18 UTC
Hi again! Sorry for the triple post. 

Cleaning the profile seems to fix the problem. I've added new files to the collection folder after it had already been scanned by Amarok and they have been detected and included in the collection. However this is not a real solution as the user shouldn't have to clean the config files anytime. Maybe some further investigation is needed to find why this problem appears.
Comment 25 Lydia Pintscher 2008-12-16 20:35:18 UTC
What is the status of this? Close as fixed?
Comment 26 missive 2008-12-16 22:43:50 UTC
> Close as fixed?

I hope not... I just built r897811, and nothing shows up in my collection at all.
Comment 27 Rishad Bharucha 2008-12-17 05:02:37 UTC
Cleaning the profile did not solve the problem for me. My collection scan still hangs at 97%. When I copy a few random songs to the home/Music folder it works though. My entire collection doesn't though and it worked fine in 1.4.x.
Comment 28 Mark Kretschmann 2008-12-23 16:26:27 UTC
*** Bug 178576 has been marked as a duplicate of this bug. ***
Comment 29 Mark Kretschmann 2008-12-24 10:47:00 UTC
SVN commit 900990 by markey:

Show a warning dialog when the collection scan was aborted due to too many
scanner crashes (currently 80), and then abort the scan.

This was left unfinished after the porting from Amarok 1 to 2.0, and now
I ported it over.

CCBUG: 176154

 M  +7 -5      ScanManager.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=900990
Comment 30 Mark Kretschmann 2008-12-24 14:18:40 UTC
*** Bug 178668 has been marked as a duplicate of this bug. ***
Comment 31 Mark Kretschmann 2008-12-26 09:02:17 UTC
*** Bug 178782 has been marked as a duplicate of this bug. ***
Comment 32 Mark Kretschmann 2008-12-27 09:27:16 UTC
*** Bug 178866 has been marked as a duplicate of this bug. ***
Comment 33 Mark Kretschmann 2008-12-28 15:53:33 UTC
*** Bug 178969 has been marked as a duplicate of this bug. ***
Comment 34 Mark Kretschmann 2008-12-28 17:16:49 UTC
*** Bug 178973 has been marked as a duplicate of this bug. ***
Comment 35 Dan Meltzer 2008-12-28 17:58:10 UTC
SVN commit 902548 by dmeltzer:

debug++.  Please test and provide the output if your collection gets messed up.
CCBUG: 176154


 M  +40 -9     ScanManager.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=902548
Comment 36 Mark Kretschmann 2008-12-28 18:16:28 UTC
Please note that you need to start Amarok with "amarok --debug" to see the debug output.
Comment 37 François Michonneau 2008-12-28 20:02:50 UTC
With revision 902592, my collection still doesn't build. I mostly get the following error messages when I try to import my AmaroK 1.4 MySQL collection or that I scan the folder where my music is:
amarok:      [ERROR!] Tried to perform escape() on uninitialized MySQLe 
amarok:      [ERROR!] Tried to perform query on uninitialized MySQLe 
Comment 38 Mark Kretschmann 2008-12-28 21:11:36 UTC
@François: This issue is completely unrelated to what we are discussing here. Your MySQL Embedded installation is simply broken.

Please see here:

http://amarok.kde.org/wiki/Development/MySQL_Embedded
Comment 39 missive 2008-12-28 21:47:11 UTC
> Tried to perform escape() on uninitialized MySQLe
> Tried to perform query on uninitialized MySQLe

Yea. I am getting that also. (I reported Bug 177878).

> http://amarok.kde.org/wiki/Development/MySQL_Embedded

I am running Ubuntu 8.10 and have libmysqlclient15-dev installed.

I tried running the command from the Tip, but have no /usr/sbin/mysqld
Is that required?

Shouldn't configure while building Amarok report a version problem, or a missing library?

Or shouldn't it report a problem at runtime that mysqle is not available?
Comment 40 Mark Kretschmann 2008-12-29 09:47:14 UTC
@missive: Please keep the conversation on topic here. You can discuss your MySQL problems in our forum, or on the mailing list.
Comment 41 Knut Johansson 2008-12-29 13:45:41 UTC
Created attachment 29724 [details]
Log of Amarok messing up the collection

Here is log from Amarok(updated) when I get a messed up collection, based on the first steps of Bug 178973. Fresh start, plus adding one folder.



BTW, I had the same mysql issues as missive, adding mysql and mysql-client fixed it for some reason. There are issues there, real feedback to user is needed.
Comment 42 Mark Kretschmann 2008-12-30 11:01:47 UTC
@Knut: I can't see anything irregular in your log.

It's definitely a different issue from the one I'm currently researching (collection scanner crashes, gets restarted automatically, some data is lost in between). The symptoms for this issue are usually missing albums, not mixed up tags.
Comment 43 Knut Johansson 2008-12-30 13:45:20 UTC
@Mark: Ok, but except for the scanner crash it seem I have the everything else. Not only messing up tags, albums and tracks go missing too. I'd call it total lack of data integrity in the collection. Added with the fact that it changes after a application restart, could be symptoms there are something fundamentally wrong.

Perhaps unduplicating of Bug 178973 then? In any case it seem to me it may be easily reproducable, as I can reproduce both with prebuilt binaries of 2.0 and with svn on two different machines
Comment 44 Mark Kretschmann 2008-12-31 11:29:08 UTC
SVN commit 903687 by markey:

Fixed problems with the collection scanner missing part of the
collection, or potentially failing altogether.

Specifically the problem was with restarting the amarokcollectionscanner
binary after it has crashed, which we are doing transparently in the
background, without the user ever noticing (normally). The scanner can
crash due to TagLib crashing, which still can happen sometimes.

The code for restarting the scanner was never properly ported over from
Amarok 1, and it would often miss a part of the data during restarting.
This should be fixed now.

PS: This took me about 4 days of intense hacking, so I'm really happy
now that it's done.

Please test :)

BUG: 176154

 M  +33 -42    collection/sqlcollection/ScanManager.cpp  
 M  +0 -3      collection/sqlcollection/ScanManager.h  
 M  +5 -2      collectionscanner/CollectionScanner.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=903687
Comment 45 Mark Kretschmann 2009-01-01 15:11:54 UTC
*** Bug 179271 has been marked as a duplicate of this bug. ***
Comment 46 Mark Kretschmann 2009-01-03 14:56:01 UTC
*** Bug 179411 has been marked as a duplicate of this bug. ***
Comment 47 a thing 2009-01-08 03:31:44 UTC
This still happens.
Comment 48 Georg Wittenburg 2009-01-11 22:46:11 UTC
I'm still seeing this bug in 2.0.1.1. Specifically, the collection scanner hangs at 96% with 100% CPU usage after adding/renaming files in the collection. The output of `amarok --debug` is

[...lots of these...]
TagLib: MPEG::Header::parse() -- Invalid sample rate.
TagLib: MPEG::Header::parse() -- Invalid sample rate.
TagLib: MPEG::Header::parse() -- Invalid sample rate.
amarok:  do-while done with error:  0
amarok:  Trying to restart the QXmlStreamReader..
amarok:  Success. Committing result to database.
amarok: BEGIN: void ScanManager::slotFinished()
amarok: END__: void ScanManager::slotFinished() - Took 0.00011s

I haven't noticed any messed up information in the collection, it's just the scanner that hangs.

This happens using the Debian experimental packages of amarok 2.0.1.1 on KDE 4.1.96. Actually `amarok -v` claims that it's version 2.0.1, but maybe there was a packaging mishap or someone forgot to update the version string.
Comment 49 Johannes Schwall 2009-01-12 09:06:37 UTC
My problem (Bug 179271) is also still in 2.0.1 (2.0.1 (KDE 4.1.85 (KDE 4.1.85 (KDE 4.2 Beta2)), Kubuntu packages)) ... several albums are not shown in the collection. Unfortunately, this makes Amarok unusable for me.
Comment 50 Mark Kretschmann 2009-01-12 12:06:14 UTC
Ok, then we'll have to work some more on the scanner. Reopening report.

Unfortunately, this is very difficult work, as each developer usually only has one collection, and thus it's hard to reproduce other people's problems :(

Maybe we need to set up some kind of testing server with ssh, for checking different collections. Ideas welcome ;)
Comment 51 Georg Wittenburg 2009-01-12 12:18:27 UTC
Any chance of providing a version of Amarok from SVN with insanely verbose debug output?
Comment 52 Patrick Jensen 2009-01-12 12:21:58 UTC
Confirming that updating to 2.0.1. did not resolve problem. Additionally, it introduced bug 178973 to me.

I now have e.g. entries under the artist Altaria with an album name from an Aerosmith album and tracks from several unrelated artist, of these tracks several share the same track number. The collection is now entirely garbled and pretty much unusable.

Could this somehow be linked to collection size? The original poster had a collection in excess of 9 000 tracks, mine is a bit over 14 000 tracks. Could this be a common theme?
Comment 53 Mark Kretschmann 2009-01-12 12:38:46 UTC
@Patrick: It is linked to collection size in so far:

Bigger collection == higher chance of taglib crashing on a file, and higher chance of triggering other obscure bugs.
Comment 54 Mark Kretschmann 2009-01-12 12:40:41 UTC
@Georg: Yes, revert the following commit:

"SVN commit 905846 by markey:

Disable spammy debug output during collection scanning."
Comment 55 Mark Kretschmann 2009-01-12 19:41:57 UTC
*** Bug 180473 has been marked as a duplicate of this bug. ***
Comment 56 Jochen Bauer 2009-01-13 14:32:42 UTC
For me, my DB was fixed by deleting the mysqle-folder in .kde/share/apps/amarok.

But as markey said me, that was no scanner bug.
Comment 57 Greyor 2009-01-13 21:29:31 UTC
I just tried deleting the mysqle folder, and it worked wonders. I've a ~10,000 song collection.

What I did was delete the mysqle folder after Amarok was closed, then restarted it, and I made the changes to the db.opt files referenced <a href="https://bugs.kde.org/show_bug.cgi?id=176154#c6">above</a> (comment #6) just for good measure. I let it rescan the collection and also reloaded my playlist, and everything worked properly.

In short, try deleting the mysqle folder and thus forcing it to rebuild the database; it did work for me, as far as I can tell.
Comment 58 Emanuele Cisotti 2009-01-13 22:19:41 UTC
it doesn't work for me.
I also tried to remove the whole .amarok folder but nothing changes.
Comment 59 Emanuele Cisotti 2009-01-13 22:21:17 UTC
I'm really sorry, I was thinking this was the bug for the "compilation" bug.
About this bug, removing the folder works for me.
Comment 60 a thing 2009-01-14 00:16:26 UTC
Deleting the MySQLE DB temporarily works as I mentioned in my duplicate bug report. However, the problem returns when updating the collection.
Comment 61 Greyor 2009-01-15 05:41:16 UTC
Right, I've not added anything to the collection since I rebuilt it, so there is the possibility as you mentioned that it may recur. I hope not, but if it does, I'll just rebuild the thing again. There's not a whole lot of turnover in my collection, so it's just a minor nuisance for now, but I'm glad Mark and the other devs are aware of it.

Love Amarok and wish only the best for all of you. If I could help with this myself I would, but I'm afraid my coding skills are not up to snuff.

If it does recur I'll confirm it, however.
Comment 62 Mark Kretschmann 2009-01-16 11:12:40 UTC
*** Bug 178124 has been marked as a duplicate of this bug. ***
Comment 63 Patrick Jensen 2009-01-24 21:32:28 UTC
I deleted the mysqle folder as suggested by Greyor, which made the collection work fine for a while once it was re-built. A few days later Amarok suddenly stopped playing anything altogether, giving the following warnings:

amarok: [EngineController] URL:"file:///home/jensen/musik/Musik/Pearl%20Jam%20-%20Live%20On%20Two%20Legs/01%20-%20Corduroy.ogg"
---
QString::arg: Argument missing: Amarok - No track playing., 0:00
--
QString::arg: Argument missing: Amarok - No track playing., 0:00
---
amarok: [Playlist::Actions] [WARNING!] engineNewTrackPlaying: "Corduroy" does not match what the playlist controller thought it should be
---
QString::arg: Argument missing: Amarok - No track playing., 0:00
---
amarok: [EngineController] [WARNING!] Phonon failed to play this URL. Error:  ""
---
amarok: [Playlist::Actions] [WARNING!] Error, can not play this track.
amarok: [Playlist::Actions] [WARNING!] Failure count:  1

("---" indicates where I omited text. I did so because the output is rather verbose and since I don't know what is relevant to the error messages, I decided to not spam too much by posting a full log.)

Unfortunately I cannot for the life of me remember which track triggered this. Apologies if this is of little relevance to the topic, I will file a separate bug report if you determine it a separate bug.
Comment 64 Greyor 2009-01-27 07:35:32 UTC
Re: Patrick -- I'm not having the same problem as you, just the problem of the collection going south as I had before.

I want to confirm that this did recur for me, as "a thing" commented above, once I added to my collection again. I added about 20 albums or so, and there I was back at square one.

I did an experiment this time, however. I tried simply deleting the mysqle folder and having it rebuild from there, but *not* changing the db.opt files in each directory to reflect the recommendations above (utf8_general_ci, etc.).

I've concluded that the db.opt files need to be modified after you delete the mysqle folder, and while Amarok is loading on that first restart. If you change the latin1_swedish_ci line to utf8_general_ci while it's loading, after it rescans the collection, everything should come back to normal.

Perhaps the db.opt files should default to utf8_general_ci when the mysqle directory structure is being rebuilt? Can a patch be committed to do that? I don't think it would solve the larger issue, but it would be a step in the right direction.

Again, just wanted to confirm that it has recurred since adding to the collection, and that as far as I can tell, editing the two db.opt files seems to be required for it to rebuild correctly.

Thanks for all your hard work, and I hope a solution can be found.
Comment 65 Mark Kretschmann 2009-01-30 22:24:42 UTC
*** Bug 180248 has been marked as a duplicate of this bug. ***
Comment 66 Mark Kretschmann 2009-01-30 22:30:54 UTC
*** Bug 182067 has been marked as a duplicate of this bug. ***
Comment 67 Mark Kretschmann 2009-02-02 23:00:50 UTC
*** Bug 182877 has been marked as a duplicate of this bug. ***
Comment 68 Mark Kretschmann 2009-02-09 13:31:51 UTC
*** Bug 183770 has been marked as a duplicate of this bug. ***
Comment 69 Mark Kretschmann 2009-02-10 14:11:01 UTC
*** Bug 176942 has been marked as a duplicate of this bug. ***
Comment 70 Dan Meltzer 2009-02-18 23:14:09 UTC
SVN commit 928105 by dmeltzer:

Improve the collectionscanner by a factor of a lot.

Treating QXmlStreamReader::NoError as a non-recoverable error is probably going to make things not work that well.  This was just one of those things that was way too easy a bug to find and fix for way too long, apparently.  I'm not certain this fixes all the issues, but it scans a lot more of my collection now.  Will backport to 2.0 branch after it gets some more testing.

CCBUG: 176154

 M  +1 -0      ChangeLog  
 M  +17 -15    src/collection/sqlcollection/ScanManager.cpp  
 M  +11 -3     src/collection/sqlcollection/ScanResultProcessor.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=928105
Comment 71 Andy 2009-02-19 00:07:52 UTC
Dan, you mention it scans more of your collection?  I haven't seen the collection count anywhere in the 2.0 series yet, is there another method I'm not aware of?

I have a huge (150K) collection, I'll test the patch tonight.
Comment 72 Seb Ruiz 2009-02-19 00:17:48 UTC
The best way to get the track count in Amarok2 is using the script console. Open the console from the script manager and issue the following command:

Amarok.Collection.query( "SELECT count(*) FROM tracks" );
Comment 73 Andy 2009-02-20 03:24:13 UTC
Not sure how to properly log this, but collection scanning gets progressively slower as the collection builds...  These files by them alone scan almost instantly.  This is just an example towards the end of scanning.

END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 4.9s 
amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) 
amarok:      Album is:  "Winning Days" 
amarok:      Artists:  QSet("The Vines") 
amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 4.1s 
amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) 
amarok:      Album is:  "All Systems Go" 
amarok:      Artists:  QSet("", "Vinnie Vincent Invasion") 
amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 4.9s 
amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) 
amarok:      Album is:  "Add It Up (1981-1993)" 
amarok:      Artists:  QSet("Violent Femmes") 
amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 16s 
amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) 
amarok:      Album is:  "New Times" 
amarok:      Artists:  QSet("Violent Femmes") 
amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 4.9s 
amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) 
amarok:      Album is:  "The Blind Leading The Naked" 
amarok:      Artists:  QSet("Violent Femmes") 
amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 5s 
amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) 
amarok:      Album is:  "Violent Femmes" 
amarok:      Artists:  QSet("Violent Femmes") 
amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 3.4s 
amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) 
amarok:      Album is:  "Violent Femmes - Deluxe Edition - Disc 1" 
amarok:      Artists:  QSet("Violent Femmes") 
amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 7.9s 
amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) 
amarok:      Album is:  "Violent Femmes [Deluxe Edition] (Disc 2)" 
amarok:      Artists:  QSet("Violent Femmes") 
 slower as the collection builds:

This scan was significantly faster than before, this one took about 20 hours, 6-7 hours with Amarok 1.4x and 6-7 days with with SVN 2.0 version.

As for the file count, before Dan's SVN commit the scan found 147K files, this scan 123K.

To log an entire scans output looks like I would have to set my output to log some 500,000 lines back, which I'm willing to do if necessary.
Comment 74 Carlos Mata 2009-02-27 19:27:30 UTC
I applied the fix suggested above (deleting db folders) which worked for me until I rescaned my library today. Now there are a lot of tracks missing in the _collection view_ of amarok.

Interesting is that when I add the missing files manually by dragging them from a file manager the ratings are shown correctly. Since the ratings are stored in the amarok db it seems that amarok has all the tracks in the db, it only does not show them in the collection view and does not select them in dynamic playlist. 

Maybe this bug has another side which is not connected to the ScanManager.
Comment 75 Tyler Mandry 2009-03-01 04:08:30 UTC
I'd like to point out that there are at least two separate issues attached to this bug. Many people have reported having problems where amarok mixed up the album and artist names for lots of their songs. I've had this problem and it causes random songs to have the right title but completely wrong artists and albums (which usually don't match each other); this causes them to disappear from their usual place and pop up somewhere else. However this seems to have NOTHING TO DO with incorrect tag reading or collection building! An interesting thing I've noticed is that quitting amarok and restarting it often seems to fix some of the files that were messed up, but then I notice new ones popping up. It looks like there's some weird messed up database query voodoo going on in there.

One variant of the tagging issue I'm currently experiencing is a lot of songs' tags not being read at all, although I can confirm that they are there. When I go and retag the song in amarok that song seems to be fixed, but I am finding more every day. None of these show up under the right spot, OR Various Artists or Unknown, but when using dynamic playlists to play random songs, these untagged songs will pop up. I'm not sure if this is a database or a tagging lib issue.
Comment 76 Eleazar 2009-03-09 03:55:09 UTC
I am having the same issue (Miss read tags) I have verified that the tags contain are the correct ID3 Tag with other apps (Both Linux and Winblows). It seems to not be able to read the tags correctly with large libraries. I also noticed that even though I fetch the correct Cover and save it, amarok does not show it in the playlist. I select "Show cover" and the correct one appears in the window. So at least it is saving the correct cover to track. I have my complete collection load in a playlist and set it to random play. What I have notice (don't if it's a coincidence) is that the next song the plays has the previous song's artist or if the song is by the same artist the same incorrect artist is used.

Don't know if this info helps or not
Comment 77 Carlos Mata 2009-03-15 23:21:58 UTC
Hello,

I wanted to notify you the the problem of messed up id3 tags, missing tracks, etc. seems to be gone for me since I last upgraded amarok (nightly build from 14.3.2009)
I just rescaned the collection and all tracks showed up (this was not working before) with their correct rating.

Thanks
Comment 78 Mark Kretschmann 2009-03-25 17:17:39 UTC
@Carlos: For me the problem also appears fixed now, after our last round of fixes for the collection scanner (done by Dan Meltzer) in SVN trunk.

So I'm closing the report now, and I guess we should say "thanks" to Dan :)


PS: Please note that we don't intend to do another 2.0.x release, as we are about to release 2.1-beta1 soon. So the 2.0.x users here will probably have to wait for 2.1.
Comment 79 Dan Meltzer 2009-03-26 01:47:40 UTC
*** Bug 188122 has been marked as a duplicate of this bug. ***
Comment 80 Seb Ruiz 2009-04-20 14:42:43 UTC
*** Bug 189803 has been marked as a duplicate of this bug. ***
Comment 81 Orcan Ogetbil 2009-06-07 18:01:19 UTC
This bug is back with 2.1. Please reopen the bug.
Comment 82 Myriam Schweingruber 2009-06-09 17:35:30 UTC
(In reply to comment #81)
> This bug is back with 2.1. Please reopen the bug.

Did you erase your .kde/sahre/apps/amarok/mysqle/ folder? I f after that you can still reproduce the bug, please open a new one and give all the necessary information on how to reproduce it