Version: 1.98 (using 4.1.3 (KDE 4.1.3), Kubuntu packages) Compiler: cc OS: Linux (x86_64) release 2.6.27-8-generic I've got all of my collection (~9000 songs) organised into folders within a single folder. When Amarok scans my collection, for about half of these, it doesn't import anything but the title and track number for the song. This leads to amarok having a huge number of unsorted music. I have checked that the id3 tags are still there, and they are, it just seems that Amarok is ignoring them, and continues to do so when I rescan my collection. I can't see any logic behind which files it does this for, most of the time it's whole albums, but some are not.
I have just noticed that Amarok has also placed some albums under the name of a different album (that is, an album that has been correctly recognised as an album by a certain artist, has the name of an album that Amarok has not identified)
*** Bug 176134 has been marked as a duplicate of this bug. ***
I've seen this problem too, however it happens with far less tracks here. One suspicion I have is that it's got to do with bad MySQL escaping, which we're currently investigating. Does it happen especially with files that have non-ASCII characters in their tags or filenames? Could you make one of the problematic files accessible to us, for reproducing?
I don't think it happens especially with non-ASCII characters, but it's hard to get an idea of the trend with such a large number of songs to look through. Here is one of the files whose album and artist is ignored: http://www.megaupload.com/?d=TIEE2IO3 Also, I store the music on an NTFS partition (a relic from my windows days), just in case that is relevant.
Also, here is a zip of 2 mp3 files, one of which (track 12) is put into the correct album and given the correct artist name, the other of which is not. http://www.megaupload.com/?d=V6II05KK
~/.kde/share/apps/amarok/mysqle/mysql/db.opt ~/.kde/share/apps/amarok/mysqle/amarok/db.opt default-character-set=latin1 default-collation=utf8_general_ci try to change the default-collation to utf8 in the two files to see the difference( from latin_swedish_ci )
and btw, please don't forget to rescan your collection :)
SVN commit 889579 by markey: Use mysql_real_escape_string() for escaping, instead of our own concoction of an escaping routine. This might fix escaping errors that could affect the following bugs: CCBUG: 176154 CCBUG: 174784 Please test. M +10 -1 MySqlEmbeddedCollection.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=889579
(In reply to comment #6) > try to change the default-collation to utf8 in the two files to see the > difference( from latin_swedish_ci ) After trying this, Amarok now reads the album and artist name for all of my files, but it does still mix up some of the albums, claiming that their tags do contain an album name (the wrong album name). I checked the tags in another program, and it disagrees with Amarok, and claims that the tags are as they should be. A confusing thing is that changing the default-collation entries back to what they were does not cause the collection to revert to what it was previously. My apologies, but I don't know what to do with the MySqlEmbeddedCollection.cpp file. I have only recently started learning how to program, but I only really know enough to know that it's a C++ source file... Do I download the Amarok source code, and replace one file with the one you posted, then compile?
You have to grab the code from svn and compile, or you have to wait till the next amarok release.
(In reply to comment #6) > ~/.kde/share/apps/amarok/mysqle/mysql/db.opt > ~/.kde/share/apps/amarok/mysqle/amarok/db.opt > > > default-character-set=latin1 > default-collation=utf8_general_ci > > try to change the default-collation to utf8 in the two files to see the > difference( from latin_swedish_ci ) > Doing this actually caused the collection to lose track of all tracks bar those that I had manually edited in Kid3. If it helps any, all of the tracks are original rips, extracted with K3B.
Please try again with latest SVN trunk. We have a new patch in for an encoding problem: SVN commit 890023 by woebbe: escape(): - fix encoding (don't use implicit const char* -> QString cast) - build with -pedantic M +4 -3 MySqlEmbeddedCollection.cpp
(In reply to comment #12) > Please try again with latest SVN trunk. We have a new patch in for an encoding > problem: > > SVN commit 890023 by woebbe: > > escape(): > - fix encoding (don't use implicit const char* -> QString cast) > - build with -pedantic > > M +4 -3 MySqlEmbeddedCollection.cpp > No can do. The svn version will not compile because of a too old version of MySQL and since my experiences in bypassing the package manager to install stuff from outside the repositories have usually ended up in severely botched installations, I will instead wait until this update finds its way into the next RC version.
*** Bug 177602 has been marked as a duplicate of this bug. ***
*** Bug 177784 has been marked as a duplicate of this bug. ***
*** Bug 176563 has been marked as a duplicate of this bug. ***
*** Bug 177746 has been marked as a duplicate of this bug. ***
Hi. My bug report was marked as a duplicate of this bug. Is there a solution to this problem?
@Rishad: Not yet, otherwise this would have been marked as FIXED.
*** Bug 177878 has been marked as a duplicate of this bug. ***
*** Bug 177918 has been marked as a duplicate of this bug. ***
Ok! My bug was marked as duplicate of this one! What I've done is to create a new folder in the collection directory and put some new music files in there and also moved some from another folder into it. Amarok not only doesn't seem to find the new files as it think the ones I moved are still in the place they where before. How can I get more useful information to discover where the error is happening. Here I am using the SVN version of Amarok.
WOAH! Something is really wrong. I just tried to rescan the collection using the button in the collection part of the configuration window. Now the collection is all messed up. Albums are showing in the wrong artists and the years displayed for the albums in the tree are wrong and musics are being shown in the wrong album. I'll try to wipe my profile to see if the things get corrected.
Hi again! Sorry for the triple post. Cleaning the profile seems to fix the problem. I've added new files to the collection folder after it had already been scanned by Amarok and they have been detected and included in the collection. However this is not a real solution as the user shouldn't have to clean the config files anytime. Maybe some further investigation is needed to find why this problem appears.
What is the status of this? Close as fixed?
> Close as fixed? I hope not... I just built r897811, and nothing shows up in my collection at all.
Cleaning the profile did not solve the problem for me. My collection scan still hangs at 97%. When I copy a few random songs to the home/Music folder it works though. My entire collection doesn't though and it worked fine in 1.4.x.
*** Bug 178576 has been marked as a duplicate of this bug. ***
SVN commit 900990 by markey: Show a warning dialog when the collection scan was aborted due to too many scanner crashes (currently 80), and then abort the scan. This was left unfinished after the porting from Amarok 1 to 2.0, and now I ported it over. CCBUG: 176154 M +7 -5 ScanManager.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=900990
*** Bug 178668 has been marked as a duplicate of this bug. ***
*** Bug 178782 has been marked as a duplicate of this bug. ***
*** Bug 178866 has been marked as a duplicate of this bug. ***
*** Bug 178969 has been marked as a duplicate of this bug. ***
*** Bug 178973 has been marked as a duplicate of this bug. ***
SVN commit 902548 by dmeltzer: debug++. Please test and provide the output if your collection gets messed up. CCBUG: 176154 M +40 -9 ScanManager.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=902548
Please note that you need to start Amarok with "amarok --debug" to see the debug output.
With revision 902592, my collection still doesn't build. I mostly get the following error messages when I try to import my AmaroK 1.4 MySQL collection or that I scan the folder where my music is: amarok: [ERROR!] Tried to perform escape() on uninitialized MySQLe amarok: [ERROR!] Tried to perform query on uninitialized MySQLe
@François: This issue is completely unrelated to what we are discussing here. Your MySQL Embedded installation is simply broken. Please see here: http://amarok.kde.org/wiki/Development/MySQL_Embedded
> Tried to perform escape() on uninitialized MySQLe > Tried to perform query on uninitialized MySQLe Yea. I am getting that also. (I reported Bug 177878). > http://amarok.kde.org/wiki/Development/MySQL_Embedded I am running Ubuntu 8.10 and have libmysqlclient15-dev installed. I tried running the command from the Tip, but have no /usr/sbin/mysqld Is that required? Shouldn't configure while building Amarok report a version problem, or a missing library? Or shouldn't it report a problem at runtime that mysqle is not available?
@missive: Please keep the conversation on topic here. You can discuss your MySQL problems in our forum, or on the mailing list.
Created attachment 29724 [details] Log of Amarok messing up the collection Here is log from Amarok(updated) when I get a messed up collection, based on the first steps of Bug 178973. Fresh start, plus adding one folder. BTW, I had the same mysql issues as missive, adding mysql and mysql-client fixed it for some reason. There are issues there, real feedback to user is needed.
@Knut: I can't see anything irregular in your log. It's definitely a different issue from the one I'm currently researching (collection scanner crashes, gets restarted automatically, some data is lost in between). The symptoms for this issue are usually missing albums, not mixed up tags.
@Mark: Ok, but except for the scanner crash it seem I have the everything else. Not only messing up tags, albums and tracks go missing too. I'd call it total lack of data integrity in the collection. Added with the fact that it changes after a application restart, could be symptoms there are something fundamentally wrong. Perhaps unduplicating of Bug 178973 then? In any case it seem to me it may be easily reproducable, as I can reproduce both with prebuilt binaries of 2.0 and with svn on two different machines
SVN commit 903687 by markey: Fixed problems with the collection scanner missing part of the collection, or potentially failing altogether. Specifically the problem was with restarting the amarokcollectionscanner binary after it has crashed, which we are doing transparently in the background, without the user ever noticing (normally). The scanner can crash due to TagLib crashing, which still can happen sometimes. The code for restarting the scanner was never properly ported over from Amarok 1, and it would often miss a part of the data during restarting. This should be fixed now. PS: This took me about 4 days of intense hacking, so I'm really happy now that it's done. Please test :) BUG: 176154 M +33 -42 collection/sqlcollection/ScanManager.cpp M +0 -3 collection/sqlcollection/ScanManager.h M +5 -2 collectionscanner/CollectionScanner.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=903687
*** Bug 179271 has been marked as a duplicate of this bug. ***
*** Bug 179411 has been marked as a duplicate of this bug. ***
This still happens.
I'm still seeing this bug in 2.0.1.1. Specifically, the collection scanner hangs at 96% with 100% CPU usage after adding/renaming files in the collection. The output of `amarok --debug` is [...lots of these...] TagLib: MPEG::Header::parse() -- Invalid sample rate. TagLib: MPEG::Header::parse() -- Invalid sample rate. TagLib: MPEG::Header::parse() -- Invalid sample rate. amarok: do-while done with error: 0 amarok: Trying to restart the QXmlStreamReader.. amarok: Success. Committing result to database. amarok: BEGIN: void ScanManager::slotFinished() amarok: END__: void ScanManager::slotFinished() - Took 0.00011s I haven't noticed any messed up information in the collection, it's just the scanner that hangs. This happens using the Debian experimental packages of amarok 2.0.1.1 on KDE 4.1.96. Actually `amarok -v` claims that it's version 2.0.1, but maybe there was a packaging mishap or someone forgot to update the version string.
My problem (Bug 179271) is also still in 2.0.1 (2.0.1 (KDE 4.1.85 (KDE 4.1.85 (KDE 4.2 Beta2)), Kubuntu packages)) ... several albums are not shown in the collection. Unfortunately, this makes Amarok unusable for me.
Ok, then we'll have to work some more on the scanner. Reopening report. Unfortunately, this is very difficult work, as each developer usually only has one collection, and thus it's hard to reproduce other people's problems :( Maybe we need to set up some kind of testing server with ssh, for checking different collections. Ideas welcome ;)
Any chance of providing a version of Amarok from SVN with insanely verbose debug output?
Confirming that updating to 2.0.1. did not resolve problem. Additionally, it introduced bug 178973 to me. I now have e.g. entries under the artist Altaria with an album name from an Aerosmith album and tracks from several unrelated artist, of these tracks several share the same track number. The collection is now entirely garbled and pretty much unusable. Could this somehow be linked to collection size? The original poster had a collection in excess of 9 000 tracks, mine is a bit over 14 000 tracks. Could this be a common theme?
@Patrick: It is linked to collection size in so far: Bigger collection == higher chance of taglib crashing on a file, and higher chance of triggering other obscure bugs.
@Georg: Yes, revert the following commit: "SVN commit 905846 by markey: Disable spammy debug output during collection scanning."
*** Bug 180473 has been marked as a duplicate of this bug. ***
For me, my DB was fixed by deleting the mysqle-folder in .kde/share/apps/amarok. But as markey said me, that was no scanner bug.
I just tried deleting the mysqle folder, and it worked wonders. I've a ~10,000 song collection. What I did was delete the mysqle folder after Amarok was closed, then restarted it, and I made the changes to the db.opt files referenced <a href="https://bugs.kde.org/show_bug.cgi?id=176154#c6">above</a> (comment #6) just for good measure. I let it rescan the collection and also reloaded my playlist, and everything worked properly. In short, try deleting the mysqle folder and thus forcing it to rebuild the database; it did work for me, as far as I can tell.
it doesn't work for me. I also tried to remove the whole .amarok folder but nothing changes.
I'm really sorry, I was thinking this was the bug for the "compilation" bug. About this bug, removing the folder works for me.
Deleting the MySQLE DB temporarily works as I mentioned in my duplicate bug report. However, the problem returns when updating the collection.
Right, I've not added anything to the collection since I rebuilt it, so there is the possibility as you mentioned that it may recur. I hope not, but if it does, I'll just rebuild the thing again. There's not a whole lot of turnover in my collection, so it's just a minor nuisance for now, but I'm glad Mark and the other devs are aware of it. Love Amarok and wish only the best for all of you. If I could help with this myself I would, but I'm afraid my coding skills are not up to snuff. If it does recur I'll confirm it, however.
*** Bug 178124 has been marked as a duplicate of this bug. ***
I deleted the mysqle folder as suggested by Greyor, which made the collection work fine for a while once it was re-built. A few days later Amarok suddenly stopped playing anything altogether, giving the following warnings: amarok: [EngineController] URL:"file:///home/jensen/musik/Musik/Pearl%20Jam%20-%20Live%20On%20Two%20Legs/01%20-%20Corduroy.ogg" --- QString::arg: Argument missing: Amarok - No track playing., 0:00 -- QString::arg: Argument missing: Amarok - No track playing., 0:00 --- amarok: [Playlist::Actions] [WARNING!] engineNewTrackPlaying: "Corduroy" does not match what the playlist controller thought it should be --- QString::arg: Argument missing: Amarok - No track playing., 0:00 --- amarok: [EngineController] [WARNING!] Phonon failed to play this URL. Error: "" --- amarok: [Playlist::Actions] [WARNING!] Error, can not play this track. amarok: [Playlist::Actions] [WARNING!] Failure count: 1 ("---" indicates where I omited text. I did so because the output is rather verbose and since I don't know what is relevant to the error messages, I decided to not spam too much by posting a full log.) Unfortunately I cannot for the life of me remember which track triggered this. Apologies if this is of little relevance to the topic, I will file a separate bug report if you determine it a separate bug.
Re: Patrick -- I'm not having the same problem as you, just the problem of the collection going south as I had before. I want to confirm that this did recur for me, as "a thing" commented above, once I added to my collection again. I added about 20 albums or so, and there I was back at square one. I did an experiment this time, however. I tried simply deleting the mysqle folder and having it rebuild from there, but *not* changing the db.opt files in each directory to reflect the recommendations above (utf8_general_ci, etc.). I've concluded that the db.opt files need to be modified after you delete the mysqle folder, and while Amarok is loading on that first restart. If you change the latin1_swedish_ci line to utf8_general_ci while it's loading, after it rescans the collection, everything should come back to normal. Perhaps the db.opt files should default to utf8_general_ci when the mysqle directory structure is being rebuilt? Can a patch be committed to do that? I don't think it would solve the larger issue, but it would be a step in the right direction. Again, just wanted to confirm that it has recurred since adding to the collection, and that as far as I can tell, editing the two db.opt files seems to be required for it to rebuild correctly. Thanks for all your hard work, and I hope a solution can be found.
*** Bug 180248 has been marked as a duplicate of this bug. ***
*** Bug 182067 has been marked as a duplicate of this bug. ***
*** Bug 182877 has been marked as a duplicate of this bug. ***
*** Bug 183770 has been marked as a duplicate of this bug. ***
*** Bug 176942 has been marked as a duplicate of this bug. ***
SVN commit 928105 by dmeltzer: Improve the collectionscanner by a factor of a lot. Treating QXmlStreamReader::NoError as a non-recoverable error is probably going to make things not work that well. This was just one of those things that was way too easy a bug to find and fix for way too long, apparently. I'm not certain this fixes all the issues, but it scans a lot more of my collection now. Will backport to 2.0 branch after it gets some more testing. CCBUG: 176154 M +1 -0 ChangeLog M +17 -15 src/collection/sqlcollection/ScanManager.cpp M +11 -3 src/collection/sqlcollection/ScanResultProcessor.cpp WebSVN link: http://websvn.kde.org/?view=rev&revision=928105
Dan, you mention it scans more of your collection? I haven't seen the collection count anywhere in the 2.0 series yet, is there another method I'm not aware of? I have a huge (150K) collection, I'll test the patch tonight.
The best way to get the track count in Amarok2 is using the script console. Open the console from the script manager and issue the following command: Amarok.Collection.query( "SELECT count(*) FROM tracks" );
Not sure how to properly log this, but collection scanning gets progressively slower as the collection builds... These files by them alone scan almost instantly. This is just an example towards the end of scanning. END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 4.9s amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) amarok: Album is: "Winning Days" amarok: Artists: QSet("The Vines") amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 4.1s amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) amarok: Album is: "All Systems Go" amarok: Artists: QSet("", "Vinnie Vincent Invasion") amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 4.9s amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) amarok: Album is: "Add It Up (1981-1993)" amarok: Artists: QSet("Violent Femmes") amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 16s amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) amarok: Album is: "New Times" amarok: Artists: QSet("Violent Femmes") amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 4.9s amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) amarok: Album is: "The Blind Leading The Naked" amarok: Artists: QSet("Violent Femmes") amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 5s amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) amarok: Album is: "Violent Femmes" amarok: Artists: QSet("Violent Femmes") amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 3.4s amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) amarok: Album is: "Violent Femmes - Deluxe Edition - Disc 1" amarok: Artists: QSet("Violent Femmes") amarok: END__: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) - Took 7.9s amarok: BEGIN: void ScanResultProcessor::processDirectory(const QList<QMap<QString, QVariant> >&) amarok: Album is: "Violent Femmes [Deluxe Edition] (Disc 2)" amarok: Artists: QSet("Violent Femmes") slower as the collection builds: This scan was significantly faster than before, this one took about 20 hours, 6-7 hours with Amarok 1.4x and 6-7 days with with SVN 2.0 version. As for the file count, before Dan's SVN commit the scan found 147K files, this scan 123K. To log an entire scans output looks like I would have to set my output to log some 500,000 lines back, which I'm willing to do if necessary.
I applied the fix suggested above (deleting db folders) which worked for me until I rescaned my library today. Now there are a lot of tracks missing in the _collection view_ of amarok. Interesting is that when I add the missing files manually by dragging them from a file manager the ratings are shown correctly. Since the ratings are stored in the amarok db it seems that amarok has all the tracks in the db, it only does not show them in the collection view and does not select them in dynamic playlist. Maybe this bug has another side which is not connected to the ScanManager.
I'd like to point out that there are at least two separate issues attached to this bug. Many people have reported having problems where amarok mixed up the album and artist names for lots of their songs. I've had this problem and it causes random songs to have the right title but completely wrong artists and albums (which usually don't match each other); this causes them to disappear from their usual place and pop up somewhere else. However this seems to have NOTHING TO DO with incorrect tag reading or collection building! An interesting thing I've noticed is that quitting amarok and restarting it often seems to fix some of the files that were messed up, but then I notice new ones popping up. It looks like there's some weird messed up database query voodoo going on in there. One variant of the tagging issue I'm currently experiencing is a lot of songs' tags not being read at all, although I can confirm that they are there. When I go and retag the song in amarok that song seems to be fixed, but I am finding more every day. None of these show up under the right spot, OR Various Artists or Unknown, but when using dynamic playlists to play random songs, these untagged songs will pop up. I'm not sure if this is a database or a tagging lib issue.
I am having the same issue (Miss read tags) I have verified that the tags contain are the correct ID3 Tag with other apps (Both Linux and Winblows). It seems to not be able to read the tags correctly with large libraries. I also noticed that even though I fetch the correct Cover and save it, amarok does not show it in the playlist. I select "Show cover" and the correct one appears in the window. So at least it is saving the correct cover to track. I have my complete collection load in a playlist and set it to random play. What I have notice (don't if it's a coincidence) is that the next song the plays has the previous song's artist or if the song is by the same artist the same incorrect artist is used. Don't know if this info helps or not
Hello, I wanted to notify you the the problem of messed up id3 tags, missing tracks, etc. seems to be gone for me since I last upgraded amarok (nightly build from 14.3.2009) I just rescaned the collection and all tracks showed up (this was not working before) with their correct rating. Thanks
@Carlos: For me the problem also appears fixed now, after our last round of fixes for the collection scanner (done by Dan Meltzer) in SVN trunk. So I'm closing the report now, and I guess we should say "thanks" to Dan :) PS: Please note that we don't intend to do another 2.0.x release, as we are about to release 2.1-beta1 soon. So the 2.0.x users here will probably have to wait for 2.1.
*** Bug 188122 has been marked as a duplicate of this bug. ***
*** Bug 189803 has been marked as a duplicate of this bug. ***
This bug is back with 2.1. Please reopen the bug.
(In reply to comment #81) > This bug is back with 2.1. Please reopen the bug. Did you erase your .kde/sahre/apps/amarok/mysqle/ folder? I f after that you can still reproduce the bug, please open a new one and give all the necessary information on how to reproduce it