Version: 2.2.0 (using 4.3.1 (KDE 4.3.1), Arch Linux) Compiler: gcc OS: Linux (i686) release 2.6.31-ARCH Problem description: ------------------------------------------------------------------------------ Yesterday, I did an system upgrade of my Arch Linux computer, including an update to amarok 2.2.0 from 2.1-1. Upon the first start of amarok after the update, the configuration screen poped up, the directories for my local collection was selected properly. After confirming the dialog, the application (seemed) to rescan the directories in question. After it was finished, several inconsistencies in the collection appeared: - Tags for several albums seem to have been mangled/mixed up in the process of updating/rescanning the collection. Example: The albums for one artist are present, yet contain no files. Adding the files by hand via the file interface of amarok shows that amarok believes to belong to a different artist, yet neither the album nor the files turn up in the list for the (wrong) artist. The tags of the files seem to be correct, though (at least according to kid3). The albums and the files were present and associated with the correct artists in the collection prior to the update. - Some albums turn up multiple times, even though the files in question are only present in location. I will attach screenshots depicting these problems below, if necessary I can provide further information (e.g. a dump of the local database, although instructions for doing this would be appreciated). Restarting the application (even after an unrelated reboot) did not solve the problem, neither did several attempts to rescan the collection. Many thanks for your efforts, please let me know if you need more information. Best regards Martin
Created attachment 37439 [details] Screenshot describing the first mentioned problem depicting problems - "Empty" albums with files attributed to wrong artists - The album does not show up as belonging to the wrongly identified album either Note that the information given by kid3 is stored within tag2, tag1 seems to be empty
Created attachment 37440 [details] Screenshot describing the second mentioned problem Problem depicted: - Search result does not contain the "correct" files, even though they are present on the system, part of the directories that got scanned and the tags are correct (According to kid3).
Did you try a rescan, then restart Amarok?
@Myriam Schweingruber Yes, I did so, with no effect (should have formulated this more clearly in my post, sorry for that). My next step is to remove/rename ~/.kde4/share/apps/amarok directory ~/.kde4/share/config/amarok_homerc ~/.kde4/share/config/amarokrc and start all over again from scratch. Would that be Ok, or do you need further information from these directories/files prior to that? Best regards Martin
Try touching the folder with the missing album first to change it's time stamp.
Touching (both the parent folders as well as the ogg files in question) had no effect, regardless of whether I initialised a rescan manually via the "rescan collection" button from the "Configure Amarok" dialog or via Tools -> Update Collection. But what did the trick was a.) actually moving the directory with one of the "wrong" albums outside the directory tree that is searched by amarok b.) rescan (via Settings -> Configure Amarok -> Collection -> Rescan Collection) c.) move the directory back to it's initial position once the rescan was finished and finally d.) rescan again, repeating the steps from b.) Since a lot of albums are affected (Last count was > 20, not counting individual files that seem to have gone missing in the collection here and there), it is probably more efficient to start from scratch and save the content of the amarok folder for future reference. I can live with this workaround but would still be interested to learn what caused this hickup in the database, hence it would be nice if you (or anybody else knowledgeable) could share your thoughts on this problem. Many thanks for your efforts & best regards Martin
Martin, I'll try to think about what could have caused this. It might be related to a regression (which will be fixed in 2.2.1) where some dirs, if they weren't picked up in an initial full rescan, would not be picked up in subsequent incremental scans, even if touched. As for tags showing up messed up -- did closing Amarok and re-opening help? In 2.2 those are probably collection browser issues (which closing and opening refreshes). A full rescan, which should have been kicked off upon first upgrade, should have fixed things in the database itself (other than that regression, which may have caused your problem).
@Jeff Mitchell Thank you very much for your quick response. Restarting amarok did not help, neither did - unsuprisingly, I know - rebooting the computer or logging out and in to my KDE session. Just for the sake of completeness: I did a system - upgrade ( including amarok) on my laptop using the very same packages as for my desktop computer (which is the one this bug report is all about) and so far everything seems to be Ok, including no peculiarities in the collection browser. I'm in the process of re-initialising the amarok settings on my desktop back to a clean state and will report back in case any problems occur. In case you or one of the other amarok devs are interested, I can provide you with a tarball containing the amarok specfic contents of my ~/.kde4 directory prior to this restart operation. Regards
Martin, That would actually be quite nice, as I can dump that out to a SQL file and load it into my MySQL server for examination. I'm interested in knowing why this would happen, since results for the vast majority of people are that 2.2 finds albums that 2.1 had missing for various reasons, not the other way around :-) However, the data might be somewhat meaningless to me without enough context -- are you ever on IRC or IM? If I can catch you there at some point then that would probably help quite a bit. Let me know (email me your details if you don't want them on this public bug report :-) ).
I had too issues with the collection after upgrading to Amarok 2.2. Songs were missing in the collection that were - according to easytag - tagged correctly. I have debugged the database and Amarok. I have found out, that those songs have been false positive recognized as duplicates. I will add a patch to this bug which solves this problem. In addition I will add a patch which simplifies some SQL queries.
Created attachment 37625 [details] handling for default tag "[mb track uuid]" This patch prevents false positive duplicate detection for MP3 files with the default tag "[mb track uuid]".
Created attachment 37626 [details] simplify some SQL queries during collection scanning This patch simplifies some SQL queries during collection scanning. Concrete it avoids selecting ID's into QStringList to convert them to (potentially very) long WHERE clauses. It uses instead SQL subselects directly in the WHERE clause. So a lot off data need not to be transferred between the SQL server and the application. Also there might be limitations to the length of SQL statements or WHERE clauses which could cause failures during collection scanning.
Re: comment #11, thanks! I didn't know that MusicBrainz had default values -- I'd only seen tracks that simply had empty values. Patch applied in 15a3865.
Egbert, Regarding comment #12 -- most of the patch is great, thanks -- I'd seen those queries in passing (didn't write them) and figured they could be sped up, but I'm not really a SQL guy and although I've used subqueries in the past I rather forgot about them. I can see this really being a nice speedup. However -- again, not being a SQL guy -- can you explain to me exactly what is being done in the last two parts of the diff? I trust that it's correct, but I'd like to have an idea what it's doing before I apply it. Thanks!
@Jeff Mitchell Please note the mail I wrote to your mitchell@kde.org address which contains a link to the dump of the contents of my home directory. If you need additional information and/or somebody else needs the dump too, please don't hesitate to drop me an email or comment here. Thanks in advance
Hello Jeff, the last 2 changes were to get debug prints also for the temporary tables that are being used during the collection scan. First off both: the temporary tables are created with the same columns as the global tables. So it is possible to select the columns for these tables using the names of the global tables. This must be done, as the temporary tables are obviously not recorded in INFORMATION_SCHEMA. This was the reason that debug prints did not appear for the contents of the temporary tables. Second off both: the temporary tables are not part of schema "amarok" but they are part of the schema "session". The easiest way to solve this problem was not to qualify the schema while generating debug prints. This should be O.K. as normally there will not be different schemes with the same tables in the amarok database. By the way, I like amarok 2 much, especially the wikipedia and the lyrics plugin and also the speedup in collection scanning. While amarok 1 needed 5-6 hours for my collection, amarok 2 does it in less than 30 minutes. Best regards, Egbert Am Samstag, 17. Oktober 2009 03:33:31 schrieb Jeff Mitchell: > https://bugs.kde.org/show_bug.cgi?id=209822 > > > > > > --- Comment #14 from Jeff Mitchell <mitchell kde org> 2009-10-17 03:33:30 > --- Egbert, > > Regarding comment #12 -- most of the patch is great, thanks -- I'd seen > those queries in passing (didn't write them) and figured they could be > sped up, but I'm not really a SQL guy and although I've used subqueries in > the past I rather forgot about them. I can see this really being a nice > speedup. > > However -- again, not being a SQL guy -- can you explain to me exactly what > is being done in the last two parts of the diff? I trust that it's > correct, but I'd like to have an idea what it's doing before I apply it. > > Thanks! >
Egbert, Thanks for clarifying. Committed in ed7fcde. I'm glad you see such large speedups in scanning with Amarok 2. Hopefully everything is still actually being scanned :-) Most people have seen huge speed increases with Amarok 2.2 compared to 2.1 but for some reason some people have seen large slowdowns. If you're a MySQL guy and are willing to help out, take a look at bug 210201. :-)
Since I believe that this has been fixed, I'm closing it. Please re-open after 2.2.1 comes out if it's still a problem.