Created attachment 48573 [details] my collection settings in Amarok Version: 2.3.1-GIT (using Devel) OS: Linux all media in the parent directory of the directory selected for the collection appears to be scanned and included in the collection ex ~/Home --MyDocuments --MyMedia -----MP3s (set as collection) --OtherDir's For me anything in the parent of the selected collections folder (MyMedia) is scanned and included in the local collection (should only include stuff in dir MP3s). Other directories in the directory structure at the same level as the parent folder are not scanned (MyDocuments, OtherDir's) Reproducible: Always Steps to Reproduce: place audio file if parent der of collections folder Actual Results: see details Expected Results: collection only includes files in the selected collections folder
My experience is similar, verified on 2.3.1 on Fedora 12. I have configured Amarok to find its music in the following folders: /stuff/audio/flac/music /stuff/audio/flac/classical /stuff/audio/mp3/music yet when I have enabled "Watch folders for changes", it picks up files in /stuff/rips/ as they are being ripped, as well as files in /stuff/audio/flac/audiobooks /stuff/audio/mp3/podcasts if I edit their tags or add files, and incorporates them into the collection. Only by forcing a full rescan will these files be excluded from my collection again, until they are touched again.
Did you move these files from the collection folder in one of the newly included folders? Then this is tracked by the Active Files Tracker and not a bug. I can't reproduce this here at all, none of my parent folders have ever been scanned
not in my case - these were files that were newly ripped
Thank you for the feedback.
Are you guys using the internal or external DB?
internal
As hinted by the directory names, the files being erroneously picked up are newly ripped, fresh off the CD, files. Or recently converted (FLAC->MP3), copied from another computer or downloaded. It does not, however, seem to pick up files _completely_ randomly. /stuff is a separate file system, but despite picking up files in subdirectories many levels above the configured directories, it does not appear to pick up files in the root file system, nor from /tmp (which is a separate file system). I use the internal DB.
Oh, and a question: Why should it matter if the files were moved from the collection or not? If I configure Amarok to find its collection in "/Collection", then if I move stuff from "/Collection/Music" to, say, "/Not-In-My-Collection/Music", what possible excuse could there be for Amarok snooping in the "/Not-In-My-Collection" directory, never mind actually showing files residing there in my collection? I had this exact thing just happening as I was experimenting. I moved a directory from "/stuff/audio/mp3/music/" (one of the three directories I've told Amarok are part of my collection) to "/stuff/audio/mp3/redundant" which isn't. Yet, after selecting Tools->Update Collection, not only does Amarok still show the files as being part of my collection, selecting "Edit Track Details" shows the new .../redundant/... path as being the path to the files. (Which is correct, but Amarok should _not_ know about this file, as it is _not_ part of the collection.) It does this even after shutting Amarok down and restarting it after updating the collection. Ok - so I extend the experiment. I tar&compress&encrypt the directory and delete the original completely from any attached storage device. I then do a full rescan of the collection. Now the files are gone from the collection. I then decrypt&decompress&untar the directory into a newly created directory "/stuff/audio/mp3/definitely-not-in-my-collection". Then Tool->Update Collection, and what do you know? The files magically appear in my collection again! I've searched the hierarchies included in my collection for links (hard and soft), to no avail. I've examined the amarokrc, and it says: [Collection Folders] -1=./stuff/audio/flac/classical,./stuff/audio/flac/music,./stuff/audio/mp3/music Use MusicLocation=true ...i.e., nothing that should tell Amarok that it's OK for it to peek in /stuff/audio/mp3/definitely-not-in-my-collection or /stuff/rips Yet, if I do a full rescan of my collection, all the offending files disappear from the collection, so clearly Amarok somewhere, deep inside, knows it should not be looking anywhere but in those listed directories.
(In reply to comment #8) > Oh, and a question: Why should it matter if the files were moved from the > collection or not? It doesn't. > Yet, after selecting Tools->Update Collection, not only does Amarok still show > the files as being part of my collection, selecting "Edit Track Details" shows > the new .../redundant/... path as being the path to the files. (Which is > correct, but Amarok should _not_ know about this file, as it is _not_ part of > the collection.) I haven't said much here so far because I've never seen this behavior in five years of working on Amarok, during which I've ended up being the main scanning guy. So I test this stuff a *lot*. Which means I have no idea how to reproduce this. That said, let's try to figure *something* out. A few things: 1) Please go into Settings and look at the checkboxes for the folders for your collection and make absolutely sure that there aren't folders checked that you don't know about. 2) What's the output of "xdg-user-dir MUSIC" -- if you don't have that, please install it. 3) Please paste or (if it's more than a few lines) attach the output of ~/.kde4/share/apps/amarok/amarokcollectionscanner_batchincrementalinput.data (you may have to adjust for the appropriate folder, i.e. .kde, .kde4, .kde4mod, etc.) 4) Please paste or attach the output of /etc/mtab 5) Have you *ever* had those dirs, or parents of those dirs, as specific parts of your collection? (The full folder list for scanning is stored in the database; it's possible that there are dirs there that weren't properly removed or something).
> I haven't said much here so far because I've never seen this behavior in five > years of working on Amarok, during which I've ended up being the main scanning > guy. So I test this stuff a *lot*. Which means I have no idea how to reproduce > this. > > That said, let's try to figure *something* out. A few things: > > 1) Please go into Settings and look at the checkboxes for the folders for your > collection and make absolutely sure that there aren't folders checked that you > don't know about. At the root-level, all boxes are clear except the one for /stuff which has a dotted checkmark, and then dotted for /stuff/audio and /stuff/audio/{flac,mp3}, finally solid checkmarks for the /stuff/audio/flac/music /stuff/audio/flac/classical /stuff/audio/mp3/music directories, and no other. This matches the amarokrc-file (see previous comment). > 2) What's the output of "xdg-user-dir MUSIC" -- if you don't have that, please > install it. This prints my home directory, /home/bkn > 3) Please paste or (if it's more than a few lines) attach the output of > ~/.kde4/share/apps/amarok/amarokcollectionscanner_batchincrementalinput.data > (you may have to adjust for the appropriate folder, i.e. .kde, .kde4, .kde4mod> ,etc.) This file is >800kb and has dirs all over the place, including my home directory, the /stuff root directory and thus also my entire backup hierarchy (/stuff/backup) and all other directories in my storage hierarchy, which means I'm not comfortable posting this file. I have a soft link from my home directory to /stuff, and I see a fair number of paths starting with /home/bkn/stuff but since /stuff is also there, this is redundant. > 4) Please paste or attach the output of /etc/mtab /dev/sda1 / ext3 rw 0 0 proc /proc proc rw 0 0 sysfs /sys sysfs rw 0 0 devpts /dev/pts devpts rw,gid=5,mode=620 0 0 tmpfs /dev/shm tmpfs rw 0 0 /dev/sda2 /stuff ext3 rw 0 0 tmpfs /tmp tmpfs rw 0 0 /dev/sdb /stuff/backup ext4 rw 0 0 none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0 gvfs-fuse-daemon /home/bkn/.gvfs fuse.gvfs-fuse-daemon rw,nosuid,nodev,user=bkn 0 0 > 5) Have you *ever* had those dirs, or parents of those dirs, as specific parts > of your collection? (The full folder list for scanning is stored in the > database; it's possible that there are dirs there that weren't properly removed > or something). Well, I most likely started out with my collection being my home dir and/or the /stuff root dir, only to narrow it down over time. (I used to have my collection in mp3 format, but I've since re-ripped it all in FLAC, but I keep the mp3s around for my portable players. Naturally, as I've done this, I've sought to exclude the now redundant mp3 versions from my Amarok collection.)
Are you able to build from git?
I've never tried, so probably not without extensive handholding.
@Jeff Mitchell - I've done it before, so I'll give it a go @bk see http://amarok.kde.org/wiki/2.0_Development_HowTo problem is you'll probably need a bunch of dev packages
(In reply to comment #13) > @bk see http://amarok.kde.org/wiki/2.0_Development_HowTo problem is you'll > probably need a bunch of dev packages Since you never did it before, I suggest you try a local installation and follow this guide line by line: http://blogs.fsfe.org/myriam/2009/09/compiling-amarok-from-git-locally-full-summary/ instead of the link quoted above, which assumes you are comfortable with compiling development versions. Also a local installation allows you to remove it very easily once you don't want to use it anymore. Just make sure to remove the distribution packages as running both versions is a bit tricky.
I built it using http://amarok.kde.org/wiki/2.0_Development_HowTo and it compiled and installed apparently fine but doesn't see any of my collection even after I rescanned it running "amarok --debug" I get: amarok: ServiceBrowser starting... amarok: BEGIN: void ServicePluginManager::collect() amarok: [PluginManager] Plugin trader constraint: "[X-KDE-Amarok-framework-version] == 56 and [X-KDE-Amarok-plugintype] == 'service' and [X-KDE-Amarok-rank] > 0" amarok: Received [ "4" ] collection plugin offers amarok: BEGIN: static Plugins::Plugin* Plugins::PluginManager::createFromService(KService::Ptr) amarok: [PluginManager] Trying to load: "amarok_service_ampache" amarok: BEGIN: void CollectionManager::init() amarok: [PluginManager] Plugin trader constraint: "[X-KDE-Amarok-framework-version] == 56 and [X-KDE-Amarok-plugintype] == 'collection' and [X-KDE-Amarok-rank] > 0" amarok: [CollectionManager] Received [ "5" ] collection plugin offers amarok: [CollectionManager] Initialising sqlcollection amarok: [ERROR!] MySQL library initialization failed. amarok: [ERROR!] "GREPME MySQLe query failed! (2000) on init" amarok: BEGIN: void MountPointManager::init() amarok: [PluginManager] Plugin trader constraint: "[X-KDE-Amarok-framework-version] == 56 and [X-KDE-Amarok-plugintype] == 'device' and [X-KDE-Amarok-rank] > 0" amarok: [MountPointManager] Received [ "1" ] device plugin offers amarok: BEGIN: static Plugins::Plugin* Plugins::PluginManager::createFromService(KService::Ptr) amarok: [PluginManager] Trying to load: "amarok_massstorage-device" amarok: [PluginManager] " " PluginManager Service Info: " " --------------------------- " " name : "Mass Storage Device" " " library : "amarok_massstorage-device" " " desktopEntryPath : "amarok_massstorage-device.desktop" " " X-KDE-Amarok-plugintype : "device" " " X-KDE-Amarok-name : "massstorage-device" " " X-KDE-Amarok-authors : ("Maximilian Kossick") " " X-KDE-Amarok-rank : "100" " " X-KDE-Amarok-version : "1" " " X-KDE-Amarok-framework-version: "56" amarok: END__: static Plugins::Plugin* Plugins::PluginManager::createFromService(KService::Ptr) - Took 0.0012s amarok: [MountPointManager] Device added and mounted, checking handlers amarok: [MountPointManager] Device added and mounted, checking handlers amarok: [MountPointManager] Device added and mounted, checking handlers amarok: [MountPointManager] Device added and mounted, checking handlers amarok: [MountPointManager] found handler for "/org/freedesktop/Hal/devices/volume_uuid_ee0e6fcb_b93b_47fe_a12a_f4a70024f286_0" amarok: BEGIN: virtual DeviceHandler* MassStorageDeviceHandlerFactory::createHandler(const Solid::Device&, const QString&, SqlStorage*) const amarok: Initialized thread, count== 1 amarok: [ERROR!] Tried to perform query on uninitialized MySQL amarok: [ERROR!] Tried to perform escape() on uninitialized MySQL amarok: [ERROR!] Tried to perform insert on uninitialized MySQL amarok: [MassStorageDeviceHandler] [WARNING!] Inserting into devices failed for type=uuid, uuid= "ee0e6fcb-b93b-47fe-a12a-f4a70024f286"
bill, it can't find your collection plugins. Myriam has an excellent guide there; make sure you've followed steps 3 and 4, and the bit about MySQL if you have it installed in a nonstandard location.
Jeff - I placed them in the std locations using these instructions http://amarok.kde.org/wiki/2.0_Development_HowTo and I had use them to compile 2.3.0 (and or 2.3.1beta) with no problems. Didn't see any errors when compiling but could have missed them (and appears I did). All "tool" tools (equalizer, script mgr, cover mgr plus the widgets) seem to work fine but none of the collection functionality is there. Looking, I do see some errors in the CMakeError.log http://pastebin.com/isEim3hP though a cursory search didn't give me an obvious solution(s), will try later to resolve (editorial: hate it when it used to work but after an update doesn't - up'ed to kde4.5rc and openSuse 11.3rc arghhhhh) here's my make install, it all looks like std directories http://pastebin.com/YV5ERpXZ
Please follow Myriam's guide. The development howto is meant for developers who already have a sandboxed environment set up.
Jeff: 1) this worked fine for me earlier in the year, why it won't now annoys me 2) I'm working my way through Myriams rq's and currently not making it through the qtbindings install. 3) this is not easy for a nondeveloper and since her rq's are Kubuntu specific I can't be sure of a few of them, refering to her 5.1
commit 664f87cccea6c895fc2abe350779c29fea82d7df Author: Jeff Mitchell <mitchell@kde.org> Date: Sat Jul 10 09:53:17 2010 -0400 dirs that were once in the collection but aren't anymore could end up being scanned. BUG: 243532 diff --git a/ChangeLog b/ChangeLog index 50f55fc..410fb5b 100644 --- a/ChangeLog +++ b/ChangeLog @@ -33,6 +33,9 @@ VERSION 2.3.2-Beta 1 Patch by Richard Longland <rlongland@hotmail.com>. BUGFIXES: + * Directories that were once part of the collection but not anymore may not + have been properly removed, leading to files outside the collection being + scanned. (BR 243532) * Fixed "genre" and other playlist groupings not working. (BR 243344) * Removed track progress effect on TrayIcon as it caused several problems (also caused by a bogus implementation of the KSNI class) diff --git a/src/core-impl/collections/sqlcollection/DatabaseUpdater.cpp b/src/core-impl/collections/sqlcollection/DatabaseUpdater.cpp index 9a00568..46597c1 100644 --- a/src/core-impl/collections/sqlcollection/DatabaseUpdater.cpp +++ b/src/core-impl/collections/sqlcollection/DatabaseUpdater.cpp @@ -743,7 +743,6 @@ DatabaseUpdater::prepareTemporaryTables() void DatabaseUpdater::prepareTemporaryTablesForFullScan() { - m_storage->query( "INSERT INTO directories_temp SELECT * FROM directories;" ); m_storage->query( "INSERT INTO urls_temp SELECT * FROM urls;" ); } diff --git a/src/core-impl/collections/sqlcollection/ScanManager.cpp b/src/core-impl/collections/sqlcollection/ScanManager.cpp index ca6ca75..5f90048 100644 --- a/src/core-impl/collections/sqlcollection/ScanManager.cpp +++ b/src/core-impl/collections/sqlcollection/ScanManager.cpp @@ -436,15 +436,33 @@ ScanManager::getDirsToScan() QList<int> deletedFolderIds; QStringList result; + QStringList collectionFolders = m_collection->mountPointManager()->collectionFolders(); for( QListIterator<QString> iter( values ); iter.hasNext(); ) { int id = iter.next().toInt(); int deviceid = iter.next().toInt(); const QString folder = m_collection->mountPointManager()->getAbsolutePath( deviceid, iter.next() ); - const uint mtime = iter.next().toUInt(); + bool validFolder = false; + bool scanRecursively = AmarokConfig::scanRecursively(); + foreach( QString cFolder, collectionFolders ) + { + cFolder += '/'; //config file paths don't have ending /, sql does + if( folder.startsWith( cFolder ) && scanRecursively ) + { + validFolder = true; + break; + } + if( folder == cFolder && !scanRecursively ) + { + validFolder = true; + break; + } + } + + const uint mtime = iter.next().toUInt(); QFileInfo info( folder ); - if( info.exists() ) + if( info.exists() && validFolder ) { m_incrementalDirs << QString( folder + "_AMAROKMTIME_" + QString::number( mtime ) ); if( info.lastModified().toTime_t() != mtime ) @@ -568,6 +586,7 @@ ScanManager::cleanTables() m_storage->query( "DELETE FROM composers;" ); m_storage->query( "DELETE FROM albums;" ); m_storage->query( "DELETE FROM artists;" ); + m_storage->query( "DELETE FROM directories;" ); //images table is deleted in DatabaseUpdater::copyToPermanentTables }
thanks, since I'm stuck at getting the qtscript things compiled and installed I won't be able to test until I can find a git rpm to install (plus I don't do patches well)
This should be fixed -- but, if you can build from git you can test it out before waiting for 2.3.2 to release. Note that you may need to do a full rescan to actually get those things in your collection that shouldn't be there dropped.
Fixed my compile problem (was mysql server version different than devel rpm's) and dl'ed yesterday's git and it worked as expected without having to rescan collection thanks,
*** Bug 244663 has been marked as a duplicate of this bug. ***
This is not fixed -- I propose re-opening this bug. I installed amarok 2.4.0 in kde 4.4.5. I deleted all amarok config files from ~/.kde (every thing found by 'find -iname \*amar\*') then started amarok anew. Without prompting, Amarok scanned my home directory for music. I went into settings and set the collection dir to /media/Music and did a full re-scan. Amarok built the collection with everything in /media not just /media/Music.
The find command in my earlier comment should have been: 'find ~/.kde -iname \*amar\*' Some more info on my problem -- I don't know why the fix isn't working for me, and I'm not sure any of the following would be a source of the problem. - I'm running kubuntu 10.04 lucid - My collection is set to the directory /media/Music - amarokcollectionscanner_batchscan.xml contains all directories in /media, not just /media/Music - /media/Music is on a different partition than / and /home - another subdir in /media is /media/Windows, a read-only ntfs partition on which I dual boot windows; amarok is finding all the sound effects from games installed on my windows partition - in my home dir there are no links to any directories I'm going to try a fresh install on a virtual machine and do some experiments.
In addition to trying a fresh install on a virtual machine, I also tried creating a new user account on my real machine. Neither experienced the behavior I encounter in my main user account on my real machine. It seems that on my main user account, all sub-dirs of /media are being scanned no matter what directories I set in Configure Collection. I tried mounting my music partition at different points outside /media as well, but /media always gets scanned. With the new user account, /media was not scanned. So it looks like I am encountering a different bug.
Here are my latest experiments to see what's going on. I have some directories set up: Test 1. /media/Music # mount point for a separate hard drive /media/Windows # mount point for windows OS, also on a separate hard drive /media/TestDir # a plain sub directory in /media I created a new user account and in amarok set the collection to /media/Music. Amarok added everything it found in /media/Windows and /media/TestDir to the collection. The behavior seems to be related to this bug. Test 2. Directories setup: $HOME/Stuff/Collection $HOME/Stuff/NotInCollection changed amarok collection config to use $HOME/Stuff/Collection, and did a full rescan. It found everything in $HOME/Stuff/Collection, and correctly did not add anything from $HOME/Stuff/NotInCollection. Is something special about having your collection on a separate hard drive or partition that's mounted on a subdir at the same directory level as another mount point? I think I found a new bug.