Version: unspecified OS: Linux [pid 31852] --- {si_signo=SIGWINCH, si_code=SI_KERNEL} (Window changed) --- [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 12/Absolute dance 12.m3u", {st_mode=S_IFREG|0644, st_size=380, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 12/Absolute dance 12.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 13/Absolute dance 13.m3u", {st_mode=S_IFREG|0644, st_size=379, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 13/Absolute dance 13.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 14/Absolute dance 14.m3u", {st_mode=S_IFREG|0644, st_size=392, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 14/Absolute dance 14.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 15/Absolute dance 15.m3u", {st_mode=S_IFREG|0644, st_size=433, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 15/Absolute dance 15.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 16/Absolute dance 16.m3u", {st_mode=S_IFREG|0644, st_size=325, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 16/Absolute dance 16.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 17/Absolute dance 17.m3u", {st_mode=S_IFREG|0644, st_size=421, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 17/Absolute dance 17.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 18/Absolute dance 18.m3u", {st_mode=S_IFREG|0644, st_size=416, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 18/Absolute dance 18.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 19/Absolute dance 19.m3u", {st_mode=S_IFREG|0644, st_size=364, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 19/Absolute dance 19.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 20/Absolute dance 20.m3u", {st_mode=S_IFREG|0644, st_size=368, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 20/Absolute dance 20.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 22/Absolute dance 22.m3u", {st_mode=S_IFREG|0644, st_size=436, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance 22/Absolute dance 22.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance classics, disc 1/Absolute dance classics.m3u", {st_mode=S_IFREG|0644, st_size=356, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance classics, disc 1/Absolute dance classics.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance classics, disc 2/Absolute dance classics.m3u", {st_mode=S_IFREG|0644, st_size=167, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance classics, disc 2/Absolute dance classics.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance vol. 6/Absolute dance vol. 6.m3u", {st_mode=S_IFREG|0644, st_size=387, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute dance vol. 6/Absolute dance vol. 6.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 [pid 31852] stat("../../var/shared/Entertainment/Music/Good/Compilations/Absolute kidz 16/Absolute kidz 16.m3u", {st_mode=S_IFREG|0644, st_size=376, ...}) = 0 [pid 31852] open("../../var/shared/Entertainment/Music/Good/Compilations/Absolute kidz 16/Absolute kidz 16.m3u", O_RDONLY|O_CLOEXEC) = 47 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/playlists", {st_mode=S_IFDIR|0700, st_size=2, ...}) = 0 On a slow link, this is SHIT. Reproducible: Always Steps to Reproduce: start amarok if your database points to playlists in your collection, and your collection is in an NFS server, the stats happen serially and if the NFS server is slow, it will take more than five minutes to start up Actual Results: amarok to start immediately Expected Results: amarok starts after I have grown a beard
not just that every time you drag and drop a song from the collection, or even CLICK IT in the playlist: [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/Dance and dense denso/Frijolero.mp3", {st_mode=S_IFREG|0644, st_size=3344544, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/Dance and dense denso/Frijolero.mp3", {st_mode=S_IFREG|0644, st_size=3344544, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/Dance and dense denso/Frijolero.mp3", {st_mode=S_IFREG|0644, st_size=3344544, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/Dance and dense denso/Frijolero.mp3", {st_mode=S_IFREG|0644, st_size=3344544, ...}) = 0 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/albumcovers/cache/", {st_mode=S_IFDIR|0700, st_size=581, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/\302\277D\303\263nde jugar\303\241n las ni\303\261as?/Gimme tha power.mp3", {st_mode=S_IFREG|0664, st_size=4004240, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/\302\277D\303\263nde jugar\303\241n las ni\303\261as?/Gimme tha power.mp3", {st_mode=S_IFREG|0664, st_size=4004240, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/\302\277D\303\263nde jugar\303\241n las ni\303\261as?/Gimme tha power.mp3", {st_mode=S_IFREG|0664, st_size=4004240, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/\302\277D\303\263nde jugar\303\241n las ni\303\261as?/Gimme tha power.mp3", {st_mode=S_IFREG|0664, st_size=4004240, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/\302\277D\303\263nde jugar\303\241n las ni\303\261as?/Gimme tha power.mp3", {st_mode=S_IFREG|0664, st_size=4004240, ...}) = 0 [pid 31852] stat("/home/rudd-o/.kde/share/apps/amarok/albumcovers/cache/", {st_mode=S_IFDIR|0700, st_size=581, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/Dance and dense denso/Frijolero.mp3", {st_mode=S_IFREG|0644, st_size=3344544, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/Dance and dense denso/Frijolero.mp3", {st_mode=S_IFREG|0644, st_size=3344544, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/Dance and dense denso/Frijolero.mp3", {st_mode=S_IFREG|0644, st_size=3344544, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/Dance and dense denso/Frijolero.mp3", {st_mode=S_IFREG|0644, st_size=3344544, ...}) = 0 [pid 31852] stat("/var/shared/Entertainment/Music/Good/Albums and singles/Molotov/Dance and dense denso/Frijolero.mp3", {st_mode=S_IFREG|0644, st_size=3344544, ...}) = 0 [pid 31852] open("/etc/fstab", O_RDONLY) = 12 [pid 31852] open("/etc/mtab", O_RDONLY) = 12 Seriously??? Why do we have a collection if you're going to touch files every time? Might as well just use unix locate or find. It's going to be faster.
And what Amarok version and KDE and Qt versions is this about? Please be precise when you report bugs.
2.4 git, kde 4.7
Thank you for the feedback.
No problem, Myriam. I hope you're having a great day.
Now the bug is even worse. Latest git amarok stats EVERY SINGLE FILE referenced by each m3u playlist on disk that was detected during scan. Amarok now takes more than ten minutes to start. How can this even be happening...
You mean all existing playlists or those in the playlist queue? I can't reproduce this here at all.
All saved playlists. I will attempt to reproduceand capture more information soon.
It's actually even worse than I thought. Upon startup, amarok attempts to open each file three times in order. Between batches of files (each one belonging to the same album, which has an m3u playlist), this happens: amarok: END__: Playlists::PlaylistFilePtr Playlists::loadPlaylistFile(const KUrl&) [Took: 1.3s] amarok: BEGIN: virtual SyncedPlaylistPtr KConfigSyncRelStore::asSyncedPlaylist(Playlists::PlaylistPtr) amarok: "UIDurl: file:../../var/shared/Entertainment/Music/Good/Albums%20and%20singles/Shakira/D%C3%B3nde%20est%C3%A1n%20los%20ladrones/D%C3%B3nde%20est%C3%A1n%20los%20ladrones.m3u" amarok: END__: virtual SyncedPlaylistPtr KConfigSyncRelStore::asSyncedPlaylist(Playlists::PlaylistPtr) [Took: 0s] amarok: BEGIN: Playlists::PlaylistFilePtr Playlists::loadPlaylistFile(const KUrl&) I will upload the log when amarok is finished loading, ok?
~@karen.dragonfear α: grep open../var/shared amaroklog | wc -l 28420 :-\
Created attachment 65831 [details] insane logfile (with --debug and strace -efile)
Tried again with commit ID 0c1641dca5e2d691f624b9e27a4c039e94051e59 (latest of the latest). Same problem. I think it is related to the playlist fixes that have been going in. I also think this is a show stopper and amarok should not be released with a bug that will render startup for medium-sized collections a cache-thrashing affair of epic proportions and many minutes.
Backtrace confirming thel ocation in amarok code: -------------------------------- amarok: END__: Playlists::PlaylistFilePtr Playlists::loadPlaylistFile(const KUrl&) [Took: 1.1s] amarok: BEGIN: virtual SyncedPlaylistPtr KConfigSyncRelStore::asSyncedPlaylist(Playlists::PlaylistPtr) amarok: "UIDurl: file:../../var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATB/Addicted%20to%20music/Addicted%20to%20music.m3u" amarok: END__: virtual SyncedPlaylistPtr KConfigSyncRelStore::asSyncedPlaylist(Playlists::PlaylistPtr) [Took: 0s] amarok: BEGIN: Playlists::PlaylistFilePtr Playlists::loadPlaylistFile(const KUrl&) ^C Program received signal SIGINT, Interrupt. 0x00007ffff6dd0fed in read () at ../sysdeps/unix/syscall-template.S:82 82 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) Missing separate debuginfos, use: debuginfo-install bespin-kwin-theme-0.1-0.3.20100909svn1228.fc15.x86_64 celt-0.11.1-2.fc16.x86_64 dirac-libs-1.0.2-6.fc16.x86_64 ffmpeg-libs-0.8.5-2.fc16.x86_64 fftw-libs-3.3-3.fc16.x86_64 flac-1.2.1-6.fc12.x86_64 gnutls-2.12.7-2.fc16.x86_64 gsm-1.0.13-2.fc12.x86_64 gstreamer-0.10.35-1.fc16.x86_64 gstreamer-plugins-base-0.10.35-3.fc16.x86_64 gstreamer-plugins-good-0.10.30-4.fc16.x86_64 herqq-1.0.0-1.fc16.x86_64 lame-libs-3.98.4-1.fc14.x86_64 lcms-libs-1.19-4.fc16.x86_64 libXdamage-1.1.3-2.fc15.x86_64 libXxf86vm-1.1.1-2.fc15.x86_64 libaio-0.3.109-4.fc16.x86_64 libasyncns-0.8-2.fc15.x86_64 libdrm-2.4.27-2.fc16.x86_64 libffi-3.0.10-1.fc16.x86_64 libgcrypt-1.5.0-2.fc16.x86_64 libgpg-error-1.10-1.fc16.x86_64 libjpeg-turbo-1.1.1-2.fc16.x86_64 liblastfm-0.3.3-3.fc15.x86_64 libmng-1.0.10-5.fc15.x86_64 libofa-0.9.3-17.fc15.x86_64 libogg-1.2.2-3.fc15.x86_64 libqzeitgeist-0.8.0-3.fc16.x86_64 librtmp-2.4-0.1.20110811gitc58cfb3e.fc16.x86_64 libsndfile-1.0.25-1.fc16.x86_64 libtasn1-2.7-2.fc15.x86_64 libtheora-1.1.1-1.fc15.x86_64 libtiff-3.9.5-1.fc16.x86_64 libv4l-0.8.5-1.fc16.x86_64 libva-1.0.14-1.fc16.x86_64 libvorbis-1.3.2-1.fc15.x86_64 libvpx-0.9.7.1-1.fc16.x86_64 mysql-embedded-5.5.17-1.fc16.x86_64 openjpeg-libs-1.4-6.fc16.x86_64 orc-0.4.16-4.fc16.x86_64 phonon-4.5.57-3.20111031.fc16.x86_64 phonon-backend-gstreamer-4.5.90-3.fc16.x86_64 qt-mobility-1.2.0-5.20110922.fc16.x86_64 qt4-theme-bespin-0.1-0.3.20100909svn1228.fc15.x86_64 qtsoap-2.7-2.fc16.x86_64 qtwebkit-2.2.0-1.fc16.x86_64 schroedinger-1.0.10-2.fc15.x86_64 speex-1.2-0.13.rc1.fc15.x86_64 strigi-libs-0.7.6-4.fc16.x86_64 taglib-1.7-1.fc16.x86_64 taglib-extras-1.0.1-2.fc15.x86_64 tcp_wrappers-libs-7.6-64.fc16.x86_64 x264-libs-0.0.0-0.31.20110811.fc16.x86_64 xvidcore-1.3.2-2.fc15.x86_64 (gdb) break open Breakpoint 1 at 0x7ffff7df2f00: file ../sysdeps/unix/syscall-template.S, line 82. (3 locations) (gdb) fg Continuing. Breakpoint 1, open64 () at ../sysdeps/unix/syscall-template.S:82 82 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) (gdb) cont Continuing. Breakpoint 1, open64 () at ../sysdeps/unix/syscall-template.S:82 82 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS) (gdb) info locals No locals. (gdb) bt #0 open64 () at ../sysdeps/unix/syscall-template.S:82 #1 0x0000003559d4d23e in open64 (__oflag=524288, __path=0x15008c8 "/var/shared/Entertainment/Music/Good/Albums and singles/ATB/Movin' melodies/Underwater world.mpc") at /usr/include/bits/fcntl2.h:92 #2 qt_safe_open (mode=438, flags=524288, pathname=0x15008c8 "/var/shared/Entertainment/Music/Good/Albums and singles/ATB/Movin' melodies/Underwater world.mpc") at ../../src/corelib/kernel/qcore_unix_p.h:154 #3 QFSFileEnginePrivate::nativeOpen (this=0x1502860, openMode=...) at io/qfsfileengine_unix.cpp:334 #4 0x0000003559d44df5 in QFSFileEngine::open (this=0x14de2f0, openMode=...) at io/qfsfileengine.cpp:227 #5 0x0000003559d03001 in QFile::open (this=0x7fffffff8970, mode=...) at io/qfile.cpp:1033 #6 0x00007ffff7435750 in Meta::Tag::generateUniqueId (path=<optimized out>) at /home/rudd-o/Projects/Third-party/amarok2/amarok/shared/MetaTagLib.cpp:169 #7 0x00007ffff7437bfe in Meta::Tag::readTags (path=...) at /home/rudd-o/Projects/Third-party/amarok2/amarok/shared/MetaTagLib.cpp:281 #8 0x00007ffff7453d14 in MetaFile::Track::Private::readMetaData (this=0x14fded0) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/core-impl/meta/file/File_p.h:138 #9 0x00007ffff7455668 in MetaFile::Track::Track (this=0x14fd050, url=...) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/core-impl/meta/file/File.cpp:189 #10 0x00007ffff74c4914 in CollectionManager::trackForUrl (this=<optimized out>, url=...) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/core-impl/collections/support/CollectionManager.cpp:466 #11 0x00007ffff7432a1e in Playlists::M3UPlaylist::loadM3u (this=0x14f5670, stream=...) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/core-impl/playlists/types/file/m3u/M3UPlaylist.cpp:186 #12 0x00007ffff7432fb3 in Playlists::M3UPlaylist::triggerTrackLoad (this=0x14f5670) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/core-impl/playlists/types/file/m3u/M3UPlaylist.cpp:111 #13 0x00007ffff74332f9 in Playlists::M3UPlaylist::M3UPlaylist (this=0x14f5670, url=<optimized out>, __in_chrg=<optimized out>, __vtt_parm=<optimized out>) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/core-impl/playlists/types/file/m3u/M3UPlaylist.cpp:59 #14 0x00007ffff743ec92 in Playlists::loadPlaylistFile (url=<optimized out>) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/core-impl/playlists/types/file/PlaylistFileSupport.cpp:142 #15 0x00007ffff75537cc in Playlists::PlaylistFileProvider::loadPlaylists (this=0x1434490) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/playlistmanager/file/PlaylistFileProvider.cpp:364 #16 0x00007ffff755152e in Playlists::PlaylistFileProvider::playlists (this=<optimized out>) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/playlistmanager/file/PlaylistFileProvider.cpp:101 #17 0x00007ffff754ba5b in PlaylistManager::loadPlaylists (this=0x1433c10, provider=<optimized out>, category=1) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/playlistmanager/PlaylistManager.cpp:142 #18 0x00007ffff754bda7 in PlaylistManager::addProvider (this=0x1433c10, provider=0x1434490, category=1) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/playlistmanager/PlaylistManager.cpp:136 #19 0x00007ffff754be89 in PlaylistManager::PlaylistManager (this=0x1433c10) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/playlistmanager/PlaylistManager.cpp:81 #20 0x00007ffff754bfb5 in PlaylistManager::instance () at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/playlistmanager/PlaylistManager.cpp:62 #21 0x00007ffff72bdc04 in Playlist::Actions::restoreDefaultPlaylist (this=0x14383d0) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/playlist/PlaylistActions.cpp:477 #22 0x00007ffff72be670 in Playlist::Actions::instance () at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/playlist/PlaylistActions.cpp:59 #23 0x00007ffff7610769 in MainWindow::createActions (this=0x7d4c80) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/MainWindow.cpp:697 #24 0x00007ffff761b533 in MainWindow::MainWindow (this=0x7d4c80) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/MainWindow.cpp:145 #25 0x00007ffff75f3c3f in App::continueInit (this=0x7fffffffb4e0) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/App.cpp:545 #26 0x00007ffff75f5348 in App::App (this=0x7fffffffb4e0) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/App.cpp:185 #27 0x00000000004095e8 in main (argc=3, argv=0x7fffffffdc68) at /home/rudd-o/Projects/Third-party/amarok2/amarok/src/main.cpp:308 (gdb) ------------------------------------------ Apparently amarok says "well this is a localfile, so why not just read the metadata from disk?" thatsfrom reading the code. extremely bizarre. why have a collection if amarok is gonna do this?
I can confirm that removing playlists from amarok config file makes amarok stop statting song files upon startup. This means that the m3u playlist handling code is not using the database at all upon startup. and, why does amarok store playlists in its CONFIGURATION file? isn't that the reason amarok has a DATABASE that can safely and transactionally save this data?
I guess the playlist list in the configuration code was a left-over from previous version, right? Changing component to Saved Playlists
(In reply to comment #14) > This means that the m3u playlist handling code is not using the database at all > upon startup. Correct, playlist-file saved playlists are completely independent from the SQL database. > > and, why does amarok store playlists in its CONFIGURATION file? isn't that the > reason amarok has a DATABASE that can safely and transactionally save this > data? These playlist files are imported from the Collection and shown in the browser for convenience. Their location, not the playlists themselves, is saved in the configuration file because this is the obvious persistent storage option besides the database.
Manuel Amador (Rudd-O): Why do you not remove those M3U files in your collection folders when you know it slows down startup combined with your NFS mounted collection? There is little more that can be done to improve the performance of M3U playlists since it's such a limited format. XSPF is a lot better since it can contain the complete metadata of the track, hence no need for reading the files.
Bart: There's TONS that can be done to improve the horrible performance associated with this bug (which is NEW, by the way, older versions didn't do this). 1) For starters: DO NOT READ the M3U files upon startup. 2) Further: DO NOT READ the MP3 files listed in those M3U files -- if metadata is needed for those files, use the database. 3) If the playlist itself doesn't have the metainfo required for Amarok to function properly, because it's "limited" or some other excuse, FETCH THE METAINFO FROM THE DATABASE (see #2 above). 3) As a temporary fix: GIVE USERS AN OPTION to skip adding M3U playlists to the Amarok configuration file. 4) To ensure quality in the future: write functional TESTS that ensure Amarok doesn't unnecessarily read files at arbitrary points. The side effects of the code base should be known and tested for, rather than left for the users to discover in frustration. --------------------- So, why don't I remove the M3U files? a) Because using Amarok (or any application, for that matter) should not require me to DELETE MY FILES. b) Because I don't feel like it. c) Because it's perfectly legitimate to have text files in directories. d) Because it's perfectly legitimate to expect a computer program that uses a database as a performance optimization, to parse these files ONCE and to not parse them again until instructed to do so. e) Because, if I wanted to use /usr/bin/find to find the files to play, I would do that, instead of using Amarok. ------------------------------------- The response I've gotten to this bug, makes me feel like you guys are ready to dismiss it as some sort of "tantrum" or "oh, this stupid luser, by having playlist files in his collection, he is being dumb... doesn't he know that?" I will reiterate: Taking ten minutes to start up is a LEGITIMATE bug. One that renders Amarok USELESS with medium-to-large collections. ESPECIALLY when the files in question are in a REMOTE drive (in which case, Amarok takes HOURS to start). Reading metadata off-disk, from files listed on playlists, especially when the metadata is available in the database, is quite possibly the lamest behavior I've seen implemented in software. It frustrates me that, after fixing the earlier bug I reported, where playlists themselves would be read upon startup (which was wrong), someone introduced this bug that is even WORSE (now all MP3 files listed in those playlists are scanned for metadata, DESPITE THE METADATA BEING IN THE DATABASE ALREADY), nobody ever BOTHERED to do basic quality control of the new code base, and now I'm being told "why don't YOU work around the bug by DELETING your files?". No way, no thanks.
Also, again, why are playlists stored in the CONFIGURATION FILE? The playlists on my collection are DATA, NOT configuration. The playlists on my collection should be indexed for search as well. The contents of the playlists should (//preferably//) be indexed as well. Finally, the paths to the playlists on my collection should get the device,relpath treatment as well. It makes *no sense* to store them in the configuration file, for a variety of reasons that *should* be pretty obvious.
(In reply to comment #18) > The response I've gotten to this bug, makes me feel like you guys are ready to > dismiss it as some sort of "tantrum" or "oh, this stupid luser, by having > playlist files in his collection, he is being dumb... doesn't he know that?" > > I will reiterate: Taking ten minutes to start up is a LEGITIMATE bug. One that > renders Amarok USELESS with medium-to-large collections. ESPECIALLY when the > files in question are in a REMOTE drive (in which case, Amarok takes HOURS to > start). Reading metadata off-disk, from files listed on playlists, especially > when the metadata is available in the database, is quite possibly the lamest > behavior I've seen implemented in software. > If anything is going to for me to give up on this bug it's your demeanor, so please stay a little positive. If you spend even 10% of the energy you put in writing these rants in a productive way we'll have this bug fixed ASAP. And yes, that means I do conciser it a valid bug that needs to get fixed. But I do need your help since I can not reproduce the exact circumstances by a lack of data (not a huge collection) and time (volunteer, remember).
(In reply to comment #18) > Bart: > > There's TONS that can be done to improve the horrible performance associated > with this bug (which is NEW, by the way, older versions didn't do this). > > 1) For starters: DO NOT READ the M3U files upon startup. > 2) Further: DO NOT READ the MP3 files listed in those M3U files -- if metadata > is needed for those files, use the database. > 3) If the playlist itself doesn't have the metainfo required for Amarok to > function properly, because it's "limited" or some other excuse, FETCH THE > METAINFO FROM THE DATABASE (see #2 above). This is already done, notice the trackForUrl call in there, this means the track is trying to be resolved by the database. In your case this is failing so it falls back to reading the track directly. This is probably the only regression and cause of this bug. > 3) As a temporary fix: GIVE USERS AN OPTION to skip adding M3U playlists to the > Amarok configuration file. We used to have this option if I can remember correctly and importing playlists was disabled by default. Not sure what happened there. > 4) To ensure quality in the future: write functional TESTS that ensure Amarok > doesn't unnecessarily read files at arbitrary points. The side effects of the > code base should be known and tested for, rather than left for the users to > discover in frustration. >
Git commit 6dbc27f57eef2b700a0933c5e4e9a0a21ffe0612 by Bart Cerneels. Committed on 23/11/2011 at 13:53. Pushed by shanachie into branch 'master'. Use non-blocking load for playlist files. CCBUG:284761 UI is still quite unresponsive with a lot of playlists on slow mounts (like NFS). M +39 -44 src/playlistmanager/file/PlaylistFileProvider.cpp M +2 -2 src/playlistmanager/file/PlaylistFileProvider.h http://commits.kde.org/amarok/6dbc27f57eef2b700a0933c5e4e9a0a21ffe0612
That is correct, Bart. While the app actually starts up much, much faster, the UI is completely unusable while the playlist load happens. Thus, consider a temporary fix of using ThreadWeaver for this job, and then consider putting this data in the database itself (with safeguards to not overwrite playlists that have been modified behind Amarok's back, when the user changes the playlist through amarok). Also, I think it's safe to change the status from UNCONFIRMED to something else.
Setting status correctly.
Git commit 4f44b8c697f53d00b669538c299eee4cdc2a2266 by Bart Cerneels. Committed on 25/11/2011 at 11:53. Pushed by shanachie into branch 'master'. Fix playlist import bug because of one stupid char BUG:284761 M +1 -1 utilities/collectionscanner/Playlist.cpp http://commits.kde.org/amarok/4f44b8c697f53d00b669538c299eee4cdc2a2266
Even though this specific issue is fixed I will change playlist importing in the next version. - There will be an option to import playlists from the collection folders. - Imported playlists will be listed seperate from regular saved playlists (in a different provider). and optionally, though I don't know yet if this will be usable and it's a lot of work: - Save the file paths in the database along with a copy of the playlist contents and sync file and sql copy automatically using PlaylistSync.
Let me test if this fixes the performance problem. BRB...
OK, amarok actually now DOES display its window and starts its main UI loop without statting all the playlists or the files referenced in them. HOWEVER, Amarok STILL tries to access the playlists and the referenced files in them after it's started up, and this happens in the main loop now. This means that, on slow NFS mounts, Amarok perceptibly freezes because the pending UI redraw tasks barely get enough time slices between blocking checks to the files. I would not consider this bug fixed. I would consider this to transform itself into a feature request for actual proper collection support for playlists (noting that playlists stored in NFS drives also have the same DeviceID in common with files, and that other collection types, such as mobile playlists, also probably need the same support). OK. I guess I've given all I can.
Manuel: Please attach your $KDEHOME/shared/config/amarokrc file. I think you might still have incorrect playlist file url's in there that are still causing issues. Bart
[Collection Folders] 9=./Good/Albums and singles,./Good/Compilations,./Good/Mixes and mashups,./Rejected/Damaged,./Rejected/Incomplete,./Rejected/Poor quality,./Unsorted/Uncut albums,./Unsorted/Untagged albums,./Unsorted/Untagged tracks Use MusicLocation=false [Loaded Playlist Files] file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/4%20Strings/Take%20me%20away%20(into%20the%20night)%202006/Take%20me%20away%20(into%20the%20night)%202006.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/4%20Strings/Turn%20it%20around/Turn%20it%20around.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/A*Teens/New%20arrival/New%20arrival.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/A*Teens/Teen%20spirit/Teen%20spirit.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATB/Addicted%20to%20music/Addicted%20to%20music.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATB/Movin'%20melodies/Movin'%20melodies.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATB/No%20silence/No%20silence.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATB/The%20DJ%20(EP)/The%20DJ%20(EP).m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATC/Touch%20the%20sky/Touch%20the%20sky.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/Da%20Capo/Da%20Capo.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/Flowers/Flowers.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/Greatest%20hits/Greatest%20hits.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/The%20Sign/The%20Sign.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/The%20bridge/The%20bridge.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Airwave/(I%20want%20to)%20believe,%20CD%201/(I%20want%20to)%20believe,%20CD%201.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Airwave/(I%20want%20to)%20believe,%20CD%202/(I%20want%20to)%20believe,%20CD%202.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alanis%20Morissette/Alanis%20Morissette%20MTV%20Unplugged/Alanis%20Morisette%20MTV%20Unplugged.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alanis%20Morissette/Supposed%20former%20infatuation%20junkie/Supposed%20former%20infatuation%20junkie.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alcazar/Alcastar/Alcastar.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alcazar/Start%20the%20fire/Start%20the%20fire.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alice%20Deejay/Who%20needs%20guitars%20anyway/Who%20needs%20guitars%20anyway.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Aqua/Aquarium/Aquarium.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/BT/Movement%20in%20still%20life/Movement%20in%20still%20life.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Bacilos/Caraluna/Caraluna.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Backstreet%20Boys/Backstreet's%20back/Backstreet's%20back.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Barry%20White/All-time%20greatest%20hits/All-time%20greatest%20hits.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Basshunter/LOL%20%3C(%5E%5E,)%3E%20(international%20edition)/LOL%20%3C(%5E%5E,)%3E%20(international%20edition).m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Bl%C3%BCmchen/Bl%C3%BCmchen%20live%20in%20Berlin/Bl%C3%BCmchen%20live%20in%20Berlin.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Blank%20&%20Jones/DJ%20Culture/DJ%20Culture.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Blink%20182/Take%20off%20your%20pants%20and%20jacket/Take%20off%20your%20pants%20and%20jacket.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Blue%20Lagoon/Clublagoon/Clublagoon.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Bob%20Sinclar/Western%20dream/Western%20dream.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Bob%20Sinclar/World,%20hold%20on/World,%20hold%20on.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Boney%20M/Remix%202005/Remix%202005.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Captain%20Jack/Greatest%20hits/Greatest%20hits.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Captain%20Jack/Music%20instructor/Music%20instructor.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Captain%20Jack/The%20captain's%20revenge/The%20captain's%20revenge.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Caramell/Supergott/Supergott.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Carlos%20Vives/La%20tierra%20del%20olvido/La%20tierra%20del%20olvido.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Cartoons/Toonage/Toonage.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Cascada/Perfect%20day/Perfect%20day.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Charly%20Lownoise%20&%20Mental%20Theo/Charlottenburg%20(the%20album)/Charlottenburg%20(the%20album).m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Cruks%20en%20karnak/13%20-%20Gracias/13%20-%20Gracias.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Cruks%20en%20karnak/Las%20desventuras%20de%20Cruks%20en% [...180 more...] I will remove those, rescan now, and let you know if it changed. Don't remove the waitingforinfo just yet. I will be back in 20 minutes.
For the record: the code should cope and fix with this circumstance, if it already does not. Otherwise during upgrade this cruft remains there and causes hard-to-debug problems, especially down the road.
OK, after rescanning, the paths of the playlists remained the same: [Loaded Playlist Files] file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/4%20Strings/Take%20me%20away%20(into%20the%20night)%202006/Take%20me%20away%20(into%20the%20night)%202006.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/4%20Strings/Turn%20it%20around/Turn%20it%20around.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/A*Teens/New%20arrival/New%20arrival.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/A*Teens/Teen%20spirit/Teen%20spirit.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATB/Addicted%20to%20music/Addicted%20to%20music.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATB/Movin'%20melodies/Movin'%20melodies.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATB/No%20silence/No%20silence.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATB/The%20DJ%20(EP)/The%20DJ%20(EP).m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/ATC/Touch%20the%20sky/Touch%20the%20sky.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/Da%20Capo/Da%20Capo.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/Flowers/Flowers.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/Greatest%20hits/Greatest%20hits.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/The%20Sign/The%20Sign.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Ace%20of%20Base/The%20bridge/The%20bridge.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Airwave/(I%20want%20to)%20believe,%20CD%201/(I%20want%20to)%20believe,%20CD%201.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Airwave/(I%20want%20to)%20believe,%20CD%202/(I%20want%20to)%20believe,%20CD%202.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alanis%20Morissette/Alanis%20Morissette%20MTV%20Unplugged/Alanis%20Morisette%20MTV%20Unplugged.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alanis%20Morissette/Supposed%20former%20infatuation%20junkie/Supposed%20former%20infatuation%20junkie.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alcazar/Alcastar/Alcastar.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alcazar/Start%20the%20fire/Start%20the%20fire.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Alice%20Deejay/Who%20needs%20guitars%20anyway/Who%20needs%20guitars%20anyway.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Aqua/Aquarium/Aquarium.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/BT/Movement%20in%20still%20life/Movement%20in%20still%20life.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Bacilos/Caraluna/Caraluna.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Backstreet%20Boys/Backstreet's%20back/Backstreet's%20back.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Barry%20White/All-time%20greatest%20hits/All-time%20greatest%20hits.m3u= file:///var/shared/Entertainment/Music/Good/Albums%20and%20singles/Basshunter/LOL%20%3C(%5E%5E,)%3E%20(international%20edition)/LOL%20%3C(%5E%5E,)%3E%20(international%20edition).m3u= [...]
For the record, amarok behavior is now this: start up print a bunch of debugging info stat() all the files in the current playlist show window freeze for a bit, window is grey scan the first saved playlist (m3u), and stat()/lstat() its referenced files paint window scan the second (doing it in the mainloop thread) process gui events scan the third... and so on... and so forth... in that order. Let me verify one more time.
yeap, confirmed. all the playlists are being statted. This does cause amarok to be slow and unresponsive the first 30 seconds or so. HOWEVER. There is a SIGNIFICANT improvement. Now, files that already are in the database are NOT STATTED anymore. Only files that do not exist in the database (because the playlist refers to a path that is not in the database) are statted (with the predictable and perfectly -ENOENT return value). This is still slow, and I would ideally prefer that not to happen. But it's much better.
(In reply to comment #34) > yeap, confirmed. all the playlists are being statted. This does cause amarok > to be slow and unresponsive the first 30 seconds or so. We can probably improve that as well. Playlists should not be read unless they are browsed by the user or when they are being synced with another playlist. But that is not the case in this bug. > > Now, files that already are in the database are NOT STATTED anymore. Only > files that do not exist in the database (because the playlist refers to a path > that is not in the database) are statted (with the predictable and perfectly > -ENOENT return value). > > This is still slow, and I would ideally prefer that not to happen. But it's > much better. Aren't those tracks in the database though? I see there is a mismatch between the [Collection Folders] entries (uses MountpointManager enumeration), but that should not break SqlCollection::trackForUrl(). Those M3U playlists are relative right?
The paths that are stat()ted are referred to by playlists that point to files that have been deleted or moved to another folder. The code essentially first tries to load the data from the databsae, and if it can't find the data from the database (which is expected after a rescan that deleted those absent files from the URLs table), then it proceeds to stat the path itself (which also fails because, well, eh, the file isn't there).
Is this still valid with Amarok 2.6?
Closing for lack of feedback. Please feel free to reopen if you can reproduce this with Amarok 2.6 or later. I can't reproduce this here with Amarok v2.6.0-422-gfc07990