Bug 291934

Summary: saved playlists folder opens very slowly
Product: [Applications] amarok Reporter: Peter Lewis <peterl>
Component: Playlists/Saved PlaylistsAssignee: Amarok Developers <amarok-bugs-dist>
Status: RESOLVED FIXED    
Severity: normal CC: bart.cerneels, maddiemadan, matej, mscho527, t.gornak
Priority: NOR    
Version: 2.6-git   
Target Milestone: ---   
Platform: openSUSE   
OS: Linux   
Latest Commit: http://commits.kde.org/amarok/6f87755c9ab292dc0c590140f133dadead7304f3 Version Fixed In: 2.8
Sentry Crash Report:

Description Peter Lewis 2012-01-19 09:47:41 UTC 
Version:           2.5.0 (using KDE 4.7.2) 
OS:                Linux

When Amarok is first loaded and a saved playlists folder containing 385 playlists is clicked to open it nothing happens for 35 seconds, other than the greying of the leading greater than symbol.
The system is a Dell with an Intel Core2 Duo CPU E7500 @ 2.93GHz processor and 4GByte of main memory.
The test is started when system activity is showing no process having heavy use. Immediately after clicking on the folder amarok uses 11% of CPU activity and tracker-miner-fs is at 1% CPU. When the folder opens amarok usage drops and tracker-store goes up to 50% for an equally long duration with tracker-miner-fs going up to 10-18% and tracker-extract joining in.

The music collection has 5,850 files in 412 disk folders. The tracks in the saved playlists under the folder being opened number 2,014 in 47 disk folders.

Why so many playlists? You can blame Mozart for that!

Reproducible: Always

Steps to Reproduce:
1) Open a System Activity tool.
2) Start Amarok from the Kickoff launcher.
3) Wait until system activity dies down with no process using more than 1% CPU.
4) Click on the folder containing the 385 stored playlists.
5) After 35 seconds the folder opens and the playlists are displayed.
6) During the 35 seconds the amarok cpu usage increases to 11%.
7) After the folder opens tracker-store process cpu usage increases to 50%.

Actual Results:  
Amarok freezes for 35 seconds. Operator wonders if it is still alive.

Expected Results:  
For preference the folder should just open immediately, as it does on subsequent open requests.
If it is impossible to pre-cache the contents of all folders, then a progress bar would be useful.

A folder with 53 playlists takes about 3 seconds.
Comment 1 Myriam Schweingruber 2012-01-19 11:22:08 UTC 
Does the playlist only contain local files or are there also streams and remote files (DAAP, NFS, Samba)?
Comment 2 Peter Lewis 2012-01-19 11:51:17 UTC 
All playlists contain only tracks stored on hard disk on the local host.
Comment 3 Mayank Madan 2012-12-01 04:58:29 UTC 
Can open the folder without any lag on v2.6.0-421-gf66a306
Comment 4 Peter Lewis 2012-12-01 09:24:20 UTC 
Do you have more than 5,850 sound files brought together into about 1200 play lists? I got the impression that it was the size of the database that caused the problem.
I changed to using Clementine shortly after my post and got down to the job of ripping my music collection onto flac files, yes I can hear the difference. My play lists are now all in m3u files.
I would be happy to return to Amarok with this new collection when it is complete. When I do I will let report back.
Comment 5 Myriam Schweingruber 2012-12-01 17:52:03 UTC 
That would only be true if these playlists were saved int he collection instead of being saved on disk. But if you rip tracks the created m3u files are written to disk, not stored in the collection. For that to happen you need to load a playlist and then save it actively in the collection. So the database can definitely be ruled out here.

Mind you, this is a corner case, 1200 playlists is a rather high number and those are scattered all over the disk and need to be gathered together first, which might cause the delay.
FWIW: I have approx. 200 playlists and have no delay on opening the saved playlist section.
Comment 6 MinSik CHO 2012-12-02 12:03:36 UTC 
reproducible with amarok 2.6
Comment 7 Mayank Madan 2012-12-19 16:50:07 UTC 
I can reproduce this with v2.6.90-26-gbcdd84c. It took about 30sec to open the folder containing about 310 playlists
Comment 8 Myriam Schweingruber 2012-12-21 14:17:34 UTC 
Setting status and version correctly.
Comment 9 Matěj Laitl 2013-01-03 12:35:01 UTC 
Will be solved when a form of https://git.reviewboard.kde.org/r/107473/ is merged.
Comment 10 Matěj Laitl 2013-04-02 10:47:13 UTC 
Git commit 6f87755c9ab292dc0c590140f133dadead7304f3 by Matěj Laitl, on behalf of Tatjana Gornak.
Committed on 02/04/2013 at 12:08.
Pushed by laitl into branch 'master'.

Biggie: PlaylistFile refactoring, code deduplication, asynchronous loading

FEATURES:
 * Playlist files are now read asynchronously when possible; patch and great
   deal of refactoring by Tatjana Gornak.

Original description:
I've started my changes with an implementation of a ASXPlaylist class,
due this work I've found that implementation of different playlist file
types has different logic, as result I end up with a rewriting
playlist's implementations.

As example of difference:
 - Constructor M3UPlaylist::M3UPlaylist( const KUrl &url ) sets a url,
   but does not load playlist, therefore loading of playlist are
   postponed till data from playlist are actualy needed (lazy loading)
   On the other hand constructor XSPFPlaylist::XSPFPlaylist( const KUrl
   &url, bool autoAppend )  loads playlist.

The main idea of proposed changes is to create a unify code for
processing playlist files:
 - lazy loading
 - common functionality was moved to PlaylistFile.
FIXED-IN: 2.8
REVIEW: 107473
DIGEST: Optimization: Amarok now reads playlist files asynchronously
        when possible, thanks to Tatjana Gornak.
CCMAIL: Tatjana Gornak <t.gornak@yandex.ru>

M  +2    -0    ChangeLog
M  +3    -1    src/CMakeLists.txt
M  +32   -12   src/DirectoryLoader.cpp
M  +23   -8    src/DirectoryLoader.h
M  +3    -0    src/browsers/playlistbrowser/PlaylistBrowserModel.cpp
M  +2    -1    src/browsers/playlistbrowser/PlaylistBrowserModel.h
M  +2    -2    src/context/applets/info/InfoApplet.cpp
M  +1    -0    src/core-impl/collections/ipodcollection/IpodPlaylistProvider.h
M  +4    -3    src/core-impl/collections/umscollection/podcasts/UmsPodcastProvider.cpp
M  +146  -4    src/core-impl/playlists/types/file/PlaylistFile.cpp
M  +91   -27   src/core-impl/playlists/types/file/PlaylistFile.h
A  +129  -0    src/core-impl/playlists/types/file/PlaylistFileLoaderJob.cpp     [License: GPL (v2+)]
C  +32   -25   src/core-impl/playlists/types/file/PlaylistFileLoaderJob.h [from: src/core-impl/playlists/types/file/PlaylistFileSupport.h - 059% similarity]
M  +20   -93   src/core-impl/playlists/types/file/PlaylistFileSupport.cpp
M  +4    -11   src/core-impl/playlists/types/file/PlaylistFileSupport.h
M  +14   -166  src/core-impl/playlists/types/file/m3u/M3UPlaylist.cpp
M  +10   -44   src/core-impl/playlists/types/file/m3u/M3UPlaylist.h
M  +31   -181  src/core-impl/playlists/types/file/pls/PLSPlaylist.cpp
M  +10   -43   src/core-impl/playlists/types/file/pls/PLSPlaylist.h
M  +72   -254  src/core-impl/playlists/types/file/xspf/XSPFPlaylist.cpp
M  +41   -57   src/core-impl/playlists/types/file/xspf/XSPFPlaylist.h
M  +2    -1    src/core-impl/podcasts/sql/SqlPodcastMeta.cpp
M  +3    -3    src/core-impl/podcasts/sql/SqlPodcastProvider.cpp
M  +13   -0    src/core/playlists/Playlist.cpp
M  +41   -4    src/core/playlists/Playlist.h
M  +0    -2    src/core/playlists/PlaylistProvider.h
M  +4    -52   src/playlist/PlaylistActions.cpp
M  +4    -1    src/playlist/PlaylistController.cpp
A  +119  -0    src/playlist/PlaylistRestorer.cpp     [License: GPL (v2+)]
A  +71   -0    src/playlist/PlaylistRestorer.h     [License: GPL (v2+)]
M  +0    -31   src/playlistmanager/PlaylistManager.cpp
M  +0    -3    src/playlistmanager/PlaylistManager.h
M  +10   -0    src/playlistmanager/SyncedPlaylist.cpp
M  +1    -0    src/playlistmanager/SyncedPlaylist.h
M  +18   -19   src/playlistmanager/file/PlaylistFileProvider.cpp
M  +1    -0    src/playlistmanager/sql/SqlPlaylist.cpp
M  +2    -3    src/playlistmanager/sql/SqlUserPlaylistProvider.cpp
M  +0    -6    src/services/gpodder/GpodderProvider.h
M  +3    -3    tests/TestDirectoryLoader.cpp
M  +1    -1    tests/TestDirectoryLoader.h
M  +1    -0    tests/core-impl/meta/multi/CMakeLists.txt
M  +10   -1    tests/core-impl/meta/multi/TestMetaMultiTrack.cpp
M  +2    -0    tests/core-impl/meta/multi/TestMetaMultiTrack.h
M  +19   -13   tests/core-impl/playlists/types/file/m3u/TestM3UPlaylist.cpp
M  +25   -18   tests/core-impl/playlists/types/file/pls/TestPLSPlaylist.cpp
M  +1    -1    tests/core-impl/playlists/types/file/pls/TestPLSPlaylist.h
M  +12   -6    tests/core-impl/playlists/types/file/xspf/TestXSPFPlaylist.cpp
M  +9    -0    tests/core/playlists/CMakeLists.txt
A  +143  -0    tests/core/playlists/TestPlaylistObserver.cpp     [License: GPL (v2+)]
A  +81   -0    tests/core/playlists/TestPlaylistObserver.h     [License: GPL (v2+)]
M  +1    -1    tests/playlistmanager/file/TestPlaylistFileProvider.cpp

http://commits.kde.org/amarok/6f87755c9ab292dc0c590140f133dadead7304f3