Bug 283022

Summary: Performance with large feed (6000 items) very poor
Product: [Applications] amarok Reporter: mogliii
Component: PodcastAssignee: Amarok Developers <amarok-bugs-dist>
Status: RESOLVED FIXED    
Severity: normal CC: bart.cerneels, codeinkyle, frank, furykerry
Priority: LO    
Version: 2.6-git   
Target Milestone: 2.9   
Platform: Ubuntu   
OS: Linux   
Latest Commit: Version Fixed In: 2.9
Sentry Crash Report:
Attachments: (backtrace) freeze after click the podcast icon to expand the list

Description mogliii 2011-09-29 09:26:35 UTC
Version:           2.4.3 (using KDE 4.7.1) 
OS:                Linux

Amarok has serious performance issues with a 6000 item podcast feed. While feed is refreshed software does not react to mouse clicks for several minutes. 
Selecting a single item takes ~30 seconds. Scrolling of the list with episodes is very unresponsive (actual scrolling after moving the scroll bar takes more than 30s). After selecting all items, clicking right to download episodes takes also very long. When episode listing is open Amarok is basically unusable.

However it does not crash and all mouse clicks are evaluated.

Reproducible: Always

Steps to Reproduce:
Suscribe to a podcast with many items. In this case the basic feed of japanesepod101.com was used (non-free subscription): itpc://www.japanesepod101.com/basic_feed/feed.xml

Actual Results:  
Amarok is not responsive. Software interaction takes between 30-180 seconds per click.

Expected Results:  
Although 6000 items is a lot the list scrolling and other interaction should not be blocked.

- iTunes also gets stuck for a few seconds when first opening the feed, but otherwise scrolling, righ click and other software usage is responsive as expected.
- gPodder only shows latest 200 items.
- Banshee does not accept itpc feeds

For testing purpose I am willing to sponsor a 2 months basic subscription ( for any developer who is willing to look into this. Please contact me by mail.

Hardware used: 2.1 GHz Dual core, 6 GB Ram (Amarok uses less than 80Mb), 3x500GB in hardware Raid5
Comment 1 Bart Cerneels 2011-09-30 06:23:32 UTC
Hi, thanks for the bug report. I've never seen any feed with this much entries. Obviously an uncommon case, as is a subscription-only feed.

I'll first try to see why it's blocking the GUI thread and reproduce using a local test-feed.
One thing though: is the server hosting that feed particularly slow? Perhaps a lot of data in the <description> tags as well?
Comment 2 mogliii 2011-10-01 16:14:12 UTC
I cant say the server is particularly slow. Download speed is quite decent (250 kb/s). Getting the whole feed list in iTunes took less than a minute, if I remember well.

About <description> not sure. There is a (limited) free feed
http://itunes.apple.com/us/podcast/learn-japanese-japanesepod101.com/id109573938
Maybe this can give you some idea about the type of files in the feed.

Or direct xml access
http://www.japanesepod101.com/feed.xml
Comment 3 zhen 2012-08-10 03:23:15 UTC
I have the same problem with Amarok 2.5.0 with KDE 4.9.0. 

I recently subscribed a free podcast with 6000+ old episodes. Once I click the title of the podcast, the whole UI get quite slow, to the extent that almost freeze. 

The  podcast location is 
http://www.justing.com.cn/justpod/justpod_ch.xml
Comment 4 zhen 2012-08-10 10:05:21 UTC
Created attachment 73082 [details]
(backtrace) freeze after click the podcast icon to expand the list

The backtrace is collected after I click the icon of the podcast to expand the list of episodes (6000+ in total). The CPU goes to 50% (2 CPUs, so a CPU is full). 

Tested in :
Ubuntu 12.04 ( with Kunbuntu backport ppa http://ppa.launchpad.net/kubuntu-ppa/backports/ubuntu/) 
KDE 4.9.0
Amarok 2.5.0 

I think that the use case is quite common. A novice podcast user subscribes a popular and content rich podcast, which had accumulates vast episodes over  time. If Amarok freeze in such situation, it basially means Amarok cannot support these large podcast which can be of great value for the end user.
Comment 5 mogliii 2012-08-10 10:51:55 UTC
zhen, could you report if amarok actually crashed? How long did you wait?
When I tried with the large feed (initial report) it was not crashing, only horribly slow. If I gave it ~15 minutes it would eventually calm down.

Also do you have a chance to try the same podcast feed with itunes or banshee, to have a performance comparison?
Comment 6 zhen 2012-08-10 11:15:11 UTC
Amarok does not crash, after I click the podcast icon, it takes around 50 seconds for the list to expand. But after that, the UI is horribly slow, especially for the operations related to the list of episode such as clicking one of the episode or scroll down the list. 

I'm not sure whether Amarok will calm down eventually. Once I had  leave the amarok  open and after quite a while the UI is still sluggish. 

I've tried other players, such as rhythmbox and bansee, they are a bit slow initially after subscribing the podcast, but once they had downloaded the list, the operation such as scrolling down the list and select any episode is quite smooth.
Comment 7 Myriam Schweingruber 2012-08-10 17:47:18 UTC
Could you please test Amarok 2.6 beta or 2.6 RC? We are about to release version 2.6 so it would be useful to test these packages.
At least 2.6 beta is available in the kubuntu beta PPA
Comment 8 zhen 2012-08-11 04:15:42 UTC
I installed the Amarok 2.5.9 in the kunbuntu beta PPA. The UI is still sluggish and almost unusable when the episode list is expanded. 

BTW: I also start to think that using the tree-style list for the podcast and episode is not a good UI. I think bug 265503 is about this. Amarok need more work in the podcast area.
Comment 9 Myriam Schweingruber 2012-08-11 07:47:04 UTC
(In reply to comment #8)
> I installed the Amarok 2.5.9 in the kunbuntu beta PPA. The UI is still
> sluggish and almost unusable when the episode list is expanded. 

Thank you for the feedback.

> BTW: I also start to think that using the tree-style list for the podcast
> and episode is not a good UI. I think bug 265503 is about this. Amarok need
> more work in the podcast area.

You are welcome to give a hand :)
Comment 10 Kyle 2012-12-03 15:17:58 UTC
If you clutter a situation enough, you will eventually notice a slowdown in process time. This doesn't even seem to be a bug but rather just lag? Could be improved with advancements in code efficiency perhaps.
Comment 11 Frank Meerkötter 2013-09-08 19:38:41 UTC
commit a5b0f2afe1b62de636798b7452e4e3abae86db72
Author: Frank Meerkötter <frank@meerkoetter.org>
Date:   Sun Sep 8 19:59:57 2013 +0200

    Fix performance issue with large podcast feeds.
    
    The podcast UI was very slow with channels with a large
    number of episodes (1000+).
    
    The root cause was that SqlPodcastChannel::tracks() was implemented
    in such a way that it was expensive to call it. Unfortunately
    it is extensively called by the PlaylistBrower...
    
    BUG: 283022
    REVIEW: 112417
Comment 12 vedant agarwala 2014-05-21 10:42:28 UTC
Git commit 22486a5d6bd7a6c9f1b88008b81a47bef2f8d6fe by Vedant Agarwala, on behalf of Frank Meerkötter.
Committed on 08/09/2013 at 17:59.
Pushed by vedanta into branch 'tagguessing'.

Fix performance issue with large podcast feeds.

The podcast UI was very slow with channels with a large
number of episodes (1000+).

The root cause was that SqlPodcastChannel::tracks() was implemented
in such a way that it was expensive to call it. Unfortunately
it is extensively called by the PlaylistBrower...
REVIEW: 112417

M  +1    -1    src/browsers/playlistbrowser/PlaylistBrowserModel.cpp
M  +11   -2    src/core-impl/podcasts/sql/SqlPodcastMeta.cpp
M  +2    -0    src/core-impl/podcasts/sql/SqlPodcastMeta.h

http://commits.kde.org/amarok/22486a5d6bd7a6c9f1b88008b81a47bef2f8d6fe