Bug 323789 - dolphin slow and resource hungry when batch renaming
Summary: dolphin slow and resource hungry when batch renaming
Alias: None
Product: dolphin
Classification: Applications
Component: general (show other bugs)
Version: 4.11.0
Platform: Ubuntu Linux
: NOR normal (vote)
Target Milestone: ---
Assignee: Dolphin Bug Assignee
Depends on:
Reported: 2013-08-20 17:01 UTC by dmatt
Modified: 2013-08-24 16:14 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In: 4.11.1


Note You need to log in before you can comment on or make changes to this bug.
Description dmatt 2013-08-20 17:01:06 UTC
When batch renaming cca 15000 files (one dovecot maildir with file names like "1376946133.M85112P1260.mail.nubox.net,W=2398,S=2351:2,", all renamed to #.eml ), then
1. dolphin process rose quickly from 50 MB to 220 MB 
2. doplhin started to use 100% of 1 core of my  Core2Duo
3. cca 3 minutes nothing else happened (memory and cpu usage stayed at high level)
4, after this time renaming started ... very slowly... i guess around two renames per second
5, now load is on both cores, around 80% together

what was expected: 10 to 20 high load seconds
what happened: I am finishing this report and only 2000 emails are renamed so this is going to take more than hour

desktop search is disabled
Comment 1 dmatt 2013-08-20 17:32:56 UTC
When I closed dolphin window, dolphin process survived and continued renaming files at higher rate 10 renames/second.
The weirdest thing is it keeps my eth0 interface busy 30KB down/55 up. Netstat show five pairs of these:
unix  3      [ ]         STREAM     CONNECTED     587726   20416/dolphin       /tmp/ksocket-ubik/dolphini20416.slave-socket
unix  3      [ ]         STREAM     CONNECTED     589129   7557/dolphini20416. 
and five of these:
unix  3      [ ]         STREAM     CONNECTED     423003   20416/dolphin 

After renaming finished, load on eth0 continued. It only stopped once I issued  "akonadictl restart". No idea what is going on here.
Comment 2 Frank Reininghaus 2013-08-23 09:17:08 UTC
Thanks for the bug report. I can confirm that renaming a large number of files makes Dolphin very slow.

The main reason is that we resort all items (which is a rather slow operation) after each successful renaming. This is quite straightforward to fix.

However, there might be some slowness remaining even if we fix that because KDirLister repeatedly re-lists the entire folder contents. This thing, and also the high memory usage (and probably the network usage, which I have no idea about) can probably only be fixed with modifications in kdelibs. Also the 3 minutes freeze before the renaming starts (which I cannot reproduce, BTW) has most likely nothing to do with the resorting. It might be related to the setup of the 15000 "move file" jobs. Unfortunately, I don't think kdelibs provides a way to have one job rename multiple files.
Comment 3 Emmanuel Pescosta 2013-08-24 16:14:59 UTC
Git commit 9cbca7241f55da4c77d037dba39c36403b9c762b by Emmanuel Pescosta.
Committed on 24/08/2013 at 16:13.
Pushed by emmanuelp into branch 'KDE/4.11'.

Make use of the "resort all items timer" in KFileItemModel::slotRefreshItems
to avoid too much expensive resorting calls, in case of many refresh items signals.

Followup to patch 111146
Related: bug 303873, bug 299565
FIXED-IN: 4.11.1
REVIEW: 111195

M  +1    -1    dolphin/src/kitemviews/kfileitemmodel.cpp