Bug 174144 - Dolphin is too slow deleting files
Summary: Dolphin is too slow deleting files
Status: RESOLVED FIXED
Alias: None
Product: dolphin
Classification: Unclassified
Component: general (show other bugs)
Version: 16.12.2
Platform: Ubuntu Packages Unspecified
: NOR normal with 40 votes (vote)
Target Milestone: ---
Assignee: David Faure
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-11-03 04:10 UTC by Jorge S. de Lis
Modified: 2009-02-18 21:49 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jorge S. de Lis 2008-11-03 04:10:17 UTC
Version:            (using KDE 4.1.2)
Installed from:    Ubuntu Packages

I've made a simple test I want to share with you.

mkdir one ; cd one ; for p in `seq 1 500` ; do mkdir a$p ; cd a$p; for i in `seq 1 10` ; do touch b$i ; mkdir c$i ; done ; cd .. ; done ; cd .. ; cp -a one two

This way I create a directory named "one" containing 500 directories. Each directory has another 10 directories and 10 empty files.

Then I clone this structure to another directory called "two".

Dolphin takes up to 19" to delete the directory "one". Please notice that about 17" of that time is spent just counting files/directories.

Then I try the same with Nautilus with the directory "two", and it takes less than 3". On this case the files/directories counting is almost unnoticeable (it takes less than a second).

Why is Dolphin counting taking so long? The scroll bar seems to be accurate on both programs, but it takes ages to get the neccessary input on Dolphin.
Comment 1 Peter Penz 2008-11-03 08:38:50 UTC
Thanks for the detailed report, the issue has been fixed by David Faure already (see http://www.kdedevelopers.org/node/3683 for details) and is available in KDE 4.1.3 / KDE 4.2.
Comment 2 Jorge S. de Lis 2008-11-03 11:03:44 UTC
Thanks, but http://dot.kde.org/1223050704/ says this is already on 4.1.2. Are they wrong?
Comment 3 Peter Penz 2008-11-03 11:15:41 UTC
Hm, I don't know honestley speaking. Maybe David can comment on this (I've added him as CC to this report).
Comment 4 Jorge S. de Lis 2008-11-05 08:43:49 UTC
I'm reopening this until someone can confirm it's really fixed. This seems to be another different issue.
Comment 5 David Faure 2008-11-07 20:23:50 UTC
Yes, this case is a bit different from the one I fixed previously, because here there are a lot of subdirectories, which means a lot of ListJobs for listing them (about 5000 listjobs). Investigating this made me improve a few things that  were slow with job handling (a QUrl bug, some slow connect statements, and some unnecessary object creations), but the global performance overall doesn't seem to have changed much. QUrl is said to be 3 times faster with Qt-4.5 so I'll re-evaluate this once we use 4.5. Meanwhile, I'd say this is the price to pay for network transparency, the generic way of listing dirs makes the local case slower, and even though I added shortcuts for local things in other areas (stat'ing and deleting), I don't think the code would be nice if the listing of subdirs was duplicated to special case local dirs.
And 5000 subdirs seems like a big enough case for 20 seconds to seem acceptable :)
Comment 6 Grissiom 2009-01-10 15:44:17 UTC
I have the same problem here. I am running with KDE4.1.3. Deleting Qt4.4.3 source code itself takes me minutes to run... Is there any way that deleting files WHILE counting files and folders? It's just my humble opinion.
Comment 7 Rasmus Steinke 2009-02-04 10:54:59 UTC
I can confirm this and honestly - this is bugging me for years.
It does not only affect deleting of dirs. Also copying of several small files is slower..
Comment 8 Rasmus Steinke 2009-02-04 10:58:22 UTC
*** This bug has been confirmed by popular vote. ***
Comment 9 alexc223 2009-02-17 20:06:04 UTC
I can also confirm this, it takes a very long time to delete a directory with sub-directories and files it in. It is also very slow to empty the trash/wastebin.

KDE 4.2.0, Arch Linux w/KDEMod 64 bit
Comment 10 David Faure 2009-02-18 02:02:28 UTC
SVN commit 927652 by dfaure:

Speed up deletion of deep directories by speeding up the slow part of it: the recursive listing
 - no need to stat() before opendir(), we can use the error codes from opendir to handle "it's a file" and "it doesn't exist"
 - no need to stat() every file in this particular case of recursive-listing-for-deletion,
      we just need to know if file/dir/symlink, which opendir tells us.
The testcase from bug 174144 (5000 subdirs) went from 20+ seconds to 13 seconds here.
Better but not perfect, since this testcase should ideally take about 3 seconds.
CCBUG: 174144


 M  +10 -36    kio/kio/deletejob.cpp  
 M  +12 -8     kioslave/file/file.cpp  
 M  +68 -54    kioslave/file/file_unix.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=927652
Comment 11 David Faure 2009-02-18 12:54:06 UTC
SVN commit 927756 by dfaure:

Implement recursive deletion inside kio_file. Brings down the 13-seconds testcase to 2 seconds,
although it kills progress information completely for the case of "deleting one directory with stuff in it".
BUG: 174144


 M  +3 -8      kio/kio/deletejob.cpp  
 M  +9 -1      kio/tests/jobtest.cpp  
 M  +31 -0     kioslave/file/file.cpp  
 M  +3 -2      kioslave/file/file.h  
 M  +1 -0      kioslave/file/file.protocol  
 M  +2 -1      kioslave/file/file_unix.cpp  
 M  +5 -3      kioslave/file/file_win.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=927756
Comment 12 Ivo Anjo 2009-02-18 18:11:52 UTC
@David Faure : Sorry for asking here, but I could not post a comment on your blog entry.

Would it be possible to also add support for this to kio_fish?
Instead of doing all the stat'ing, just sending a "rm -rf" to the remote server?
Should I open a new bug report for this?
Comment 13 David Faure 2009-02-18 18:57:43 UTC
Yes please open a different bug report for kio_fish; the kio_fish author will have to look into how to implement recursive deletion.
Comment 14 alexc223 2009-02-18 19:34:56 UTC
Will the improvements made be present in 4.2.1? Just thinking 4.3 is quite a long way off to have this bug sticking around.
Comment 15 David Faure 2009-02-18 21:49:24 UTC
I haven't backported yet, better wait for possible regressions to pop their head. A bit of slowness is surely much better than bugs/crashes/regressions in a stable release....