Bug 27308 - large directories cause slow konqueror response
Summary: large directories cause slow konqueror response
Status: RESOLVED FIXED
Alias: None
Product: dolphin
Classification: Applications
Component: general (show other bugs)
Version: 16.12.2
Platform: Compiled Sources Solaris
: NOR normal
Target Milestone: ---
Assignee: Peter Penz
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2001-06-16 12:18 UTC by jester
Modified: 2011-12-12 18:16 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In: 4.8.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description jester 2001-06-16 12:16:47 UTC
(*** This bug was imported into bugs.kde.org ***)

Package:           konqueror
Version:           2.1.1 (using KDE 2.1.2 )
Severity:          wishlist
Installed from:    compiled sources
Compiler:          gcc version 2.95.3 20010315 (release)
OS:                SunOS 5.7 sun4u
OS/Compiler notes: 

I have some directories with a few thousand files (8000 or so each of 10-50 kb). When viewing this with konqueror in detailed list mode a "kdeinit +kcminit" process starts up which eats as much CPU time as it can get. The konqueror window itself does not freeze or crash but takes a few minutes to be redrawn. It can be quitted normally with the "close window" button. After that the CPU-eating process is gone as well.

This looks like it may be related to bug 19409 and a few others but I decided to submit another report as 

1) the fix to that bug was unclear to me (was there any change made or did the fix say "just live with it?) and 

2) konqueror does not actually freeze here and can be exited normally. It "just" takes up one CPU to list a long directory and presumably check MIME types of the files in it.

There should be a feature to prevent excessive CPU usage even when viewing long directories.

(Submitted via bugs.kde.org)
(Called from KBugReport dialog)
Comment 1 Carsten Pfeiffer 2001-06-16 23:33:35 UTC
On Samstag 16. Juni 2001 14:16 jester@mpia.de wrote:

> I have some directories with a few thousand files (8000 or so each of
> 10-50 kb). When viewing this with konqueror in detailed list mode a
> "kdeinit +kcminit" process starts up which eats as much CPU time as it can
> get. The konqueror window itself does not freeze or crash but takes a few
> minutes to be redrawn. It can be quitted normally with the "close window"
> button. After that the CPU-eating process is gone as well.

there have been some perfornance improvements in that area. Please test if 
2.2beta (to be released soon) is faster for you. Additional question: are you 
using the Mimetype-filter plugin for Konqueror? That one eats some more CPU 
cycles.

Cheers
Carsten Pfeiffer
Comment 2 Sebastian Jester 2001-06-19 16:17:41 UTC
Unfortunately I can't install 2.2beta myself since I use KDE in a network and
our sysop has other worries than KDE most of the time. I don't have the
mimetype plugin installed.

On Sun 17 Jun 2001 Carsten Pfeiffer wrote:

> On Samstag 16. Juni 2001 14:16 jester@mpia.de wrote:
> 
> > I have some directories with a few thousand files (8000 or so each of
> > 10-50 kb). When viewing this with konqueror in detailed list mode a
> > "kdeinit +kcminit" process starts up which eats as much CPU time as it can
> > get. The konqueror window itself does not freeze or crash but takes a few
> > minutes to be redrawn. It can be quitted normally with the "close window"
> > button. After that the CPU-eating process is gone as well.
> 
> there have been some perfornance improvements in that area. Please test if 
> 2.2beta (to be released soon) is faster for you. Additional question: are you 
> using the Mimetype-filter plugin for Konqueror? That one eats some more CPU 
> cycles.
> 
> Cheers
> Carsten Pfeiffer
> 

Sebastian Jester
Comment 3 Manuel Amador (Rudd-O) 2003-02-11 19:20:03 UTC
This is the problem: Viewing large directories (My music folder, e.g.) makes 
konqueror freeze when entering the directory, and evidently makes me want to 
stop using konqueror to navigate that folder.  While the initial entering the 
directory phase passes, the stop button illuminates, BUT you can't click on 
it.  Then Konqueror starts adding icons to the list (apparently re-sorting them 
with every batch of newcomer icons), (this is slow) and when it finishes, all 
is good.  Entering entering a subfolder, is good also.  But going up or back is 
insane.  Plus, konqueror doesn't remember which folder I selected previously, 
making it a pain to find the former folder I was using.  And to top it all, all 
konqueror windows freeze when one is frozen (and my music folder maks my 
konquerors freeze for tons of time perhaps 10 seconds).
Comment 4 Manuel Amador (Rudd-O) 2003-02-11 19:21:15 UTC
it seems an architecture issue.  It feels as if konqueror's gui was handled by 
a single process/thread, and while another thread is busy fetching data for 
konqueror's gui thread, the gui thread stalls.
Comment 5 Sashmit Bhaduri 2003-11-28 00:25:17 UTC
Can you try with latest head?
Comment 6 Michael Jahn 2004-07-10 13:21:35 UTC
Although this is not bugging me (hey, my music is sorted *g*) I tried to reproduce. This is still valid with 3.3 beta 1. Upon entering a very large directory (13000 files, 196,5 MB) konqueror's gui froze and you were not able to do anything but wait for about 10s.
Comment 7 Jo Sta 2007-12-06 19:02:13 UTC
This is still an issue with 3.5.8. Even on a very fast new dual-core PC, opening a directory with more than 1000 entries, can take over 1 minute. For comparison, opening the same size of directory with IE7 on Windows-XP, takes only 5 seconds for an initial view to appear.  

I see this report has been open since 2001. It would be nice to know is anybody able to confirm that they are working on a fix for KDE 4? If a fix is difficult, what about adding a user option to generate only an extremely basic file list (without file last-modification times, file sizes, etc) similar to the output of "ls" (not like "ls -l") so as to avoid calling detailed file status checking functions, such as fstat(), which tend to be very slow?
Comment 8 Jaime Torres 2008-05-22 12:33:02 UTC
In konqueror 4.0.3 in Linux with 1200 entries, it shows all the files in less than 5 seconds (in a virtualbox f9), and then it takes 30 seconds to assign the icon for each file type using only 50% of cpu (without preview).
We'll have to wait to have a fully working solaris port of kde4 to test this.
Comment 9 groot 2008-07-08 13:04:06 UTC
Created directory with 10000 empty files in it (on local disk). This lists in less than 5 seconds. During those 5 seconds the menus are unresponsive. If i switch on previews, though, that konqueror window becomes slow and unresponsive entirely while it goes and does a preview for all of them.

Updated version from 2.1 to SVN HEAD, because it's still happening.
Comment 10 Jonas Vejlin 2009-06-19 21:24:02 UTC
Is this still valid on kde 4.2 or later?
Comment 11 Jo Sta 2009-06-21 00:19:58 UTC
Much better in 4.2.3 but still intolerable. It takes 10 seconds to open a directory with 1500 entries of many different filetypes. During that time, konqueror is useless - it does not respond to mouse clicks, scrolling requests or anything else. I can open the same directory in other browsers e.g. dillo in about 1 second. 

konqueror has so many serious bugs it is not possible to recommend it for serious use. See e.g. 153575 repeatable out-of-memory crash (unfixed since 2007), 153578 badly incorrect rendering (unfixed since 2007), and there are hundreds more similarly serious bugs. None of the bugs has even been acknowledged, accepted or assigned by anybody claiming to be one of the developers. This gives a bad impression of KDE.
Comment 12 Samuel Brack 2011-01-06 02:39:21 UTC
This is confirmed in 4.5.4 and seems to be more serious than it looks on the first view. Konqueror seems to do all the work in just one thread, which is very inefficient in the modern world of multicore CPUs.
How I reproduced it: 1,5k files seem not to impress Konqueror on a relativley new AMD Athlon II with 3 cores, so I created 100k empty files (with "touch testfile{1..100000}.test" in my shell) and let konqueror open them. Konqueror used one core for several minutes and did not response.
Comment 13 Dawit Alemayehu 2011-05-08 17:45:32 UTC
First of all I can duplicate the problem if I do as mentioned in comment #12. But it is really unrealistic to expect a GUI to be snippy when reading 10K, let alone a 100K , files in any given a directory. I do not even want to know how much longer it would have taken if I had the previewing enabled. However, it is reasonable to expect it not to lock the user interface up when doing so, which to me is the real bug.

The second but most important point is that this is not a Konqueror bug, at least not in KDE 4. That is because file management including directory listing is handled by the dolphin kpart component Konqueror uses to show file listings.
I know it is difficult for most users to wrap their heads around how Konqueror works, but most of the bugs reported against it are in other components it uses despite always getting blamed for it. It is no different than blaming a web browser for all crashes and security issues in Adobe's flash plugin. Anyhow, reassigning this to dolphin/general where it has a better chance of getting resolved...