Bug 283366 - unacceptably large delay to open file picker when system has numerous potential mount points
Summary: unacceptably large delay to open file picker when system has numerous potenti...
Status: RESOLVED DUPLICATE of bug 272361
Alias: None
Product: kfile
Classification: Applications
Component: general (show other bugs)
Version: 4.8.4
Platform: OpenSUSE Linux
: NOR major (vote)
Target Milestone: ---
Assignee: kdelibs bugs
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2011-10-05 04:45 UTC by Felix Miata
Modified: 2018-04-09 20:43 UTC (History)
9 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
screenshot of first open of gwenview (105.55 KB, image/png)
2011-10-05 21:07 UTC, Felix Miata
Details
120 DPI screenshot: default picker window opens with the left pane so narrow that most entries are indistinguishable from others (48.65 KB, image/png)
2012-03-31 19:33 UTC, Felix Miata
Details
.xsession-errors (84.67 KB, text/plain)
2012-08-18 20:35 UTC, Felix Miata
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Felix Miata 2011-10-05 04:45:45 UTC
https://bugzilla.novell.com/show_bug.cgi?id=675370 is original downstream bug opened in February. My many /etc/fstabs have many noauto entries for cifs & nfs. Typically, mounted entries at any given time represent a small fraction of total entries. My 24/7 server currently has 141 fstab lines, of which 126 contain the string noauto. Minimum time to open file picker dialog is in the neighborhood of 20-25 seconds. Cutting down the number of total entries, and particularly noauto entries, cuts down the file open time proportionally. KDE3 didn't and doesn't do this. IIRC, 4.5 didn't either.

The picker is populating the left pane with virtually everything it finds in fstab. It should only put there what's actually available via current mounting, not what might be mounted if the hosts represented there were actually running.
Comment 1 Felix Miata 2011-10-05 21:07:28 UTC
Created attachment 64258 [details]
screenshot of first open of gwenview

This is nuts. Note how long the scroller is from the 100+ empty mountpoints.
Comment 2 Rajko M. 2011-10-25 04:22:21 UTC
(In reply to comment #0)
> ...
> The picker is populating the left pane with virtually everything it finds in
> fstab. It should only put there what's actually available via current
> mounting, not what might be mounted if the hosts represented there were 
> actually running. 

You computer lab is not the most common use case :)

How to access the rest, the stuff what might be mounted? 

Open /etc/fstab see mount point name and then mount it manually. Well, it is 2011 and that is considered as waste of time and energy, plus it requires knowledge that is not wide spread that there is console application that can be used to mount media, that there are programs like mount and how to use them. 

Have another application that will list all and mount from there? Possible, but still it doesn't tell user that there is some method to add/remove media and where to look for such application.  

List limited number of items, like those mounted or first five, and give user button with hint 'Add/Remove media' or 'View media list' which will open separate window (application) to add/remove media.
Comment 3 Felix Miata 2011-10-25 05:53:58 UTC
(In reply to comment #2)
> How to access the rest, the stuff what might be mounted? 
 
The main thing I care about is that fstab lines containing noauto respect noauto. If I want potential mounts listed that way mounted, I expect to mount them myself if and when I see fit, same as it used to be. Listing in fstab with noauto is good reason for autowhateverfs to assume they do not exist unless they show up in mtab due to a manual mount. How things not listed there might be handled is not what this regression is about.
Comment 4 Stefan Brüns 2012-03-23 12:13:12 UTC
This are actually 2 bugs:

1. unacceptable delay with a large number of mount points:
This may be solved in 4.8.1, previously solid was reparsing mtab for each item in the list.

2. showing "noauto" fstab entries in sidepane
This is IMHO a matter of taste - users might want to show up noauto entries in the sidepane to access these in the first place. In your case, with a huge number this becomes sort of ugly. Proposed solution:

Entries can be hidden - this is done using vi ~/.kde4/share/apps/kfileplaces/bookmarks.xml .
Comment 5 Felix Miata 2012-03-27 15:19:11 UTC
(In reply to comment #4)
> This are actually 2 bugs:
 
> 1. unacceptable delay with a large number of mount points:
> This may be solved in 4.8.1, previously solid was reparsing mtab for each
> item in the list.

I don't see any apparent improvement in openSUSE 12.2M2's 4.8.1-1.1.
 
> 2. showing "noauto" fstab entries in sidepane
> This is IMHO a matter of taste - users might want to show up noauto entries
> in the sidepane to access these in the first place. In your case, with a
> huge number this becomes sort of ugly. Proposed solution:

> Entries can be hidden - this is done using vi
> ~/.kde4/share/apps/kfileplaces/bookmarks.xml .

It's not obvious how simply opening it. :-(
Comment 6 Stefan Brüns 2012-03-31 16:07:06 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > This are actually 2 bugs:
>  
> > 1. unacceptable delay with a large number of mount points:
> > This may be solved in 4.8.1, previously solid was reparsing mtab for each
> > item in the list.
> 
> I don't see any apparent improvement in openSUSE 12.2M2's 4.8.1-1.1.

can you please try the following command:

$> time solid-hardware query 'is NetworkShare'

This should give you a list of all available NFS shares. If this is slow, the bug is in Solid, if is (almost) instant, the slowness is due to the graphical representation in the file dialog.

> > 2. showing "noauto" fstab entries in sidepane
> > This is IMHO a matter of taste - users might want to show up noauto entries
> > in the sidepane to access these in the first place. In your case, with a
> > huge number this becomes sort of ugly. Proposed solution:
> 
> > Entries can be hidden - this is done using vi
> > ~/.kde4/share/apps/kfileplaces/bookmarks.xml .
> 
> It's not obvious how simply opening it. :-(

Clarification: Hiding can be done using Right Click on the entry in the file dialog. This is persistent and saved in bookmarks.xml.
Comment 7 Felix Miata 2012-03-31 19:33:17 UTC
Created attachment 70044 [details]
120 DPI screenshot: default picker window opens with the left pane so narrow that most entries are indistinguishable from others

(In reply to comment #6)
> can you please try the following command:
 
> $> time solid-hardware query 'is NetworkShare'
 
> This should give you a list of all available NFS shares. If this is slow,
> the bug is in Solid, if is (almost) instant, the slowness is due to the
> graphical representation in the file dialog.

I tried on a different host (gx28b; NVidia G84 using Nouveau, probably the fastest video card in the building) that hadn't had 4.8.1 before, with no PIMware running. The delay was down to about 11 seconds. "time solid-hardware query 'is NetworkShare'" printed output to Konsole as fast as Konsole could draw it, virtually instantly.

real 0m0.109s
user 0m0.025s
sys 0m0.019s

I'll have to check on hosts with slower video chips running 4.8.1 as time permits.

> > > Entries can be hidden - this is done using vi
> > > ~/.kde4/share/apps/kfileplaces/bookmarks.xml .
 
> Clarification: Hiding can be done using Right Click on the entry in the file
> dialog. This is persistent and saved in bookmarks.xml.

I still don't know what you're suggesting I try to do. Are you suggesting I right click on all 113 noauto entries each time the dialog opens, since which are actually available varies according to which hosts are online? Unless you can explain better something I can figure out how to do, the insane number of noauto entries populating the picker in random order, obfuscating the actually available items in the list, remains a big usability issue, notwithstanding the persistently narrow pane problem shown in the screenshot.
Comment 8 Felix Miata 2012-08-18 02:24:31 UTC
Even with a typically short fstab and everything in it mounted, just the presence of numerous empty NFS and CIFS mount points creates the very same long and unacceptable delay. For routine use even KDE4.9 remains unacceptable because of this. -> major
Comment 9 David Faure 2012-08-18 19:39:09 UTC
Please help us reproduce the bug. What should we write in our fstab in order to trigger this? Any debug output in ~/.xsession-errors during the delay, especially after enabling debug output in `kdebugdialog`? Any chance you can attach gdb to the running process and get a backtrace?

I'm trying to determine at least if this is a bug in kfile (kfileplacesmodel.cpp) or (more likely) in solid itself?
Comment 10 Felix Miata 2012-08-18 20:35:17 UTC
Created attachment 73283 [details]
.xsession-errors

(In reply to comment #9)
> Any debug output in ~/.xsession-errors during the delay,
> especially after enabling debug output in `kdebugdialog`?

How made:
1-deleted .xsession-errors
2-logged into 4.8.4 on openSUSE 12.2RC2
3-opened Konsole
4-changed current profile from white on black to black on white
5-opened a virgin KSnapshot
6-saved the fresh snapshot to an NFS mount, which took the picker about 13 seconds to initially get open and populated
7-logged out

(In reply to comment #9)
> Please help us reproduce the bug. What should we write in our fstab in order
> to trigger this?

I recently noticed no changes to fstab are required. To reproduce seems to be dependent on doing something like as follows:
1-create /nfs and /smbmnt (I have both on all my Linux hosts)
2-in each of /nfs & /smbmnt, create separate directories for ~30 hosts (I have 39 on my LAN server)
3-in each of the "host" directories, create 4-5 (nfs) or 4-6 (smbmnt) directories for each host's shares
4-(possibly optional) mount just a few shares to one or two hosts

What I have here are two 24/7 systems running KDE3 on one and OS/2 on the other. Also on the LAN are two STBs for HD video reception and playback, both of which support both Samba and NFS and make their shares available both ways. The STBs need not be powered up to duplicate the problem. I have more than 30 mostly multiboot systems running no more than 2 or 3 at a time for testing distros, KDE and Mozillas, and these are the hosts represented in /nfs and /smbmnt.

(In reply to comment #9)
> ...especially after enabling debug output in `kdebugdialog`? Any chance you can
> attach gdb to the running process and get a backtrace?

I need details on how to get you what you're asking for here, if you still need it.
Comment 11 Stefan Brüns 2012-08-19 10:16:35 UTC
I did a Valgrind trace a few weeks ago. 
IIRC, the problem is caused by fileplaces
doing a relayout on addition of every item,
and then querying Solid for e very item in the list.
Therefor, the dialog has O(n^2)  behaviour.
Comment 12 Felix Miata 2012-08-19 10:22:34 UTC
(In reply to comment #9)
> What should we write in our fstab in order to trigger this? 

My 100+ fstab lines for shares take these two forms (far more of the first):

host2:/home /nfs/host2/home nfs noauto,ro,nfsvers=3,nosuid,soft,rsize=8192,wsize=8192 0 0

//host3/X /smbmnt/host3/X cifs ro,credentials=/etc/samba/host3.cred,sec=lanman,servern=HOST3,domain=MYDOMAIN,port=139,dir_mode=0555,file_mode=0664,noauto 0 0
Comment 13 postdoc38 2012-10-26 12:38:17 UTC
I am suffering from this as well. I have only one (!) noauto line in my fstab, and it still takes about 15 seconds to open dolphin, or the "save-as" dialog for that matter.
Comment 14 postdoc38 2012-10-26 14:06:33 UTC
Not sure this also applies to your problem, Felix, but I solved mine: the file picker runs a "net" script at startup (some kind of samba administration thing) and waits for this to complete or timeout before proceeding. I had an own "net" script in my path that was tailing a log, so never completing, hence the long wait.

Hope this helps...
Comment 15 Nate Graham 2018-04-09 20:22:16 UTC

*** This bug has been marked as a duplicate of bug 272361 ***
Comment 16 Nate Graham 2018-04-09 20:43:29 UTC
The lag for unavailable mounts is tracked with Bug 272361.

The issue of excessive numbers of mounts being displayed is tracked with Bug 297219.