Bug 471018

Summary:	Terrible browsing performance with mounted remote FS on links with high latency
Product:	[Frameworks and Libraries] frameworks-kio	Reporter:	Andrey <batonxleba>
Component:	general	Assignee:	KIO Bugs <kio-bugs-null>
Status:	CONFIRMED ---
Severity:	normal	CC:	kdelibs-bugs, kfm-devel, mason2371, voidpointertonull+bugskdeorg
Priority:	NOR	Keywords:	efficiency
Version:	5.110.0
Target Milestone:	---
Platform:	Gentoo Packages
OS:	Linux
Latest Commit:		Version Fixed In:
Sentry Crash Report:

Description Andrey 2023-06-14 11:17:22 UTC

SUMMARY
Performance with mounted remote FS like SSHFS or CIFS on links with 200+ ms latency is terrible: whole window freezes for 5-15 sec on opening folder (especially if there is a lot of sub-folders in there) and delays on any action (even on mouse scrolling) still persists during a minute or two after folder's file list was loaded. This also have an impact on desktop or trash bin performance during that lag.

STEPS TO REPRODUCE
1. mount any cifs/sshfs with 200+ ms latency (mount foo@192.168.0.1:/ /mnt/)
2. browse it in dolpin
3. "enjoy" with extremely high delays in file manager's work during content load (up to 1-2 min total)

SOFTWARE/OS VERSIONS
KDE Plasma Version: 5.26.5
KDE Frameworks - Version 5.102.0
Qt - Version 5.15.8 (built against 5.15.8)

ADDITIONAL INFORMATION
I suppose that the main reason for this problem is that the Dolphin tries to count directory size, so I propose you to consider to add an option which disables this feature completely (globally or as a view for a certain folder).

Thanks.

Comment 1 Pedro V 2024-01-02 11:03:35 UTC

I'm not sure if the issue I have is exactly the same, but given that the symptom is familiar, figured I'd do some digging.
I tend to use Krusader with sftp which is a better target to reproduce such troubles anyway given that it doesn't even require mounting. Found `sftp://testhost/usr/share/man/man1/` to be a good test target on an Ubuntu host with high latency.

Running `strace -p $(pidof sftp-server)` on the target host shows the slow iteration with readlink and newfstatat alternating.
What's interesting though is that it starts with a large batch of newfstatat, and apparently the slow iteration is only done over symbolic links. This seems to be confirmed by being able to check out directories consisting of many regular files rather quick.

I think my issue lies in SFTPWorker::listDir -> SFTPWorker::createUDSEntry -> sftp_stat being called for each symlink back to back with synchronous I/O. Likely a more generic solution is desired though than dealing with this specifically, or additional framework features would be needed which would need to be implemented for each KIO slave individually.
These approaches could help:
- Asynchronous I/O support to allow KIO slaves to queue up all I/O work in one pass so there's only one latency hit per operation group
- Progressive directory listing especially as the file list is already available early, and filling in the details later would be more desirable than waiting for everything even though the extra info may not be needed at all

Likely no easy feat due to the need to work on slaves individually, but apparently sftp was getting some related goods already with Harald Sitter introducing some light coroutine usage and mentioning "sftp_aio is currently being worked on upstream" which is definitely an encouraging direction.

Setting CONFIRMED as it's really a reproducible problem, and moving to frameworks-kio as this isn't really Dolphin specific, at least I don't think that's doing anything interesting to make the problem worse than what I can observe with Krusader.
This isn't really a bug though as it's more of a design shortcoming, so recommending dropping importance to wishlist.