sftp in Dolphin 1.7 (KDE 4.7.4) is too slow when upload a file on a SSH server on a 100 Mbit LAN: sftp from commandline (upload and download) transfer at 11 MB/sec sftp from dolphin (download) transfer at a speed between 2 MB/sec and 10 MB/sec sftp from dolphin (upload) transfer at a speed between 700 KB/sec and 1 MB/sec
an upgrade: sftp on dolphin (upload) with KDE 4.8.3 the transfers run at a speed between 3 MB/sec and 4 MB/sec better than before, but you can do more
Same here 4.11.1, on 100MBit just reaching 800 - 900kb/s.
I just tried with dolphin 15.08.3, it's definitely an issue. On a gigabit network, dolphin copies up at about 6MB/s, sftp at around 45MB/s. The source of slowness is definitely cpu-bound : in htop, kio-scp runs at 100% of one core during the transfer.
We do not have an async sftp upload API yet. It is on the TODO list ...
Is it still in the TODO list ? ;) I've experienced the same issue with Dolphin 17.12 (KDE Neon user Edition) The upload transfer rate is about 2/3 MB/s with Dolphin and 18/19 MB/s with command line.
*** Bug 320231 has been marked as a duplicate of this bug. ***
Two PCs, both with gigabit LAN cards, both connected to a gigabit hub/switch/router, both with OpenSSH Server installed. Steps to replicate: 1. PC-1 has IP address 192.168.0.101 2. PC-2 has IP address 192.168.0.102 3. On PC-1, open Konsole (or whatever terminal application you use). 4. cd ~ (in my case, my home directory is /home/scott). 5. fallocate -l 1G test.img 6. On PC2: Open Dolphin, press CTRL + L (to make address bar editable). 7. On PC2, enter the following in the address bar: sftp://192.168.0.101/home/scott 8. On PC2, drag the file test.img to the desktop. 9. OBSERVE: it's painfully slow (around 6MB/s). It's pretty much the same result whether the transfer is done using sftp://192.168.0.101 or smb://192.168.0.101 within Dolphin. Now repeat the above, only this time using scp from the terminal: 1. On PC2, open Konsole (or whatever terminal application you use). 2. Enter: scp 192.168.0.1:/home/scott/test.img /tmp 3. OBSERVE: It utilizes the bandwidth of the network, around 110MB/s. It's approximately 22 times faster using scp from the terminal! What needs to be done to confirm this issue?
Confirming doesn't really help; what helps is investigating and fixing it! :)
Thanks Nate. Are there any good resources/references for someone with some C/C++ experience to get involved (perhaps with KIO to start with)? I'd love to get involved, even if it takes a while (this was was reported over 6 years ago after all).
I'm pretty desperate to find someone willing to dig into these kinds of KIO issues and would be happy to personally help you get started! Check out https://community.kde.org/Get_Involved/development Feel free to send me email too whenever you need a hand.
Nice one - and thank you very much! I should warn you that I'm probably quite "rusty" at the moment, but I've been keen to find something I can sink my teeth in to.
I have fixed this sync API problem into Ultracopier, you can look source to see algorithm
Would you be interesting in submitting a patch for KIO? I'd be happy to help guide you through the process!
libssh-0.8.1 has been released and kio_sftp will be updated to gain performance improvements.
Fixed with libssh 0.8 https://www.libssh.org/2018/09/21/libssh-0-8-3/
Reply to comment #14: I am not convinced that this bug should be closed based on "performance improvements" in libssh. If libssh was the cause of this issue, then using SFTP from the terminal would also suffer from the same performance issue (i.e. on a wired gigabit network, we'd be seeing transfer rates of 10MB/s instead of 60+MB/s). We would also see similar performance issues in other file managers (that also use the same version of libssh).
Reply to comment #13: > Would you be interesting in submitting a patch for KIO? I'd be happy to help guide you through the process! If this question was directed at me - yes, I definitely would. First step for me will be setting up a development environment. I'll read through the links you provided.
As others have experienced, dolphin transfers my files roughly 6 times slower than using pure scp or any other modern protocol. This should not be marked as resolved due to some minor upstream protocol changes.
(In reply to Scott Deagan from comment #16) > Reply to comment #14: > > If libssh was the cause of this issue, then using > SFTP from the terminal would also suffer from the same performance issue Alas, it wouldn't. sftp, the terminal application, is based on openssh, not libssh. > We would also see similar performance issues in other > file managers (that also use the same version of libssh). I do not know of any other file managers that use libssh. I'm honestly not sure what you want to happen here. I, personally, haven't seen libssh perform on the same level as openssh, so I figure the biggest fixes need to be made in libssh. Having a bug open here will not make that happen. I'm seeing comparisons being made with scp, which is a completely different protocol to begin with. Apples and oranges. We can't really use openssh because openssh has no library interface to build integrated GUIs on top of. So what's to be done here exactly?
Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please mark the bug as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone!
What info is needed here? I'm interested to know if the problem affect only small file, or big file too. If somebody can give me a simple KIO file transfer based on KUrl I can implement to Ultracopier to see if have same problem (to determine if the problem is KIO layer, Ultracopier have multiple mechanism of performance that's mostly fix the common GUI slow down)
> So what's to be done here exactly?
Hi, Async open/close and of course read/write (but I think you have do this). http://ultracopier-wiki.first-world.info/wiki/General_documentation As you can see here: https://ultracopier.first-world.info/articles/security-speed-ultracopier-supercopier-teracopy-copyhandler.html You can by this way transfert over network. In general the copy software just do lineare transfer without prepare the next copy with previous opening file. And wait the end of close() call before start the next file.
New information was added with comment 23; changing status for inspection.
Can confirm this. I'm uploading zip archive 128MB via SFTP to remote server. The results: 1) sftp console utility upload speed is 3.2 MB/s 2) dolphin upload speed is 350 KB/s
Created attachment 149131 [details] sftp console utility upload speed sftp console utility upload speed
Created attachment 149132 [details] dolphin upload speed dolphin upload speed
Kubuntu 20.04, Plasma 5.18 LTS
Unfortunately Kubuntu 20.04 ships outdated software that is no longer supported. Notably kio-extras and probably also libssh.
I also tested Ubuntu 22.04 with Plasma 5.24. Bug still exists
Created attachment 149137 [details] kubuntu 22.04
Same issue here, always present in Manjaro KDE (3+ years). It seems in my case the upload speed is actually capped at ~274kb/s. SSH directly is 3-4Mb/s.
November 2022, kubuntu 22.04, bug still exists, scp is faster than dolphin approximately 20 times. SCP upload speed 10.7 Mb/s and dolphin speed 500-600 Kb/s
Created attachment 153550 [details] scp console kubuntu 22.04 upload speed
Created attachment 153551 [details] dolphin kubuntu 22.04 upload speed
I have tested it myself using openSUSE TW with Plasma 5.26.2, Frameworks 5.99, where I copied a ~ 300 MB png file via a wired ethernet connection to a local RaspberryPi server: * Dolphin 22.08.3: ~ 10 MB/s * scp: ~ 41 MB/s
Note to the comment above: * Dolphin 22.08.3: ~ 10 MB/s ---> with fish:// * Dolphin 22.08.3: ~ 17 MB/s ---> with sftp://
Same issue over here. Using Dolphin with fish, it uploads files at 100 KB/s. However, using scp over command line, it uploaded files at 8 MB/s. This occurred while accessing the remote server over a WireGuard VNP connection. When I'm on the same network as the remote server and I don't need VPN connection, things improve quite a bit, but it's still significantly slower than scp command. Using Arch Linux and Dolphin 23.04.2. If there is anything else I can do to provide the info needed sort this issue out, please let me know.