Bug 393047 - SFTP transfers handled by kio_sftp are much slower than those handled by scp
Summary: SFTP transfers handled by kio_sftp are much slower than those handled by scp
Status: RESOLVED UPSTREAM
Alias: None
Product: kio
Classification: Frameworks and Libraries
Component: sftp (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Andreas Schneider
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-04-12 16:04 UTC by eduard
Modified: 2018-04-27 09:01 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description eduard 2018-04-12 16:04:15 UTC
Description of problem:
On a 100Mbps network, I can transfer files with an average throughput of 11.1 MB/s with sftp get command in Konsole. Transferring the exact files using Dolphin and kio_sftp gives an average throughput of only 1.2 MB/s.

Version-Release number of selected component (if applicable):
unknown

How reproducible:
Always

Steps to Reproduce:
0.a. Setup a gigabit network with at least two machines and a multiGB file to download on one side.
0.b. Set up ssh access on machine 2
1. On machine1 open Dolphin
2. Click on the "address" bar. The bar becomes editable
3. enter "sftp://<user>@<ip_of_machine2>:/home/<user>"
4. Enter password when prompted for one
5. After directory loads, "copy"(Ctrl-C) a large local, and "paste" (Ctrl-V) it in the Dolphin window. This starts the transfer.
6. Use the "Network Management" widget to observe transfer speed
7. Try to copy the same or a similar file with scp:
 $ scp large.file <user>@<ip_of_machine2>:/home/<user>/
8 Observe the transfer speed either by watching scp output, or using the "Network Management" widget.

Actual results:
kio_sftp: about 1.2 MiB/s average throughput
scp: about 11.1 MiB/s average throughput

Expected results:
Same throughput using both scp and kio_sftp, and same cpu usage.

Additional info:
I transferred one contiguous 65 GiB file. Speeds remain constant in both escenarios. Dolphin scenario almost 2 days downloading some 215GB.

I read someplace in bugs.kde.org a similar problem solved by increassing some buffer size for samba kio slave.

I copied this bug description from https://bugzilla.redhat.com/show_bug.cgi?id=985113 replacing statistics with my own data.
Comment 1 Andreas Schneider 2018-04-16 05:58:12 UTC
Please open a bug at: https://bugs.libssh.org and ask for a complete async sftp API.
Comment 2 eduard 2018-04-16 15:44:59 UTC
I did open the bug. Is there anything that can be done with buffer sizes or is it a totally unrelated problem as I guess?
Comment 3 Andreas Schneider 2018-04-27 09:01:44 UTC
It isn't related to the buffer size. The API is blocking ...