Bug 103661 - fish transfer slow (max 1.2 Mb/sec vs scp 10 Mb/sec)
Summary: fish transfer slow (max 1.2 Mb/sec vs scp 10 Mb/sec)
Status: RESOLVED FIXED
Alias: None
Product: kio
Classification: Frameworks and Libraries
Component: fish (show other bugs)
Version: unspecified
Platform: Compiled Sources Linux
: NOR normal with 237 votes (vote)
Target Milestone: ---
Assignee: Jörg Walter
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-04-11 15:26 UTC by Akos Tompos
Modified: 2009-02-14 10:14 UTC (History)
9 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Akos Tompos 2005-04-11 15:26:41 UTC
Version:            (using KDE KDE 3.4.0)
Installed from:    Compiled From Sources
OS:                Linux

Try this:

Choose two machines running linux with ssh server, on the same LAN.
Chosse a big file (100+ Mb)

Then try to copy the file between the machines using Konqeror (fish://)
The speed of copying usally tops somewhere around 1 Mb/s based on machine speed.
Than try the same using scp.
If you have a good machine (SATA HDD etc), you can manage get the speed up to 
11 Mb/s

I think this is a bug
Comment 1 Thiago Macieira 2005-04-12 00:53:57 UTC
Did you compile kio_fish with debugging? If so, it might be spewing debugging information to ~/.xsession-errors and causing a noticeable slow-down.
Comment 2 Akos Tompos 2005-04-12 09:08:53 UTC
I've compiled it like this:
configure --prefix=/opt/kde --disable-debug
make
make install

I don't think this is the problem.
My ~/.xsession-errors file is only 2.8K

xset:  bad font path element (#63), possible causes are:
    Directory does not exist or has wrong permissions
    Directory missing fonts.dir
    Incorrect font server address or syntax
xset:  bad font path element (#63), possible causes are:
    Directory does not exist or has wrong permissions
    Directory missing fonts.dir
    Incorrect font server address or syntax
xset:  bad font path element (#63), possible causes are:
    Directory does not exist or has wrong permissions
    Directory missing fonts.dir
    Incorrect font server address or syntax
xset:  bad font path element (#63), possible causes are:
    Directory does not exist or has wrong permissions
    Directory missing fonts.dir
    Incorrect font server address or syntax
xset:  bad font path element (#63), possible causes are:
    Directory does not exist or has wrong permissions
    Directory missing fonts.dir
    Incorrect font server address or syntax
startkde: Starting up...
kbuildsycoca running...
kdecore (KLibLoader): WARNING: KLibrary: /opt/kde/lib/kde3/kcm_kdnssd.so: undefined symbol: init_kdnssd
ALSA lib pcm_hw.c:549:(snd_pcm_hw_start) SNDRV_PCM_IOCTL_START failed: Broken pipe
ALSA lib pcm_hw.c:549:(snd_pcm_hw_start) SNDRV_PCM_IOCTL_START failed: Broken pipe
ALSA lib pcm_hw.c:549:(snd_pcm_hw_start) SNDRV_PCM_IOCTL_START failed: Broken pipe
konqueror: WARNING: Unknown class  in session saved data!
konqueror: WARNING: Unknown class  in session saved data!
konqueror: WARNING: Unknown class  in session saved data!
konqueror: WARNING: Unknown class  in session saved data!
ALSA lib pcm_hw.c:549:(snd_pcm_hw_start) SNDRV_PCM_IOCTL_START failed: Broken pipe
ALSA lib pcm_hw.c:549:(snd_pcm_hw_start) SNDRV_PCM_IOCTL_START failed: Broken pipe
kio (KIOConnection): ERROR: Header read failed, errno=104
kio (KIOConnection): ERROR: Header has invalid size (-1)
X Error: BadWindow (invalid Window parameter) 3
  Major opcode:  20
  Minor opcode:  0
  Resource id:  0x2600022
X Error: BadWindow (invalid Window parameter) 3
  Major opcode:  20
  Minor opcode:  0
  Resource id:  0x0
ALSA lib pcm_hw.c:549:(snd_pcm_hw_start) SNDRV_PCM_IOCTL_START failed: Broken pipe
X connection to :0.0 broken (explicit kill or server shutdown).
X Error: BadWindow (invalid Window parameter) 3
  Major opcode:  20
  Minor opcode:  0
  Resource id:  0x2c00011
X Error: BadWindow (invalid Window parameter) 3
  Major opcode:  20
  Minor opcode:  0
  Resource id:  0x0
X Error: BadWindow (invalid Window parameter) 3
  Major opcode:  20
  Minor opcode:  0
  Resource id:  0x0
QClipboard: Unknown SelectionClear event received.
X Error: BadWindow (invalid Window parameter) 3
  Major opcode:  20
  Minor opcode:  0
  Resource id:  0x2c003a4
X Error: BadWindow (invalid Window parameter) 3
  Major opcode:  20
  Minor opcode:  0
  Resource id:  0x0
Comment 3 Andre Woebbeking 2005-04-14 12:13:05 UTC
I can confirm this:

fish: -> local is fast (up to 11 MBit/s)
local -> fish: is slow (1.2 MBit/s)

I'm running SuSE 9.1 with KDE 3.4 (from SuSE supplementary).
Comment 4 Joseph Roback 2005-05-14 09:39:23 UTC
I can confirm this also, but with a 10Mbps+ internet connection:

Copying file from my workstation, which is on a 250Mbps internet connection (during not peak hours), to a remote machine on a 10Mbps internet connection

file:
007-TLBS20Apr05.ogg (~17.8 MB)

scp:
007-TLBS20Apr05.ogg                           100%   18MB 505.5KB/s   00:36

fish:
66.6KB/s (screen: http://www.roback.cc/tmp/fish_bug.jpg)

I repeated the tests 3 times with the results within ~5kb of the original tests. I seen this happen since 3.3.x. :(

KDE-3.4
openssh-3.9_p1

Tested on both Gentoo & Kubuntu with same results. My .xsession-errors file is empty also.
Comment 5 Joseph Roback 2005-05-14 09:46:29 UTC
I receive the same slow results with the sftp:// kio slave also. Maybe it is not a fish specific problem...
Comment 6 jeroen 2006-03-17 10:05:30 UTC
Still in kde 3.5 I find a rate of 1 Mbps with fish when scp gets to 10 Mbps.
Comment 7 Gino Peeters 2006-03-17 11:10:53 UTC
I can also confirm that this problem still exists in KDE 3.5.1 (gentoo):

local <- fish://remote : 10 MB/s
local -> fish://remote : 1 MB/s
local -> fish://localhost : very fast  (NO traffic in ethereal monitoring any)
local -> fish://127.0.0.1 : about 500 KB/s (traffic in ethereal monitoring any)

Note that on both machines the CPU is almost idle during the slow transfers.
Comment 8 Matthias Wieser 2006-04-12 12:34:04 UTC
This bug still exists in KDE 3.5.2

sftp://: ~3MB/s
scp: 8-9MB/s
Comment 9 mauceri 2006-04-29 15:56:27 UTC
I can confirm this bug in KDE 3.5.2. Uploads via fish always get stalled. I can't finish an upload any more.
Comment 10 Matthias Wieser 2006-06-17 19:44:43 UTC
kio-Sftp is 2.6 times slower than commandline sftp client:

Commandline sftp:
---
sftp mw@xxxx
cd xyz
get amarok-1.4.0-23.2.i586.rpm
[...]
100%  18MB  2.6MB/s   00:07
---

kio-sftp:
---
~1MByte/second
---

I used IPTraf to count the transfered bytes and packets:

sftp commandline client:
---
Packets incoming:  13312
Bytes incoming: 19795472
(~2000 packets/s)
Packets outgoing:   6814
Bytes outgoing:   499884
(~1000 packets/s)

kio-sftp:
---
Packets incoming:  13978
Bytes incoming: 19821409
(~770 packets/s)
Packets outgoing:   7293
Bytes outgoing:   514770
(~400 packets/s)

Ping is 1.60 ms (== 1/625)

Could it be that kio sftp waits for an ack after each package, while the commandline sftp client has some more intelligent design to achieve those high packet rates (>2000 per second)?
Comment 11 Popsy 2006-06-24 16:51:21 UTC
Same problem with Kubuntu 5.10 and 6.06.
scp ~10MB/s
fish ~1.1MB/s
Comment 12 Fernando Monera Daroqui 2006-07-07 01:12:40 UTC
Both fish and sftp are slow, but fish is much more slower than sftp. At least, sftp is useable.

Gentoo up-to-date. Kde 3.5.3.
Comment 13 Maciej Dems 2006-10-18 18:13:02 UTC
*** This bug has been confirmed by popular vote. ***
Comment 14 copong 2007-07-02 11:31:42 UTC
Same problem on Kubuntu Feisty and Edgy. No way it goes at more than 1.1MB/s. If I open two sessions, each session will still do 1.1MB/s. Idle CPU on both computers and a 100Mb/s ethernet link LAN.
Comment 15 Chris Wadge 2007-10-05 05:48:55 UTC
Confirmed in KDE 3.5.5 (Debian Etch). Standard sftp/fish tools are faster by a multiple of 3-5x.
Comment 16 Thorsten Staerk 2008-01-02 17:10:51 UTC
For me, this has changed with the latest KDE 3.5. Anyone, please re-open if it persists.
Comment 17 copong 2008-01-02 17:30:50 UTC
Still going on on Gutsy for me.
Comment 18 Chris Wadge 2008-01-02 19:21:28 UTC
Works for me, same speeds in Konqueror using fish or sftp as sftp client, KDE 3.5.8 (Debian Lenny/Testing).
Comment 19 Bartemius Crouch 2008-03-18 14:43:08 UTC
Same problem here, gentoo up to date, kde compiled from source v 3.5.8:
scp ~10MB/s
fish ~1.1MB/s
Comment 20 Andre Duffeck 2008-06-11 15:56:11 UTC
Reopening, fish is still slow (~1MB/s) in both KDE 3.5.8 and KDE4 trunk.
Comment 21 Alessandro Guido 2008-09-09 18:56:34 UTC
fish and sftp kioslaves performance are really bad. please fix this!
Comment 22 Ralf Kleineisel 2009-01-13 12:50:49 UTC
This bug exists in Fedora 10 (kde 4.1.3-1). I get ~40 MB/s across a Gigabit-LAN with scp but only ~12 MB/s with fish://.

Sometimes the transfer stalls completely (after a few hundred MB) when transferring a big (1 GB) file. 
Comment 23 Thorsten Staerk 2009-01-13 20:15:22 UTC
kio_fish no longer stalls for a file transfer, see bug 147948.‎
Comment 24 Thorsten Staerk 2009-01-15 16:33:58 UTC
The command iptraf shows us that the average package size is around 75 bytes while it could be 1500. This explains imo why it is so slow.
Comment 25 Pascal d'Hermilly 2009-02-01 02:56:52 UTC
In 4.2 final I get the following transfer rates:
fish: 8 MB/s
sftp: 5.5 MB/s
scp: 9.5 MB/s
Comment 26 Thorsten Staerk 2009-02-02 05:22:21 UTC
Because of some overhead, fish may be slower than scp. But as long as fish is more than 50% slower than scp, I would like to have this bug open.

I see that scp gets a high CPU utilization (per time, not per copied byte) while during a fish transfer, the CPU utilization is very low. iptraf shows us that as well fish as scp almost only send very small packages.
Comment 27 Thorsten Staerk 2009-02-14 10:14:21 UTC
OK, I copied a 2.2 GB file using KDE 4.1 and using scp.

scp:  34 MB/s
fish: 25 MB/s

fish should become better, but it is understandable that it is slow in my environment - it outputs a lot of debugging messages that go into files that make the whole system slow. The time needed by fish is not a (>2) multiple of what scp takes. Closing this bug. Do not re-open for KDE 3.5.
Comment 28 Thorsten Staerk 2009-02-14 10:14:48 UTC
Now really closing this bug :)