Bug 64647 - configure number of ftp connections opened
Summary: configure number of ftp connections opened
Status: RESOLVED FIXED
Alias: None
Product: kio
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Mandrake RPMs Linux
: NOR wishlist
Target Milestone: ---
Assignee: David Faure
URL:
Keywords:
: 117264 134732 176054 187992 234983 244164 (view as bug list)
Depends on:
Blocks:
 
Reported: 2003-09-21 15:01 UTC by José JORGE
Modified: 2015-11-23 10:42 UTC (History)
17 users (show)

See Also:
Latest Commit:
Version Fixed In: 4.8.1


Attachments
Limit number of slaves per protocol and number of connections per server (21.01 KB, patch)
2009-02-11 23:12 UTC, Andreas Hartmetz
Details
Limit number of slaves per protocol and number of connections per server (21.01 KB, patch)
2009-02-11 23:12 UTC, Andreas Hartmetz
Details
Update the patch to version 4.3.4 (22.00 KB, patch)
2010-02-07 12:23 UTC, Alex Hermann
Details

Note You need to log in before you can comment on or make changes to this bug.
Description José JORGE 2003-09-21 15:01:59 UTC
Version:           3.1.4 (using KDE 3.1.3)
Installed from:    Mandrake Linux Cooker i586 - Cooker
Compiler:          gcc version 3.3.1 (Mandrake Linux 9.2 3.3.1-2mdk)
OS:          Linux (i686) release 2.4.22-4mdk

when I connect to an FTP site with login/password, Konqueror opens more than one connection in order to get thumbnails, etc.
We should be able to change the max number of connections he can open :
on my isp (free.fr), I can't open more than two ftp connections at the same time. This gives me errors when uploading something while konqueror is still querying remote directories!

thanks
Comment 1 Nicolas Laplante 2004-02-11 18:54:41 UTC
I got a similar problem, with KDE 3.2 from Gentoo packages. But this time, when I browse the FTP site of my company, kio_ftp seems to open multiple connections and then BAM, I cannot connect to ANYTHING in my company's domain, including HTTP, POP3, FTP and SMTP. The firewall of our hosting company blocks clients which open more than 3 connections. So I have to contact them and ask them to remove my IP from their 'black' IP list.

This should be resolved IMHO.
Comment 2 José JORGE 2006-12-19 14:34:22 UTC
This is still a problem as of KDE 3.5.5
Comment 3 José JORGE 2007-09-24 14:13:54 UTC
This will be the oldest bug of KDE ;-)

As far as I've understood, it maybe a kio slave problem : kioslave opens several connections, and this is no more configurable (it was before). Or a konqueror problem : konqueror launches several ioslaves.
Comment 4 Tommi Tervo 2007-09-24 14:42:58 UTC
*** Bug 134732 has been marked as a duplicate of this bug. ***
Comment 5 George Goldberg 2008-01-07 04:42:28 UTC
*** Bug 117264 has been marked as a duplicate of this bug. ***
Comment 6 George Goldberg 2008-01-07 04:48:09 UTC
See also bug #27780 for the issue generally to all ioslaves.
Comment 7 Xavier Brochard 2008-03-06 12:58:19 UTC
*** This bug has been confirmed by popular vote. ***
Comment 8 Andreas Hartmetz 2009-02-11 23:12:48 UTC
Created attachment 31233 [details]
Limit number of slaves per protocol and number of connections per server
Comment 9 Andreas Hartmetz 2009-02-11 23:12:50 UTC
Created attachment 31234 [details]
Limit number of slaves per protocol and number of connections per server
Comment 10 Andreas Hartmetz 2009-02-11 23:59:55 UTC
Ahem... my patch actually only works if applications call scheduleJob() voluntarily. It has more impact if Scheduler::doJob() simply calls scheduleJob().
Comment 11 Andreas Hartmetz 2009-02-12 00:26:18 UTC
after the things scheduleJob() usually does, that is. then it *actually* works.
Comment 12 Andreas Hartmetz 2009-02-12 02:46:41 UTC
I mean after the things doJob() usually does, of course. And I've been using it to fix session loading in Kate which failed before when trying to load 20+ files via sftp.
Comment 13 David Faure 2009-02-19 19:35:47 UTC
*** Bug 176054 has been marked as a duplicate of this bug. ***
Comment 14 Thomas Capricelli 2010-01-19 20:25:40 UTC
I have the same problem while connecting to a corporate ftp (with login/passwd). I can't use konqueror because i keep getting errors "530 Sorry, there are too many connections from this host" even though i'm only using konqueror and one tab. I can only go to the first page, and not further.

This is using kde/konqueror from 4.4rc1

Also I think #28361 is a (very old) duplicate.
Comment 15 Thomas Capricelli 2010-01-19 20:26:56 UTC
I think mot of those are duplicate:
https://bugs.kde.org/buglist.cgi?quicksearch=ftp+connections
Comment 16 Alex Hermann 2010-02-07 12:23:15 UTC
Created attachment 40580 [details]
Update the patch to version 4.3.4

Ported the patch from Andreas Hartmetz to version 4.3.4. Seems to work fine.

Is there any possibility something like this will be implemented?

A nice addition would be to provide a config file where the maximum number of connections is configurable by protocol.
Comment 17 Andreas Hartmetz 2010-02-23 02:02:49 UTC
Alex Hermann: The number of connections is still not *configurable* but it's a small fixed number now. This should help.
The fix now in trunk is, at least technically, much better than the patch you've been using.
Comment 18 Andreas Hartmetz 2010-02-23 02:07:07 UTC
*** Bug 187992 has been marked as a duplicate of this bug. ***
Comment 19 José JORGE 2010-02-23 09:11:15 UTC
I don't know how many is the small fixed number, but 1 is the only way to get sure everybody can use kio ftp. This what every FTP client does.
Comment 20 Malte S. Stretz 2010-06-17 11:46:53 UTC
Still got this issue in KDE 4.4.4.  Dolphin displays the error message "421 Too many connections (8) from this IP".  I'm the only person accessing the server from this IP (but both browsed and opened a file with Kate).  netstat confirmed that eight or so connections are open to port 21 on that server:

$ netstat -antp | grep :21
tcp        0      0 10.67.0.30:58608        80.239.150.40:21        ESTABLISHED 2587/kwriteEa2586.s
tcp        0      0 10.67.0.30:48589        80.239.150.40:21        TIME_WAIT   -               
tcp        0      0 10.67.0.30:46211        80.239.150.40:21        ESTABLISHED 2574/dolphinDS2078.
tcp        0      0 10.67.0.30:48578        80.239.150.40:21        ESTABLISHED 2645/dolphinhO2078.
tcp        0      0 10.67.0.30:48585        80.239.150.40:21        TIME_WAIT   -               
tcp        0      0 10.67.0.30:46208        80.239.150.40:21        ESTABLISHED 2572/dolphinNa2078.
tcp        0      0 10.67.0.30:46212        80.239.150.40:21        ESTABLISHED 2575/dolphinBu2078.
tcp        0      0 10.67.0.30:46209        80.239.150.40:21        ESTABLISHED 2573/dolphinnX2078.
tcp        0      0 10.67.0.30:48587        80.239.150.40:21        TIME_WAIT   -               
tcp        0      0 10.67.0.30:46210        80.239.150.40:21        ESTABLISHED 2570/dolphinke2078.
tcp        0      0 10.67.0.30:48588        80.239.150.40:21        ESTABLISHED 2647/dolphinhx2078.
tcp        0      0 10.67.0.30:48584        80.239.150.40:21        TIME_WAIT   -               

Bug 28361 talks about some kind of KIO scheduler, can't that one use a single control connection to the FTP if the tuple (IP, Port, User) are the same?  Might slow down one process if another does a long listing, but would be a lot more social.

Especially in NAT networks where multiple persons might be trying to connect to that server and one single KDE instance is eating up all the available connections.
Comment 21 Søren Holm 2010-07-02 10:50:48 UTC
I'm running KDE 4.4.5 here. And it's not fixed. I agree with #20 that a single connection to any (ip,user,port) should be enough. There is most likely only one network connection anyway.
Comment 22 Conrad Hoffmann 2010-07-22 11:44:57 UTC
*** Bug 234983 has been marked as a duplicate of this bug. ***
Comment 23 Conrad Hoffmann 2010-07-24 14:05:14 UTC
This should be adjustable via the file "ftp.protocol" (mine is in /usr/share/kde4/services). It has a line "maxInstancesPerHost=2". Obviusly, setting this to 1 should fix the problem. I am not at home and cannot confirm that it really works. Can someone test this please?
Comment 24 gambas 2010-07-24 14:13:23 UTC
I copied the ftp.procotol file in ~/.kde4/share/services, and modified maxInstancesPerHost to 1, but it changed nothing. I cannot browse correctly ftp://ftp.free.fr without constantly but randomly having the error message "too many connections from your internet address".
Comment 25 Conrad Hoffmann 2010-07-24 14:17:16 UTC
Although I doubt it makes any difference, can you try and modify the original file (not in your home dir)?
Comment 26 gambas 2010-07-24 14:19:10 UTC
I tried by directly modifying /usr/share/kde4/services/ftp.protocol as root and by killing all kio_ftp processes without browsing ftp://ftp.free.fr, but nothing changed again.
Comment 27 gambas 2010-07-24 14:19:54 UTC
Oops. Read "before" instead of "without".
Comment 28 Conrad Hoffmann 2010-07-24 14:24:47 UTC
I have absolutely no problems browsing ftp.free.fr and downloading files. Can you restart KDE and try again? And maybe tell me exactly how you produce the error message (just browsing, multiple downloads...)?
Comment 29 Xavier Brochard 2010-07-24 23:16:12 UTC
I think that gambas in #26 refer to the same situation as the original submitter: browsing his own personal account on free.fr (with login/password). In this case you cannot upload (see bug description).
As for gambas, setting maxInstancesPerHost=1 doesn't have any effect for me.
Comment 30 Søren Holm 2010-07-24 23:22:10 UTC
KDE 4.5 rc2 at least keeps itself below 4 connection, which is the limit I have been hitting.
Comment 31 Dawit Alemayehu 2011-05-11 18:59:00 UTC
As of KDE 4.5.1 there will be at most 2 connections to ftp servers. Moreover, starting with KDE 4.7 the connection to ftp server will be reused more efficiently due to several long standing bugs being fixed. See

https://git.reviewboard.kde.org/r/101244/
https://git.reviewboard.kde.org/r/101333/

The ability to configure these maximum number of connections to a given site has been around for a long time. Two entries in the ftp.protocol file, 'maxInstancesPerHost=' and 'maxInstances=', can be modified to limit number of ftp ioslaves that can be instantiated per host and the total number of ftp ioslaves that can be running at one given time respectively.
Comment 32 Andreas Hartmetz 2011-05-11 19:15:27 UTC
I am pretty sure that this bug is about a GUI to configure the number of connections. Protocol files are not supposed to be edited by users - on normal systems they are owned by root.
If this wasn't about GUI I'd have closed the bug after committing the new KIO scheduler. I really doubt that slave-on-hold played a major role here one way or another.
Comment 33 Dawit Alemayehu 2011-05-11 20:01:29 UTC
(In reply to comment #32)
> I am pretty sure that this bug is about a GUI to configure the number of
> connections. Protocol files are not supposed to be edited by users - on normal
> systems they are owned by root.

Ahh... I did not see anything related to a GUI configuration, but I now see patches for setting max ioslave instances per host and per session from you. I did not notice that before. My mistake...

> If this wasn't about GUI I'd have closed the bug after committing the new KIO
> scheduler. I really doubt that slave-on-hold played a major role here one way
> or another.

Not entirely correct. The fact that slave-on-hold feature did not even work for kio_ftp plays a role in the fact that too many connections are made to the server resulting in the user hitting the limit set by the server. Anyhow, that is not the point here. If it is a GUI config related issue, then I should not have closed it.
Comment 34 Dawit Alemayehu 2012-03-05 15:46:54 UTC
Git commit 7a5ecbd5316b880f86a2088f420057cdb8d4e2ba by Dawit Alemayehu.
Committed on 16/12/2011 at 20:34.
Pushed by adawit into branch 'KDE/4.8'.

Made the maximum number of connections allowed per host user configurable.
FIXED-IN:4.8.1
REVIEW:104120

M  +22   -10   kio/kio/scheduler.cpp

http://commits.kde.org/kdelibs/7a5ecbd5316b880f86a2088f420057cdb8d4e2ba
Comment 35 Nick Shaforostoff 2012-04-11 19:53:03 UTC
i think an explanation from https://git.reviewboard.kde.org/r/104120/ on how to use your fix should go to official dolphin documentation
Comment 36 Nick Shaforostoff 2012-04-11 19:55:56 UTC
and also dolphin should suggest to add the limit for the specific host when he detects 'too many connections' problem
Comment 37 Marcin Gryszkalis 2013-01-20 00:58:36 UTC
*** Bug 244164 has been marked as a duplicate of this bug. ***
Comment 38 H.H. 2015-02-04 23:00:01 UTC
this should be a gui option. It can't be, that a user has to google, and has to find this bug report and reviewboard entry to limit the connection count.

perhaps in systemsettings under "connections"?
Comment 39 brazzmonkey 2015-04-15 06:43:49 UTC
Indeed, a GUI option would be nice.
Comment 40 brazzmonkey 2015-11-23 10:42:07 UTC
is this really supposed to be fixed?
I've encountered this bug for about 10 years, no matter what.
Of course, Filezilla does the job, but it's really inconvenient when it comes to editing remote files...

I'm currently using KDE 4.14.2 on Linux Mint 17.2. I'd appreciate any guidance to limit kio_ftp slave connection number. Thanks.