Version: unspecified (using KDE 4.7.2) OS: Linux (Tested against latest 4.7 branch) I'm behind a squid proxy that needs authentication to be used. It offers NTLM and Basic as authentication schemes, but kio_http is unable to authenticate correctly with NTLM and falls back to Basic. This is the relevant messages exchanged between client and proxy server: GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org Proxy-Connection: keep-alive User-Agent: Mozilla/5.0 (X11; Linux i686) KHTML/4.7.1 (like Gecko) Konqueror/4.7 Accept: text/html, text/*;q=0.9, image/jpeg;q=0.9, image/png;q=0.9, image/*;q=0.9, */*;q=0.8 Accept-Encoding: gzip, deflate, x-gzip, x-deflate Accept-Charset: utf-8,*;q=0.5 Accept-Language: en-US,en;q=0.9 HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE3 Date: Mon, 24 Oct 2011 13:35:59 GMT Content-Type: text/html Content-Length: 1311 Expires: Mon, 24 Oct 2011 13:35:59 GMT X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="XXXXXXX - Inserire credenziali per la navigazione" X-Cache: MISS from proxy.name X-Cache-Lookup: NONE from proxy.name:8080 Via: 1.0 proxy.name:8080 (squid/2.7.STABLE3) Connection: close GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org Proxy-Connection: close User-Agent: Mozilla/5.0 (X11; Linux i686) KHTML/4.7.1 (like Gecko) Konqueror/4.7 Accept: text/html, text/*;q=0.9, image/jpeg;q=0.9, image/png;q=0.9, image/*;q=0.9, */*;q=0.8 Accept-Encoding: gzip, deflate, x-gzip, x-deflate Accept-Charset: utf-8,*;q=0.5 Accept-Language: en-US,en;q=0.9 Proxy-Authorization: NTLM TlRMTVNTUAABAAAABQIAAAAAAAAAAAAAAAAAAAAAAAA= HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE3 Date: Mon, 24 Oct 2011 13:35:59 GMT Content-Type: text/html Content-Length: 1311 Expires: Mon, 24 Oct 2011 13:35:59 GMT X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: Basic realm="XXXXXXX - Inserire credenziali per la navigazione" X-Cache: MISS from proxy.name X-Cache-Lookup: NONE from proxy.name:8080 Via: 1.0 proxy.name:8080 (squid/2.7.STABLE3) Connection: close GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org Proxy-Connection: close User-Agent: Mozilla/5.0 (X11; Linux i686) KHTML/4.7.1 (like Gecko) Konqueror/4.7 Accept: text/html, text/*;q=0.9, image/jpeg;q=0.9, image/png;q=0.9, image/*;q=0.9, */*;q=0.8 Accept-Encoding: gzip, deflate, x-gzip, x-deflate Accept-Charset: utf-8,*;q=0.5 Accept-Language: en-US,en;q=0.9 Proxy-Authorization: Basic Y29ib1xhaWFjb3Z... cutted HTTP/1.0 200 OK Date: Mon, 24 Oct 2011 13:36:13 GMT Server: Apache Content-Location: index.en.html Vary: negotiate,accept-language,Accept-Encoding TCN: choice Last-Modified: Mon, 24 Oct 2011 11:38:20 GMT ETag: "37f1-4b009dc0ee300" Accept-Ranges: bytes Cache-Control: max-age=86400 Expires: Tue, 25 Oct 2011 13:36:13 GMT Content-Encoding: gzip Content-Length: 4617 Content-Type: text/html Content-Language: en X-Cache: MISS from proxy.name X-Cache-Lookup: MISS from proxy.name:8080 Via: 1.1 proxy.name:8080 (squid/2.7.STABLE3) Connection: close Reproducible: Always Steps to Reproduce: .... Actual Results: Basic authentication scheme is used Expected Results: NTLM authentication scheme have to be used This happens because after the first 407 response from the proxy, the subsequent client request is sent with "Proxy-Connection: close" header. Forcing kio to send always "Proxy-Connection: keep-alive" header solves the problem. For instance this is what happens if i use firefox against the same proxy: GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110928 Iceweasel/3.6.23 (like Firefox/3.6.23) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: it,en-us;q=0.7,en;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Proxy-Connection: keep-alive HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE3 Date: Mon, 24 Oct 2011 14:09:30 GMT Content-Type: text/html Content-Length: 1311 Expires: Mon, 24 Oct 2011 14:09:30 GMT X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="XXXXXXXXX - Inserire credenziali per la navigazione" X-Cache: MISS from proxy.name X-Cache-Lookup: NONE from proxy.name:8080 Via: 1.0 proxy.name:8080 (squid/2.7.STABLE3) Connection: close GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110928 Iceweasel/3.6.23 (like Firefox/3.6.23) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: it,en-us;q=0.7,en;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Proxy-Connection: keep-alive Proxy-Authorization: NTLM TlRMTVNTUAABAAAAB4IIAAAAAAAAAAAAAAAAAAAAAAA= HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE3 Date: Mon, 24 Oct 2011 14:09:47 GMT Content-Type: text/html Content-Length: 1311 Expires: Mon, 24 Oct 2011 14:09:47 GMT X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: NTLM TlRMTVNTUAACAAAADgAOADAAA.... cutted X-Cache: MISS from proxy.name X-Cache-Lookup: NONE from proxy.name:8080 Via: 1.0 proxy.name:8080 (squid/2.7.STABLE3) Connection: keep-alive Proxy-Connection: keep-alive GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110928 Iceweasel/3.6.23 (like Firefox/3.6.23) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: it,en-us;q=0.7,en;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Proxy-Connection: keep-alive Proxy-Authorization: NTLM TlRMTVNTUAADAAAAGAAYAG4AAAAYABg.... cutted HTTP/1.0 200 OK Date: Mon, 24 Oct 2011 14:09:47 GMT Server: Apache Content-Location: index.it.html Vary: negotiate,accept-language,Accept-Encoding TCN: choice Last-Modified: Mon, 24 Oct 2011 12:04:29 GMT ETag: "3abb-4b00a3993ed40" Accept-Ranges: bytes Cache-Control: max-age=86400 Expires: Tue, 25 Oct 2011 14:09:47 GMT Content-Encoding: gzip Content-Length: 4977 Content-Type: text/html Content-Language: it X-Cache: MISS from proxy.name X-Cache-Lookup: MISS from proxy.name:8080 Via: 1.1 proxy.name:8080 (squid/2.7.STABLE3) Connection: keep-alive Proxy-Connection: keep-alive
Setting Priority to high. I think it would be good to specify in the popup window that ask for username/password the authentication scheme we are going to use. I was convinced i was using NTLM before sniffing the network traffic...
Follow the following steps to enable persistent proxy connection: 1.) In Konqueror, select Settings->Configure Konqueror 2.) In the configuration dialog, select Proxy. 3.) Under options, make sure "Use persistent connections to proxy" is checked. That should solve your problem. For KDE 4.7.3, that option will no longer have any effect on kio_http since it will completely ignore it and always use persistent connection unless the server tells it otherwise. For KDE 4.8 the GUI has been updated and the option mentioned above is no longer there. That should solve your problem.
> Follow the following steps to enable persistent proxy connection: > > 1.) In Konqueror, select Settings->Configure Konqueror > 2.) In the configuration dialog, select Proxy. > 3.) Under options, make sure "Use persistent connections to proxy" is checked. > Done what you mentioned above in KDE 4.6.5 and i have got the exactly same problem described in comment #0: kio_http sends "proxy-connection: close" after the first 407 response from the proxy and the authentication downgrades to Basic. > That should solve your problem. For KDE 4.7.3, that option will no longer have > any effect on kio_http since it will completely ignore it and always use > persistent connection unless the server tells it otherwise. For KDE 4.8 the GUI > has been updated and the option mentioned above is no longer there. May be you missed what i wrote in the first line of this report "Tested against latest 4.7 branch" ;- Anyway i checked out today kde/4.7 from git and recompiled it: i can (re)confirm the issue. So, reopening the bug...
(In reply to comment #3) > > That should solve your problem. For KDE 4.7.3, that option will no longer have > > any effect on kio_http since it will completely ignore it and always use > > persistent connection unless the server tells it otherwise. For KDE 4.8 the GUI > > has been updated and the option mentioned above is no longer there. > > May be you missed what i wrote in the first line of this report "Tested against > latest 4.7 branch" ;- Ah... I indeed missed that. I only saw the version 4.7.2 and did not notice you tested against the branch which is effectively 4.7.3. > Anyway i checked out today kde/4.7 from git and recompiled it: i can > (re)confirm the issue. So, reopening the bug... Right. The keepAlive flag needs to be reset when retrying a 401 or 407 request. That is indeed a logical error on our part. Will fix it soon.
Git commit c2eb8c4225e2d7b6394e7f1c4a805f1f27763430 by Dawit Alemayehu. Committed on 25/10/2011 at 20:14. Pushed by adawit into branch 'KDE/4.7'. If m_request.isKeepAlive flag is false and the request has to be retried, e.g. in case of HTTP authentication, then make sure the isKeepAlive flag is reset to true since we have already carried out what the server requested and closed the previous connection. Otherwise, proxied connections that rely on authentication scheme that requires persistent connection such as NTLM will fail. BUG: 284870 FIXED-IN: 4.7.3 M +2 -0 kioslave/http/http.cpp http://commits.kde.org/kdelibs/c2eb8c4225e2d7b6394e7f1c4a805f1f27763430
Bad regression by the previous commit, it causes infinite loop. Tested against up to date KDE/4.7 branch. Reverting only the mentioned commit make the loop goes away but the authentication scheme is downgraded to Basic as before. GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org Proxy-Connection: keep-alive User-Agent: Mozilla/5.0 (X11; Linux i686) KHTML/4.7.2 (like Gecko) Konqueror/4.7 Pragma: no-cache Cache-control: no-cache Accept: text/html, text/*;q=0.9, image/jpeg;q=0.9, image/png;q=0.9, image/*;q=0.9, */*;q=0.8 Accept-Encoding: gzip, deflate, x-gzip, x-deflate Accept-Charset: utf-8,*;q=0.5 Accept-Language: en-US,en;q=0.9 HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE3 Date: Wed, 26 Oct 2011 12:39:30 GMT Content-Type: text/html Content-Length: 1311 Expires: Wed, 26 Oct 2011 12:39:30 GMT X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="XXXXXXX ......" X-Cache: MISS from proxy.name X-Cache-Lookup: NONE from proxy.name:8080 Via: 1.0 proxy.name:8080 (squid/2.7.STABLE3) Connection: close GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org Proxy-Connection: keep-alive User-Agent: Mozilla/5.0 (X11; Linux i686) KHTML/4.7.2 (like Gecko) Konqueror/4.7 Pragma: no-cache Cache-control: no-cache Accept: text/html, text/*;q=0.9, image/jpeg;q=0.9, image/png;q=0.9, image/*;q=0.9, */*;q=0.8 Accept-Encoding: gzip, deflate, x-gzip, x-deflate Accept-Charset: utf-8,*;q=0.5 Accept-Language: en-US,en;q=0.9 Proxy-Authorization: NTLM TlRMTVNTUAABAAA... HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE3 Date: Wed, 26 Oct 2011 12:39:30 GMT Content-Type: text/html Content-Length: 1311 Expires: Wed, 26 Oct 2011 12:39:30 GMT X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: NTLM TlRMTVNTUAACAAAADgAOADAAA..... X-Cache: MISS from proxy.name X-Cache-Lookup: NONE from proxy.name:8080 Via: 1.0 proxy.name:8080 (squid/2.7.STABLE3) Connection: keep-alive Proxy-Connection: keep-alive GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org Proxy-Connection: keep-alive User-Agent: Mozilla/5.0 (X11; Linux i686) KHTML/4.7.2 (like Gecko) Konqueror/4.7 Pragma: no-cache Cache-control: no-cache Accept: text/html, text/*;q=0.9, image/jpeg;q=0.9, image/png;q=0.9, image/*;q=0.9, */*;q=0.8 Accept-Encoding: gzip, deflate, x-gzip, x-deflate Accept-Charset: utf-8,*;q=0.5 Accept-Language: en-US,en;q=0.9 Proxy-Authorization: NTLM TlRMTVNTUAABAA........ HTTP/1.0 407 Proxy Authentication Required Server: squid/2.7.STABLE3 Date: Wed, 26 Oct 2011 12:39:30 GMT Content-Type: text/html Content-Length: 1311 Expires: Wed, 26 Oct 2011 12:39:30 GMT X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0 Proxy-Authenticate: NTLM Proxy-Authenticate: Basic realm="XXXXXXX - Inserire credenziali per la navigazione" X-Cache: MISS from proxy.name X-Cache-Lookup: NONE from proxy.name:8080 Via: 1.0 proxy.name:8080 (squid/2.7.STABLE3) Connection: close GET http://www.debian.org/ HTTP/1.1 Host: www.debian.org Proxy-Connection: keep-alive User-Agent: Mozilla/5.0 (X11; Linux i686) KHTML/4.7.2 (like Gecko) Konqueror/4.7 Pragma: no-cache Cache-control: no-cache Accept: text/html, text/*;q=0.9, image/jpeg;q=0.9, image/png;q=0.9, image/*;q=0.9, */*;q=0.8 Accept-Encoding: gzip, deflate, x-gzip, x-deflate Accept-Charset: utf-8,*;q=0.5 Accept-Language: en-US,en;q=0.9 Proxy-Authorization: NTLM TlRMTVNTUAABAA......... ....and so on
Ok. (In reply to comment #6) > Bad regression by the previous commit, it causes infinite loop. > Tested against up to date KDE/4.7 branch. > Reverting only the mentioned commit make the loop goes away but the > authentication scheme is downgraded to Basic as before. Hmm... That is the correct fix. IOW, it corrects the previous bug, but exposes another one. I already see the problem from the output you provided. NTLM is a two stage authentication scheme and as such there will be two 407 response from the server, but the second response from the client (kio_http) should be completely different from the first one but does not seem to be from what you posted above. To get to the bottom of this new issue, can you please follow the instructions in the link below and generate the kio_http debug messages and post it here or send it directly to my email address in this bug report ? http://techbase.kde.org/Development/Tutorials/Debugging/Debugging_IOSlaves#How_to_get_debug_output Please save the output from both debug areas (7103 and 7113) to the same output file.
Created attachment 64898 [details] Debug log with both debug areas 7103 and 7113 enabled
Git commit c95849d0a4d20357377aec8f40be71f24256afc0 by Dawit Alemayehu. Committed on 27/10/2011 at 00:20. Pushed by adawit into branch 'KDE/4.7'. Fix NTLM authentication yet again, but this time added a unit test to make sure the regression does not occur again. BUG: 284870 FIXED-IN: 4.7.3 M +6 -7 kioslave/http/httpauthentication.cpp M +28 -13 kioslave/http/tests/httpauthenticationtest.cpp http://commits.kde.org/kdelibs/c95849d0a4d20357377aec8f40be71f24256afc0
(In reply to comment #8) > Created an attachment (id=64898) [details] > Debug log with both debug areas 7103 and 7113 enabled Let me know if the comment shown in comment #9 does not fix your problem.
The patch works fine, thanks! Just a question: is there any reason to not always send "Connection/Proxy-Connection: keep-alive" in our requests given that we are an HTTP/1.1 client? As i understand from RFC this is just a way to inform remote side, that could only speak HTTP/1.0, that we support keep-alive connections. I have quick tested other browsers too and it seems they behave this manner. But may be i missed something....
(In reply to comment #11) > The patch works fine, thanks! > > Just a question: is there any reason to not always send > "Connection/Proxy-Connection: keep-alive" in our requests given that we are an > HTTP/1.1 client? Starting with KDE 4.7.3 that is exactly what kio_http does. The option in the proxy configuration dialog will be ignored from that version onwards. For KDE 4.8, the proxy configuration dialog has been completely redesigned and the persistent proxy option thingy has been removed. See https://git.reviewboard.kde.org/r/102802/.
*** Bug 202379 has been marked as a duplicate of this bug. ***