Created attachment 5189 [details]
Our proxy autoconfiguration file

As you see this is a no-thrills file...

I was bored. I created a minimalist proxy autoconfiguration file:

### snip ###
function FindProxyForURL(url, host)
{
   return "DIRECT";
}
### snap ###

tried to use it with konqueror and: It works!

So, the whole thing must be a bug that's related to the *.pac file itself. Is there any way of activating debugging when konqueror loads & parses the proxy autoconfiguration file?

Come on, give me some straws to grasp on!

Anybody? Would the real coder please stand up?

Anybody?

Anybody? Would the real coder please stand up?
I'm willing to do some work here, but if you keep ignoring me, there's nothing you can do.

Hello Ralf,

I think I am one of the those "real coders" responsible for this code. There are conditions that can cause what seems like a freeze that goes away after a second if you use the "Automatic Detection" feature. I am trying to find a solution for that. However, you seem to be using a proxy configuration url not automatic detection using WPAD. So I have one question for you, what did you mean by "when one tries to use the very same URL...". Did you mean the same URL as the proxy server ? I personally cannot duplicate the problem, but then none of those entries work for me so I immediately get an error page about a proxy server not being found.

Regarding the PAC file you posted, the only thing I see that will not work in konqueror is that it does not yet support multiple proxies (PROXY) entries, but that should not cause any problems as it simply ignores all but the first PROXY stuff.

Could you comment out the isInNet(...) section and see if it works then ? To me it seems to be related to a DNS resolution problems. Does this configuration work from other browsers on the same computer, e.g. Mozilla ?

* Dawit Alemayehu <adawit@kde.org>:

> I think I am one of the those "real coders" responsible for this
> code. There are conditions that can cause what seems like a freeze
> that goes away after a second if you use the "Automatic Detection"
> feature. 

How can I use the "Automatic Detection" feature if I don't know hwat
it does? The M$ docs are very shady, and I still have to find a real
documentation. Care to share a link?

> I am trying to find a solution for that. However, you seem to be
> using a proxy configuration url not automatic detection using WPAD.

The very same config can also be retrieved using WPAD -- I think.
Unfortunately, I'm not entirely sure if that's implemented correctly.
I derived what I had to do by sniffing traffic...

The machine is also responding to the name of wpad.charite.de and
serves two files:

lrwxrwxrwx    1 root     root           38 Apr 23  2003 wpad.dat -> http://spidergirl.charite.de/index.pac
lrwxrwxrwx    1 root     root           38 Apr 23  2003 wspad.dat -> http://spidergirl.charite.de/index.pac

which point to the very same index.pac.

> So I have one question for you, what did you mean by "when one tries
> to use the very same URL...".

The same auto configuration URL works with Mozilla and IE, but not
with my most recent konqueror, it DOES work with older konquerors. 

> Regarding the PAC file you posted, the only thing I see that will not
> work in konqueror is that it does not yet support multiple proxies
> (PROXY) entries, but that should not cause any problems as it simply
> ignores all but the first PROXY stuff.

OK.

> Could you comment out the isInNet(...) section and see if it works
> then?

I could try that,

> To me it seems to be related to a DNS resolution problems.
> Does this configuration work from other browsers on the same
> computer, e.g. Mozilla ?

Yes, and I did already say that in my bugreport:

"We have a proxy autoconfiguration URL which can be used with IE,
Mozilla/Netscape and of course konqueror.
Unfortuately, the recent konqueror freezes when one tries to use the
very same URL"

* Ralf Hildebrandt <ralf.hildebrandt@charite.de>:

> > Could you comment out the isInNet(...) section and see if it works
> > then?
> 
> I could try that,

Interesting. If I remove the isInNet(...) section, it works.
If I use the original file with the isInNet(...) section, konqueror
freezes.

So, any ideas on how I could debug the isInNet() function?

How do we proceed from here?

You are probably using a buggy DNS server or atleast a DNS server that does not respond to IPv6 address resolution requests. IPv6 support was recently added to KDE and I bet that is what is causing your problem. For details see the bug report below specially from comment #11 onwards:

http://bugs.kde.org/show_bug.cgi?id=70363

Unfortunately the fix or rather the workaround in KDE will not be available until KDE 3.3 is released. In the meantime the only solution is to disable ipv6 support in the kernel. I do not have IPv6 support enabled in my kernel, hence I was unable to duplicate your problem using the .pac file you attached to this bug report. That tells me that IPv6 support in KDE is the likely culprit. As to the other browsers not having a problem, I suspect none of them have ipv6 support or in the least it is not enabled by default.

An answer! At least!!

The local DNS used to resolve hostnames is djb's dnscache listening on 127.0.0.1

I checked, and my kernel has no ipv6 support compiled in:

* checked .config
* dmesg| grep -i v6
  -> no output
* lib/modules/kernelname/kernel/net/ contains no ipv6 modules:

lib/modules/2.6.7-bk4/kernel/net/xfrm/xfrm_user.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4
/lib/modules/2.6.7-bk4/kernel/net/ipv4/ah4.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/ipip.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/ip_gre.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/ipcomp.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/esp4.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_TCPMSS.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ip_nat_ftp.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_conntrack.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/iptable_mangle.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_MASQUERADE.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/iptable_filter.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ip_conntrack.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ip_nat_irc.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_MARK.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ip_conntrack_ftp.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_mark.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_owner.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_length.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ip_conntrack_irc.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_tcpmss.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_state.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ip_tables.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_REJECT.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/iptable_nat.ko
/lib/modules/2.6.7-bk4/kernel/net/ipv4/netfilter/ipt_mac.ko
/lib/modules/2.6.7-bk4/kernel/drivers

Again, no usable answer. But I'm used to that.
Can anybody give me a hint what may be bad with the isInNet(...) function?

The problem is that KDE tries IPv6 queries first (for this it doesn't matter what your kernel got compiled in) and if these are ignored instead of rejected on your firewall or server, then you get a timeout aka freeze.

KDE 3.2.3 and KDE 3.3 got support for environment variable KDE_NO_IPV6=1 (to be set before your KDE starts, e.g. in an env script) - that might help you here. 

So if I were to wait (for how long) then it would "unlock" at some point?

I will try KDE_NO_IPV6=1...

I don't know how the long the timeout is, sorry. You can also debug this using ethereal btw

> Again, no usable answer. But I'm used to that. 

There is no need to be sarcastic. How do you expect to get an answer when we 
cannot duplicate the problem using the same proxy.pac file you posted ? We do 
not have a crystal ball you know.

> Can anybody give me a hint what may be bad with the isInNet(...) function? 

The isInNet function simply uses KExtendedSocket::resolve to do a dns query. 
It is getting stuck waiting for response and hence the lockup. Anyways, I 
have asked the author of the KExtendedSocket to comment on this bug...

A friendly reminder instead of rude sarcasm will get you a long way specially 
when dealing with people who are volunteering their time and knowledge...

KExtendedSocket::resolve is used to resolve an IP into its string form. I don't see how it could be related. Don't you mean KExtendedSocket::lookup, which works the other way around, name->IP?

Anyways, what happens is really simple: it sends a request and it awaits an answer. If none come in a minute, it'll report failure. The current KDE 3.2.x code will timeout after two minutes, however, since it does two queries. That means that your script should "unlock" after 14 minutes, because you have 7 queries. We don't cache replies in the resolver code either because we believe the DNS server should be working.

The simple solution is to make sure DNS works, regardless of the query type. It is plain wrong and broken to discard a request just because you don't know what it is: an error should instead be reported. 

So, what I ask of you to help fix this problem is to run a tcpdump or ethereal session on port 53 and see what happens. Open Konqueror with this script, browse some site and wait to see what happens. Then attach here the session output here. Please try this on an otherwise idle system, to avoid catching other DNS lookups.

As for the proxy script code, I'd recommend that the hostname resolution be done once and cached. I can even write some routines to test if a given IP address belongs to a network/netmask, if required.

As for the 
KDE_NO_IPV6=1
workaround: I tried that - doesn't work.

I filled 
KDE_NO_IPV6=1
into /etc/environment.

After starting KDE, I get in a konsole window:
$ set | grep KDE
KDE_FULL_SESSION=true
KDE_MULTIHEAD=true
KDE_NO_IPV6=1

Nevertheless, the autoconfiguration still causes a freeze.

* Thiago Macieira <thiago.macieira@kdemail.net>:

> Anyways, what happens is really simple: it sends a request and it
> awaits an answer. If none come in a minute, it'll report failure. The
> current KDE 3.2.x code will timeout after two minutes, however, since
> it does two queries. That means that your script should "unlock"
> after 14 minutes, because you have 7 queries. We don't cache replies
> in the resolver code either because we believe the DNS server should
> be working.

I made a tcpdump (loopback interface due to the fact that
/etc/resolv.conf looks like this:
search charite.de
nameserver 127.0.0.1

# tcpdump -vv -i lo port 53
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
10:07:54.157962 IP (tos 0x0, ttl  64, id 36661, offset 0, flags [DF], length: 62) localhost.32801 > localhost.domain: [bad udp cksum 49d6!] 64869+ A? proxy.charite.de. (34)
10:07:54.248946 IP (tos 0x0, ttl  64, id 369, offset 0, flags [DF], length: 103) localhost.domain > localhost.32801:  64869 q: A? proxy.charite.de. 2/0/0 proxy.charite.de. CNAME[|domain]
10:07:54.313743 IP (tos 0x0, ttl  64, id 36817, offset 0, flags [DF], length: 67) localhost.32801 > localhost.domain: [bad udp cksum b263!] 64870+ A? spidergirl.charite.de. (39)
10:07:54.313956 IP (tos 0x0, ttl  64, id 370, offset 0, flags [DF], length: 83) localhost.domain > localhost.32801:  64870 q: A? spidergirl.charite.de. 1/0/0 spidergirl.charite.de. A[|domain]

These are the queries for the host in the proxy autoconfig. URL
http://proxy.charite.de/ (proxy.charite.de which is a CNAME to
spidergirl.charite.de) 

After that I surfed to my.yahoo.com:

10:07:55.293341 IP (tos 0x0, ttl  64, id 37797, offset 0, flags [DF], length: 58) localhost.32801 > localhost.domain: [bad udp cksum b1fe!] 59043+ A? my.yahoo.com. (30)
10:07:55.322920 IP (tos 0x0, ttl  64, id 37826, offset 0, flags [DF], length: 70) localhost.32802 > localhost.domain: [bad udp cksum efb3!] 60132+ PTR? 11.1.42.141.in-addr.arpa. (42)
10:07:55.323545 IP (tos 0x0, ttl  64, id 371, offset 0, flags [DF], length: 101) localhost.domain > localhost.32802:  60132 q: PTR? 11.1.42.141.in-addr.arpa. 1/0/0 11.1.42.141.in-addr.arpa. PTR[|domain]
10:07:55.392411 IP (tos 0x0, ttl  64, id 37896, offset 0, flags [DF], length: 58) localhost.32802 > localhost.domain: [bad udp cksum 5083!] 25092+ A? my.yahoo.com. (30)
10:07:55.393438 IP (tos 0x0, ttl  64, id 37897, offset 0, flags [DF], length: 71) localhost.32803 > localhost.domain: [bad udp cksum ed7e!] 60133+ PTR? 1.111.42.141.in-addr.arpa. (43)
10:07:55.393980 IP (tos 0x0, ttl  64, id 372, offset 0, flags [DF], length: 103) localhost.domain > localhost.32803:  60133 q: PTR? 1.111.42.141.in-addr.arpa. 1/0/0 1.111.42.141.in-addr.arpa. (75)
10:07:55.716116 IP (tos 0x0, ttl  64, id 373, offset 0, flags [DF], length: 107) localhost.domain > localhost.32802:  25092 q: A? my.yahoo.com. 2/0/0 my.yahoo.com. CNAME[|domain]
10:07:55.716469 IP (tos 0x0, ttl  64, id 374, offset 0, flags [DF], length: 107) localhost.domain > localhost.32801:  59043 q: A? my.yahoo.com. 2/0/0 my.yahoo.com. CNAME[|domain]
10:07:57.031751 IP (tos 0x0, ttl  64, id 39535, offset 0, flags [DF], length: 61) localhost.32803 > localhost.domain: [bad udp cksum e096!] 58172+ A? eur.i1.yimg.com. (33)
10:07:57.034127 IP (tos 0x0, ttl  64, id 39538, offset 0, flags [DF], length: 60) localhost.32804 > localhost.domain: [bad udp cksum 3f13!] 60097+ A? us.i1.yimg.com. (32)
10:07:57.037959 IP (tos 0x0, ttl  64, id 39541, offset 0, flags [DF], length: 60) localhost.32805 > localhost.domain: [bad udp cksum 67e2!] 7064+ A? us.i1.yimg.com. (32)
10:07:57.055551 IP (tos 0x0, ttl  64, id 375, offset 0, flags [DF], length: 115) localhost.domain > localhost.32803:  58172 q: A? eur.i1.yimg.com. 3/0/0 eur.i1.yimg.com. CNAME[|domain]

Everything works. This is with my "pruned" proxy autoconfig. script
which doesn't contain the isinNet() calls.

Now I swith back to the script WITH the isinNet() calls (which also
resides on proxy.charite.de). After performing this change in
konqueror, these lookups took place:

10:14:49.649214 IP (tos 0x0, ttl  64, id 58993, offset 0, flags [DF], length: 62) localhost.32813 > localhost.domain: [bad udp cksum 61f0!] 58177+ A? proxy.charite.de. (34)
10:14:49.649392 IP (tos 0x0, ttl  64, id 413, offset 0, flags [DF], length: 103) localhost.domain > localhost.32813:  58177 q: A? proxy.charite.de. 2/0/0 proxy.charite.de. CNAME[|domain]

Immediately after that, konqueror became unusable.
No more lookups.

Today I retried with Konqueror 3.3 (Using KDE 3.3.0).
And, alas, the same problem.

Nothing has been fixed, konqueror still freezes when using the proxy autoconfiguration URL that contains the isinNet() function.

Is there anything wrong with my tcpdump output? Anything I may have forgotten or omitted?

Hey Ralf,

Try putting that "KDE_NO_IPV6=1" variable in /etc/profile instead of /etc/environment.....

I had a similar problem using a proxy.pac file I made that does ad filtering (via John LoVeraso's no-ads.pac) and also uses the hash method of returning public proxy servers.....I had some isInNet(host, "xxx.xxx.xxx.xxx", "255.255.255.0") for blocking Class C ad servers (ad requests get redirected to a "blackhole" server which is a localhost:<port> address that returns either nothing at all or a small gif image)...

What was happening to me was Konq had problems directing everything correctly, some ads would get blocked, others wouldn't.......After putting that "KDE_NO_IPV6=1" variable into /etc/profile, I restarted the KDE desktop, and now Konq is using the pac file without any problems....

HTH :-)
---CTWaley


PS: Don't forget to export the KDE_NO_IPV6 variable, too...

I am not sure if I should open a new PR because of the KDE vesion but I add my comment first here. This is for 3.3.2, look what I can say:

When using automatic proxy it does not matter if dhcpd or dns based Konqueror gets it and parse it and use it correctly but freeze for a certain time between a couple of seconds to a minute when open a new URL in the same or new window or tab. Within this period I can not switch between the tabs not even scroll the window, konquerors menu is also dead. But it comes soon back and acts normal until open a new URL

Same happens when using wpad.dat as manual proxy configured script.

J

Dawit said before:
does not yet support multiple proxies (PROXY)

is this still the case and is it only for more the one PROXY or this 

proxy _IP_:8080;SOCKS _IP_:1080 ;
or
proxy _IP_:8080;SOCKS _IP_:1080;DIRECT;
or
proxy _IP_:8080;DIRECT;

also do not go through?

Thank's
João

IN #24 I reported wrong, Konqueror is NOT using wpad.dat correctly even if it is parsed (If errors in it they are corerctly noticed and adviced) but Konqueror definitly do not use the configurations in set in wpad.dat.

I retried this today using KDE 3.4.0 and Konqueror 3.4.0 -- still the same.

Indeed: "does not yet support multiple proxies (PROXY)"

Once I removed that from our script, they were usable with konqueror again. 
 

BTW you removed "that" what?

is this understood as multiple proxy proxy _IP_:8080;DIRECT;

?

I used to have the script return multiple proxies.
I removed that and have it return ONE PROXY directive only and now it works.

Which kind of sucks, since there's no fallback mechanism possible anymore.

Created attachment 10279 [details]
wpad.dat - almost  accepted

indeed, but it sucks anyway.
I have at this moment no chance to compile 3.4 and we use 3.3.latest on FreeBSD
would you mind to check my wpad.dat?
Konqueror decide well for the first direct statements but the proxy selection later is ignored and ever takes the first one.
You can get it here attached above.

I was able to pinpoint the command in my proxy.pac that made konqueror freeze:

-    dnsDomainIs(host, "127.0.0.1") ||
-    (url.substring(0, 4) == "smb:"))
+    dnsDomainIs(host, "127.0.0.1"))

So, if I remove the 
(url.substring(0, 4) == "smb:")
konqueror happily eats & digests my proxy.pac.

Still there with Konqueror 3.5.1 on KDE 3.5.1

I found something:
==================

WHen I use the URL http://proxy.charite.de/proxy.pac
the proxy autoconfiguration file is being loaded & used OK.

When I use http://proxy.charite.de/, which creates a REDIRECT to http://proxy.charite.de/proxy.pac, konqueror freezes.

The behaviour is shown in the attachment.

Created attachment 14925 [details]
wget output that shows the redirection

*** This bug has been confirmed by popular vote. ***

I close this bug in favor of a new bug I submitted, with a proper description and a better analysis of the problem, it's bug #123356