Bug 53515

Summary: misc. NEW HTML render problems in KDE 3.1 with specific site(s)
Product: [Applications] konqueror Reporter: adminimail
Component: khtmlAssignee: Konqueror Developers <konq-bugs>
Status: RESOLVED WORKSFORME    
Severity: normal CC: bruggie, n.v.d.maas, w.k.havinga
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: problem with www.nosnieuws.nl

Description adminimail 2003-01-28 14:56:19 UTC
Version:            (using KDE KDE 3.1)
Installed from:    Compiled From Sources
Compiler:          GCC 3.2.1 Gentoo 1.4 RC2
OS:          Linux

These sites worked fine in KDE 3.0.5

Introduced in 3.1 RC6 (Bug reported for RC6 as well)

http://www.nosnieuws.nl	->Middle frame missing

Introduced in 3.1 final

http://www.tweakers.net	->Top menu's expand vertically over entire screen on mouse-over

Older problem also in KDE 3.0.x:

http://www.tweakers.net	->Pictures beside headlines don't render


These sites attract large amount of visitors. The first one is from the national Dutch news/broadcasting organisation. The render problems are showstoppers in the process of converting some windows desktops to KDE 3.1 :(
Comment 1 W K Havinga 2003-01-30 17:38:22 UTC
I can confirm all these problems, and have some additional information. 
 
First of all, the 'top menu expands vertically over the entire screen' problem did certainly NOT exist 
in RC6, so it has been introduced between RC6 and 3.1 final (see bug 53116 - it's a dupe of this 
one). 
 
The "some pictures don't render" problem has been pinned down by Odysseus (a tweakers.net 
newsposter). I will translate his post here, as he does not seem to have posted it to bugs.kde.org 
yet. 
 
Let's check the output of a non-working Konqueror http-request (captured using ethereal): 
 
--[ Client request ]-- 
GET /ext/i/1032119313.gif HTTP/1.1 
Connection: Keep-Alive 
User-Agent: Mozilla/5.0 (compatible; Konqueror/3.1; Linux) 
Referer: http://www.tweakers.net/ 
Accept: image/x-krl, image/x-portable-bitmap, image/x-xbm, image/x-ico, 
 image/png, image/x-portable-pixmap, image/jpeg, image/x-xpm, image/x-eps, 
 image/tiff, image/x-bmp, image/gif 
Accept-Encoding: x-gzip, x-deflate, gzip, deflate, identity 
[more irrelevant headers etc.] 
 
--[ Server answers ]-- 
 
HTTP/1.1 406 Not Acceptable 
Date: Thu, 09 Jan 2003 19:25:21 GMT 
Server: Apache/1.3.27 (Unix) PHP/4.2.2 mod_gzip/1.3.19.1a 
Alternates: {"i.dsp" 1 {type application/x-httpd-php} {length 4248}} 
Connection: close 
Transfer-Encoding: chunked 
Content-Type: text/html; charset=iso-8859-1 
 
1b0 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> 
<HTML><HEAD> 
<TITLE>406 Not Acceptable</TITLE> 
</HEAD><BODY> 
<H1>Not Acceptable</H1> 
An appropriate representation of the requested resource /ext/i/1032119313.gif could not be found 
on this server.<P> 
[etc.] 
 
So apparently Konqueror is asking for pictures only, while Apache sends the picture using the 
"application/x-httpd-php" mime-type (the .gif images are probably constructed using PHP..). One 
could argue this is a server misconfiguration, *however*: tnet is probably not the only site that has 
this problem, and besides: Mozilla, Opera, IE etc. all render the page just fine, so that makes me 
wonder....  
Adding "application/x-httpd-php, " to the acceptHeader (kdelibs/kioslave/http/http.cc:2165 or 
thereabout) fixes the problem (tested by several tweakers.net visitors :) 
 
It's up to the developpers to decide if they want to include this somewhat dirty workaround (or 
something similar/more clean), but at least this info should clarify the problem. 
Comment 2 Roger Larsson 2003-03-15 17:07:05 UTC
Works for me with KDE CVS (future KDE 3.2) from 2003-03-12 
Comment 3 Stephan Binner 2003-03-17 10:53:45 UTC
Here, HEAD doesn't render it fine.  
Please report only one bug per report in future. 
Comment 4 Roger Larsson 2003-03-25 14:00:12 UTC
Retested with HEAD from tonight. 
 
nosnieuws: Only problem I see is on the news site where the scrolling text looks like 
it has a second raw with most of it hidden (only the top of characters visible) 
 
tweakers:  
	top menu OK 
	tab menu "FRONTPAGE", "NIEUWS", ... NO dropdown menu - (java script) BUG 
	the GIF pictures beside headline don't render (as reported) 
Comment 5 Thomas Mitzkat 2003-05-10 09:56:58 UTC
the menus on left side do not work/expand at http://www.signal-iduna.de -> Konqueror 3.1.x 
positioning of pther tables and menues on main frame works not properly at 
http://www.signal-iduna.de -> Konqueror 3.1.x 
 
Comment 6 Moritz Moeller-Herrmann 2003-06-08 16:41:24 UTC
How does one test this bug report? 50 % Closed. IMHO it should be closed as invalid... Only the 
second reported bug does still show in KDE-CVS-2003-6-3 
 
Comment 7 Vincent Wagelaar 2003-07-31 02:27:26 UTC
bug: tweakers.net 
 
With Konq CVS 20030727 the top menu works but I can only click on the 
first item of the menu. The menus a bit down further don't show. 
 
The images still don't show. 
Comment 8 Niek van der Maas 2003-10-31 14:45:22 UTC
Running KDE CVS from a few days ago (3.1.93):

www.nosnieuws.nl -> everything works OK
www.tweakers.net -> all menus are now rendering OK, but images still not working.

I talked about the images with Tim Jansen on IRC, and we concluded the problem is that Konqi doesn't send */* as Accept value when requesting images. Note that this does only happen when an image is requested from a HTML page inside <img> tags.  If an image is directly requested by Konqi (by pasting the URL in the location bar), */* and image/* are send in the Accept field.

For example: request http://www.tweakers.net/ext/i/1067542219.gif from inside a <img> tag:

GET /ext/i/1067542219.gif HTTP/1.1.
Connection: Keep-Alive.
User-Agent: Mozilla/5.0 (compatible; Konqueror/3.1; Linux) (KHTML, like Gecko).
Referer: http://tweakers.net/.
Accept: image/x-portable-bitmap, image/x-pcx, image/x-xbm, image/x-targa, image/x-ico, image/png, image/x-portable-pixmap, image/jpeg, image/x-xpm, image/x-eps, image/tiff, image/x-bmp, image/gif.
Accept-Encoding: x-gzip, x-deflate, gzip, deflate, identity.
Accept-Charset: iso-8859-1, utf-8;q=0.5, *;q=0.5.
Accept-Language: en.
Host: www.tweakers.net.
Cookie: -snip-.


Reply:

HTTP/1.1 406 Not Acceptable.
Date: Fri, 31 Oct 2003 12:57:15 GMT.
Server: Apache/1.3.28 (Unix) PHP/4.3.3 mod_gzip/1.3.19.1a.
Alternates: {"i.dsp" 1 {type application/x-httpd-php} {length 4263}}.
Connection: close.
Transfer-Encoding: chunked.
Content-Type: text/html; charset=iso-8859-1.
-snip-.


Here is how a IE6 (Crossover) request on the same image looks like:

GET /ext/i/1067542219.gif HTTP/1.1.
Accept: */*.
Referer: http://tweakers.net/.
Accept-Language: en-us.
Accept-Encoding: gzip, deflate.
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98).
Host: www.tweakers.net.
Connection: Keep-Alive.
Cookie: -snip-.


Reply:

HTTP/1.1 200 OK.
Date: Fri, 31 Oct 2003 13:05:20 GMT.
Server: Apache/1.3.27 (Unix) PHP/4.3.1 mod_gzip/1.3.19.1a.
X-Powered-By: PHP/4.3.1.
Expires: Sun, 31 Dec 2006 21:21:59 GMT.
Connection: close.
Transfer-Encoding: chunked.
Content-Type: image/gif.
.
1660.
GIF89a-snip-.


Please also take a look at this message:  http://ftp.ics.uci.edu/pub/ietf/http/hypermail/1997q1/0007.html
"Since most browsers send '*/*' as part of the Accept header, 406's should
be extremely rare."

So please, add */* (and perhaps image/* too) as a default Accept value in HTTP requests for images.
A possible fix is adding the following lines in kdelibs/kioslave/http/http.cc:

-    if (!acceptHeader.isEmpty())
+    if (!acceptHeader.isEmpty()) {
+      if(acceptHeader.contains("*/*") == 0)
+        header += "*/*, ";
       header += acceptHeader;
+    }

But I think there is a better fix for this problem. Any ideas?
Comment 9 Dirk Mueller 2003-11-04 00:05:19 UTC
Subject: kdelibs/khtml

CVS commit by mueller: 

add */* to accept header for images
CCMAIL: 53515-done@bugs.kde.org


  M +3 -0      ChangeLog   1.80
  M +3 -0      misc/loader.cpp   1.156


--- kdelibs/khtml/ChangeLog  #1.79:1.80
@@ -1,4 +1,7 @@
 2003-11-03  Dirk Mueller  <mueller@kde.org>
 
+        * misc/loader.cpp (buildAcceptHeader): add */* to Accept-Header
+        for image requests with low priority to fix www.tweakers.net (#53515).
+
         * css/parser.y: lowercase the media we get via CSS stylesheets. 
         Its true that the media is case insensitive in CSS, but it is

--- kdelibs/khtml/misc/loader.cpp  #1.155:1.156
@@ -465,4 +465,7 @@ static QString buildAcceptHeader()
     if (result.endsWith(", "))
         result.truncate(result.length()-2);
+    if ( result.length() )
+        result += ";q=0.5";
+    result += ",*/*;q=0.1";
     return result;
 }


Comment 10 John van Spaandonk 2003-11-26 10:12:09 UTC
Created attachment 3409 [details]
problem with www.nosnieuws.nl

The only remaining problem for me in this report is www.nosnieuws.nl

It works better (I can read the site now!), but all's not well yet.
There is no scrolling text at the top of the page (javascript problem)
and I the bottom of the page is not reachable by scrolling.
The stipples at the top of the page are supposed to be the date in Dutch:
"Woensdag 26 September".

See attached picture or try out for yourself.

The javascript debugging window says:

Error: http://www.omroep.nl/nos/nieuws/hoofdpunten/hoofdpunten.html:
SyntaxError: Parse error at line 17
Error: http://www.omroep.nl/nos/nieuws/scripts/sitestat.js: SyntaxError: Parse
error at line 51
Comment 11 Niek van der Maas 2004-01-05 19:30:05 UTC
Hi, I updated my CVS build from KDE and it seems like the images on www.tweakers.net are broken again in HEAD. I think (not sure) it's a result of this commit:
http://webcvs.kde.org/cgi-bin/cvsweb.cgi/kdelibs/khtml/misc/loader.cpp.diff?r1=1.161&r2=1.162&f=h
It's also possible that these problems are because I upgraded from Qt 3.2 to 3.3b1.
Someone with a recent CVS build who can confirm this?
Comment 12 Otto Bruggeman 2004-01-05 19:45:48 UTC
Looks like this bug is back, so reopening for Niek who cant do it himself (he reported it to me on IRC) and i can confirm this on http://www.tweakers.net/nieuws/30418 where the image about the linux penetration is not loaded.

So looks like someone reverted this fix:
http://webcvs.kde.org/cgi-bin/cvsweb.cgi/kdelibs/khtml/misc/loader.cpp.diff?r1=1.155&r2=1.156&f=h
Comment 13 Jonathan Brugge 2004-01-06 02:33:39 UTC
The image Otto mentions in comment #12 loads correctly here. It's probably due to the problems tweakers.net has had during the last days with its image server...there was a disk crash and server instability. Note that the loading of images inside newsposts (as reported in the above comment) has never been a problem - the pricewatch-graphs and the headline-annoucements were. Those (and the image mentioned in comment #12) still load correctly here (CVS as of 4th Januari 2004). It's possible the above comments come from the crashing image server, though I'm not sure about that. For additional information, you can contact me directly at jonathan [at] tweakers.net.

There are some other issues with www.tweakers.net, but these are in other areas...I'll open another bugreport about that, if it isn't already reported.

Jonathan Brugge (= the newsposter 'odysseus' as mentioned in comment #1)
Comment 14 Otto Bruggeman 2004-01-06 06:48:49 UTC
Yup i found that later last night it worked again, forgot to send a message about it. So it looks indeed like it was a problem on tweakers.net's side. But i'll let Niek comment again before i close this bug (I'll do so this afternoon at around 6pm if i dont hear from Niek before that).
Comment 15 Niek van der Maas 2004-01-06 08:12:03 UTC
Ah, it seems everything works again. So it was a tweakers.net problem, and had nothing to do with my recent CVS update of KDE as I thought. I think this report can be closed.
Comment 16 George Staikos 2004-01-06 08:18:41 UTC
He said close it.