Bug 198428

Summary: Wrong default user agent string in Konqueror with webkit-kde kpart
Product: [Applications] konqueror Reporter: Sterling X. Winter <sterling.winter>
Component: generalAssignee: webkit-devel
Status: RESOLVED FIXED    
Severity: normal CC: adawit, ingmar, kamikazow, kde, stijn
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: Version Fixed In: 4.6
Sentry Crash Report:

Description Sterling X. Winter 2009-06-30 17:25:21 UTC
Version:            (using KDE 4.2.4)
Compiler:          gcc-4.3 
OS:                Linux
Installed from:    Compiled From Sources

When using Konqueror 4.2.4 with the WebKit kpart from SVN, "KHTML" is still present in the default user agent string instead of "WebKit".

Steps to reproduce:

1. Build webkit-kde from SVN and re-compile Konqueror (i.e. kdebase and konq-plugins) against it.
2. Go to System Settings --> Advanced --> File Associations, expand the text/html node, select the Embedding tab, and move "WebKit (webkitpart)" to the top and click Apply.
3. Launch Konqueror and make sure Tools --> Change Browser Identification is set to "Default Identification".
4. Visit a website which echoes back your browser's user agent string (e.g. http://www.microsystools.com/services/http-user-agent/ ).

Expected user agent string would be something like:

Mozilla/5.0 (compatible; Konqueror/4.2; Linux; X11; en-US, en) WebKit/4.2.4 (like Gecko)

Observed user agent string is:

Mozilla/5.0 (compatible; Konqueror/4.2; Linux; X11; en-US, en) KHTML/4.2.4 (like Gecko)
Comment 1 Urs Wolfer 2009-06-30 21:44:20 UTC
Really strange. In the debug output I can see the correct useragent, as well as on http://show-ip.net/browserinfo/

User-Agent: Mozilla/5.0 (Konqueror/4.3; Linux) AppleWebKit/527+ (KHTML, like Gecko, Safari/419.3) 

No idea where your site gets the user agent from...
Comment 2 Sterling X. Winter 2009-07-01 12:51:25 UTC
(In reply to comment #1)

From what I can tell, the substring "KHTML" is hard-coded into the default user agent string in kdelibs/kio/kio/kprotocolmanager.cpp. Are you sure you've got Konqueror configured to use the default, and if so, where is the new default being set?
Comment 3 Urs Wolfer 2009-07-01 21:11:33 UTC
It gets overwritten in QString KWebPage::userAgentForUrl

That's the way QtWebKit suggests it. It also looks like it gets accepted in KIO. Do you still think kprotocolmanager does something wrong?
Comment 4 Sterling X. Winter 2009-07-06 13:58:08 UTC
I've narrowed the bad behavior down a little.

The incorrect user agent (with "KHTML/") is always sent in the following instances:

* When entering a URL in the location bar
* When opening a URL from the Bookmarks menu, from Bookmarks in the Navigation Panel, or from Bookmark Editor
* When viewing a web page, right-clicking on a link in the page and selecting "Open in New Tab"

I can make Konqueror send the correct user agent (with "AppleWebKit/"), but only when left-clicking on a link (such as the links to user agent services in the previous comments on this bug).

The logic in KWebPage::userAgentForUrl and KProtocolManager::userAgentForHost looks correct to me, so perhaps KIO is caching something per-KPart instance, or KWebPage::userAgentForUrl is being called inconsistently?
Comment 5 Dawit Alemayehu 2009-09-01 04:28:33 UTC
This is a Konqueror issue. It happens simply because everytime you enter a URL in the location bar or the search bar, Konqueror attempts to determine the mime-type by doing a KIO::get so it can embed the appropriate part based on the mime-type. Unfortunately, that means the default user agent string from KIO will be used since the webkit part will not have a chance to re-write the default string. IOW, this cannot be fixed here and it is not trivial to fix without a significant change in Konqueror...
Comment 6 Dawit Alemayehu 2009-10-05 17:27:37 UTC
As a workaround there is a little documented trick you can use to change the built-in default agent string in KDE. 

Simply add a "UserAgent=" entry shown below to the default section of either $KDEHOME/share/config/kioslaverc or $KDEHOME/share/config/kio_httprc files and execute the command 'kdeinit4' from KRunner (ALT+F2): 

UserAgent=Mozilla/5.0 (X11; U; Linux; en-US) AppleWebKit/527+ (KHTML, like Gecko, Safari/419.3) konqueror/4.3.1

Doing that will ensure the correct string is sent at all times. However, if you switch the engine back to KHTML, you have to remove that line or you will have the same problem in reverse.

Perhaps in the future we might add a hack to the webkitpart to automatically do the above workaround...
Comment 7 Dawit Alemayehu 2009-10-23 17:35:41 UTC
Actually the last paragraph in comment #6 is not possible to do for the very same reason this issue exists. The webkit part gets loaded after Konqueror does its stat to determine the mime-type for the typed in url. As such the only option is to use the workaround outlined in comment #6 with one change. Instead of hard-coding the konqueror version number to "4.3.1" or any other value, use the "KDE_VERSION" keyword. That is

Mozilla/5.0 (X11; U; Linux; en-US) AppleWebKit/527+ (KHTML, like
Gecko, Safari/419.3) konqueror/KDE_VERSION
Comment 8 Allan Sandfeld 2009-10-25 00:34:18 UTC
I would suggest fixing the user-agent configuration instead. We have a configuration dialog for the user-agent string, and it could take into consideration which html-engine is default, or you could extend the API to set an override while webkitpart is being used.
Comment 9 Dawit Alemayehu 2009-10-25 04:38:41 UTC
On Saturday 24 October 2009 18:34:20 Allan Sandfeld wrote:
> https://bugs.kde.org/show_bug.cgi?id=198428
> 
> 
> Allan Sandfeld <kde@carewolf.com> changed:
> 
>            What    |Removed                     |Added
> ---------------------------------------------------------------------------
> - CC|                            |kde@carewolf.com
> 
> 
> 
> 
> --- Comment #8 from Allan Sandfeld <kde carewolf com>  2009-10-25 00:34:18
>  --- I would suggest fixing the user-agent configuration instead. We have a
>  configuration dialog for the user-agent string, and it could take into
>  consideration which html-engine is default, or you could extend the API to
>  set an override while webkitpart is being used.

Well it is more complicated than that and as I already stated not a trivial 
thing to fix because 

#1. The useragent configuration is decoupled from the application. It is tied 
to KIO rather than any one specific application. If a single application 
changes this setting, all applications will be affected! The only way 
applications can provide their own user-agent string is to pass that 
information through the KIO meta data system and ignore the KDE (read: KIO) 
wide configuration.

#2. The process Konqueror uses to find and use the appropriate kpart has a huge 
effect in which user-agent string gets sent to remote sites. I have explained 
how this process is currently handled by Konqueror in comment #5. Namely, it 
always does a stat on user typed/selected url in order to find out the 
request's mime-type. It does this to make sure that it invokes the correct 
kpart to handle the request.

Anyhow, I agree that the user-agent configuration dialog can and should 
definitely be modified to allows users to easily change the default user-agent 
string. Currently, it does not...
Comment 10 Dawit Alemayehu 2010-09-09 16:10:07 UTC
*** Bug 250664 has been marked as a duplicate of this bug. ***
Comment 11 Dawit Alemayehu 2010-09-28 16:31:45 UTC
SVN commit 1180675 by adawit:

- Added support for reading the "X-KDE-Default-UserAgent-String" property from
  KDE browsing engine components to ensure the correct user-agent string is used 
  and shown by default.

- Make sure the runtime and not compile time KDE version number is 
  used when constructing user-agent strings.

- Used QStringBuilder for minor performance improvement.

BUG:198428


 M  +211 -52   kprotocolmanager.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1180675