Bug 98695

Summary:	rewritten webarchiver plugin handles frames and stylesheets
Product:	[Applications] konqueror	Reporter:	Matthias Grimrath <maps4711>
Component:	webarchiverplugin	Assignee:	Malte Starostik <malte>
Status:	CONFIRMED ---
Severity:	wishlist	CC:	christian.loose, jjm, mueller
Priority:	NOR
Version:	unspecified
Target Milestone:	---
Platform:	Compiled Sources
OS:	Linux
Latest Commit:		Version Fixed In:
Sentry Crash Report:
Attachments:	new webarchiver source khtml patch for rewritten webarchiver plugin

Description Matthias Grimrath 2005-02-06 15:16:20 UTC

Version:            (using KDE KDE 3.3.1)
Installed from:    Compiled From Sources
OS:                Linux

Hi,

this is *not* a wish, but I didn't know a better way to reach
the right people.

I have rewritten the webarchiver plugin. I think / hope you will
appreciate it.  Compared to the old code, the new plugin

* saves framessets

* saves nested stylesheets, both <link href=""> and @import with no depth limit

* saves background images and other URLs in *all* stylesheet, i.e.
  - external <link href...>
  - inline <style>...</style>
  - even style attribute on HTML elements

* handles <base href...> tags

* writes a <!DOCTYPE> header if KHTML could parse one in the original document

* writes xhtml tags


Unfortunately, I needed to change a bit in KHTML:

* When looking for @import and url() statements in the internal
  stylesheet structures, the values returned by CSSImportRule::href()
  and CSSValue::cssText() now return those as originally downloaded.

  According to the comments in css/cssparser.cpp this change is
  actually a bugfix.


* Added a StyleSheet::baseURL() method


IMHO these changes are binary compatible and minor. They made implementing the
improved webarchiverplugin much easier.


I developed the new plugin in KDE 3.3.2.

Comments welcome. Please tell me the right mailing list and where to send the patches / files.


Regards,

Matthias Grimrath

Comment 1 Matthias Grimrath 2005-02-06 15:18:34 UTC

Created attachment 9444 [details]
new webarchiver source

ok, forget about my inquiry where to send the patches

Comment 2 Matthias Grimrath 2005-02-06 15:19:48 UTC

Created attachment 9445 [details]
khtml patch for rewritten webarchiver plugin

Comment 3 Maksim Orlovich 2005-02-06 15:27:54 UTC

Either kfm-devel or kde-core-devel would probably do. Thanks for your hard work on this... What kind of sucks is that we just entered a freeze for 3.4, though all this is probably classified as a bug fix. And you should also be able to get a CVS account to put this in, or if the freeze-enforcers think it's too large, to at least put into kdeplayground or something

Comment 4 Matthias Grimrath 2005-02-06 21:33:30 UTC

On Sunday 06 February 2005 15:27, Maksim Orlovich wrote:
> Thanks for your hard work on this... What kind of sucks is that we
> just entered a freeze for 3.4, though all this is probably
> classified as a bug fix. 
> And you should also be able to get a CVS 
> account to put this in, or if the freeze-enforcers think it's too
> large, to at least put into kdeplayground or something

The code has only alpha quality and 1200 lines, so I recommend not to 
put it in a stable release yet.

Matthias Grimrath

Comment 5 Matthias Wieser 2006-06-06 11:41:41 UTC

What about KDE 3.5.4?

Comment 6 Matthias Grimrath 2006-06-08 22:09:23 UTC

Good question.

Once I posted my patch to kfm-devel but since then I have kept a low profile.

I guess I should try again.

Comment 7 Dirk Mueller 2006-06-20 21:04:00 UTC

hmm, we should commit this.

Comment 8 Giovanni Venturi 2007-03-22 00:03:08 UTC

when? it's interesting...

Comment 9 jos poortvliet 2007-06-08 16:04:04 UTC

Has this been comitted yet? Some ppl use this ;-)

Comment 10 Matthias Grimrath 2007-06-13 22:06:10 UTC

No. But I will maintain the patches published at kde-apps.org (probably ;-)
Hm, I guess I should push harder for inclusion :-)

Comment 11 Maksim Orlovich 2007-06-13 22:29:42 UTC

I think the easiest might for you to just get an SVN account and do it, I don't think Malte is doing much KDE stuff anymore, and honestly, it's hard enough to keep track of stuff one has to do to remember outside contributions like this one... Except even 4.0 seems to be in a app freeze :(, but may be a good chunk of this can be considered bugfixes --- can it be done incrementally, or is it a totally different codebase?

Comment 12 Matthias Grimrath 2007-06-16 19:32:06 UTC

> can it be done incrementally, or is it a totally different codebase?

It varies, but it should be possible to integrate different parts of it
over time.

essential:

kio_http/kdelibs:
   * medium change: offline-browsing: While my patches do not change the
     codebase I had to tweak and extend it quite a bit. The main problem
     was that the cache strategy was changed at several places basically
     randomly independent of what was selected at the cache preferences.
   * small change: referrer-URL: I changed some sources to be able
     to tell whether a kio_http job was a top-level request or not. It is
     still not perfect (especially if javascript is involved) but better
     than before.
   * medium change: slave generation: In the original code IMHO too many
     parallel kio_http slaves are created. To reduce resource usage I had
     to tweak the scheduler that spawns new slaves. I consider the
     original behaviour buggy, because if I specify to only have one
     kio_http slave in $KDEDIR/share/services/http.protocol I do not
     expect to have 6+ kio_http slaves after viewing a moderately complex
     website.

   offline-browsing, referrer-URL and slave generation are quite
   interwoven so I would not like to separate them :-)

webarchiver/kdelibs:
   * small change: Added a method to KHTML to get data the webarchiver
     needs. Optimized a bit. These changes are binary compatible.

webarchiver/kdeaddons
   * big change: I started modifying the orignal code and ended up
     rewriting it. For example the recursions into frames
     and stylesheets got too complex for the old code base.


non-essential:

kio_http/kdebase:
   * small change: I extended the KDE Cache Preference dialog to be able
     to select all 4 available strategies. Improved the tooltips (at
     least I hope so :-)
   * small change: Changed the favicon loader to also look at the cache.
     The original KDE sources explicitly disable the cache. I don't know if
     there is a reason for that, but for me this just messed up offline
     browsing so I turned it off.
     
kio_http/kdeaddons:
   * small change: I extended the KDE Cache selection menu addon similiar
     to the KDE Cache Preference dialog.

kio_http/kde-i18n-de:
   * German translations for the tooltips of the KDE Cache Preference dialog.

Comment 13 jos poortvliet 2008-02-26 12:46:32 UTC

I agree with Maksim, please just apply for an SVn account and commit this so it can be there for 3.5.10 (would love a port to KDE 4.x as well).
http://techbase.kde.org/index.php?title=Contribute/Get_a_SVN_Account

Comment 14 Jonathan Marten 2009-01-15 20:43:42 UTC

Is this change the same as committed to trunk as r869740?
If so, this bug can be closed as FIXED.