Bug 167739 - Konqueror autoscroll high cpu
Summary: Konqueror autoscroll high cpu
Status: RESOLVED FIXED
Alias: None
Product: konqueror
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Ubuntu Linux
: NOR normal with 20 votes (vote)
Target Milestone: ---
Assignee: Konqueror Developers
URL:
Keywords:
: 171516 171594 (view as bug list)
Depends on:
Blocks:
 
Reported: 2008-07-30 14:37 UTC by Joseph Reagle
Modified: 2008-10-01 22:46 UTC (History)
8 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Oprofile log, scrolling through webpage without horizontal scrollbar (854.44 KB, text/plain)
2008-08-23 18:50 UTC, Manuel Massing
Details
Oprofile log, scrolling through webpage with horizontal scrollbar (880.07 KB, text/x-log)
2008-08-23 18:52 UTC, Manuel Massing
Details
possible patch (1.13 KB, patch)
2008-09-16 15:00 UTC, Germain Garand
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joseph Reagle 2008-07-30 14:37:16 UTC
Version:           4.1.00 (using KDE 4.1.0)
Installed from:    Ubuntu Packages
OS:                Linux

When auto-scrolling in Konqueror 4.1.00 it uses 100% of one of my CPUs, in Konqueror 3.5 it did fine at around 20%.
Comment 1 Dominik Tritscher 2008-07-30 22:47:18 UTC
I can't reproduce that with the same version, also from Kubuntu packages.
Does this occure with all websites?
Konqueror did not use more then 20% on one core (Intel Core Duo T2400) auto-scrolling some regular page (like bugs.kde.org).
Auto-scrolling on a site containing flash didn't caused more then 50% CPU load.
Comment 2 Joseph Reagle 2008-07-30 23:12:39 UTC
On Wednesday 30 July 2008, Dominik Tritscher wrote:
> I can't reproduce that with the same version, also from Kubuntu packages.
> Does this occure with all websites?
> Konqueror did not use more then 20% on one core (Intel Core Duo T2400) 

auto-scrolling some regular page (like bugs.kde.org).
> Auto-scrolling on a site containing flash didn't caused more then 50% CPU 

load.

Thanks for trying to reproduce, I provide as much info below as possible.

# Setup

Kubuntu 8.04 with PPA debs for KDE 4.1

Konqueror
 *** 4:3.5.9-0ubuntu7.2 0
 *** 4:4.1.0-0ubuntu1~hardy1~ppa2

module: i915
direct rendering: Yes

# Testing

URL = Most sever bugs on bugs.kde.org

Konq 3.5.9 :~10% on oneCPU
Konq 4.1.0:~60% on one CPU
 
This happens under 3.5.9 or 4.1 KDE window manager.
Comment 3 Rui G. 2008-07-31 14:37:32 UTC
Did you try changing the smooth scrolling setting in the Konqueror Web Browsing settings? I read in some bug reports that disabling smooth scrolling made scrolling smoother ( or less cpu intensive ). It could depend on the hardware.
Comment 4 Joseph Reagle 2008-07-31 16:23:17 UTC
On Thursday 31 July 2008, Rui G. wrote:
> Did you try changing the smooth scrolling setting in the Konqueror Web 

Browsing settings? I read in some bug reports that disabling smooth 
scrolling made scrolling smoother ( or less cpu intensive ). It could 
depend on the hardware.

Great, I disliked smooth scrolling and didn't know I could turn it off. But 
sadly it does nothing to the disprortionate increase in CPU use from KDE 
3.5.9 .
Comment 5 Maksim Orlovich 2008-07-31 19:06:36 UTC
Do the pages you have problems on have 2 scrollbars for you?
Comment 6 Joseph Reagle 2008-07-31 22:49:53 UTC
On Thursday 31 July 2008, Maksim Orlovich wrote:
> ------- Additional Comments From maksim kde org  2008-07-31 19:06 -------
> Do the pages you have problems on have 2 scrollbars for you?


A single vertical scroll bar on the right side of [1].

[1]:http://bugs.kde.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=critical&bug_severity=grave&bug_severity=major&order=bugs.bug_severity
Comment 7 Joseph Reagle 2008-08-04 15:34:18 UTC
Because of issues with KDE 4.1, I'm trying running 3.5.9 with kwin4.1 using kwin --replace. I notice that even with everything else 3.5.9 the high CPU happens when I scroll in Konqueror. So perhaps this bug should be reassigned to kwin.
Comment 8 Maksim Orlovich 2008-08-04 15:56:07 UTC
Are you using composite ("Desktop effects") perhaps?
Comment 9 Joseph Reagle 2008-08-04 16:29:03 UTC
On Monday 04 August 2008, Maksim Orlovich wrote:
> ------- Additional Comments From maksim kde org  2008-08-04 15:56 -------
> Are you using composite ("Desktop effects") perhaps?


Yes, the KDE 4.1 windows manager was using them.
Comment 10 Maksim Orlovich 2008-08-04 16:32:25 UTC
Disabling them might help --- at the very least it avoids one more layer of buffering/one more copy, and puts lets pressure on GPU memory (which KDE4 is a real pig about, too, I should go back to improving that, really...).
Comment 11 Manuel Massing 2008-08-23 18:47:46 UTC
I can confirm excessive CPU usage by Xorg while scrolling vertically, 
if and only if a horizontal scrollbar is present. It is reproducable
whether smooth scrolling is enabled or not. Compositing effects are
disabled. KDE 4.1, debian packages from experimental, Xorg 7.3, xserver
1.4.2, QT 4.4.0

E.g.: Opening "Most hated bugs" page from the KDE bug tracker. This 
does not fit on my 1280x1024 screen, so a horizontal scroll bar is
present. Scrolling using the mouse wheel or scrollbar results in 
100% cpu usage and very slow and jerky response.

If I resize konqueror to accomodate the horizontal span of the same 
document (i.e. the horizontal scrollbar disappears), Xorg cpu usage is 
about 20%, reaction is fluid.

Caveat: I'm running nvidia proprietary driver version 177.68, GeForce 7600 Go, but I am not sure this is an nvidia issue (no performance problems otherwise).

Attached are oprofiles of Xorg, in both cases konqueror scrolling through the same webpage ("Most hated bugs") for 60 seconds, with (xorg-slow-scrolling.log) and without (xorg-fast-scrolling.log) horizontal scrollbars. Note the factor 10 difference in the
number of samples at fbCompositeSolidMask_nx8888x8888Cmmx, which indicates that a different (and inefficient) code path is taken (be it in the driver, Xorg, QT or KDE) when a horizontal scrollbar is present. 

I doubt this is konqueror specific.

May be related to http://bugs.kde.org/show_bug.cgi?id=163626
Comment 12 Manuel Massing 2008-08-23 18:50:40 UTC
Created attachment 27001 [details]
Oprofile log, scrolling through webpage without horizontal scrollbar
Comment 13 Manuel Massing 2008-08-23 18:52:08 UTC
Created attachment 27002 [details]
Oprofile log, scrolling through webpage with horizontal scrollbar
Comment 14 Manuel Massing 2008-08-23 18:56:47 UTC
Sorry, misread the initial bug report, I could only reproduce high
cpu usage in the Xorg xserver, not in konqueror.
Comment 15 Maksim Orlovich 2008-08-23 19:04:18 UTC
Thanks for the analysis. I am actually aware of the 2 scrollbar case (hence the comment #5) --- the  problem there is that the QWidget::scroll function actually forces the entire view to repaint, instead of moving the rendered picture in the 
backbuffer and repainting a small stripe, since it gets worried about the /other/ scrollbar getting in the way. I don't know how to fix this yet,
though, especially since it's tied with the Qt internals.

That fbCompositeSolidMask_nx8888x8888Cmmx is hot is interesting, though.

As for NVidia drivers, it's mostly a problem with 8600, which shows much worse performance, sometimes visibly slow performance than far cheaper and far older hardware (e.g. my 7100).
Comment 16 Manuel Massing 2008-09-12 16:55:20 UTC
Sorry to follow up so late:

fbCompositeSolidMask_nx8888x8888Cmmx apparently is the libfb software fallback taken for anti-aliased font rendering, but the newer nvidia 177.70 beta driver takes a hardware accelerated rendering path again.

However, performance is still extremly sluggish (saturating my Core Duo 1.66),
and these fundamental repainting inefficiencies you explained seem to  affect multiple parts of KDE (essentially all webpages viewed through akregator, mail scrolling in kmail and the 2-scrollbar case in konqueror).

Do you think this should be filed as a QT bug?
Comment 17 Germain Garand 2008-09-15 05:10:52 UTC
SVN commit 861059 by ggarand:

QWidgetPrivate::scrollRect doesn't account for it's rect argument
but rather examine the whole widget surface when pondering if
it can apply accelerated blitting, resulting in unnecessarily slow
and non-blitting QWidget::scroll calls.

CCBUG: 167739



 A             0248-fix-qwidget-scroll-slowness.diff  
 M  +1 -1      README  


--- trunk/qt-copy/patches/README #861058:861059
@@ -1,5 +1,5 @@
 Please assign the numbers incrementally, and don't reuse them. The next one:
-#0248
+#0249
 
 This directory contains patches for Qt that haven't been accepted by TrollTech
 yet. All patches in this directory itself shouldn't make qt-copy incompatible
Comment 18 Manuel Massing 2008-09-16 09:30:50 UTC
Just a quick update: I tested the scrollRect patch on KDE 4.1.1, but
unfortunately scrolling with two scrollbars in konqueror is still a lot slower
than with a single-scrollbar - I haven't found the time to do any real
debugging/profiling with the patch yet, though.

Comment 19 Manuel Massing 2008-09-16 13:30:39 UTC
Hmm, with 2 scrollbars, I get QWidgetPrivate->data.crect == rect (apparently the size of the scroll area), so the patch #248 doesn't affect the code path.

I tentatively changed patch #248 to use clipRect() (parent-clipped version
of data.crect) instead of data.crect for intersection, which seems in line with
the patch intent and yields fast scrolling - mind you, just a guess, as I 
know nothing about qt internals. In this scenario it seems to behave correctly.

In QWidgetPrivate::scrollRect (this=0x9a61870, rect=@0xbfc051f0, dx=0, dy=-10):

(gdb) print clipRect()
$10 = {x1 = 0, y1 = 0, x2 = 1250, y2 = 771}
(gdb) print data.crect
$11 = {x1 = 0, y1 = 0, x2 = 2034, y2 = 6036}
(gdb) print rect
$12 = (const QRect &) @0xbfc051f0: {x1 = 0, y1 = 0, x2 = 2034, y2 = 6036}
Comment 20 Germain Garand 2008-09-16 14:53:00 UTC
no, you need also tip of branch (r861088).
I only fixed the most glaring bug, but scrollRect() remains severly under-optimised and doesn't deal correctly with parent widgets overing

it should e.g substract them, invalidate them, then iterate over the rects() of the remaining region, recursively scrolling them

> I tentatively changed patch #248 to use clipRect() (parent-clipped version
> of data.crect) instead of data.crect for intersection, which seems in line
> with the patch intent 

I think they meant scrollRect to invalidate all that is in the passed rectangle, so this wouldn't do.
Comment 21 Germain Garand 2008-09-16 15:00:22 UTC
Created attachment 27442 [details]
possible patch

just naively implementing what I said... it should work with 4.1.1's khtml, but completely untested..
Comment 22 Germain Garand 2008-09-16 15:09:05 UTC
re #19: on second thought, I think you are right... it does make full sense to use the parent-clipped version.

and only in the overlaping case should it attempt to scroll the sub-rectangles.
Comment 23 Germain Garand 2008-09-16 18:21:30 UTC
SVN commit 861610 by ggarand:

use the widget's clip-to-parent rectangle as suggested by Manuel Massing
for catching even more cases.

further optimisations could be achieved by scrolling the subrects
in the overlapped case, but the patch would be more complex
due to the Q_WS_QWS code path interactions. Not my fight.



 M  +19 -2     0248-fix-qwidget-scroll-slowness.diff  


--- trunk/qt-copy/patches/0248-fix-qwidget-scroll-slowness.diff #861609:861610
@@ -13,12 +13,29 @@
 ===================================================================
 --- src/gui/painting/qbackingstore.cpp (révision 860438)
 +++ src/gui/painting/qbackingstore.cpp (copie de travail)
-@@ -523,7 +523,7 @@
+@@ -523,7 +523,8 @@
      }
  
      bool overlapped = false;
 -    bool accelerateScroll = accelEnv &&  isOpaque() && !(overlapped = isOverlapped(data.crect));
-+    bool accelerateScroll = accelEnv &&  isOpaque() && !(overlapped = isOverlapped(rect & data.crect));
++    QRect cRect = clipRect();
++    bool accelerateScroll = accelEnv &&  isOpaque() && !(overlapped = isOverlapped(rect & cRect));
  
  #if defined(Q_WS_QWS)
      QWSWindowSurface *surface;
+@@ -558,13 +559,13 @@
+ #ifdef Q_WS_QWS
+         QWSWindowSurface *surface = static_cast<QWSWindowSurface*>(wbs->windowSurface);
+         const QRegion clip = surface->clipRegion().translated(-toplevelOffset)
+-                             & clipRect();
++                             & cRect;
+         const QRect scrollRect = rect & clip.boundingRect();
+         const QRect destRect = scrollRect.translated(dx, dy)
+                                & scrollRect
+                                &  clip.boundingRect();
+ #else
+-        QRect scrollRect = rect & clipRect();
++        QRect scrollRect = rect & cRect;
+ 
+         QRect destRect = scrollRect.isValid() ? scrollRect.translated(dx,dy).intersected(scrollRect) : QRect();
+ 
Comment 24 Maksim Orlovich 2008-09-23 18:34:26 UTC
*** Bug 171516 has been marked as a duplicate of this bug. ***
Comment 25 Germain Garand 2008-09-24 16:09:46 UTC
*** Bug 171594 has been marked as a duplicate of this bug. ***
Comment 26 Germain Garand 2008-09-24 22:28:58 UTC
SVN commit 864463 by ggarand:

introduce some heavy hacks to avoid QWidget::scroll's performance bug
on unpatched Qt.

BUG: 167739


 M  +102 -1    khtmlview.cpp  


--- trunk/KDE/kdelibs/khtml/khtmlview.cpp #864462:864463
@@ -95,6 +95,7 @@
 //#define DEBUG_FLICKER
 
 //#define DEBUG_PIXEL
+#define FIX_QT_BROKEN_QWIDGET_SCROLL
 
 #include <limits.h>
 #ifdef Q_WS_X11
@@ -243,6 +244,14 @@
         smoothScrolling = false;
         smoothScrollModeIsDefault = true;
         shouldSmoothScroll = false;
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+        oldVScrollUpdatesEnabled = true;
+        oldHScrollUpdatesEnabled = true;
+        oldHScrollOpaquePE = false;
+        oldVScrollOpaquePE = false;
+        brokenQWidgetScroll = false;
+        shouldBeBlitting = false;
+#endif
         complete = false;
         firstLayoutPending = true;
         firstRepaintPending = true;
@@ -314,6 +323,22 @@
         smoothScrolling = true;
         smoothScrollTimer.start(sSmoothScrollTick);
         shouldSmoothScroll = false;
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+        if (view->horizontalScrollBar()->isVisible() && view->verticalScrollBar()->isVisible()) {
+            if (!dx) {
+                oldHScrollOpaquePE = view->horizontalScrollBar()->parentWidget()->testAttribute( Qt::WA_OpaquePaintEvent );
+                view->horizontalScrollBar()->parentWidget()->setAttribute( Qt::WA_OpaquePaintEvent );
+                oldHScrollUpdatesEnabled = view->horizontalScrollBar()->parentWidget()->updatesEnabled();
+                view->horizontalScrollBar()->parentWidget()->setUpdatesEnabled( false );            
+            }
+            if (!dy) {
+                oldVScrollOpaquePE = view->verticalScrollBar()->parentWidget()->testAttribute( Qt::WA_OpaquePaintEvent );
+                view->verticalScrollBar()->parentWidget()->setAttribute( Qt::WA_OpaquePaintEvent );
+                oldVScrollUpdatesEnabled = view->verticalScrollBar()->parentWidget()->updatesEnabled();
+                view->verticalScrollBar()->parentWidget()->setUpdatesEnabled( false );
+            }
+        }
+#endif
     }
 
     void stopScrolling()
@@ -326,6 +351,16 @@
         updateContentsXY();
         smoothScrolling = false;
         shouldSmoothScroll = false;
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+        if (!oldHScrollOpaquePE && view->horizontalScrollBar()->parentWidget()->testAttribute( Qt::WA_OpaquePaintEvent ))
+            view->horizontalScrollBar()->parentWidget()->setAttribute( Qt::WA_OpaquePaintEvent, false );
+        if (!oldVScrollOpaquePE && view->verticalScrollBar()->parentWidget()->testAttribute( Qt::WA_OpaquePaintEvent ))
+            view->verticalScrollBar()->parentWidget()->setAttribute( Qt::WA_OpaquePaintEvent, false );
+        if (!view->horizontalScrollBar()->parentWidget()->updatesEnabled() && oldHScrollUpdatesEnabled)
+            view->horizontalScrollBar()->parentWidget()->setUpdatesEnabled( true );
+        if (!view->verticalScrollBar()->parentWidget()->updatesEnabled() && oldVScrollUpdatesEnabled)
+            view->verticalScrollBar()->parentWidget()->setUpdatesEnabled( true );
+#endif
     }
 
     void updateContentsXY()
@@ -400,6 +435,14 @@
     bool smoothScrolling                          :1;
     bool smoothScrollModeIsDefault                :1;
     bool shouldSmoothScroll                       :1;
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    bool oldHScrollUpdatesEnabled                 :1;
+    bool oldVScrollUpdatesEnabled                 :1;
+    bool oldHScrollOpaquePE                       :1;
+    bool oldVScrollOpaquePE                       :1;
+    bool brokenQWidgetScroll                      :1;
+    bool shouldBeBlitting                         :1;
+#endif
     bool complete                              :1;
     bool firstLayoutPending                    :1;
     bool firstRepaintPending                    :1;
@@ -686,6 +729,10 @@
     widget()->resize(w, h);
     if (!widget()->isVisible())
         updateScrollBars();
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    if (!horizontalScrollBar()->isVisible() || !verticalScrollBar()->isVisible())
+        d->brokenQWidgetScroll = false;
+#endif
 }
 
 int KHTMLView::contentsX() const
@@ -874,6 +921,13 @@
 
     if (!r.isValid() || r.isEmpty()) return;
 
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    if (d->shouldBeBlitting && r.width() == v.width() && r.height() == v.height()) {
+        d->brokenQWidgetScroll = true;
+    }
+    d->shouldBeBlitting = false;
+#endif
+
     if (d->haveZoom()) {
         p.scale( d->zoomLevel/100., d->zoomLevel/100.);
 
@@ -3852,6 +3906,17 @@
             w = v->widget();
     }
 
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    bool hideScrollBars = false;
+    if (horizontalScrollBar()->isVisible() && verticalScrollBar()->isVisible()) {
+        if (!d->brokenQWidgetScroll) {
+            d->shouldBeBlitting = true;
+        } else {
+            hideScrollBars = true;
+        }
+    }
+#endif
+
     if ( d->staticWidget ) {
 
         // now remove from view the external widgets that must have completely
@@ -3861,10 +3926,20 @@
 
         if ( d->staticWidget == KHTMLViewPrivate::SBPartial
                                 && m_part->xmlDocImpl() && m_part->xmlDocImpl()->renderer() ) {
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+            if (hideScrollBars) {
+                horizontalScrollBar()->parentWidget()->lower();
+                verticalScrollBar()->parentWidget()->lower();
+            }
+#endif
             // static objects might be selectively repainted, like stones in flowing water
             QRegion r = static_cast<RenderCanvas*>(m_part->xmlDocImpl()->renderer())->staticRegion();
             r.translate( -contentsX(), -contentsY());
             QVector<QRect> ar = r.rects();
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+            if (ar.size() == 1 && ar[0].width() >= visibleWidth() && ar[0].height() >= visibleHeight())
+                d->shouldBeBlitting = false;
+#endif
             for (int i = 0; i < ar.size() ; ++i) {
                 widget()->update( ar[i] );
             }
@@ -3873,19 +3948,45 @@
             for (int i = 0; i < ar.size() ; ++i) {
                 w->scroll( dx, dy, ar[i].translated(off) );
             }
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+           if (hideScrollBars) {
+                horizontalScrollBar()->parentWidget()->raise();
+                verticalScrollBar()->parentWidget()->raise();
+           }
+#endif
             d->scrollExternalWidgets(dx, dy);
-        } else
+        } else {
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+            d->shouldBeBlitting = false;
+#endif
             // we can't avoid a full update
             widget()->update();
+        }
         return;
     }
     if (d->firstRepaintPending)
         return;
+
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    if (hideScrollBars) {
+        horizontalScrollBar()->parentWidget()->lower();
+        verticalScrollBar()->parentWidget()->lower();
+    }
+#endif
+
     if (m_kwp->isRedirected()) {
         w->scroll(dx, dy, QRect(off.x(), off.y(), visibleWidth(), visibleHeight()));
     }  else {
         widget()->scroll(dx, dy, widget()->rect() & viewport()->rect());
     }
+
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    if (hideScrollBars) {
+        horizontalScrollBar()->parentWidget()->raise();
+        verticalScrollBar()->parentWidget()->raise();
+    }
+#endif
+
     d->scrollExternalWidgets(dx, dy);
 }
 
Comment 27 auxsvr 2008-10-01 10:41:05 UTC
Regarding scrolling problems, has anyone else seen konqueror scroll beyond the limits of the web page? I start scrolling to the right for a while, then it starts scrolling to the left on its own, beyond the web page area and doesn't stop. The window becomes garbled and useless after that.
Comment 28 René Krell 2008-10-01 17:50:10 UTC
(In reply to comment #27)
> Regarding scrolling problems, has anyone else seen konqueror scroll beyond the
> limits of the web page? I start scrolling to the right for a while, then it
> starts scrolling to the left on its own, beyond the web page area and doesn't
> stop. The window becomes garbled and useless after that.
> 

Yes, I have seen such an effect accidentally, but cannot reproduce it surely.
Exactly what you say, even in KDE 4.1.1. I'm not sure whether a bug report exists for that, but not being able to reproduce it will be hard to report it.
Comment 29 Maksim Orlovich 2008-10-01 17:59:44 UTC
That sounds a bit like bug #166870, which is fixed in 4.1.2
Comment 30 René Krell 2008-10-01 18:08:04 UTC
(In reply to comment #29)
> That sounds a bit like bug #166870, which is fixed in 4.1.2

Yes, sounds like this. I don't know, how I run into that, but it didn't even stop, there were only more and more lines on the representation area, and it looked "scrambled" more and more. Not nice - I hope this is fixed, let's look forward to 4.1.2 :-)
Comment 31 auxsvr 2008-10-01 22:46:11 UTC
Yes, it's bug #166870, thanks. Good to see it's already been fixed.