Bug 167739

I can't reproduce that with the same version, also from Kubuntu packages.
Does this occure with all websites?
Konqueror did not use more then 20% on one core (Intel Core Duo T2400) auto-scrolling some regular page (like bugs.kde.org).
Auto-scrolling on a site containing flash didn't caused more then 50% CPU load.

On Wednesday 30 July 2008, Dominik Tritscher wrote:
> I can't reproduce that with the same version, also from Kubuntu packages.
> Does this occure with all websites?
> Konqueror did not use more then 20% on one core (Intel Core Duo T2400) 

auto-scrolling some regular page (like bugs.kde.org).
> Auto-scrolling on a site containing flash didn't caused more then 50% CPU 

load.

Thanks for trying to reproduce, I provide as much info below as possible.

# Setup

Kubuntu 8.04 with PPA debs for KDE 4.1

Konqueror
 *** 4:3.5.9-0ubuntu7.2 0
 *** 4:4.1.0-0ubuntu1~hardy1~ppa2

module: i915
direct rendering: Yes

# Testing

URL = Most sever bugs on bugs.kde.org

Konq 3.5.9 :~10% on oneCPU
Konq 4.1.0:~60% on one CPU
 
This happens under 3.5.9 or 4.1 KDE window manager.

Did you try changing the smooth scrolling setting in the Konqueror Web Browsing settings? I read in some bug reports that disabling smooth scrolling made scrolling smoother ( or less cpu intensive ). It could depend on the hardware.

On Thursday 31 July 2008, Rui G. wrote:
> Did you try changing the smooth scrolling setting in the Konqueror Web 

Browsing settings? I read in some bug reports that disabling smooth 
scrolling made scrolling smoother ( or less cpu intensive ). It could 
depend on the hardware.

Great, I disliked smooth scrolling and didn't know I could turn it off. But 
sadly it does nothing to the disprortionate increase in CPU use from KDE 
3.5.9 .

Do the pages you have problems on have 2 scrollbars for you?

On Thursday 31 July 2008, Maksim Orlovich wrote:
> ------- Additional Comments From maksim kde org  2008-07-31 19:06 -------
> Do the pages you have problems on have 2 scrollbars for you?


A single vertical scroll bar on the right side of [1].

[1]:http://bugs.kde.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_severity=critical&bug_severity=grave&bug_severity=major&order=bugs.bug_severity

Because of issues with KDE 4.1, I'm trying running 3.5.9 with kwin4.1 using kwin --replace. I notice that even with everything else 3.5.9 the high CPU happens when I scroll in Konqueror. So perhaps this bug should be reassigned to kwin.

Are you using composite ("Desktop effects") perhaps?

On Monday 04 August 2008, Maksim Orlovich wrote:
> ------- Additional Comments From maksim kde org  2008-08-04 15:56 -------
> Are you using composite ("Desktop effects") perhaps?


Yes, the KDE 4.1 windows manager was using them.

Disabling them might help --- at the very least it avoids one more layer of buffering/one more copy, and puts lets pressure on GPU memory (which KDE4 is a real pig about, too, I should go back to improving that, really...).

I can confirm excessive CPU usage by Xorg while scrolling vertically, 
if and only if a horizontal scrollbar is present. It is reproducable
whether smooth scrolling is enabled or not. Compositing effects are
disabled. KDE 4.1, debian packages from experimental, Xorg 7.3, xserver
1.4.2, QT 4.4.0

E.g.: Opening "Most hated bugs" page from the KDE bug tracker. This 
does not fit on my 1280x1024 screen, so a horizontal scroll bar is
present. Scrolling using the mouse wheel or scrollbar results in 
100% cpu usage and very slow and jerky response.

If I resize konqueror to accomodate the horizontal span of the same 
document (i.e. the horizontal scrollbar disappears), Xorg cpu usage is 
about 20%, reaction is fluid.

Caveat: I'm running nvidia proprietary driver version 177.68, GeForce 7600 Go, but I am not sure this is an nvidia issue (no performance problems otherwise).

Attached are oprofiles of Xorg, in both cases konqueror scrolling through the same webpage ("Most hated bugs") for 60 seconds, with (xorg-slow-scrolling.log) and without (xorg-fast-scrolling.log) horizontal scrollbars. Note the factor 10 difference in the
number of samples at fbCompositeSolidMask_nx8888x8888Cmmx, which indicates that a different (and inefficient) code path is taken (be it in the driver, Xorg, QT or KDE) when a horizontal scrollbar is present. 

I doubt this is konqueror specific.

May be related to http://bugs.kde.org/show_bug.cgi?id=163626

Created attachment 27001 [details]
Oprofile log, scrolling through webpage without horizontal scrollbar

Created attachment 27002 [details]
Oprofile log, scrolling through webpage with horizontal scrollbar

Sorry, misread the initial bug report, I could only reproduce high
cpu usage in the Xorg xserver, not in konqueror.

Thanks for the analysis. I am actually aware of the 2 scrollbar case (hence the comment #5) --- the  problem there is that the QWidget::scroll function actually forces the entire view to repaint, instead of moving the rendered picture in the 
backbuffer and repainting a small stripe, since it gets worried about the /other/ scrollbar getting in the way. I don't know how to fix this yet,
though, especially since it's tied with the Qt internals.

That fbCompositeSolidMask_nx8888x8888Cmmx is hot is interesting, though.

As for NVidia drivers, it's mostly a problem with 8600, which shows much worse performance, sometimes visibly slow performance than far cheaper and far older hardware (e.g. my 7100).

Sorry to follow up so late:

fbCompositeSolidMask_nx8888x8888Cmmx apparently is the libfb software fallback taken for anti-aliased font rendering, but the newer nvidia 177.70 beta driver takes a hardware accelerated rendering path again.

However, performance is still extremly sluggish (saturating my Core Duo 1.66),
and these fundamental repainting inefficiencies you explained seem to  affect multiple parts of KDE (essentially all webpages viewed through akregator, mail scrolling in kmail and the 2-scrollbar case in konqueror).

Do you think this should be filed as a QT bug?

SVN commit 861059 by ggarand:

QWidgetPrivate::scrollRect doesn't account for it's rect argument
but rather examine the whole widget surface when pondering if
it can apply accelerated blitting, resulting in unnecessarily slow
and non-blitting QWidget::scroll calls.

CCBUG: 167739



 A             0248-fix-qwidget-scroll-slowness.diff  
 M  +1 -1      README  


--- trunk/qt-copy/patches/README #861058:861059
@@ -1,5 +1,5 @@
 Please assign the numbers incrementally, and don't reuse them. The next one:
-#0248
+#0249
 
 This directory contains patches for Qt that haven't been accepted by TrollTech
 yet. All patches in this directory itself shouldn't make qt-copy incompatible

Just a quick update: I tested the scrollRect patch on KDE 4.1.1, but
unfortunately scrolling with two scrollbars in konqueror is still a lot slower
than with a single-scrollbar - I haven't found the time to do any real
debugging/profiling with the patch yet, though.

Hmm, with 2 scrollbars, I get QWidgetPrivate->data.crect == rect (apparently the size of the scroll area), so the patch #248 doesn't affect the code path.

I tentatively changed patch #248 to use clipRect() (parent-clipped version
of data.crect) instead of data.crect for intersection, which seems in line with
the patch intent and yields fast scrolling - mind you, just a guess, as I 
know nothing about qt internals. In this scenario it seems to behave correctly.

In QWidgetPrivate::scrollRect (this=0x9a61870, rect=@0xbfc051f0, dx=0, dy=-10):

(gdb) print clipRect()
$10 = {x1 = 0, y1 = 0, x2 = 1250, y2 = 771}
(gdb) print data.crect
$11 = {x1 = 0, y1 = 0, x2 = 2034, y2 = 6036}
(gdb) print rect
$12 = (const QRect &) @0xbfc051f0: {x1 = 0, y1 = 0, x2 = 2034, y2 = 6036}

no, you need also tip of branch (r861088).
I only fixed the most glaring bug, but scrollRect() remains severly under-optimised and doesn't deal correctly with parent widgets overing

it should e.g substract them, invalidate them, then iterate over the rects() of the remaining region, recursively scrolling them

> I tentatively changed patch #248 to use clipRect() (parent-clipped version
> of data.crect) instead of data.crect for intersection, which seems in line
> with the patch intent 

I think they meant scrollRect to invalidate all that is in the passed rectangle, so this wouldn't do.

Created attachment 27442 [details]
possible patch

just naively implementing what I said... it should work with 4.1.1's khtml, but completely untested..

re #19: on second thought, I think you are right... it does make full sense to use the parent-clipped version.

and only in the overlaping case should it attempt to scroll the sub-rectangles.

SVN commit 861610 by ggarand:

use the widget's clip-to-parent rectangle as suggested by Manuel Massing
for catching even more cases.

further optimisations could be achieved by scrolling the subrects
in the overlapped case, but the patch would be more complex
due to the Q_WS_QWS code path interactions. Not my fight.



 M  +19 -2     0248-fix-qwidget-scroll-slowness.diff  


--- trunk/qt-copy/patches/0248-fix-qwidget-scroll-slowness.diff #861609:861610
@@ -13,12 +13,29 @@
 ===================================================================
 --- src/gui/painting/qbackingstore.cpp (révision 860438)
 +++ src/gui/painting/qbackingstore.cpp (copie de travail)
-@@ -523,7 +523,7 @@
+@@ -523,7 +523,8 @@
      }
  
      bool overlapped = false;
 -    bool accelerateScroll = accelEnv &&  isOpaque() && !(overlapped = isOverlapped(data.crect));
-+    bool accelerateScroll = accelEnv &&  isOpaque() && !(overlapped = isOverlapped(rect & data.crect));
++    QRect cRect = clipRect();
++    bool accelerateScroll = accelEnv &&  isOpaque() && !(overlapped = isOverlapped(rect & cRect));
  
  #if defined(Q_WS_QWS)
      QWSWindowSurface *surface;
+@@ -558,13 +559,13 @@
+ #ifdef Q_WS_QWS
+         QWSWindowSurface *surface = static_cast<QWSWindowSurface*>(wbs->windowSurface);
+         const QRegion clip = surface->clipRegion().translated(-toplevelOffset)
+-                             & clipRect();
++                             & cRect;
+         const QRect scrollRect = rect & clip.boundingRect();
+         const QRect destRect = scrollRect.translated(dx, dy)
+                                & scrollRect
+                                &  clip.boundingRect();
+ #else
+-        QRect scrollRect = rect & clipRect();
++        QRect scrollRect = rect & cRect;
+ 
+         QRect destRect = scrollRect.isValid() ? scrollRect.translated(dx,dy).intersected(scrollRect) : QRect();
+ 

*** Bug 171516 has been marked as a duplicate of this bug. ***

*** Bug 171594 has been marked as a duplicate of this bug. ***

SVN commit 864463 by ggarand:

introduce some heavy hacks to avoid QWidget::scroll's performance bug
on unpatched Qt.

BUG: 167739


 M  +102 -1    khtmlview.cpp  


--- trunk/KDE/kdelibs/khtml/khtmlview.cpp #864462:864463
@@ -95,6 +95,7 @@
 //#define DEBUG_FLICKER
 
 //#define DEBUG_PIXEL
+#define FIX_QT_BROKEN_QWIDGET_SCROLL
 
 #include <limits.h>
 #ifdef Q_WS_X11
@@ -243,6 +244,14 @@
         smoothScrolling = false;
         smoothScrollModeIsDefault = true;
         shouldSmoothScroll = false;
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+        oldVScrollUpdatesEnabled = true;
+        oldHScrollUpdatesEnabled = true;
+        oldHScrollOpaquePE = false;
+        oldVScrollOpaquePE = false;
+        brokenQWidgetScroll = false;
+        shouldBeBlitting = false;
+#endif
         complete = false;
         firstLayoutPending = true;
         firstRepaintPending = true;
@@ -314,6 +323,22 @@
         smoothScrolling = true;
         smoothScrollTimer.start(sSmoothScrollTick);
         shouldSmoothScroll = false;
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+        if (view->horizontalScrollBar()->isVisible() && view->verticalScrollBar()->isVisible()) {
+            if (!dx) {
+                oldHScrollOpaquePE = view->horizontalScrollBar()->parentWidget()->testAttribute( Qt::WA_OpaquePaintEvent );
+                view->horizontalScrollBar()->parentWidget()->setAttribute( Qt::WA_OpaquePaintEvent );
+                oldHScrollUpdatesEnabled = view->horizontalScrollBar()->parentWidget()->updatesEnabled();
+                view->horizontalScrollBar()->parentWidget()->setUpdatesEnabled( false );            
+            }
+            if (!dy) {
+                oldVScrollOpaquePE = view->verticalScrollBar()->parentWidget()->testAttribute( Qt::WA_OpaquePaintEvent );
+                view->verticalScrollBar()->parentWidget()->setAttribute( Qt::WA_OpaquePaintEvent );
+                oldVScrollUpdatesEnabled = view->verticalScrollBar()->parentWidget()->updatesEnabled();
+                view->verticalScrollBar()->parentWidget()->setUpdatesEnabled( false );
+            }
+        }
+#endif
     }
 
     void stopScrolling()
@@ -326,6 +351,16 @@
         updateContentsXY();
         smoothScrolling = false;
         shouldSmoothScroll = false;
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+        if (!oldHScrollOpaquePE && view->horizontalScrollBar()->parentWidget()->testAttribute( Qt::WA_OpaquePaintEvent ))
+            view->horizontalScrollBar()->parentWidget()->setAttribute( Qt::WA_OpaquePaintEvent, false );
+        if (!oldVScrollOpaquePE && view->verticalScrollBar()->parentWidget()->testAttribute( Qt::WA_OpaquePaintEvent ))
+            view->verticalScrollBar()->parentWidget()->setAttribute( Qt::WA_OpaquePaintEvent, false );
+        if (!view->horizontalScrollBar()->parentWidget()->updatesEnabled() && oldHScrollUpdatesEnabled)
+            view->horizontalScrollBar()->parentWidget()->setUpdatesEnabled( true );
+        if (!view->verticalScrollBar()->parentWidget()->updatesEnabled() && oldVScrollUpdatesEnabled)
+            view->verticalScrollBar()->parentWidget()->setUpdatesEnabled( true );
+#endif
     }
 
     void updateContentsXY()
@@ -400,6 +435,14 @@
     bool smoothScrolling                          :1;
     bool smoothScrollModeIsDefault                :1;
     bool shouldSmoothScroll                       :1;
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    bool oldHScrollUpdatesEnabled                 :1;
+    bool oldVScrollUpdatesEnabled                 :1;
+    bool oldHScrollOpaquePE                       :1;
+    bool oldVScrollOpaquePE                       :1;
+    bool brokenQWidgetScroll                      :1;
+    bool shouldBeBlitting                         :1;
+#endif
     bool complete                              :1;
     bool firstLayoutPending                    :1;
     bool firstRepaintPending                    :1;
@@ -686,6 +729,10 @@
     widget()->resize(w, h);
     if (!widget()->isVisible())
         updateScrollBars();
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    if (!horizontalScrollBar()->isVisible() || !verticalScrollBar()->isVisible())
+        d->brokenQWidgetScroll = false;
+#endif
 }
 
 int KHTMLView::contentsX() const
@@ -874,6 +921,13 @@
 
     if (!r.isValid() || r.isEmpty()) return;
 
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    if (d->shouldBeBlitting && r.width() == v.width() && r.height() == v.height()) {
+        d->brokenQWidgetScroll = true;
+    }
+    d->shouldBeBlitting = false;
+#endif
+
     if (d->haveZoom()) {
         p.scale( d->zoomLevel/100., d->zoomLevel/100.);
 
@@ -3852,6 +3906,17 @@
             w = v->widget();
     }
 
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    bool hideScrollBars = false;
+    if (horizontalScrollBar()->isVisible() && verticalScrollBar()->isVisible()) {
+        if (!d->brokenQWidgetScroll) {
+            d->shouldBeBlitting = true;
+        } else {
+            hideScrollBars = true;
+        }
+    }
+#endif
+
     if ( d->staticWidget ) {
 
         // now remove from view the external widgets that must have completely
@@ -3861,10 +3926,20 @@
 
         if ( d->staticWidget == KHTMLViewPrivate::SBPartial
                                 && m_part->xmlDocImpl() && m_part->xmlDocImpl()->renderer() ) {
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+            if (hideScrollBars) {
+                horizontalScrollBar()->parentWidget()->lower();
+                verticalScrollBar()->parentWidget()->lower();
+            }
+#endif
             // static objects might be selectively repainted, like stones in flowing water
             QRegion r = static_cast<RenderCanvas*>(m_part->xmlDocImpl()->renderer())->staticRegion();
             r.translate( -contentsX(), -contentsY());
             QVector<QRect> ar = r.rects();
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+            if (ar.size() == 1 && ar[0].width() >= visibleWidth() && ar[0].height() >= visibleHeight())
+                d->shouldBeBlitting = false;
+#endif
             for (int i = 0; i < ar.size() ; ++i) {
                 widget()->update( ar[i] );
             }
@@ -3873,19 +3948,45 @@
             for (int i = 0; i < ar.size() ; ++i) {
                 w->scroll( dx, dy, ar[i].translated(off) );
             }
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+           if (hideScrollBars) {
+                horizontalScrollBar()->parentWidget()->raise();
+                verticalScrollBar()->parentWidget()->raise();
+           }
+#endif
             d->scrollExternalWidgets(dx, dy);
-        } else
+        } else {
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+            d->shouldBeBlitting = false;
+#endif
             // we can't avoid a full update
             widget()->update();
+        }
         return;
     }
     if (d->firstRepaintPending)
         return;
+
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    if (hideScrollBars) {
+        horizontalScrollBar()->parentWidget()->lower();
+        verticalScrollBar()->parentWidget()->lower();
+    }
+#endif
+
     if (m_kwp->isRedirected()) {
         w->scroll(dx, dy, QRect(off.x(), off.y(), visibleWidth(), visibleHeight()));
     }  else {
         widget()->scroll(dx, dy, widget()->rect() & viewport()->rect());
     }
+
+#ifdef FIX_QT_BROKEN_QWIDGET_SCROLL
+    if (hideScrollBars) {
+        horizontalScrollBar()->parentWidget()->raise();
+        verticalScrollBar()->parentWidget()->raise();
+    }
+#endif
+
     d->scrollExternalWidgets(dx, dy);
 }
 

Regarding scrolling problems, has anyone else seen konqueror scroll beyond the limits of the web page? I start scrolling to the right for a while, then it starts scrolling to the left on its own, beyond the web page area and doesn't stop. The window becomes garbled and useless after that.

(In reply to comment #27)
> Regarding scrolling problems, has anyone else seen konqueror scroll beyond the
> limits of the web page? I start scrolling to the right for a while, then it
> starts scrolling to the left on its own, beyond the web page area and doesn't
> stop. The window becomes garbled and useless after that.
> 

Yes, I have seen such an effect accidentally, but cannot reproduce it surely.
Exactly what you say, even in KDE 4.1.1. I'm not sure whether a bug report exists for that, but not being able to reproduce it will be hard to report it.

That sounds a bit like bug #166870, which is fixed in 4.1.2

(In reply to comment #29)
> That sounds a bit like bug #166870, which is fixed in 4.1.2

Yes, sounds like this. I don't know, how I run into that, but it didn't even stop, there were only more and more lines on the representation area, and it looked "scrambled" more and more. Not nice - I hope this is fixed, let's look forward to 4.1.2 :-)

Yes, it's bug #166870, thanks. Good to see it's already been fixed.