Bug 331156 - Display of HTML-Message extremely slow
Summary: Display of HTML-Message extremely slow
Status: RESOLVED FIXED
Alias: None
Product: kmail2
Classification: Applications
Component: general (show other bugs)
Version: 4.12.2
Platform: unspecified Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 330664 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-02-15 14:13 UTC by Axel Braun
Modified: 2014-07-31 11:57 UTC (History)
11 users (show)

See Also:
Latest Commit:
Version Fixed In: 4.13.3


Attachments
Example mail that loads extremely slow (189.65 KB, application/mbox)
2014-02-15 14:15 UTC, Axel Braun
Details
gdb backtrace whilst opening Axel's 'example mail' (30.80 KB, text/plain)
2014-02-23 10:30 UTC, Jon Moon
Details
Backtrace from opening one of my problem e-mails (52.86 KB, text/plain)
2014-02-23 10:31 UTC, Jon Moon
Details
sample mail causing cpu/ram spike and time lag (298.41 KB, text/plain)
2014-04-23 19:26 UTC, Orion
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Axel Braun 2014-02-15 14:13:40 UTC
I run KDE 4.12.2 on ThinkPad T520 (core-i7), OS: openSUSE 13.1 (x86_64) with 8GB RAM and a 1 TB SSD - so in general a quick system.

But displaying a HTML mail takes about 5-8 seconds until it displays (not even loading external references and pictures). I switched Nepomuk EMail-Indexing off, but the result remains the same. This is quite annoying....

Reproducible: Always

Steps to Reproduce:
1. display a HTML mail
2.
3.
Comment 1 Axel Braun 2014-02-15 14:15:12 UTC
Created attachment 85164 [details]
Example mail that loads extremely slow
Comment 2 Laurent Montel 2014-02-15 14:40:36 UTC
work fine here
Do you have specific option ? Adblock activate ?
etc.
Comment 3 Axel Braun 2014-02-15 16:15:48 UTC
Adblocker activated in Konqueror? No 
Special settings? not AFAIK
BTW, this is one of the messages that does not load external content, although the sender is added to the addressbook and external content is permitted
Comment 4 T Kleindienst 2014-02-18 12:26:19 UTC
Same behavior over here. html mail takes about 4-6 seconds to display.
Opensuse 13.1. Checked with KDE 4.11 and 4.12 (same results)

remark: $home is on NFS
Comment 5 Christux 2014-02-19 21:10:30 UTC
Same here! KMAIL needs several seconds to display messages as HTML! 
CPU usage is 100% while loading and kmail is unresponsive to user input.
I'm using KDE 4.12.2!

Turning of file indexer did not help, deleting kmail related files did not help!

Any solution proposals?

Thank you for your help!
Comment 6 Laurent Montel 2014-02-20 06:16:09 UTC
no solution because I can not reproduce it.
I need to know why.
Try to attach gdb to kmail display a message, make a break in gdb and look at what kmail does.
Regards
Comment 7 Axel Braun 2014-02-20 07:30:26 UTC
(In reply to comment #6)
> no solution because I can not reproduce it.
> I need to know why.

Sure.....

> Try to attach gdb to kmail display a message, make a break in gdb and look
> at what kmail does.

...but I have no idea about gdb. I can offer you a teamviewer session on my PC, if that helps.
Comment 8 Laurent Montel 2014-02-21 06:26:39 UTC
Teamviewer can be help us to debug it indeed.
Comment 9 Jon Moon 2014-02-23 10:27:59 UTC
I was just about to raise a similar bug and came across this one which I believe to be the same.

In my case some e-mails, typically with a lot of nested HTML quotes from previous e-mails in the thread, can cause Kmail to lock up (w. CPU at 100%) for extremely long periods of time - in extreme cases order of a minute - effectively making kmail unusable (especially when I am trying to quickly read lots of replies on the same thread!).  This is not a new behaviour ,but has got a lot worse recently (likely changing factor s  the nature of the e-mails I'm receiving).

With the sample mail above, then similar to Axel it takes ~5 seconds to open.

I've tried gdb on both my case and the test mail above - and the backtrace is basically the same, with the pertinent bit of the backtrace being -

<79 frames of recursion into JSC::Bindings::convertValueToQVariant>
#80 0x00007fef7d771b9f in JSC::Bindings::convertValueToQVariant(OpaqueJSContext const*, OpaqueJSValue const*, QMetaType::Type, int*, WTF::HashSet<OpaqueJSValue*, WTF::PtrHash<OpaqueJSValue*>, WTF::HashTraits<OpaqueJSValue*> >*, int, OpaqueJSValue const**) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#81 0x00007fef7d771b9f in JSC::Bindings::convertValueToQVariant(OpaqueJSContext const*, OpaqueJSValue const*, QMetaType::Type, int*, WTF::HashSet<OpaqueJSValue*, WTF::PtrHash<OpaqueJSValue*>, WTF::HashTraits<OpaqueJSValue*> >*, int, OpaqueJSValue const**) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#82 0x00007fef7d774317 in JSC::Bindings::convertValueToQVariant(OpaqueJSContext const*, OpaqueJSValue const*, QMetaType::Type, int*, OpaqueJSValue const**) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#83 0x00007fef7d6698a7 in QWebFrame::evaluateJavaScript(QString const&) () from /usr/lib/x86_64-linux-gnu/libQtWebKit.so.4
#84 0x00007fef842adf77 in MessageViewer::HTMLQuoteColorer::process (this=this@entry=0x7fff80c51a20, htmlSource=..., extraHead=...) at ../../messageviewer/viewer/htmlquotecolorer.cpp:133
#85 0x00007fef842bde78 in MessageViewer::ObjectTreeParser::processTextHtmlSubtype (this=0x7fff80c51bf0, curNode=0x19e5f430) at ../../messageviewer/viewer/objecttreeparser.cpp:1229

In both cases a call from MessageViewer::HTMLQuoteColorer into QWebFrame::evaluateJavaScript is generating a deep recursion into JSC::Bindings::convertValueToQVariant.

I am suspicious that this is related to the following upstream bug -

https://bugs.webkit.org/show_bug.cgi?id=39717

However there's not enough information on that ticket to be sure, and either way its old, has been closed as invalid and certainly doesn't have traction.

I will attach both backtraces.  I'm afraid I can't provide my problem e-mails for confidentiality reasons.

If there's anything else I can do to help let me know - I would really like to see this fixed, as it really affects my work.

As the problem functionality is conditional on KDEPIM being compiled with Webkit functionality (#ifndef KDEPIM_NO_WEBKIT) then as a workaround I'll investigate getting a version of KDEPIM installed with that functionality disabled.
Comment 10 Jon Moon 2014-02-23 10:30:33 UTC
Created attachment 85283 [details]
gdb backtrace whilst opening Axel's 'example mail'

See 'Jon Moon 2014-02-23 10:27:59 UTC' for more context.
Comment 11 Jon Moon 2014-02-23 10:31:47 UTC
Created attachment 85284 [details]
Backtrace from opening one of my problem e-mails

See 'Jon Moon 2014-02-23 10:27:59 UTC' for more context
Comment 12 Axel Braun 2014-04-07 07:39:31 UTC
As the number of cc's rises - anything new on this bug?
Comment 13 Orion 2014-04-23 15:23:17 UTC
I confirm this bug on KDE 14.3 with lag of +5 seconds + very high cpu + large usage of RAM >500 Mb and growing.

remove "html mail" preference and all works well - even if you set a contact/sender (in Kaddressbook) to be allowed html mail format instead of default ... the lag then totally disappears with said user(s) even though the mail is shown in HTML. 

further the RAM usage then stays around 115 Mb
Comment 14 Orion 2014-04-23 15:25:14 UTC
I should add I am on Kubuntu 14.04 
and that I suspect this bug to be a duplicate of 330664
Comment 15 Orion 2014-04-23 19:26:35 UTC
Created attachment 86232 [details]
sample mail causing cpu/ram spike and time lag

after more experimenting: the culprit seems to be a certain type of html mail that, at least in my version, just shows up as a link to a webpage unless you have html mail set to default. 

If I set Kmail to "non html" as default BUT use Kaddressbook/Contact to allow the sender of the attached mail in HTML format - then I see the CPU spike and rise in RAM once I click on this mail.
Comment 16 Axel Braun 2014-04-26 18:47:33 UTC
(In reply to comment #13)

> remove "html mail" preference and all works well - even if you set a

What do you mean with 'hmtl-mail'? The setting `Folder -> prefer HTML instead of Text' ?
That leaves mostly an unreadable HTML-mail OR a reference to an external website (Newsletters mostly)
Comment 17 Orion 2014-04-27 06:16:17 UTC
(In reply to comment #16)
> (In reply to comment #13)
> 
> > remove "html mail" preference and all works well - even if you set a
> 
> What do you mean with 'hmtl-mail'? The setting `Folder -> prefer HTML
> instead of Text' ?
> That leaves mostly an unreadable HTML-mail OR a reference to an external
> website (Newsletters mostly)
Yes that is what I mean. Those mail where you get a "click here to view mail" option then works fine, others as you say just point to a web site. It is NOT a good workflow at all but the only way I found to avoid getting the cpu spikes, viewing lag and RAM growth
Comment 18 Axel Braun 2014-04-27 16:43:30 UTC
problem persists in 4.13
Comment 19 Beguam 2014-05-03 15:33:21 UTC
Hello,

Simply try with disabling AdBlock in Kmail : Configure (or Settings ? my system is in French) > Configure Kmail > Security > AdBlock.

It works much more faster for me.
Comment 20 Axel Braun 2014-05-03 16:57:19 UTC
(In reply to comment #19)
 
> Simply try with disabling AdBlock in Kmail : Configure (or Settings ? my
> system is in French) > Configure Kmail > Security > AdBlock.

It was always disabled in my case
Comment 21 Orion 2014-05-04 07:17:06 UTC
I have never used the AdBlock function either so that is not what is causing the performance problem. It is quite clear that it is related to "totally html" mail e.g newsletters
Comment 22 Jon Moon 2014-05-26 18:10:53 UTC
Consistent with my earlier analysis, following patch does seem to work around the problem.  Not sure what is lost by not having the html quote coloring.

jmoon@jmoon-w530:~/kde$ diff -c10 kdepim-4.13.1{-orig,}/messageviewer/viewer/htmlquotecolorer.cpp 
*** kdepim-4.13.1-orig/messageviewer/viewer/htmlquotecolorer.cpp        2014-05-07 11:01:39.000000000 +0100
--- kdepim-4.13.1/messageviewer/viewer/htmlquotecolorer.cpp     2014-05-26 18:41:06.013342901 +0100
***************
*** 26,45 ****
--- 26,46 ----
  #include <QWebPage>
  #include <QWebFrame>
  #include <QWebElement>
  #endif
  
  using namespace MessageViewer;
  
  HTMLQuoteColorer::HTMLQuoteColorer()
  {
  }
+ #define KDEPIM_NO_WEBKIT
  
  QString HTMLQuoteColorer::process( const QString &htmlSource, QString&extraHead )
  {
  #ifndef KDEPIM_NO_WEBKIT
      // Create a DOM Document from the HTML source
      QWebPage page(0);
      page.settings()->setAttribute( QWebSettings::JavascriptEnabled, false );
      page.settings()->setAttribute( QWebSettings::JavaEnabled, false );
      page.settings()->setAttribute( QWebSettings::PluginsEnabled, false );
Comment 23 Christux 2014-05-26 18:52:46 UTC
(In reply to comment #22)
> Consistent with my earlier analysis, following patch does seem to work
> around the problem.  Not sure what is lost by not having the html quote
> coloring.
> 
> jmoon@jmoon-w530:~/kde$ diff -c10
> kdepim-4.13.1{-orig,}/messageviewer/viewer/htmlquotecolorer.cpp 
> *** kdepim-4.13.1-orig/messageviewer/viewer/htmlquotecolorer.cpp       
> 2014-05-07 11:01:39.000000000 +0100
> --- kdepim-4.13.1/messageviewer/viewer/htmlquotecolorer.cpp     2014-05-26
> 18:41:06.013342901 +0100
> ***************
> *** 26,45 ****
> --- 26,46 ----
>   #include <QWebPage>
>   #include <QWebFrame>
>   #include <QWebElement>
>   #endif
>   
>   using namespace MessageViewer;
>   
>   HTMLQuoteColorer::HTMLQuoteColorer()
>   {
>   }
> + #define KDEPIM_NO_WEBKIT
>   
>   QString HTMLQuoteColorer::process( const QString &htmlSource,
> QString&extraHead )
>   {
>   #ifndef KDEPIM_NO_WEBKIT
>       // Create a DOM Document from the HTML source
>       QWebPage page(0);
>       page.settings()->setAttribute( QWebSettings::JavascriptEnabled, false
> );
>       page.settings()->setAttribute( QWebSettings::JavaEnabled, false );
>       page.settings()->setAttribute( QWebSettings::PluginsEnabled, false );

Hey John,

I also have this issues and was never able to fix it, I have tried everything from removing all 
kde settings to disabling some settings as mentioned by the others. Anyway, I guess you have 
build KMail with this patch on your machine. I would really love to test this patch as well to 
so if it fixes the problem, could you tell me which libraries do I have to install to build kmail
from scratch with this patch applied, a link to a "how to compile KDE Kmail" would be sufficient  :-).

Thank you, Christian
Comment 24 Axel Braun 2014-05-26 20:20:11 UTC
(In reply to comment #23)

> I guess you have 
> build KMail with this patch on your machine. I would really love to test
> this patch as well to 
> so if it fixes the problem, could you tell me which libraries do I have to
> install to build kmail
> from scratch with this patch applied, a link to a "how to compile KDE Kmail"
> would be sufficient  :-).

Easiest is probably if you branch KMail in the build service and apply the patch.
Comment 25 Laurent Montel 2014-06-25 06:05:42 UTC
Git commit b164601eb63eb62d3aa439c506072a0bbedeff74 by Montel Laurent.
Committed on 25/06/2014 at 06:03.
Pushed by mlaurent into branch 'KDE/4.13'.

Fix Bug 331156 - Display of HTML-Message extremely slow

FIXED-IN: 4.13.3

It's not a fix because it's in webkit the problem and there will not new update of webkit.
I added a variable to disable/enable htmlquote (it's disable by default).
=> we will lose html quoted color :(
Too bad.
I will add a gui in 4.14 to configure it.

M  +3    -0    messageviewer/settings/messageviewer.kcfg.cmake
M  +6    -4    messageviewer/viewer/htmlquotecolorer.cpp

http://commits.kde.org/kdepim/b164601eb63eb62d3aa439c506072a0bbedeff74
Comment 26 Axel Braun 2014-07-29 19:27:54 UTC
Just installed 4.13.3 - really much faster, thanks a lot!
Comment 27 Orion 2014-07-31 09:53:44 UTC
to me it also seems that the "memory leak" is gone, KMail stays steady at around 135 Mb RAM that I found acceptable

Kubuntu 14.04 with KDE 4.13.3 

Great bugfix!!
Comment 28 Allen Winter 2014-07-31 11:57:25 UTC
*** Bug 330664 has been marked as a duplicate of this bug. ***