Bug 113794

It looks like the search also hangs forever(with 0% CPU used) when doing searches.  I have to re-click 'Search' in order to complete the search.  Is this what you're seeing?

actually, i don't know which interface is better, but the old one allowed me to search for a specific quote in the conversation. the new one only give me on which day this quote has been said... and god knows how much you can talk with someone on one day. that you gotta read through it (ctrl-f is not even available)

the old interface could have just been modified to give a configurable context of 3, 5 or 10 lines. anyway... maybe hightlighting would help the new one ? 

I think it is related to the History slowness because I have that option that get the last 7 message from the history when I open a new chat window.

it takes between 15 and 30 seconds for me to open a chat window, depending on who I'm clicking on. which seems to directly depends with how much discussion I had with this person. Even though I click on the same person three times in a row after closing the window, reopening it always take as long.

thanks

Matthias> ok, I will try to fix this little bug.

Mathieu> highlighting is already used to show the word you find. About slowness, do you speak of the slowness of the search ? or the slowness of the chatwindow opening. Even though history was modified, the way it loads the last history items in your chatwindow wasn't changed actually.

Actually, I already took a crack at in subversion but forgot to mention it here.  The signal used to start the search is the same as the one that cancels the search was being sent twice and the progress bar wasn't getting set to 'done' when a search was cancelled.  That buglet's fixed now.  I don't see exactly what Mathieu is complaining about.  Nothing's slow here but I only have a few months of light IM history on my primary computer.

opening history window, doing a search and opening chat window. I'm very surprised loading method for last history message for chat windows has not changed. It's much slower.

would also be possible that my kopete is slower because I have a kolab addressbook ?

Just browsing the history becomes quite slow when the number (and length) of messages increase. Some IRC logs give me the following results:

77 messages => >0.27s to display
205 messages => >2.4s to display

kopete: BEGIN: void HistoryDialog::dateSelected(QListViewItem*)
kopete:   BEGIN: QValueList<Kopete::Message> HistoryLogger::readMessages(QDate)
kopete:   END__: QValueList<Kopete::Message> HistoryLogger::readMessages(QDate) - Took 0.07s
kopete:   BEGIN: void HistoryDialog::setMessages(QValueList<Kopete::Message>)
kopete:     msgs size=77
kopete:   END__: void HistoryDialog::setMessages(QValueList<Kopete::Message>) - Took 0.2s
kopete: END__: void HistoryDialog::dateSelected(QListViewItem*) - Took 0.27s


kopete: BEGIN: void HistoryDialog::dateSelected(QListViewItem*)
kopete:   BEGIN: QValueList<Kopete::Message> HistoryLogger::readMessages(QDate)
kopete:   END__: QValueList<Kopete::Message> HistoryLogger::readMessages(QDate) - Took 1.7s
kopete:   BEGIN: void HistoryDialog::setMessages(QValueList<Kopete::Message>)
kopete:     msgs size=205
kopete:   END__: void HistoryDialog::setMessages(QValueList<Kopete::Message>) - Took 0.67s
kopete: END__: void HistoryDialog::dateSelected(QListViewItem*) - Took 2.4s

I have also observed slowness when browsing the history (opening the window or clicking the previous button).

A did a little debugging and found out that kopetemessage.cpp spents lots of time in Message::setBody() on this statement:

theBody.replace( QRegExp( QString::fromLatin1(".*<body.*>\\s+(.*)\\s+</body>.*") ), QString::fromLatin1("\\1") );

If have a few messages (decompiled source code) in my history that stall kopete for _minutes_. Here ist the gdb bt:

#0  0xb6917b05 in QGArray::at () from /usr/lib/libqt-mt.so.3
#1  0xb69d72d8 in QMemArray<int>::operator[] () from /usr/lib/libqt-mt.so.3
#2  0xb6d09b6b in QRegExpEngine::matchHere () from /usr/lib/libqt-mt.so.3
#3  0xb6d0aee1 in QRegExpEngine::badCharMatch () from /usr/lib/libqt-mt.so.3
#4  0xb6d0b2e7 in QRegExpEngine::match () from /usr/lib/libqt-mt.so.3
#5  0xb6d0f04a in QRegExp::search () from /usr/lib/libqt-mt.so.3
#6  0xb6d18a4c in QString::replace () from /usr/lib/libqt-mt.so.3
#7  0xb7e4a960 in Kopete::Message::setBody (this=0xbfb248a8, body=@0xbfb24944,
    f=Kopete::Message::RichText) at kopetemessage.cpp:214
#8  0xb5b9e5f1 in HistoryLogger::readMessages () from /usr/lib/kde3/kopete_history.so

My system is Debian unstable, libqt3-mt 3.3.5-3, kopete 3.4.3-3.

I wrote a short test program that invokes the regexp on the same string but I was not able reproduce the behaviour, it took only seconds.

Created attachment 13826 [details]
Message

Here is a sample 349 line message. Send this to someone, close the chat window
and try to view the history. On my sytem it takes minutes until the history
window is displayed.

Thanks.  I've been doing performance tuning for the Real Job for the last few days, so I'll profile kopete too and see if I can get this taken care of.

Hi, try this:

Index: kopetemessage.cpp
===================================================================
--- kopetemessage.cpp   (revision 485489)
+++ kopetemessage.cpp   (working copy)
@@ -216,7 +216,7 @@
        {
                //This is coming from the RichTextEditor component.
                //Strip off the containing HTML document
-               theBody.replace( QRegExp( QString::fromLatin1(".*<body.*>\\s+(.*)\\s+</body>.*") ), QString::fromLatin1("\\1") );
+               theBody.replace( QRegExp( QString::fromLatin1("<body.*>\\s+(.*)\\s+</body>") ), QString::fromLatin1("\\1") );

                //Strip <p> tags
                theBody.replace( QString::fromLatin1("<p>"), QString::null );


The .* are really not needed, right? Cuts down processing time of #9 from 51s to 0.39s.

I don't think this is equivalent.  The first strips everything outside the body as well, while the second just strips the <body> </body> tags, so if the RichTextEditor component feeds us anything outside of the <body> tags, it won't get nuked.  If it doesn't put anything there then the new regexp is fine.

<html><body>text</body></html> would become <html>text</html>

Usability wise even 0.39 s per entry seems too long. I usually have my history size set to 200 items. Maybe replacing it by a short code snippet that uses QString::find("<body") would be more feasible?

Created attachment 13840 [details]
Fixed patch for regexp handling

#12: You're right, that change was not correct. This is what I intended. I've
just added some unit testing (libkopete/tests/) and it seems to work.

*** Bug 122141 has been marked as a duplicate of this bug. ***

*** Bug 122748 has been marked as a duplicate of this bug. ***

Tommi > How much does this patch improve speed ?

Created attachment 15311 [details]
Performance improvement

Here's an improved patch. Any realistic logs take about 15% processing time
compared to unpatched Kopete.

For example one log that took 3 seconds to load now takes about 0.45 seconds.

> Here's an improved patch. Any realistic logs take about 15% processing time
> compared to unpatched Kopete.
>
> For example one log that took 3 seconds to load now takes about 0.45 seconds.


didn't make much difference to me. kopete normally takes about 26
seconds to load all history; with the patch it took 25.

*** This bug has been confirmed by popular vote. ***

SVN commit 566905 by rantala:

Commit the patch partially.

Reduces time taken by HistoryLogger::readMessages(QDate) from 25 seconds
to 0.5 seconds with a 200kb log file.

CCBUG: 113794


 M  +10 -2     kopetemessage.cpp  


--- trunk/KDE/kdenetwork/kopete/libkopete/kopetemessage.cpp #566904:566905
@@ -83,7 +83,11 @@
 	{
 		//This is coming from the RichTextEditor component.
 		//Strip off the containing HTML document
-		this->body.replace( QRegExp( QLatin1String(".*<body.*>\\s+(.*)\\s+</body>.*") ), QLatin1String("\\1") );
+		if (this->body.contains(QLatin1String("<body"))) {
+			QRegExp rx( QLatin1String("<body[^>]*>\\s+(.*)\\s+</body>") );
+			if (rx.indexIn(this->body) != -1)
+				this->body = rx.cap(1);
+		}
 
 		//Strip <p> tags
 		this->body.replace( QLatin1String("<p>"), QString::null );
@@ -192,7 +196,11 @@
 	{
 		//This is coming from the RichTextEditor component.
 		//Strip off the containing HTML document
-		theBody.replace( QRegExp( QLatin1String(".*<body.*>\\s+(.*)\\s+</body>.*") ), QLatin1String("\\1") );
+		if (theBody.contains(QLatin1String("<body"))) {
+			QRegExp rx( QLatin1String("<body[^>]*>\\s+(.*)\\s+</body>") );
+			if (rx.indexIn(theBody) != -1)
+				theBody = rx.cap(1);
+		}
 
 		//Strip <p> tags
 		theBody.replace( QLatin1String("<p>"), QString::null );

sugoi, thank you very much

I'm using Kopete 0.12.3 on Ubuntu Edgy and I'm still having problems with slow appearing chat windows because of history plugin. Any chance we got a release to fix this soon?

I cannot reproduce the initial bug with KDE 4.0.99. Search is snappy enough, however clicking on a large conversation in the history viewer can take up to 3 seconds to show up, using a phenom 3-core processor @ 2.1 GHz. Not sure if this can be improved somehow...

I agree that the inital problem is solved, using kopete trunk version.