Bug 93158 - Search folders cause huge slowdowns
Summary: Search folders cause huge slowdowns
Status: RESOLVED UNMAINTAINED
Alias: None
Product: kmail
Classification: Applications
Component: search (show other bugs)
Version: 1.10.90
Platform: Unlisted Binaries Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords: triaged
: 111377 147634 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-11-12 12:10 UTC by automailer
Modified: 2015-04-12 09:58 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description automailer 2004-11-12 12:10:44 UTC
Version:            (using KDE KDE 3.3.1)
Installed from:    Unspecified Linux

When using Kmail with greater 5k messages and around 200 on an disconected IMAP server my system becomes very hard to user since KMAIL creates quite some system load when checking email and even worse accesses the harddisk so much that doing anything that requires hard disk access becomes absolutely unusable in a productive enivorment. - That behaviour last much too long.


Cheers,
Thorsten
Comment 1 automailer 2004-11-12 12:28:47 UTC
I took some more time to experiment. When deleting ALL Search folders the problem seems to disappear. So maybe it's a problem with inefficient Indexing of mails.

Cheers,
Thorsten
Comment 2 Don Sanders 2004-12-22 09:00:57 UTC
If you have some statistics or feel like generating some I'd be happy to look 
at them.

Comment 3 automailer 2005-01-02 23:53:17 UTC
Setting:P4: 2,8 GHZ
Distribution is Fedora Core 3
KMail: 1.7.1
File System is ReiserFS
With /home/ Directory on a different partition then the root directory hover ont he same harddrive
Physical Memory is 768MB
SWAP is 800 MB

Problem has also ocured with Yoper 2.1

Setting: Message corps of about 10k Messages, consuming around 800 MB disk space.

Since I have deleted all previous search folders for the sake of usability I created the following setup for testing. All search folders search the entire message corps.

1. Hannah: FROM contains Hannah (207 messages found)
2. New: Status is new and Subject does not contain SPAM (0 messages found)
3. Tony: From contains Tony (6 messages found)

When doing the testing setup I originally wanted to create a folder called All-Mail which used the rule Message does not contain 1213212234. However that absolutely showed the problematic behauviour again. The process table show User loads for contact of up to 90% and a lt of HDD access. After "finding" the first 4000 some messages the search becomes slower and slower at which point I decided to kill it.


There is on Disconected IMAP account in this setup. On the first Mail check there where 3 new messages on the Server which caused system loads of aroun 90% when opening. However when I sent three more test messages the system works as expected. This is very much like the situation I used to have in normal operation. Starting up KMail works fine, reading Email is OK but then when keeping the programm open and going web surfing at some point KMail starts to consume a lot of ressources. 

I would prefer doing specific tests instead of just experimenting without a clue where to go. So I will stop this round here. If you need any further information or testing please contact me an I'll be happy to try my best to answer your questions or perform any necessary tests.


Cheers & Thanks for working on such a great programm!
Thorsten
Comment 4 automailer 2005-01-12 01:19:25 UTC
Just as a point of Information:
I have imported the abobe message body into Zoe (http://www.zoe.nu) which is appraently based on Lucene search technology (http://jakarta.apache.org/lucene/docs/index.html).

This combination provides almost instant search results for the above queries (of course after spending a lot of time indexing).

This kind of search technology makes very interesting features possible:

1. A new tab in kaddressbook showing the entire comunication history with the contact (yes nice and ordered as well as searchable)
2. Broswing the archive by date
3. Tags to structure Emails. For example Project A with a little extra info for KMail this tag could also tell the Programm that this was done for Organization B thus showing the Mail when looking for project A or organization B.
4. Easy re-submission of Mails on set dates and maybe even simple conditions.

Most of this seems rather easy when being able to search huge corpses of Email spending only very limited amounts of recourses.

With the Zoe aproach of offering an XML-RPC interface the search could even be delegated to a specialized server.


Cheers,
Thorsten
Comment 5 Don Sanders 2005-01-12 06:35:56 UTC
Thanks for your excellent information Thorsten. Please give me some time to 
process this information before responding.

Comment 6 Thomas McGuire 2007-07-09 18:05:15 UTC
I can confirm this with KMail 1.9.7.
Comment 7 Thomas McGuire 2007-07-09 18:05:32 UTC
*** Bug 147634 has been marked as a duplicate of this bug. ***
Comment 8 Thomas McGuire 2007-07-11 20:39:20 UTC
*** Bug 111377 has been marked as a duplicate of this bug. ***
Comment 9 Dominik Tritscher 2008-09-14 21:42:15 UTC
In KDE4 (using SVN trunk r860647) kio_imap4 still needs a lot of cpu power (usually between 80-100% on one core), no matter whether search folders are defined or not.
Comment 10 Laurent Montel 2015-04-12 09:58:43 UTC
Thank you for taking the time to file a bug report.

KMail2 was released in 2011, and the entire code base went through significant changes. We are currently in the process of porting to Qt5 and KF5. It is unlikely that these bugs are still valid in KMail2.

We welcome you to try out KMail 2 with the KDE 4.14 release and give your feedback.