Version: (using KDE KDE 3.2.2) Installed from: Debian stable Packages Searching emails in kmail is slow. Kmail seems to need to look at each message. It would be nice to have an index or some other method by which searching could be sped up.
Erik, KMail does use an index, of course, albeit only for selected headers of the message. There is a fulltext index as well, but it is currently disabled. I assume you are referring to searches over the body of mails?
> you are referring to searches over the body of mails? Even headers-only searches are very slow in KMail on large folders - they are O(#messages), fulltext is really the only solution to that.
*** Bug 84159 has been marked as a duplicate of this bug. ***
I've written a small prototype maildir indexer at http://www.danielnaber.de/tmp/. It uses Lupy, a Python version of Lucene. There's also a C++ version of Lucene which could be integrated into KMail. Lucene is a powerful but easy to use fulltext search library. If anybody is willing to work on this I can help you with the Lucene part.
*** Bug 88408 has been marked as a duplicate of this bug. ***
I just did a search for a match in the "From" field. I use an IMAP account with subfolders containing about 10,000 emails. This search took about 3,5 minutes resulting in 52 hits. A second search (for another From match was clearly taking advantage of some cache and took only a few seconds. Shutting down kmail and repeating the search I got the long search time again. I did the same search with a freshly started Mozilla Thunderbird and got my result after about 30 seconds. A second search on another From match took only a few seconds. Conclusion: Non-cached kmail searches are substantially slower than in competing email clients. Since Thunderbird is also open source I would suggest that the developer take a peek in the code.
OK, maybe you are seeing some cached behaviour on the server too (eg files in memory?). Can we put some science into this? Pick a standard search, based on From: Run it with KMail (report time) Run it with KMail again (report time) Shutdown KMail Run it with Thunderbird (report time) Run it with Thunderbird again (report time) Shutdown Thunderbird Run it with KMail (report time) Run it with KMail again (report time) Shutdown KMail Run it with Thunderbird (report time) Run it with Thunderbird again (report time) Shutdown Thunderbird
> Conclusion: Non-cached kmail searches are substantially slower than in > competing email clients. Since Thunderbird is also open source I would > suggest that the developer take a peek in the code. Thanks for the suggestion but this is not comparable. You're right about the initial time which is simply because we have to make sure the folder is complete when you start your search. If you first select your folder and then start a search you get the same effect. We'll work on this but please keep in mind that imap searching is brand new in kmail.
On Monday 21 February 2005 12:37, Brad Hards wrote: > ------- You are receiving this mail because: ------- > You are a voter for the bug, or are watching someone who is. > > http://bugs.kde.org/show_bug.cgi?id=79808 > > > > > ------- Additional Comments From bradh frogmouth net 2005-02-21 12:37 > ------- OK, maybe you are seeing some cached behaviour on the server too > (eg files in memory?). Can we put some science into this? Pick a standard search, based on From: "Mads" Run it with KMail (report time) 3:59 - 256 hits of 9962 messages Run it with KMail again (report time) 0:13 Shutdown KMail Run it with Thunderbird (report time) 0:34 - 257 hits Run it with Thunderbird again (report time) 0:03 Shutdown Thunderbird Run it with KMail (report time) 3:56 Run it with KMail again (report time) 0:12 Shutdown KMail Run it with Thunderbird (report time) 0:13 Run it with Thunderbird again (report time) 0:03 Shutdown Thunderbird Run it with Horde/IMP 0.06 (257 hits) Run it with Horde/IMP again 0.03 The horde webmail is installed on the mail server. KMail and Thunderbird is using an uncrypted connection to the mailserver Søren<html><head><meta name="qrichtext" content="1" /></head><body style="font-size:10pt;font-family:Georgia"> <p>On Monday 21 February 2005 12:37, Brad Hards wrote:</p> <p>> ------- You are receiving this mail because: -------</p> <p>> You are a voter for the bug, or are watching someone who is.</p> <p>></p> <p>> http://bugs.kde.org/show_bug.cgi?id=79808</p> <p>></p> <p>></p> <p>></p> <p>></p> <p>> ------- Additional Comments From bradh frogmouth net 2005-02-21 12:37</p> <p>> ------- OK, maybe you are seeing some cached behaviour on the server too</p> <p>> (eg files in memory?). Can we put some science into this?</p> <p></p> <p><span style="font-family:Courier New">Pick a standard search, based on From: "Mads"</span></p> <p><span style="font-family:Courier New">Run it with KMail (report time) 3:59 - 256 hits of 9962 messages</span></p> <p><span style="font-family:Courier New">Run it with KMail again (report time) 0:13</span></p> <p><span style="font-family:Courier New">Shutdown KMail</span></p> <p><span style="font-family:Courier New">Run it with Thunderbird (report time) 0:34 - 257 hits</span></p> <p><span style="font-family:Courier New">Run it with Thunderbird again (report time) 0:03</span></p> <p><span style="font-family:Courier New">Shutdown Thunderbird</span></p> <p><span style="font-family:Courier New">Run it with KMail (report time) 3:56</span></p> <p><span style="font-family:Courier New">Run it with KMail again (report time) 0:12</span></p> <p><span style="font-family:Courier New">Shutdown KMail</span></p> <p><span style="font-family:Courier New">Run it with Thunderbird (report time) 0:13</span></p> <p><span style="font-family:Courier New">Run it with Thunderbird again (report time) 0:03</span></p> <p><span style="font-family:Courier New">Shutdown Thunderbird</span></p> <p><span style="font-family:Courier New">Run it with Horde/IMP 0.06 (257 hits)</span></p> <p><span style="font-family:Courier New">Run it with Horde/IMP again 0.03</span></p> <p></p> <p><span style="font-family:Courier New">The horde webmail is installed on the mail server.</span></p> <p><span style="font-family:Courier New">KMail and Thunderbird is using an uncrypted connection to the mailserver</span></p> <p></p> <p><span style="font-family:Courier New">Søren</span></p> </body></html>
On Monday 21 February 2005 15:28, Carsten Burghardt wrote: > ------- You are receiving this mail because: ------- > You are a voter for the bug, or are watching someone who is. > > http://bugs.kde.org/show_bug.cgi?id=79808 > > > > > ------- Additional Comments From burghardt kde org 2005-02-21 15:27 > ------- > > > Conclusion: Non-cached kmail searches are substantially slower than in > > competing email clients. Since Thunderbird is also open source I would > > suggest that the developer take a peek in the code. > > Thanks for the suggestion but this is not comparable. > You're right about the initial time which is simply because we have to > make sure the folder is complete when you start your search. If you first > select your folder and then start a search you get the same effect. We'll > work on this but please keep in mind that imap searching is brand new in > kmail. I thought you did this check on startup. KMail takes about 40 seconds to launch on my Gentoo Athlon 1800+ system before I can read my new mails. Most of the time is spent on "Checking xxxx account for new messages". Launch time for Thunderbird is about eight seconds before I can read the first mail. I repeated the shutdown/startup procedure several times. I like KMail, but Thunderbird is definately a competitive client. IMap supports serverside searching. Is kmail supporting this feature ? best regards Søren
On Monday 21 February 2005 16:44, Soren Holstebroe wrote: > I thought you did this check on startup. KMail takes about 40 seconds to > launch on my Gentoo Athlon 1800+ system before I can read my new mails. > Most of the time is spent on "Checking xxxx account for new messages". This a check for new mail. You can reduce this time when you either do not check for new mails on startup or reduce the folders that are checked. > Launch time for Thunderbird is about eight seconds before I can read the > first mail. I repeated the shutdown/startup procedure several times. > I like KMail, but Thunderbird is definately a competitive client. > IMap supports serverside searching. Is kmail supporting this feature ? Yes, automatically.
In case this helps: the search function in Sylpheed used to satisfy all my mail search needs - and is blazing fast. Even when it has to look inside the message. Since I was told over and over open src is about not reinventing the wheel all the time...
*** Bug 113496 has been marked as a duplicate of this bug. ***
Allen Winter said in bug 113496: "run configure with the --enable-indexlib option and you can try it out."
See also Bug 93844 (Tools->Find Messages should not search (binary) attachments).
I came here to file a bug asking for faster search in kmail, and found this bug. kmail is fine until you start getting several thousand mails in your folders - but now I find it really slow. I saw one blogger saying that he used to use a mail client that used CLucene (http://clucene.sourceforge.net/) to index mails and it was lightning-fast. CLucene is used in sword/bibletime which I use and is very fast there. Maybe worth looking at?
Possible workaround: Use kerry, a KDE front-end to the beagle search engine. However, kerry seems to have plenty of bugs of its own...
*** Bug 91209 has been marked as a duplicate of this bug. ***
*** Bug 230586 has been marked as a duplicate of this bug. ***
Please reporters check KMail2 and report against it so we can see if this is still valid. Meanwhile reassigning to kmail2 product
Fixed in 4.13 with baloo