Bug 183348

Summary: Kmail filter extremely slow
Product: [Applications] kmail Reporter: Simon <gao>
Component: generalAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED UNMAINTAINED    
Severity: normal CC: dsemblano, holy, ilya.hegai, kde
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: Version Fixed In:

Description Simon 2009-02-05 19:41:45 UTC
Version:            (using KDE 4.2.0)
Compiler:          GCC-4.1.2 
OS:                Linux
Installed from:    Compiled From Sources

Create one mail filter based on subject line words and move the qualified emails to a separate subfolder from InBox.

When openning Kmail and Inbox, Kmail takes at least 40 minutes to go through 600 messages. This basically makes Kmail filters very annoying to use. 

Another problem with filer is all the new emails should be filtered on the fly while downloading, instead of downloading to Inbox and then going through Inbox for filtering.
Comment 1 Matthew Moore 2009-02-11 20:20:40 UTC
Confirmed, kmail is very slow for me as well.  I'm using ubuntu 8.10 and have installed kmail 4.1.3 and it's dog slow on a fairly fast (2gb DDR 800 RAM, Intel dual core cpu and a decent motherboard) PC.
Comment 2 Jaime Torres 2009-02-12 22:12:02 UTC
Please, could you check if you have in the rules one using clamav?
This rule is quite slow.
If it is the case, could you try without it to see if it is faster?
Comment 3 Matthew Moore 2009-02-12 23:26:12 UTC
Nope, all filters are simply filters which either match the subject or the from/to/cc and copy them to a folder.  I don't use (or have installed) clamav on either my mailserver or my PC.
Comment 4 Jaime Torres 2009-10-14 09:59:24 UTC
Is it better in a more recent version?
Also, to only check the incoming messages (and not all), please use the advanced tab in the filter editor and mark "Apply this filter for incoming messages".
Comment 5 Ilya Hegai 2009-10-14 10:27:50 UTC
(In reply to comment #4)
> Is it better in a more recent version?
> Also, to only check the incoming messages (and not all), please use the
> advanced tab in the filter editor and mark "Apply this filter for incoming
> messages".

still no progress in 1.12.2, checkbox you mentioned is selected for every filters

as for me, I see slow filters work when filters are moving mails from Inbox to nested folders, spam checking filter is appropriate
Comment 6 Ilya Hegai 2009-10-14 10:43:47 UTC
(In reply to comment #4)
> Is it better in a more recent version?
> Also, to only check the incoming messages (and not all), please use the
> advanced tab in the filter editor and mark "Apply this filter for incoming
> messages".

One more question: does it make sense to read kmail source code now, or it's time waste because of migration to akonadi in upcoming 4.4?
Comment 7 Jaime Torres 2009-10-14 11:04:43 UTC
kmail will use akonadi in 4.5 (only the addressbook is ready for 4.4).
You can see the future of kmail in a svn branch that I can not find nor remember.
Comment 8 Daniel Semblano 2010-10-12 18:16:05 UTC
I can confirm this: using IMAP, gmail account, when creating a filter based on subject or from fields, running filters are very slow and makes impossible to read any message. I'm on KDE 4.5.1, Kubuntu.
Comment 9 Laurent Montel 2015-04-12 09:46:04 UTC
Thank you for taking the time to file a bug report.

KMail2 was released in 2011, and the entire code base went through significant changes. We are currently in the process of porting to Qt5 and KF5. It is unlikely that these bugs are still valid in KMail2.

We welcome you to try out KMail 2 with the KDE 4.14 release and give your feedback.
Comment 10 Tim Holy 2016-06-18 13:21:44 UTC
I'm using Kubuntu 16.04 (Kontact 5.1.3). I copied several hundred thousand emails over from an older install. I realized I had made a mistake in ordering the priority of my filters, so I modified them and then selected a folder with about 33K messages in it, selected all messages, and hit Ctrl-J. The progress was extremely slow, under 10 messages per second, running at least 2 threads (one kontact and one mysql) constantly. Note that, because this is a new installation, I have only 6 filters defined so far (in contrast to my hundreds of filters in my old installation), and the filter rules are extremely simple (all single-rule, most looking at List-Id).

While preparing to comment here, I went and selected Help>Kontact Introduction to get my version number. When I came back to the folder, the filtering had stopped. I'm not certain whether it finished or terminated. If it finished, then I have a theory: display of folder contents is interfering with progress in filtering.
Comment 11 Tim Holy 2016-06-20 18:13:23 UTC
Just a follow-up: I tried it again, and after selecting 33K emails and starting filtering, I switched to a different mail directory (one with very few emails in it) for display. This did *not* appear to help anything.

But because I was watching it carefully I noticed more detail this time. The filtering was stuck on "Mail Filter Agent...0%" for several (10ish?) minutes, with `mysqld`, `akonadiserver`, `akonadi_maildir_resource`, and `kontact` being among the main processes. None of them were using a lot of CPU, this seems to be a disk-bound process. Then, after no visible sign of progress for many minutes, all of a sudden it ran through all 33K messages in fewer than 10 seconds.

So I'd guess the main filtering is plenty fast. It seems to be some "preparatory" step that is really slow, and because it gives no visible sign of progress it's a likely source of complaints/trouble for users (I was pretty close to trying to cancel/reboot my laptop by the time it started making progress).

Should this bug be changed from RESOLVED to something else? Or should I file this observation as a separate bug report?