Bug 126182

Summary: online IMAP filters do not always start
Product: [Applications] kmail Reporter: Jason Clinton <me>
Component: filteringAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED FIXED    
Severity: normal CC: anaselli, andreas.petzold+kdebugs, curdyben, eli, gabriel_ambuehl, ismail, j, kde-bugzilla, kde, krrm, lecit, marc.deichmann, marcus, Michael, mikecoan, morozov_ml, ovit.debian, peter, philantrop, raymond.pittigher, roeland, simone, wbsoft, zeekec
Priority: NOR    
Version: 1.9.1   
Target Milestone: ---   
Platform: Debian testing   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: Backtrace from most recent crash
Valgrind log

Description Jason Clinton 2006-04-24 17:14:55 UTC
Version:           1.9.1 (using KDE KDE 3.5.2)
Installed from:    Debian testing/unstable Packages
OS:                Linux

I have several online IMAP accounts configured in KMail and, more often than not, when I start KMail and new mail is checked, the IMAP filters will not be applied to the new mail in my accounts. If I apply the filters manually, they work fine. I can restart KMail and about 25% of the time, automatic online filtering will work as expected. I haven't been able to find any coorelation or cause.
Comment 1 Andy 2006-05-01 23:01:21 UTC
I have noticed the same behavior in Kmail 1.9 on CentOS 4.3 using KDE 3.5.
If you restart kmail a couple times the filters start working again.
Comment 2 Ismail Onur Filiz 2006-05-06 08:44:57 UTC
It happens to me pretty frequently too.
Comment 3 Ismail Donmez 2006-05-06 13:08:01 UTC
I also see this.
Comment 4 Dánial Olsen 2006-10-07 02:09:18 UTC
Same here. Running Kmail 1.9.4 with KDE 3.5.4 on Arch Linux.
Comment 5 Steve-Kai Vyska 2006-10-11 15:48:25 UTC
With me the same cyrus 2-2, KMail 1.9.4 with Kontact 1.2.3, KDE 3-5-4
Comment 6 Tobias Ussing 2006-10-22 16:06:52 UTC
same here with latest svn(as of today), that is revision 598101
Comment 7 CSights 2006-11-14 15:04:53 UTC
I experience the same problem with dimap using kmail version 1.9.5.

Bug 127998 and Bug 122923 have a similar description as this bug and may be duplicates
Comment 8 Philip Rodrigues 2006-11-14 20:04:38 UTC
*** Bug 127998 has been marked as a duplicate of this bug. ***
Comment 9 Philip Rodrigues 2006-11-14 20:05:20 UTC
*** Bug 122923 has been marked as a duplicate of this bug. ***
Comment 10 Philip Rodrigues 2006-11-14 20:06:42 UTC
Thanks for pointing those out, CSights! Note that bug 122923 reports the same behaviour for *cached* imap.
Comment 11 Ismail Donmez 2007-03-22 13:40:38 UTC
I can still reproduce this with latest 3.5 SVN.
Comment 12 Kurt Bennater 2007-05-23 14:00:42 UTC
IMAP filtering used to work here until KDE 3.5.6. Since my yesterday's update to 3.5.7 (=KMail 1.9.7), incoming mails in IMAP folders are not filtered any longer. That is really annoying, please fix this ASAP!
Comment 13 Kurt Bennater 2007-05-23 23:01:08 UTC
My preliminary investigations indicate that with KMail 1.9.7, only the first of a bunch of incoming mails is filtered. So if there are many simultaneously incoming mails or if KMail is started and there is more than one mail to process, the filter seems to neglect all mails but the first. As I said in my previous post, this is a newly introduced bug in my case.
Comment 14 Daniel Hahler 2007-05-24 00:09:40 UTC
I can confirm the observations by Kurt with KDE 3.5.7 (Kubuntu).
Comment 15 Philip Rodrigues 2007-05-24 00:10:39 UTC
I'm seeing almost exactly the same as Kurt (comment 13). Although sometimes even the first message isn't filtered. Probably related to bug 145819
Comment 16 Gael Beaudoin 2007-05-24 09:02:47 UTC
Same problem with KDE 3.5.7 (kubuntu) : filters on IMAP account are not applied.

If I apply them manually on multiple emails, only one is email is effectivly processed (althouth all selected emails are crossed-out).
Comment 17 Tais P. Hansen 2007-05-24 09:09:00 UTC
Another confirm with KDE 3.5.7/Kmail 1.9.7 on Slackware.
Comment 18 Thomas McGuire 2007-05-24 17:24:14 UTC
*** Bug 145819 has been marked as a duplicate of this bug. ***
Comment 19 Gabriel Ambuehl 2007-05-24 22:14:50 UTC
In 3.5.7 filters NEVER do anyhting. Not manually started, either.
Comment 20 Jacco 2007-05-25 12:48:59 UTC
KDE 3.5.7 on OpenSuse 10.2: only one email is filtered on my IMAP accounts, like in comment #16.
Comment 21 Thomas McGuire 2007-05-25 20:40:55 UTC
*** Bug 145870 has been marked as a duplicate of this bug. ***
Comment 22 Philip Rodrigues 2007-05-27 00:31:59 UTC
For the record, it seems to me like the cause is r650963. If anyone can try building a kmail from immediately before this commit, and see if the problem exists there, it might make a useful data point.
Comment 23 Alexey Morozov 2007-06-04 12:23:10 UTC
I've just applied the reverted r650963 commit to kmail-1.9.7 (KDE-3.5.7), and yes I can confirm that filter processing have been restored.

Although it *seems* a bit slower and more flickering than before on my 1000+ IMAP Inbox, but I've just got the entire INBOX correctly processed.
Comment 24 Alexey Morozov 2007-06-04 14:41:54 UTC
Just noticed:

KMail with reverted r650963 applies filters to the new messages, except those which just arrived: that is if KMail receives a new message or messages they won't be processed during this transaction, only in the next check cycle.

Does anybody actually understand how KMail filters are processed?
Comment 25 Alexey Morozov 2007-06-05 07:24:24 UTC
I have to conclude: filter processing is seriously broken in KMail :-(. 

Now I have a simple filter (actually it's a default anti-spam filter which checks for X-Spam-Flag == YES and moves the message to trash). The header check simply doesn't work, although similar checks perfectly work in other filters :-(

Could anybody explain how they are expected to work, step by step?
Comment 26 Thomas McGuire 2007-06-07 18:36:47 UTC
*** Bug 146215 has been marked as a duplicate of this bug. ***
Comment 27 Thomas McGuire 2007-06-13 17:28:43 UTC
*** Bug 146716 has been marked as a duplicate of this bug. ***
Comment 28 Thomas McGuire 2007-06-16 11:21:15 UTC
*** Bug 146833 has been marked as a duplicate of this bug. ***
Comment 29 Wulf C. Krueger 2007-06-16 19:45:43 UTC
I'd really like fix this issue in Gentoo. 

Could some KDE developer confirm that reverting r650963 is the correct way to fix this problem?
Comment 30 Alexey Morozov 2007-06-21 06:46:26 UTC
No, actually it's not.

The revert allows you to /somewhat/ workaround the problem, but if you need _fast_ and reliable filter processing you need to review the messages queueing/processing code and perhaps accurately rewrite it for the current environment. Unfortunately the processing model is rather complex and I have no time to fully understand it.
Comment 31 Marcus D. Hanwell 2007-07-09 21:52:38 UTC
Looking at the commit history for the ported commit, coolo stated not to port unless it proves stable. It would seem that this patch is not stable at least for us. Gentoo will revert as it seems to give more usable filtering in kmail. If there is a better patch available we would of course be interested in hearing about it.
Comment 32 Daniel Hahler 2007-07-09 22:26:20 UTC
Just for quick reference, you can see here what has been changed in r650963: http://websvn.kde.org/?view=rev&revision=650963
Comment 33 eli 2007-07-20 07:05:28 UTC
This bug exists in kmail 1.9.7 at least in Fedora 7.
Comment 34 Kevin Kofler 2007-07-20 14:24:54 UTC
Yes, Fedora 7 is known to be affected, see:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=244930
Comment 35 Gary Krueger 2007-09-04 18:18:25 UTC
In conjunction with Bug #135376 (https://bugs.kde.org/show_bug.cgi?id=135376), this is a royal pain.
Comment 36 Thomas McGuire 2007-09-06 15:37:30 UTC
*** Bug 149600 has been marked as a duplicate of this bug. ***
Comment 37 Markus Grombein 2007-09-17 21:49:01 UTC
With Kubuntu 7.10 the problem is gone. The filters now work as expected for me. KMail version is 1.9.6 (enterprise 0.20070907.709405) (KDE 3.5.7).
Comment 38 Gabriel Ambuehl 2007-09-17 22:36:42 UTC
On Monday 17 September 2007 21:49:04 Markus Grombein wrote:
> expected for me. KMail version is 1.9.6 (enterprise 0.20070907.709405) (KDE
> 3.5.7).


I can confirm that. Since Kubuntu went to the enterprise branch, I haven't 
seen any issue with filters anymore.
Comment 39 Garry Williams 2007-09-18 00:24:24 UTC
On Monday 17 September 2007 16:36:48 Gabriel Ambuehl wrote:
> On Monday 17 September 2007 21:49:04 Markus Grombein wrote:
> > expected for me. KMail version is 1.9.6 (enterprise
> > 0.20070907.709405) (KDE 3.5.7).
>
> I can confirm that. Since Kubuntu went to the enterprise branch, I
> haven't seen any issue with filters anymore.

I would like to know what I'm doing wrong, then.  I have:

    $ kmail --version
    Qt: 3.3.8
    KDE: 3.5.7-21.fc7 Fedora
    KMail: 1.9.7

and this bug is still plaguing me.  (Only one message is filtered
after I start kmail and none are filtered thereafter.)
Comment 40 Tais P. Hansen 2007-09-18 07:22:24 UTC
Kubuntu seems to have downgraded to the previous version of kmail. The problem is also sort of gone in Gentoo because they choose to revert the change that broke filtering.
Comment 41 Angelo Naselli 2007-09-26 10:22:10 UTC
confirm a similar problem on mandriva:
http://qa.mandriva.com/show_bug.cgi?id=34059
Comment 42 Chris Woodworth 2007-10-09 01:45:09 UTC
Created attachment 21776 [details]
Backtrace from most recent crash
Comment 43 Chris Woodworth 2007-10-09 01:46:15 UTC
Kubuntu 7.10
Linux HAL421 2.6.22-13-generic #1 SMP Thu Oct 4 17:18:44 GMT 2007 i686 GNU/Linux
Qt: 3.3.7
KDE: 3.5.7
KMail: 1.9.6 (enterprise 0.20070907.709405)

Filters with IMAP have been broken for a looong while. I have Spamassassin running server side - so it's not that. I have only 2 filters, both manual - Mark as Spam & Mark as Ham. The Spam filter moves email not caught by SA to the (IMAP) Spam folder - for later SA (re)training. The Ham filter moves SA false positives to a Ham folder (for training) AND to my local Inbox.

Kmail crashes whether the filter is applied to individual or multiple emails - not all the time - but with enough frequency that Kmail is all but unusable for me. It also crashes occasionally when deleting emails - by clicking the trashcan button or the delete key.

Debugging data attached.
Comment 44 Chris Woodworth 2007-10-09 01:48:12 UTC
Created attachment 21777 [details]
Valgrind log
Comment 45 Will Stephenson 2007-10-09 10:17:24 UTC
Chris, could you also attach a backtrace of the crash?
Comment 46 Chris Woodworth 2007-10-09 16:28:31 UTC
I did, it's comment 42.
Comment 47 Gary Krueger 2007-10-30 15:49:59 UTC
This issue goes away for me when I downgrade kdepim (kmail) from 1.9.7 to 1.9.4.
Comment 48 Alexey Morozov 2007-11-06 08:40:46 UTC
Well, it's surely a solution. But, well, one may wish to solve the problem w/o significant downgrading, because newer versions of KMail have brought some new useful features :-).

BTW, did you downgraded just KMail or entire kdepim ?
Comment 49 Gary Krueger 2007-11-06 16:37:29 UTC
Howdy Alexey!

I downgraded kdepim and kdeaddons (required when downgrading kdepim).

I did so, because pirut makes it easy.

I found that it resolved a number of issues for me:
#77862
#135376
#149941
#150038

It also improved (but didn't entirely correct):
#126182

But, that isn't a big issue compared to having to manually filter each 
message.  I mean, that almost made filtering a pointless activity.  And 
more than 90% of my incoming e-mails are handled by filters.

Not only that, but when I had upgraded to 1.9.7, the recipient auto-fill 
went screwy.  That now seems to work correctly with 1.9.4.

The only other thing that I use regularly, is the Summary page for RSS 
feeds.  And that seems no different.

Repeating calendar events don't appear for subsequent dates in both 
1.9.4 and 1.9.7.

I don't use the "To-do List", the "Journal", the "News", the "Notes", 
nor the "Synchronization" features.  So, if there are any features lost 
there, I wouldn't know it.

And, I haven't noticed any lost features.  I'm sure there are some.  
But, I don't see any such evidence.

I just need to make sure to not update 1.9.7 back into the system.

I hope this helps you to make an informed decision.

On Tue 6 November 2007 2:40 am, Alexey Morozov wrote:
[bugs.kde.org quoted mail]
Comment 50 Angelo Naselli 2007-11-06 16:45:44 UTC
> Repeating calendar events don't appear for subsequent dates in both 
> 1.9.4 and 1.9.7.

A friend of mine seems not to have problems with KMail 1.9.6.

Angelo
Comment 51 Jacco 2008-01-08 04:16:32 UTC
Has anybody tried if this has been fixed in KDE 3.5.8?
Comment 52 Angelo Naselli 2008-01-08 09:51:26 UTC
To be honest i was sick and tired to try little changes from svn branch
and left it to try the enterprise branch that is more stable as far as
I'm concerned... and it works there.

Angelo
Comment 53 Thomas McGuire 2008-01-12 16:22:50 UTC
*** Bug 155424 has been marked as a duplicate of this bug. ***
Comment 54 Dmitri Koulikoff 2008-01-12 18:17:26 UTC
Does anybody know what with this bug in 4.0?
Comment 55 Thomas McGuire 2008-01-20 21:43:21 UTC
*** Bug 156241 has been marked as a duplicate of this bug. ***
Comment 56 Ingo Klöcker 2008-02-10 23:45:16 UTC
SVN commit 773384 by kloecker:

Listen to the closed() and expunged() signals of the temporary filter folder (mSrcFolder), so that we can re-open the folder whenever it was closed.

This fixes the problem that filtering with online IMAP stopped after the first message had been processed.

CCBUGS: 126182


 M  +23 -1     actionscheduler.cpp  
 M  +2 -0      actionscheduler.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=773384
Comment 57 Ingo Klöcker 2008-02-11 00:16:44 UTC
This bug (or more precisely the problem reported in comment #12 and later) should now be fixed. Just in time for KDE 3.5.9.

Porting to KDE 4 is pending. Therefore I do not yet mark the bug as fixed.
Comment 58 Tais P. Hansen 2008-02-11 14:03:25 UTC
I've applied the patch to my the Gentoo kmail-3.5.8 ebuild and I can confirm filters working with imap. 

Thanks a lot!
Comment 59 Tais P. Hansen 2008-02-15 10:23:40 UTC
Seems like the automatic filtering stops working when I use manual filtering, ie. Apply All Filters (ctrl-j) or the built-in bogofilter spam/ham classify filters.
Comment 60 Ingo Klöcker 2008-02-17 13:38:04 UTC
SVN commit 776110 by kloecker:

Forward ported revision 773384 from the KDE 3.5 branch

Listen to the closed() and expunged() signals of the temporary filter folder (mSrcFolder), so that we can re-open the folder whenever it was closed.

This fixes the problem that filtering with online IMAP stopped after the first message had been processed.

CCBUGS: 126182


 M  +21 -0     actionscheduler.cpp  
 M  +2 -0      actionscheduler.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=776110
Comment 61 Ingo Klöcker 2008-02-17 13:39:54 UTC
Fixed in KDE 3.5.9 and KDE 4.x.
Comment 62 Ingo Klöcker 2008-02-17 13:41:37 UTC
I suggest to open a new bug report for the issue mentioned in comment #59.
Comment 63 Gabriel Ambuehl 2008-02-22 10:39:30 UTC
I have opened http://bugs.kde.org/show_bug.cgi?id=158211 for the issue raised 
in comment #59