Bug 299968

I am using 4.8.3 now and can confirm this nasty bug back to 4.8.0. Some of the incoming mails are filtered some are not. (They are cron job reports or positive spam and the filter criteria fits to all.)

When I select all messages in the message list, the filter is not working. When I filter the messages with after  "CROM" or "***SPAM" and apply the appropriate filter or all filters its working fine. 

But it was not my intend to do this every day multiple times by hand :-(
If its not fixed soon I will switch to another mail client.

"If its not fixed soon I will switch to another mail client." so bye !
I use kmail all the time. It works fine.
So ok perhaps there is a bug but without a test case I can't fix.
What is your filter which doesn't work ?
What do you have in "Filter log viewer" ? 
etc.

Hello, the filter criteria has not changed since years and is very simple.

If "X-Cron-Env" contains "<SHELL" then move to Folder "Cron". Which is the case in all CRON-Jobs.
Spamassassin Filter: If "X-Spam-Flag" = "Yes" then move to Folder "Spam".

I get ~20 messages per hour. When I receive them from the mailbox it may be 200 or 400 messages. Sometime 20 oder 30 are not filtered sometimes more sometimes less.

Yes I have a Log-Viewer put it is not always willing to show up. In the moment it is very unwilling and not cooperative. :-(  Upps now it is up. You have to be pationed. 8 GB and six cores are not so much power. It can take some minutes to start. I would like to send you the log file. But the "Save as", "Clear" or "Abort" buttons are also not so cooperative. Maybe they will do something in ten minutes, an hour or two. Or maybe next year? Can you wait? Are the scoll bars are on holiday?

The filter works on some incoming messages and on some not. It works by hand under special circumstances. That is selecting not all massages - which I do by searching for "CRON". Applying all filters to all messages Control-A Control-J does nothing but reports that "n of m discussion groups are updated. (I am using German version so English message may be a bit different.) A lot reports are "0 von 0 Diskussiongruppen wurden aktualisiert". In some cases n and m are increased and m is running ahead of n. Why does kamail2 not know at start how many messages are to filter. I had not figured out, what "Diskussionsgruppen" mean. Never defined such things.

Wow. After several minutes. The log-Viewer is running really fast and it seems that "save as" works now. You have to be patient. But it hangs in the file selection dialog. As I mentioned above me PC is not so powerful. Uh, not the file dialog has closed.But the log viewer is a little bit sleepy.

It had saved the log in kmail-filter.html (1 MByte) - big deal - but now it is a bit stressed after the work. I thing it has retired. - Uh, it wants to save the file again but the confirmation window for overwriting the old file is empty and doesn't react. Upps - after some minutes it shows its question.

Oh! How wonderful now the CRON-Jobs an other messages are moved to the right folder. After nearly six hours all messages are sorted! 

Never saw such a poor program. How was the trick done?

Have a good night.

Created attachment 71111 [details]
Screen shot showing inbox after receiving mail

This Screen shot shows the inbox with some not filtered messages.

Created attachment 71112 [details]
Screen shot showing cron folder after receiving mail

This screen shot shows the cron folder afer receiving the messages. A lot messages are in the folders but some are stil in the inbox.

Created attachment 71113 [details]
kmail filter log after receiving messages

This is the filter log after receiving messsages.

Created attachment 71114 [details]
A filtered mail

This mail was correctly filtered to cron folder.

Created attachment 71115 [details]
This mail was not filtered

This mail stayed in the inbox and was not filtered and moved th cron folder.

(In reply to comment #2)
> "If its not fixed soon I will switch to another mail client." so bye !
> I use kmail all the time. It works fine.
> So ok perhaps there is a bug but without a test case I can't fix.
> What is your filter which doesn't work ?
> What do you have in "Filter log viewer" ? 
> etc.

I created a test case. I send 100 messages with a delay of to seconds to my account.

for ((i=0; i<100 ; i++)) ; do echo "Test #`printf %4.4d $i`" | mail -s "Filter Test #`printf %4.4d $i`" thomas ; sleep 2 ; done

I created two filters. One to add a header field with a time stamp and one to move incoming messages to a folder named "Test".

Three messages 0056, 0053, 0003 where not filtered. Some messages showed up several times in the Test folder (#0038). Some were filtered more than once (#0038 3 times) and showed multiple times in the list. See mbox attached.

Some messages were missing in the inbox and the folder "Test" (#0096 .. #0099) but appeared 
later. Some duplicates (see #0038) were later cone and only three stayed in the folder "Test". 

Maybe this is a bug and not a feature. If it's a feature I don't need it. Please switch it off!

Created attachment 71204 [details]
Test folder after rreiving 100 test messages

This is the mbox export of the Test folder after receiving the messages. Some messages are filtered more than once, three - not in the export - are not filtered and some (at least 4) are missing but show up later.

Created attachment 71205 [details]
Test folder some minutes later

This is the Test folder some minutes later. Some duplicates are gone, some missed messages show up.

Created attachment 71206 [details]
Three messages not moved to folder Test and not filtered in any way.

This is the mbox export of three unfiltered messages.

After nearly an hour there is only one copy of message #0038 left in the list. But this message passed the filter three times as the three entries "X-Filter-Test" prove.. 

From thomas@arend-rhb.de  Sat May 19 00: 12:57 2012
Return-Path: <thomas@arend-rhb.de>
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on k2.arend.tksd
X-Spam-Status: No, score=-1.8 required=4.5 tests=AWL,BAYES_00,NO_RELAYS autolearn=ham version=3.3.1
X-Spam-Spammy: 0.955-5711--1994h-14398s--0d--HContent-Transfer-Encoding:7bit, 0.917-1648--1625h-6115s--0d--H*c:us-ascii
X-Spam-Hammy: 0.000-1990--32152h-1s--0d--H*p:D*arend-rhb.de, 0.000-5--75h-0s--0d--H*UA:mailx
X-Original-To: thomas
Delivered-To: thomas@arend-rhb.de
Received: by k2.arend.tksd (Postfix, from userid 1000) id B5C6428A17; Sat, 19 May 2012 00:08:45 +0200 (CEST)
Date: Sat, 19 May 2012 00:08:45 +0200
To: thomas@arend-rhb.de
Subject: Filter Test #0038
User-Agent: Heirloom mailx 12.2 01/07/07
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Message-ID: <20120518220845.B5C6428A17@k2.arend.tksd>
From: Thomas Arend <thomas@arend-rhb.de>
X-UIDL: 7@<"!D\P!!p1#"!d*6"!
X-Filter-Test: Sat, 19 May 2012 00:12:57 +0200
X-Filter-Test: Sat, 19 May 2012 00:12:57 +0200
X-Filter-Test: Sat, 19 May 2012 00:12:58 +0200

Test #0038

With KDE 4.9.1 no action is carried out on the Filter. The log reports that the rule matched but the mail resides in the inbox.

-----  

[21:18:54] Applying filter action: Move Into Folder ""
[21:18:54] Evaluating filter rules: (match all of the following) "X-Spam-Flag" <contains> "YES"
[21:18:55] 1 = "X-Spam-Flag" <contains> "YES" (YES)
[21:18:55] Filter rules have matched.
------------------------------
[21:18:55] Applying filter action: Move Into Folder ""
[21:18:55] Evaluating filter rules: (match all of the following) "X-Spam-Flag" <contains> "YES"
[21:18:55] 1 = "X-Spam-Flag" <contains> "YES" (YES)
[21:18:55] Filter rules have matched.
------------------------------
[21:18:55] Applying filter action: Move Into Folder ""

> [21:18:55] Applying filter action: Move Into Folder ""

Check the action in your filters settings. The field right to "Move into folder" is empty ?

This looks like filters are misconfigured (could happen if the akonadi db was recreated or if migrating failed).

@ András Manţia
How can a filter be misconfigured when it was newly created?
Why does it work on some messages and on some not? All messages are from the same type.

 Applying filter action: Move Into Folder "" suggest that. Please copy the failing filter setting from $HOME/.kde[4]/share/config/akonadi_mailfilter_agentrc

Here are my most common filters.

Some messages are moved some not.

The filter protocol is useless, because it does not log an identifier for the message. You can't match a message to a log entry. 

So I don't no what happens:
Case 1: Filters are not applied.
Case 2: Filter rules fit, but check fails. Bug in the filter test.  
Case 3: Filter is applied, check positive, but command is not executed.

Without a working logging you can also ask a crystal ball. 

Thomas

[Filter #0]
Applicability=0
AutomaticName=false
ConfigureShortcut=true
ConfigureToolbar=true
Enabled=true
Icon=/usr/share/pixmaps/gnome-spider.png
StopProcessingHere=true
ToolbarName=Mail ist SPAM
action-args-0=632
action-name-0=transfer
actions=1
apply-on=check-mail,manual-filtering
contentsA=YES
fieldA=X-Spam-Flag
funcA=contains
identifier=QHIAWqUq7GpbNJLG
name=Mail ist SPAM
operator=and
rules=1

[Filter #1]
Applicability=0
AutomaticName=false
ConfigureShortcut=true
ConfigureToolbar=false
Enabled=true
Icon=system-run
StopProcessingHere=true
ToolbarName=CRON-Jobs
action-args-0=588
action-name-0=transfer
actions=1
apply-on=check-mail,manual-filtering
contentsA=<SHELL
fieldA=X-Cron-Env
funcA=contains
identifier=uuXAxfNMdyY0bg1k
name=CRON-Jobs
operator=and
rules=1

In KMail shipped with 4.9.98 (4.10 RC3, openSUSE packaged) incoming IMAP but already read messages are not filtered at all.
Use case e.g. when someone read mail using both webmail interface and KMail, when message is marked as read in webmail, it stays unfiltered in inbox. When unread, message is filtered.

I can reproduce it 100% time so far.

Fixed in KDE 4.10.1 and later by Wolfgang Rohdewald.
http://commits.kde.org/kdepim/185c572e079ef40bee2225e582c11dbdbb35840f

I have upgraded to 4.10.1. Filters are working much better! I still have one account, however, where the incoming messages are are not filtered until I do a manual filtering.

This is the same account that is not getting it's identity set properly. I have a filter for this account to set the identity, and a couple upgrades ago, the filter broke, setting it to a different identity in my config.  What information do you need from me to trouble shoot this?