Bug 283548

Summary: Only first filter in chain works for incoming messages
Product: [Frameworks and Libraries] Akonadi Reporter: Andrew Gaydenko <a>
Component: Mail Filter AgentAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED NOT A BUG    
Severity: normal CC: amantia, anderslund, aspotashev, auxsvr, bugs, ht990332, mmtsales, sergio, sven.burmeister, winter
Priority: NOR    
Version: 4.9   
Target Milestone: ---   
Platform: Arch Linux   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: filter log
Sample filter log

Description Andrew Gaydenko 2011-10-07 15:36:59 UTC
Version:           4.7 (using KDE 4.7.1) 
OS:                Linux

It is for 4.7.2, but selection list hasn't this version.

I didn't use filtering with 4.7.1 because of "conflict resolution dialog lock up" bug which is fixed in 4.7.2. So I have tried to use filtering now - in fact it is just bogofilter antispam filtering.


Manual filtering does work as expected (Apply All Filters in context menu). But it doesn't work for incoming messages. I have tried:

- all except for IMAP
- all POP3 accounts only
- Local Folders only

The problem doesn't depend on POP3 destination folder (dedicated one or default inbox folder).

The first phase - piping through bogofilter - does work for incoming messages as expected. But the second filter - spam handling itself (checking for X-Bogosity and moving spam to dedicated folder) - works in manual mode only. Piping filter hasn't "stop processing here" flag checked, of course.

Reproducible: Always

Steps to Reproduce:
Spam yourself.

Actual Results:  
The spam message isn't moved to spam folder.


Expected Results:  
The spam message is moved to spam folder.
Comment 1 Hussam Al-Tayeb 2011-10-10 08:36:42 UTC
ah. I get it now. "Bogofilter Check" filter works but it doesn't jump to "Spam Handling".
Comment 2 Andrew Gaydenko 2011-10-10 12:23:25 UTC
@developers

Can we already change status from UNCONFIRMED to NEW?
Comment 3 Andrew Gaydenko 2011-10-13 19:58:10 UTC
A little addition. I have dedicated folder - zJunk - as a target folder for spam handling filter. Sometimes (rather rare, say, 0-3 times a day with hundred or two received messages) just clicking on (not-spam) not read message header in some input folder moves the message to this zJunk folder (not marking as read). The folder isn't presented somewhere in configuration except for spam handling.
Comment 4 Andrew Gaydenko 2011-11-03 03:37:35 UTC
The bug still exists in 4.7.3 also. I think there are concurrency pitfalls in akonadi design.
Comment 5 Andrew Gaydenko 2011-12-08 13:41:40 UTC
The bug still exists in 4.7.4 also.

Why the issue is still UNCONFIRMED?
Comment 6 Hussam Al-Tayeb 2011-12-08 14:02:31 UTC
Because not every feature of a piece of software is used by many people which is unfortunate. It leads to bugs not being noticed by a lot of users. In this case, most users don't use local spam filtering and won't stumble onto this bug.
Comment 7 Andrew Gaydenko 2011-12-08 14:09:22 UTC
Hussam, in fact the question is more for the KDE team - you have confirmed the issue, but the team is still ignoring the confirmation during two months.
Comment 8 Allen Winter 2011-12-08 14:21:11 UTC
we don't have the manpower to go through and verify each bug and mark them as NEW.  I will do that here just to make you feel better, but unfortunately that doesn't mean your bug will be fixed any faster.
Comment 9 Andrew Gaydenko 2011-12-14 18:13:42 UTC
Allen, thanks for your respecting users' feeling :-) Can we at least hope akonadi is more robust (does some redesign take place) in 4.8? At this case the issue can be "auto-cured"..
Comment 10 Hussam Al-Tayeb 2011-12-14 18:24:48 UTC
I'm using akonadi from git master branch. The bug is still there.
Comment 11 S. Burmeister 2012-02-10 09:52:49 UTC
Works for me with KDE 4.8.0. Bogofilter rules are the last 4 in a chain of >10 and the filter moves spam from my imap folder to a local folder.
Comment 12 Andrew Gaydenko 2012-02-10 10:10:01 UTC
Yes, for me filtering started to work also - unfortunately, some kind of working only. In particular, bogofilter-related filters sometimes work as expexted, but sometimes partly only (say, moving to a folder assign for spam doesn't work, while setting as read does).

Now a situation is even worse - when the issue isn't clearly 100% reproducible.

At my case mostly POP3 accounts are in use.
Comment 13 Tobias Koenig 2012-02-11 08:51:39 UTC
Hej,

can you enable the filter log 'Tools -> Filter Log Viewer' and attach the log file to this report, please?
This makes it easier for us to understand what's going on.

Ciao,
Tobias
Comment 14 Tobias Koenig 2012-02-11 23:40:31 UTC
needs info
Comment 15 Andrew Gaydenko 2012-02-11 23:53:55 UTC
Tobias, I have started the log and will attach it later.
Comment 16 Andrew Gaydenko 2012-02-13 02:20:42 UTC
Created attachment 68742 [details]
filter log

Somewhere at the end of the log the was situation with messages which were marked as read but weren't moved to assigned folder (I have few pop3 boxes with auto-checking turned on).
Comment 17 Andrew Gaydenko 2012-02-22 13:59:25 UTC
Tobias, 

Changing to NEW as far as a log was attached.
Comment 18 Sérgio Basto 2012-03-01 17:03:28 UTC
yeah spam filters doesn't work , because have 2 rules on Fedora 16 updated 
cat /var/log/rpmpkgs | grep ^kde
kde-filesystem-4-38.fc15.x86_64
kde-i18n-3.5.10-15.fc15.noarch
kde-i18n-Portuguese-3.5.10-15.fc15.noarch
kde-l10n-4.7.4-1.fc16.noarch
kde-l10n-Portuguese-4.7.4-1.fc16.noarch
kde-partitionmanager-1.0.3-5.20120205svn.fc16.x86_64
kde-plasma-activitymanager-0.5-2.fc16.x86_64
kde-plasma-daisy-0.0.4.25-2.fc15.x86_64
kde-plasma-ihatethecashew-0.4-3.fc15.x86_64
kde-plasma-qstardict-1.0-2.fc16.x86_64
kde-plasma-quickaccess-0.8.1-5.fc16.x86_64
kde-plasma-runcommand-2.4-1.fc16.x86_64
kde-plasma-smooth-tasks-0.0.1-0.2.wip20101106.fc15.x86_64
kde-plasma-translatoid-1.30-3.svn01092011.fc15.x86_64
kde-plasma-yawp-0.4.2-1.fc16.x86_64
kde-settings-4.7-14.fc16.noarch
kde-settings-kdm-4.7-14.fc16.noarch
kde-settings-ksplash-4.7-14.fc16.noarch
kde-settings-plasma-4.7-14.fc16.noarch
kde-settings-pulseaudio-4.7-14.fc16.noarch
kde-style-phase-4.7.4-1.fc16.x86_64
kdeaccessibility-4.7.4-1.fc16.x86_64
kdeaccessibility-libs-4.7.4-1.fc16.x86_64
kdeadmin-4.7.4-1.fc16.x86_64
kdeartwork-4.7.4-1.fc16.x86_64
kdeartwork-kxs-4.7.4-1.fc16.x86_64
kdeartwork-screensavers-4.7.4-1.fc16.x86_64
kdeartwork-sounds-4.7.4-1.fc16.noarch
kdeartwork-wallpapers-4.7.4-1.fc16.noarch
kdebase-4.7.4-2.fc16.x86_64
kdebase-libs-4.7.4-2.fc16.x86_64
kdebase-runtime-4.7.4-2.fc16.x86_64
kdebase-runtime-debuginfo-4.7.4-2.fc16.x86_64
kdebase-runtime-flags-4.7.4-2.fc16.noarch
kdebase-runtime-libs-4.7.4-2.fc16.x86_64
kdebase-workspace-4.7.4-9.fc16.x86_64
kdebase-workspace-devel-4.7.4-9.fc16.x86_64
kdebase-workspace-libs-4.7.4-9.fc16.x86_64
kdeedu-4.7.4-1.fc16.noarch
kdeedu-math-4.7.4-1.fc16.noarch
kdegames-4.7.4-1.fc16.x86_64
kdegames-libs-4.7.4-1.fc16.x86_64
kdegames-minimal-4.7.4-1.fc16.x86_64
kdegames3-3.5.10-10.fc15.x86_64
kdegames3-libs-3.5.10-10.fc15.x86_64
kdegraphics-4.7.4-1.fc16.noarch
kdegraphics-libs-4.7.4-1.fc16.noarch
kdegraphics-strigi-analyzer-4.7.4-1.fc16.x86_64
kdegraphics-thumbnailers-4.7.4-1.fc16.x86_64
kdelibs-4.7.4-1.fc16.x86_64
kdelibs-common-4.7.4-1.fc16.x86_64
kdelibs-debuginfo-4.7.4-1.fc16.x86_64
kdelibs-devel-4.7.4-1.fc16.x86_64
kdelibs3-3.5.10-31.fc16.i686
kdelibs3-3.5.10-31.fc16.x86_64
kdemultimedia-4.7.4-2.fc16.x86_64
kdemultimedia-common-4.7.4-2.fc16.noarch
kdemultimedia-dragonplayer-4.7.4-2.fc16.x86_64
kdemultimedia-extras-freeworld-4.7.0-2.fc16.x86_64
kdemultimedia-juk-4.7.4-2.fc16.x86_64
kdemultimedia-kio_audiocd-4.7.4-2.fc16.x86_64
kdemultimedia-kmix-4.7.4-2.fc16.x86_64
kdemultimedia-kscd-4.7.4-2.fc16.x86_64
kdemultimedia-libs-4.7.4-2.fc16.x86_64
kdenetwork-4.7.4-1.fc16.x86_64
kdenetwork-common-4.7.4-1.fc16.noarch
kdenetwork-fileshare-samba-4.7.4-1.fc16.x86_64
kdenetwork-kdnssd-4.7.4-1.fc16.x86_64
kdenetwork-kget-4.7.4-1.fc16.x86_64
kdenetwork-kget-libs-4.7.4-1.fc16.x86_64
kdenetwork-kopete-4.7.4-1.fc16.x86_64
kdenetwork-kopete-libs-4.7.4-1.fc16.x86_64
kdenetwork-kppp-4.7.4-1.fc16.x86_64
kdenetwork-krdc-4.7.4-1.fc16.x86_64
kdenetwork-krdc-libs-4.7.4-1.fc16.x86_64
kdenetwork-krfb-4.7.4-1.fc16.x86_64
kdenetwork-krfb-libs-4.7.4-1.fc16.x86_64
kdenlive-0.8.2.1-1.fc16.x86_64
kdepim-4.7.4-1.fc16.x86_64
kdepim-libs-4.7.4-1.fc16.x86_64
kdepim-runtime-4.7.4-1.fc16.x86_64
kdepim-runtime-libs-4.7.4-1.fc16.x86_64
kdepimlibs-4.7.4-2.fc16.x86_64
kdepimlibs-akonadi-4.7.4-2.fc16.x86_64
kdepimlibs-devel-4.7.4-2.fc16.x86_64
kdeplasma-addons-4.7.4-1.fc16.x86_64
kdeplasma-addons-libs-4.7.4-1.fc16.x86_64
kdesdk-4.7.4-1.fc16.x86_64
kdesdk-cervisia-4.7.4-1.fc16.x86_64
kdesdk-common-4.7.4-1.fc16.noarch
kdesdk-dolphin-plugins-4.7.4-1.fc16.x86_64
kdesdk-kapptemplate-4.7.4-1.fc16.x86_64
kdesdk-kcachegrind-4.7.4-1.fc16.x86_64
kdesdk-kioslave-4.7.4-1.fc16.x86_64
kdesdk-kmtrace-4.7.4-1.fc16.x86_64
kdesdk-kmtrace-libs-4.7.4-1.fc16.x86_64
kdesdk-kompare-4.7.4-1.fc16.x86_64
kdesdk-kompare-libs-4.7.4-1.fc16.x86_64
kdesdk-kpartloader-4.7.4-1.fc16.x86_64
kdesdk-kstartperf-4.7.4-1.fc16.x86_64
kdesdk-kuiviewer-4.7.4-1.fc16.x86_64
kdesdk-okteta-4.7.4-1.fc16.x86_64
kdesdk-okteta-libs-4.7.4-1.fc16.x86_64
kdesdk-poxml-4.7.4-1.fc16.x86_64
kdesdk-scripts-4.7.4-1.fc16.noarch
kdesdk-strigi-analyzer-4.7.4-1.fc16.x86_64
kdesdk-umbrello-4.7.4-1.fc16.x86_64
kdesvn-1.5.5-3.fc15.x86_64
kdetoys-4.7.4-1.fc16.x86_64
kdetv-0.8.9-13.fc12.x86_64
kdeutils-4.7.4-1.fc16.x86_64
kdeutils-ark-4.7.4-1.fc16.x86_64
kdeutils-ark-libs-4.7.4-1.fc16.x86_64
kdeutils-common-4.7.4-1.fc16.noarch
kdeutils-filelight-4.7.4-1.fc16.x86_64
kdeutils-kcalc-4.7.4-1.fc16.x86_64
kdeutils-kcharselect-4.7.4-1.fc16.x86_64
kdeutils-kdf-4.7.4-1.fc16.x86_64
kdeutils-kfloppy-4.7.4-1.fc16.x86_64
kdeutils-kgpg-4.7.4-1.fc16.x86_64
kdeutils-kremotecontrol-4.7.4-1.fc16.x86_64
kdeutils-kremotecontrol-libs-4.7.4-1.fc16.x86_64
kdeutils-ktimer-4.7.4-1.fc16.x86_64
kdeutils-kwalletmanager-4.7.4-1.fc16.x86_64
kdeutils-minimal-4.7.4-1.fc16.x86_64
kdeutils-printer-applet-4.7.4-1.fc16.x86_64
kdeutils-superkaramba-4.7.4-1.fc16.x86_64
kdeutils-superkaramba-libs-4.7.4-1.fc16.x86_64
kdeutils-sweeper-4.7.4-1.fc16.x86_64
kdevplatform-1.2.3-2.fc16.x86_64
kdevplatform-libs-1.2.3-2.fc16.x86_64
kdewebdev-3.5.10-11.fc16.x86_64
kdewebdev-libs-3.5.10-11.fc16.x86_64
Comment 19 Andrew Gaydenko 2012-03-01 17:12:32 UTC
To users with affected KMail: please, don't say "me too" only. Instead add your bug's votes also!
Comment 20 Graeme Hewson 2012-04-28 11:27:31 UTC
I found the problem in my case only happened when I was applying filters to selected POP3 accounts. See my update to bug 285802.
Comment 21 Andrew Gaydenko 2012-10-03 09:43:23 UTC
Is there any real  hope the issue will be resolved? (without any claim, just wonder).
Comment 22 Sérgio Basto 2012-10-04 16:26:39 UTC
Should be modify to critical bug. 
Very critical indeed .
Still have mutil problems in filters on 4.9.2 Fedora 17 .
Comment 23 S. Burmeister 2012-10-04 17:36:11 UTC
(In reply to comment #22)
> Should be modify to critical bug. 
> Very critical indeed .
> Still have mutil problems in filters on 4.9.2 Fedora 17 .

Since people do not supply logs etc. it does not seem to be that critical.

For me with imap (online not offline) and multiple filters work as expected. So those it does not work for should supply the means to reproduce and fix the bug instead of just adding useless comments or lists to the bug report.
Comment 24 Andrew Gaydenko 2012-10-04 17:55:23 UTC
(In reply to comment #23)
> Since people do not supply logs etc...
I have attached the log. It was just ignored.
> ... it does not seem to be that critical.
Votes say it is critical. For me this bug plus few others are sufficient to be a reason to migrate to another mail client. Yes, agree, for such migrated users pity multi-year kmail2's not-ready-for-use state isn't critical any more.
> So those it does not work for should supply the means to reproduce and fix
> the bug instead of just adding useless comments or lists to the bug report.
Please, point your comment place which is useful. In another words, follow you mantra yourself, please. Or. at least, try to be correct wrt facts. Nothing personal. Good luck.
Comment 25 S. Burmeister 2012-10-04 19:13:47 UTC
If something is critical to somebody that somebody is committed to fixing it. I do not see that commitment. No way to reproduce, no logs, no fixes supplied.

And since you seem to fail to get the point a little reminder: for those that do not have the issue (including me) it is impossible to supply any information on what does not work including logs etc. Seems straight forward, does it not?
Comment 26 Andrew Gaydenko 2012-10-04 19:57:15 UTC
(In reply to comment #25)
> If something is critical to somebody that somebody is committed to fixing
> it. 

It isn't a fact and it isn't even a tendency. It is your fantasy.

> I do not see that commitment. No way to reproduce, no logs, no fixes
> supplied.

And - what? Everybody has own way and opportunities to supply feedback. And, again, do you read comments? I have supplied a log an age ago.

> And since you seem to fail to get the point a little reminder: for those
> that do not have the issue (including me) it is impossible to supply any
> information on what does not work including logs etc. Seems straight
> forward, does it not?

If you haven't an issue and haven't a decision for others' issue why do you teach affected users to go? Please, stop trolling. I will not keep the conversation. I'll just report to administration .
Comment 27 S. Burmeister 2012-10-04 20:15:17 UTC
(In reply to comment #26)
> If you haven't an issue and haven't a decision for others' issue why do you
> teach affected users to go? Please, stop trolling. I will not keep the
> conversation. I'll just report to administration .

Because people like you wonder why things don't get fixed and add complaints to reports instead of useful info. I thought I'd help you to help the devs, but I see that that's futile.
Comment 28 Sérgio Basto 2012-10-04 20:26:29 UTC
One of the secondary effects is SPAM, filters doesn't work as expect (And using spam wizard). With some pop3 accounts. What logs do you want ? . Shouldn't be hard to replicate since, many people report it.
Comment 29 S. Burmeister 2012-10-04 20:39:45 UTC
From a dev's email to a mailinglist (today):

"I read about the POP3/disconnected IMAP filtering problems.

What I can say I extensively use both and have filters for both. The few 
things I noticed so far are:
- after resume from hibernation, the filter agent can be busy (not sure why) 
for a few minutes
- after resume from hibernation if some mail arrived meantime to the POP3 
inbox, they might not be filtered
- ocassionally (but not more than 1/day), I get a conflict dialog

What I didn't try yet is spam filtering on online IMAP.

In any case, there will be a KDEPIM development sprint next weekend. Test 
the latest version (if from source, even better, if not, try the latest 
release), gather as much information/logs as possible, find how problems can 
be reproduced and report them as bugs, so I can look at them at the sprint."

Same dev, different email:

"> Is in kmail a possibility/option to write debug-logs ?
> To get a chance to send some info ?

Sure, there is. There is the filter log (see the Tools menu), then you can 
enabled the different log areas for KDE applications with "kdebugdialog". 
Here you should enable the one for the akonadi filter agent (and disable 
most of the others to not get a too big log) and restart akonadi from a 
console with "akonadictl restart". You will see the debug logs then."
Comment 30 Myriam Schweingruber 2012-10-06 11:20:48 UTC
Could we please all calm down? There is a log file attached, although rather old, please also get a log file for KDE 4.9.2 ASAP as the developer sprint is to be held next week.

For a bug to bet set to critical there are some criteria this bug does not fulfill, like data loss, which is not the case.
Comment 31 Sérgio Basto 2012-10-07 14:10:45 UTC
(In reply to comment #28)
> One of the secondary effects is SPAM, filters doesn't work as expect (And
> using spam wizard). With some pop3 accounts. What logs do you want ? .
> Shouldn't be hard to replicate since, many people report it.

Hi, SPAM filters seems (now) works as they should  , so maybe you could close the bug with close with Current release 4.9.2

I have update
http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kmail2_reset_nepomuk_and_akonadi_system (not finish yet)

After a reset seems things are working more or less ...
Comment 32 Sérgio Basto 2012-10-07 14:20:04 UTC
> After a reset seems things are working more or less ...
Some filter rules seems just effective works after, log out and log in a kdm session. 
I don't know if helps or not but also did rm -rf ~/.kde/cache-*~/.kde/ socket-* and ~/.kde/tmp-*
Comment 33 Anders Lund 2012-10-11 07:11:46 UTC
I have this problem in KDE 4.9.2. I am not sure if all spam messages are affected, but I will investigate and add my findings here.
Comment 34 András Manţia 2012-10-13 18:49:15 UTC
Please test with 4.9.2 and report if still fails (with filter log).
Comment 35 Hussam Al-Tayeb 2012-10-14 00:45:52 UTC
works for me with 4.9.2 (was broken in the past).
Comment 36 Anders Lund 2012-10-14 08:55:06 UTC
Trying to pay attention today to what happens...
I have a message that was filtered, and correctly moved to spam and marked as read.
Later this message reappeared in my inbox, now marked as read, and with a indicator in the header that it is spam.
It turns out that it is still in the IMAP inbox, so akonadi didn't move it, but copy it to spam.

I have quite a few of theese during a day :(

Sometimes the graphical spam indicator is not shown, even though the header exists in the email when I view the source.

Judging from the filter, akonadi thinks it moves the message into spam each time it reappears, but there is only one copy in spam, and one in inbox.

So in my case it looks like the problem is not the filter chain, but akonadis general confused-ness about what is what and where it is.
Comment 37 Anders Lund 2012-10-14 08:59:18 UTC
Created attachment 74535 [details]
Sample filter log

Not sure if this is usable, it looks like everything is fine - but it is not. Maybe this proves that the problem (at least in my case) is not caused by the filter.

however, I am using IMAP. The problem might be different when using POP3.
Comment 38 András Manţia 2012-10-14 09:07:08 UTC
Can you also get the IMAP log for the case when the filter moves the mail?
export KIMAP_LOGFILE=/tmp/imap.log
akonadictl restart
and then see the imap.log.XXX files.
Comment 39 S. Burmeister 2012-10-14 09:57:32 UTC
Isn't spam emails re-appearing in the inbox a different issue than this bug report refers to?
Comment 40 Anders Lund 2012-10-14 10:09:35 UTC
(In reply to comment #39)
> Isn't spam emails re-appearing in the inbox a different issue than this bug
> report refers to?

Yes. But it can easily be confused, since if a message in inbox looks like it should be in spam, it is a near thought that it was marked as read, but not moved.

However, the move is the first action, and the filter log shows that all actions was applied, or tried to be applied.
Comment 41 Denis Kurz 2016-09-24 20:45:20 UTC
This bug has only been reported for versions older than KDEPIM 4.14 (at most akonadi-1.3). Can anyone tell if this bug still present?

If noone confirms this bug for a recent version of akonadi (part of KDE Applications 15.08 or later), it gets closed in about three months.
Comment 42 Andrew Gaydenko 2016-09-24 21:02:52 UTC
As far as I don't use kmail anymore, I can not estimate current state. So closing the issue as invalid.