Version: 1.5.94 (using KDE 3.1.94 (3.2 Beta 2), compiled sources) Compiler: gcc version 2.96 20000731 (Red Hat Linux 7.2 2.96-112.7.2) OS: Linux (i686) release 2.4.22 I was using KDE-3.1.4 before with a spam filter set up as: Name: SpamOracle_marked? Match any of the following: X-Spam doesn't contain yes X-Spam doesn't contain no Action: remove header X-Spam pipe through spamoracle mark to incoming + manual filtering Name: SpamOracle_isSpam match all of the following X-Spam contains yes Action: remove header X-Spam file into folder Junk-mail This worked, but I have now upgraded to 3.2, and the mails are no more filtered. Can it be, that if an incoming mail does not contain the X-Spam header at all, the first rules are not matched any more ?
Another hint: If I mark the junkmails in the inbox and press CTRL-J (apply filters), that still works and then moves the mails to my junk-mail folder.
This is the same bug as #67078. I have updated from CVS now and I can still reproduce this. Do you want me to test something ?
This works here like a charm. What I don't understand is the following: Match any of the following: X-Spam doesn't contain yes X-Spam doesn't contain no So the filter should get triggered in any case, the test evaluates to true no matter what X-Spam contains. You could use as well size >= 0. For me it makes no difference whether a X-Spam header is in the message or not. The filter is triggered. You write, you've updated from cvs. Can you start KMail in a console from command line and watch the output. You should be able to see lines like this: kmail: ####### KMFilter::process: going to apply action file into folder "drafts" If you see differences between filtering on incoming messages and manual filtering then I were interested in the output as well as in the filter definitions in your /home/<user>/.kde/share/config/kmailrc file.
Subject: Re: filters don't work anymore as in 3.1.4 On Wednesday 31 December 2003 14:40, Andreas Gungl wrote: > ------- Additional Comments From a.gungl@gmx.de 2003-12-31 14:40 ------- > This works here like a charm. What I don't understand is the following: > > Match any of the following: > X-Spam doesn't contain yes > X-Spam doesn't contain no > > So the filter should get triggered in any case, the test evaluates to true > no matter what X-Spam contains. You could use as well size >= 0. I think I found this on some spamorcale/kmail HOWTO page when I set it up. I already modified the filter to use size>0 > > For me it makes no difference whether a X-Spam header is in the message or > not. The filter is triggered. You write, you've updated from cvs. Can you > start KMail in a console from command line and watch the output. You should > be able to see lines like this: > > kmail: ####### KMFilter::process: going to apply action file into folder > "drafts" I did that and attached the log file. You only find the ... action pipe through "spamoracle mark" ... twice - this was when I did CTRL-J on two different mails _after_ they have bee received _without_ being sent through the filter. > > If you see differences between filtering on incoming messages and manual > filtering then I were interested in the output as well as in the filter > definitions in your /home/<user>/.kde/share/config/kmailrc file. also attached. Created an attachment (id=3899) kmail.log.gz Created an attachment (id=3900) kmail.filters.gz
Martin, in your (second, extended) log I've found: kmail: ####### KMFilterMgr::process: going to be apply filter rule Match any of the following: "<size>" <greater> "0" kmail: Can you countercheck with: - Match ALL of the following - "<size>" <greaterOrEqual> "0" (I mean size >= 0, don't know the exact phrase in the moment.) If that doesn't shed some light on the problem, then it's getting really difficult. I think, we'll have to add more debug output. At least the rule is used and somehow it's not considered to match the message. I hope, we can solve the issue before 3.2 is tagged.
Subject: Re: filters don't work anymore as in 3.1.4 On Wednesday 31 December 2003 22:15, Andreas Gungl wrote: > ------- Additional Comments From a.gungl@gmx.de 2003-12-31 22:15 ------- > Martin, in your (second, extended) log I've found: > > kmail: ####### KMFilterMgr::process: going to be apply filter rule Match > any of the following: "<size>" <greater> "0" > kmail: > > Can you countercheck with: > - Match ALL of the following > - "<size>" <greaterOrEqual> "0" (I mean size >= 0, don't know the exact > phrase in the moment.) > > If that doesn't shed some light on the problem, then it's getting really > difficult. I think, we'll have to add more debug output. At least the rule > is used and somehow it's not considered to match the message. I hope, we > can solve the issue before 3.2 is tagged. First of all - A happy new year! I changed now the rule to >=0, I have still the same problem but some changes also: The interesting sections are: kmail: ####### KMFilterMgr::process: going to be apply filter rule Match all of the following: "<size>" <greater-or-equal> "0" kmail: kmail: ####### KMFilterMgr::process: filter rule did match kmail: ####### KMFilter::process: going to apply action remove header "X-Spam" kmail: ####### KMFilter::process: going to apply action pipe through "spamoracle mark" kmail: ####### KMFilterMgr::process: going to be apply filter rule Match all of the following: "X-Spam" <contains> "yes" "Subject" <contains-not> "[Bug" kmail: kmail: ####### KMFilterMgr::process: going to be apply filter rule Match all of the following: " Interesting here is, that on incoming mail the filter gets now applied, but it seems that kmail did not recognize the added X-Spam header afterwards (in the rule after this one). But more important, it seems that what comes out of spamoracle with that filter is NOT the mail which is then stored in kmail, because if I look at the mail-source (pressing "V"), I don't find the X-Spam header! So maybe that is the source of the whole problem. Was there a change in the handling of receiving mails from 3.1.4 to 3.2, so that the received mail is somehow only temporary and only this one is modified by spamorcale ? Now look what happend when I pressed CTRL-J: ... kmail: kmail: ####### KMFilterMgr::process: going to be apply filter rule Match all of the following: "<size>" <greater-or-equal> "0" kmail: kmail: ####### KMFilterMgr::process: filter rule did match kmail: ####### KMFilter::process: going to apply action remove header "X-Spam" kmail: ####### KMFilter::process: going to apply action pipe through "spamoracle mark" kmail: ####### KMFilterMgr::process: going to be apply filter rule Match all of the following: "X-Spam" <contains> "yes" "Subject" <contains-not> "[Bug" kmail: kmail: ####### KMFilterMgr::process: filter rule did match kmail: ####### KMFilter::process: going to apply action remove header "X-Spam" kmail: ####### KMFilter::process: going to apply action file into folder "JUNK-mail" kmail: ####### KMFilter::process: going to apply action mark as "U" k So it seems by applying the filters manually, the mail is then really modified and therefore also used by the next filter. I attach the full log. Created an attachment (id=3912) kmail.log.goe0.gz
Subject: Re: filters don't work anymore as in 3.1.4 This looks like a bug which has been considered to be fixed. ############### (commited 22.12.2003) CVS commit by kloecker: Fix bug 69860 (output of pipe through sometimes not used). This also fixes ad hoc filters. Based on patch by Andreas Gungl. Reviewed by Don. ############### I need some more information to debug that problem. First I need to know what change triggered the rule to use spamoracle. Was it "size >= 0" or "Match all ...". Second I would like to know how the message looks like after spamoracle was applied. Can you create a small shell script (one-liner: spamoracle mark | tee ~/filter result ) and use that in the Pipe Through action. On Donnerstag, 1. Januar 2004 13:47, Martin Koller wrote: > On Wednesday 31 December 2003 22:15, Andreas Gungl wrote: > > ------- Additional Comments From a.gungl@gmx.de 2003-12-31 22:15 > > ------- Martin, in your (second, extended) log I've found: > > > > kmail: ####### KMFilterMgr::process: going to be apply filter rule > > Match any of the following: "<size>" <greater> "0" > > kmail: > > > > Can you countercheck with: > > - Match ALL of the following > > - "<size>" <greaterOrEqual> "0" (I mean size >= 0, don't know the exact > > phrase in the moment.) > > > > If that doesn't shed some light on the problem, then it's getting > > really difficult. I think, we'll have to add more debug output. At > > least the rule is used and somehow it's not considered to match the > > message. I hope, we can solve the issue before 3.2 is tagged. > > First of all - A happy new year! > > I changed now the rule to >=0, I have still the same problem but some > changes also: > > The interesting sections are: > > kmail: ####### KMFilterMgr::process: going to be apply filter rule Match > all of the following: > "<size>" <greater-or-equal> "0" > kmail: > kmail: ####### KMFilterMgr::process: filter rule did match > kmail: ####### KMFilter::process: going to apply action remove header > "X-Spam" kmail: ####### KMFilter::process: going to apply action pipe > through "spamoracle mark" > kmail: ####### KMFilterMgr::process: going to be apply filter rule Match > all of the following: > "X-Spam" <contains> "yes" > "Subject" <contains-not> "[Bug" > kmail: > kmail: ####### KMFilterMgr::process: going to be apply filter rule Match > all of the following: > " > > Interesting here is, that on incoming mail the filter gets now applied, > but it seems that kmail did not recognize the added X-Spam header > afterwards (in the rule after this one). > > But more important, it seems that what comes out of spamoracle with that > filter is NOT the mail which is then stored in kmail, because if I look > at the mail-source (pressing "V"), I don't find the X-Spam header! > So maybe that is the source of the whole problem. > > Was there a change in the handling of receiving mails from 3.1.4 to 3.2, > so that the received mail is somehow only temporary and only this one is > modified by spamorcale ? > > Now look what happend when I pressed CTRL-J: > > ... > kmail: > kmail: ####### KMFilterMgr::process: going to be apply filter rule Match > all of the following: > "<size>" <greater-or-equal> "0" > kmail: > kmail: ####### KMFilterMgr::process: filter rule did match > kmail: ####### KMFilter::process: going to apply action remove header > "X-Spam" kmail: ####### KMFilter::process: going to apply action pipe > through "spamoracle mark" > kmail: ####### KMFilterMgr::process: going to be apply filter rule Match > all of the following: > "X-Spam" <contains> "yes" > "Subject" <contains-not> "[Bug" > kmail: > kmail: ####### KMFilterMgr::process: filter rule did match > kmail: ####### KMFilter::process: going to apply action remove header > "X-Spam" kmail: ####### KMFilter::process: going to apply action file > into folder "JUNK-mail" > kmail: ####### KMFilter::process: going to apply action mark as "U" > k > > So it seems by applying the filters manually, the mail is then really > modified and therefore also used by the next filter. > > I attach the full log. > > > Created an attachment (id=3912) > --> (http://bugs.kde.org/attachment.cgi?id=3912&action=view) > kmail.log.goe0.gz
Subject: Re: filters don't work anymore as in 3.1.4 On Thursday 01 January 2004 14:28, Andreas Gungl wrote: > This looks like a bug which has been considered to be fixed. > > ############### (commited 22.12.2003) > CVS commit by kloecker: > > Fix bug 69860 (output of pipe through sometimes not used). This also fixes > ad hoc filters. Based on patch by Andreas Gungl. Reviewed by Don. > ############### > > I need some more information to debug that problem. First I need to know > what change triggered the rule to use spamoracle. Was it "size >= 0" or > "Match all ...". Second I would like to know how the message looks like > after spamoracle was applied. Can you create a small shell script > (one-liner: spamoracle mark | tee ~/filter result ) and use that in the > Pipe Through action. OK, I think we've got it: The filter on incoming message does pass spamoracle without modification - that is due to the fact, that spamoracle does not recognize the input as a mail I think. I checked why, and have seen that the incoming message does not have the very first line - From dermalptch1zx@rogers.com Thu Jan 1 14:45:45 2004 The input to spamoracle starts with the Return-Path: <dermalptch1zx@rogers.com> Header. When I save the received mail as plain text, I see this first "From" line, and when I feed this by hand through spamoracle, it then does get the X-Spam header!! The second issue regarding the ">=0" match: I changed it back to match "any", and it still works (matches). I double checked and changed also back to ">0", which does _not_ match. So this seems to be a second bug. - -- Best regards/Sch
Martin, can you check that KMail from KDE 3.1.x sent the complete mail to spamoracle (including the From line)? I can confirm that the From line in the current version is not sent to the pipe. Reading your last comment I'm not sure whether you piped the messages manually (outside KMail) through spamoracle. If so, that could have helped finding the problems. I'll try to fix the size issue until the weekend, if it's not possible I'll file a new report.
I've filed a new report for the size problem (#71614). Regarding the "From line" I've found that the line is only a delimiter in mbox files. It doesn't really belong to the message which becomes obvious when you look at messages stored in maildir format. So this is IMO a problem of spamoracle. You may consider to write a wrapper script which provides a dummy "From line" to make spamoracle work. Or you may consider to switch to another tool like bogofilter or spamassassin. Is it okay for you when I close the report with respect to the new one for the size problem? Anyway, thanks for your patience and your help on tracking it down.
Subject: Re: filters don't work anymore as in 3.1.4 On Thursday 01 January 2004 21:46, Andreas Gungl wrote: > > Regarding the "From line" I've found that the line is only a delimiter in > mbox files. It doesn't really belong to the message which becomes obvious > when you look at messages stored in maildir format. So this is IMO a > problem of spamoracle. You may consider to write a wrapper script which > provides a dummy "From line" to make spamoracle work. Or you may consider > to switch to another tool like bogofilter or spamassassin. Andreas, I found now that in fact I was wrong with my assumption. It is not the From line what puzzles spamoracle - it is the line delimiter! The mail which did not work had DOS delimiters. If I save the mail, which was not correctly parsed by spamoracle, as UNIX fileformat, it then works. I also checked with exactly the same mail 1) kmail from KDE 3.1.3 => piped to spamoracle with UNIX delimiters 2) kmail from KDE 3.2 CVS => piped to spamoracle with DOS delimiters ??? Why that change ? Can this be reverted ? > > Is it okay for you when I close the report with respect to the new one for > the size problem? No, see above. > Anyway, thanks for your patience and your help on > tracking it down. You're welcome. I really want to solve that problem, as it worked perfectly with KDE 3.1.x, and I think this bug will puzzle a lot of other people. - -- Best regards/Sch
Do I understand correctly that filtering incoming messages (from a POP server?) doesn't work, but that filtering messages with Ctrl+J does work? If yes, then I think I know what the problem is and how to fix it.
Subject: Re: filters don't work anymore as in 3.1.4 On Saturday 03 January 2004 17:41, Ingo Kl
The fix didn't make it into KDE 3.2 but it's possible it will be ported back to 3.2.1 - in any case it will be included in the next KDE PIM release.