Bug 115287 - DSpam Filter for Anti-Spam Wizard
Summary: DSpam Filter for Anti-Spam Wizard
Status: RESOLVED FIXED
Alias: None
Product: kmail
Classification: Applications
Component: general (show other bugs)
Version: 1.8.1
Platform: Gentoo Packages Linux
: NOR wishlist
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-10-28 22:14 UTC by Charlie
Modified: 2007-09-14 12:17 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Adds DSpam header support to kmail KDE4 branch (9.34 KB, patch)
2006-05-29 05:17 UTC, Kovid Goyal
Details
DSpam header support for KDE3.5 branch (9.77 KB, patch)
2006-05-29 05:18 UTC, Kovid Goyal
Details
Fix a typo in the KDE4 patch (9.34 KB, patch)
2006-05-29 20:53 UTC, Kovid Goyal
Details
local dspam kmail.antispamrc filter (704 bytes, text/plain)
2007-05-31 09:49 UTC, Jenya
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Charlie 2005-10-28 22:14:44 UTC
Version:           1.8.1 (using KDE KDE 3.4.1)
Installed from:    Gentoo Packages
Compiler:          GCC 3.4.4 
OS:                Linux

DSpam looks to be a promising filter for spam. Would be nice to have it integrated into Kmail's Anti-Spam Wizard.
Comment 1 Kovid Goyal 2006-05-29 05:17:43 UTC
Created attachment 16319 [details]
Adds DSpam header support to kmail KDE4 branch

Adds support for DSpam headers, both X-DSpam-Probability and
X-DSpam-Confidence. The confidence is displayed as a % next to the spam meter
Comment 2 Kovid Goyal 2006-05-29 05:18:30 UTC
Created attachment 16320 [details]
DSpam header support for KDE3.5 branch
Comment 3 Kovid Goyal 2006-05-29 05:33:00 UTC
You can see a screenshot and download the gentoo ebuild for kmail-3.5.2 at http://www.kde-look.org/content/show.php?content=39974
Comment 4 Kovid Goyal 2006-05-29 20:53:04 UTC
Created attachment 16341 [details]
Fix a typo in the KDE4 patch
Comment 5 Andreas Grundler 2006-06-03 23:22:21 UTC
*** This bug has been confirmed by popular vote. ***
Comment 6 Andreas Gungl 2006-06-07 09:44:43 UTC
I plan to include your patch into the 3.5.4 release. However I have to make sure whether the string changes in your patch are accepted.
In any case I will include the detection pattern for the antispam.rc file.
Comment 7 Kovid Goyal 2006-06-07 17:21:58 UTC
Thanks. 
Comment 8 Andreas Gungl 2006-06-28 09:16:41 UTC
SVN commit 555658 by gungl:

- anti-spam wizard support for BsFilter (from Yuya Nishihara <yuya at tcha dot org>)
- anti-spam wizard support for DSpam (from Kovid Goyal <kovid@theory.caltech.edu>)
- better feedback during filtering, msg count is visible in the statusbar

FEATURE: 115287
FEATURE: 128060
GUI:



 M  +46 -3     kmail.antispamrc  
 M  +10 -5     kmcommands.cpp  
 M  +15 -8     kmheaders.cpp  


--- branches/KDE/3.5/kdepim/kmail/kmail.antispamrc #555657:555658
@@ -1,5 +1,5 @@
 [General]
-tools=8
+tools=10
 
 [Spamtool #1]
 Ident=spamassassin
@@ -71,7 +71,7 @@
 [Spamtool #4]
 Ident=gmx-antispam
 Version=1
-Priority=60
+Priority=71
 VisibleName=GMX Spam Filter
 Executable=echo
 URL=http://www.gmx.net/de/produkte/virenschutz/index.html
@@ -142,7 +142,7 @@
 [Spamtool #8]
 Ident=spamassassin
 Version=1
-Priority=40
+Priority=41
 VisibleName=SpamAssassin 3.x (Perl)
 Executable=spamassassin -V | grep "SpamAssassin version 3"
 URL=http://spamassassin.org
@@ -162,3 +162,46 @@
 ScoreType=Adjusted
 ScoreValueRegexp=(?:hits|score)=([\d\.-]+)\s
 ScoreThresholdRegexp=required=([\d\.-]+)\s
+
+[Spamtool #9]
+Ident=bsfilter
+Version=1
+Priority=55
+VisibleName=Bsfilter
+Executable=bsfilter --help > /dev/null
+URL=http://bsfilter.org/
+PipeFilterName=Bsfilter Check
+PipeCmdDetect=bsfilter --pipe --insert-flag --insert-probability
+ExecCmdSpam=bsfilter --add-spam --update
+ExecCmdHam=bsfilter --add-clean --update
+DetectionHeader=X-Spam-Flag
+DetectionPattern=Yes
+DetectionPattern2=
+DetectionOnly=0
+UseRegExp=0
+SupportsBayes=1
+SupportsUnsure=0
+ScoreName=Bsfilter
+ScoreHeader=X-Spam-Probability
+ScoreType=Decimal
+ScoreValueRegexp=(.+)
+ScoreThresholdRegexp=
+
+[Spamtool #10]
+Ident=dspam
+Version=1
+Priority=70
+VisibleName=DSpam 3.x
+URL=http://dspam.nuclearelephant.org
+HeadersOnly=yes
+DetectionHeader=X-DSPAM-Result
+DetectionPattern=Spam
+DetectionOnly=0
+UseRegExp=0
+ScoreName=DSpam
+SupportsUnsure=0
+ScoreHeader=X-DSPAM-Probability
+ConfidenceHeader=X-DSPAM-Confidence
+ScoreType=Decimal
+ScoreValueRegexp=([\d\.]+)\s*
+ScoreConfidenceRegexp=([\d\.]+)\s*
--- branches/KDE/3.5/kdepim/kmail/kmcommands.cpp #555657:555658
@@ -56,6 +56,7 @@
 
 #include <qtextcodec.h>
 #include <qpopupmenu.h>
+#include <qeventloop.h>
 
 #include <libemailfunctions/email.h>
 #include <kdebug.h>
@@ -1472,13 +1473,17 @@
     if( msg->parent() )
       kmkernel->filterMgr()->tempOpenFolder(msg->parent());
 
-  int counter = 0;
+  int msgCount = 0;
+  int msgCountToFilter = msgList.count();
   for (KMMessage *msg = msgList.first(); msg; msg = msgList.next()) {
+    int diff = msgCountToFilter - ++msgCount;
+    if ( diff < 10 || !( msgCount % 20 ) ) {
+      QString statusMsg = i18n("Filtering message %1 of %2");
+      statusMsg = statusMsg.arg( msgCount ).arg( msgCountToFilter );
+      KPIM::BroadcastStatus::instance()->setStatusMsg( statusMsg );
+      KApplication::kApplication()->eventLoop()->processEvents( QEventLoop::ExcludeUserInput, 50 );
+    }
     msg->setTransferInProgress(false);
-
-    if ( !( ++counter % 20 ) )
-      KApplication::kApplication()->processEvents( 50 );
-
     int filterResult = kmkernel->filterMgr()->process(msg, mFilter);
     if (filterResult == 2) {
       // something went horribly wrong (out of space?)
--- branches/KDE/3.5/kdepim/kmail/kmheaders.cpp #555657:555658
@@ -37,6 +37,7 @@
 #include <kdebug.h>
 
 #include <qbuffer.h>
+#include <qeventloop.h>
 #include <qfile.h>
 #include <qheader.h>
 #include <qptrstack.h>
@@ -1354,10 +1355,16 @@
     START_TIMER(filter);
 
     KCursorSaver busy( KBusyPtr::busy() );
-    int counter = 0;
+    int msgCount = 0;
+    int msgCountToFilter = msgList->count();
     for (KMMsgBase* msgBase=msgList->first(); msgBase; msgBase=msgList->next()) {
-      if ( !( ++counter % 20 ) )
-        KApplication::kApplication()->processEvents( 50 );
+      int diff = msgCountToFilter - ++msgCount;
+      if ( diff < 10 || !( msgCount % 20 ) ) {
+        QString statusMsg = i18n("Filtering message %1 of %2");
+        statusMsg = statusMsg.arg( msgCount ).arg( msgCountToFilter );
+        KPIM::BroadcastStatus::instance()->setStatusMsg( statusMsg );
+        KApplication::kApplication()->eventLoop()->processEvents( QEventLoop::ExcludeUserInput, 50 );
+      }
       int idx = msgBase->parent()->find(msgBase);
       assert(idx != -1);
       KMMessage * msg = msgBase->parent()->getMsg(idx);
@@ -1365,12 +1372,12 @@
       msg->setTransferInProgress(true);
       if ( !msg->isComplete() )
       {
-	FolderJob *job = mFolder->createJob(msg);
-	connect(job, SIGNAL(messageRetrieved(KMMessage*)),
-		SLOT(slotFilterMsg(KMMessage*)));
-	job->start();
+        FolderJob *job = mFolder->createJob(msg);
+        connect(job, SIGNAL(messageRetrieved(KMMessage*)),
+                     SLOT(slotFilterMsg(KMMessage*)));
+        job->start();
       } else {
-	if (slotFilterMsg(msg) == 2) break;
+        if (slotFilterMsg(msg) == 2) break;
       }
     }
     END_TIMER(filter);
Comment 9 Thomas McGuire 2007-05-18 20:38:47 UTC
Everybody: 
Could someone please attach one example mail which is scanned by DSPAM and therefore has the dspam header available? I need to test the "confidence" part.

Kovid Goyal:
Kovid, are you still there?
There is a little problem with the "confidence" support.
For non-dspam mails, for example mails scanned by spamassassin, it displays "confidence =-2%" in the tooltip.
This is with KDE4.
Could you fix this?

Comment 10 Kovid Goyal 2007-05-18 20:53:06 UTC
Here are the Dspam headers from a mail
X-DSPAM-Result: Innocent
X-DSPAM-Processed: Fri May 18 11:38:50 2007
X-DSPAM-Confidence: 0.9895
X-DSPAM-Improbability: 1 in 9406 chance of being spam
X-DSPAM-Probability: 0.0000
X-DSPAM-Signature: 2,464df2ba223371850694299
Comment 11 Thomas McGuire 2007-05-19 15:28:26 UTC
OK, I fixed it in rev. 666324.
Comment 12 Jenya 2007-05-31 09:49:38 UTC
Created attachment 20740 [details]
local dspam kmail.antispamrc filter

I use dspam locally on workstation in the office where no server side filtering
can be done since server side is an exchange server :(
Comment 13 Thomas McGuire 2007-06-07 18:51:43 UTC
>I use dspam locally on workstation in the office where no server side filtering can be done since server side is an exchange server :( 
Please clarify: Do you want to add a new spam tool (local dspam)?
If so, please open a new bug report.