Bug 358041

Summary: spamassassin filter deletes every incoming mail
Product: [Applications] kmail2 Reporter: Tom Mittelstädt <tom-kde.bugs>
Component: generalAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED WORKSFORME    
Severity: critical    
Priority: NOR    
Version: 5.1   
Target Milestone: ---   
Platform: Ubuntu   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Tom Mittelstädt 2016-01-15 19:39:57 UTC
From time to time, the generated spamassassin filter rules deletes every incoming mail.

The filter log says (localized in german. if you could provide me information how to get the english output i'll post it):
[19:49:29] Filterung beginnt bei Nachricht „Re: xxx“ von „xxx “ um „Freitag, 15. Januar 2016 19:45:47 UTC+01:00“ :
[19:49:29] Filterregeln werden geprüft: (auf alle Nachrichten zutreffen) "<size>" <greater> "0"
[19:49:29] Filterregeln wurden angewandt.
[19:49:29] Filteraktion anwenden: Durch Programm leiten "spamassassin -L"
[19:49:31] Filterregeln werden geprüft: (alle folgenden Bedingungen müssen zutreffen) "X-Spam-Flag" <equals> "YES"
[19:49:31] 1 = "X-Spam-Flag" <equals> "YES" (YES)
[19:49:31] Filterregeln wurden angewandt.
[19:49:31] Filteraktion anwenden: In Ordner verschieben "Lokale Ordner/Papierkorb"
[19:49:31] Filteraktion anwenden: Markieren als "Gelesen"

The problem: [19:49:31] 1 = "X-Spam-Flag" <equals> "YES" (YES)

However, *there is no X-Spam-Flag header the mail*. It seems as if the mail has not been touched by spamassassin at all.

If I send the mail manually through spamassassin it is recognized as ham.

The problem started with KF5.

Reproducible: Sometimes

Steps to Reproduce:
receive emails. from time to time emails are apperently not processed by spamassassin, but marked as spam.

Actual Results:  
the mail is marked as spam

Expected Results:  
processing by spamassassin and correct header-filter-result behaviour.

I've lost emails (and was asked why I did not respond), so I set the severity to critical.
Comment 1 Tom Mittelstädt 2016-01-21 08:51:39 UTC
I changed the spamassassin ruleset to use "spamc" together with the daemonized version of spamassassin. What a performance boost! The time to scan one mail decreased from >10s to <1s.
Maybe this fixes the problem, too, that mails are not processed at all (maybe because of a timeout?)

However, kmail should not sort out mails where no spamassassin header is set at all.
Comment 2 Tom Mittelstädt 2016-01-23 09:05:32 UTC
The problem is still present, however now there a logs.

Fri Jan 22 17:38:42 2016 [1168] info: spamd: connection from ip6-localhost [::1]:39820 to port 783, fd 5
Fri Jan 22 17:38:43 2016 [1168] info: spamd: creating default_prefs: /home/monster/.spamassassin/user_prefs
Fri Jan 22 17:38:43 2016 [1168] info: spamd: failed to create readable default_prefs: /home/monster/.spamassassin/user_prefs
Fri Jan 22 17:38:43 2016 [1168] info: spamd: processing message (unknown) for monster:888
Fri Jan 22 17:38:43 2016 [1168] info: spamd: identified spam (7.9/5.0) for monster:888 in 0.3 seconds, 1 bytes.
Fri Jan 22 17:38:43 2016 [1168] info: spamd: result: Y 7 - EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS scantime=0.3,size=1,user=monster,uid=888,required_score=5.0,rhost=ip6-localhost,raddr=::1,rport=39820,mid=(unknown),autolearn=no autolearn_force=no
Fri Jan 22 17:38:43 2016 [987] info: prefork: child states: II
Comment 3 Tom Mittelstädt 2016-01-23 09:27:22 UTC
To sum up kmails behavior:

- There is an incoming email (OK)
- kmail sends it through spamassassin/spamc (OK)
- spamassassin's own log says:
EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS 
- kmail detects the spam flag in spamassassin's report (OK)
- kmail moves the email eg to trash. (OK)

The problem:

- The incoming mail was not empty. It is a normal email.
- Obviously kmail sends an empty file through spamassassin/spamc.
- The empty file is recognized as spam.
- Spamassassin generates its report, and kmail uses that X-Spam-Flag header for further filtering.

Interesting fact: The mail is not replaced with spamassassins report. It stays untouched, but is moved to trash.

So it seems as if the "pipe this email through a program" filter does not work reliably.
Maybe a race condition?
Comment 4 Tom Mittelstädt 2016-01-23 09:41:06 UTC
maybe related: https://bugs.kde.org/show_bug.cgi?id=255388
Comment 5 Justin Zobel 2022-10-19 22:11:05 UTC
Thank you for reporting this bug in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the issue with a recent software version?

If you can reproduce the issue, please change the status to "CONFIRMED" when replying. Thank you!
Comment 6 Bug Janitor Service 2022-11-03 05:07:12 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 7 Bug Janitor Service 2022-11-18 05:17:27 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!