Confirmed from a imap + spamassasin user, i.e. it's not bogofilter at fault.

+1 

the spamassassin makes the described problems. the spamassassin makes the described problems. Thanks to info from 'Rabauke' (special thanks ;-) ), to solves the disable in "KMail → Settings → Filter" → the function: "spamassasin check" the problem for now.

Same for me on Kubuntu 11.10 after the update to KDE SC 4.8 from  kubuntu-backports ppa this morning.  Happens with bogofilter + 2 imap accounts.

Achim

I updated today kde, akonadi-runtime also on version 1.7.1  … which is not the problem eliminated! I installed to test this tool in bogofilter and kmail activated. with bogofilter it works so far, at least reliable. mails are filtered and delivered completely. It seems as could be cause of the problem-spamassassin?

Grüße aus dem Länd'le

It happens with bogofilter and akonadi 1.7.1 as well.

I even removed the bogofilter rules and let kmail set them up again in case something changed in the way it pipes through it.

It does not happen to emails that have been marked as read before kmail retrieves them from the imap server.

hm, strange. I've had no failures with bogofilter. sometimes look like the remains resistant. already read emails (I use 4 imap accounts) were not affected, which were time with spamassassin been requested.

When installing bogofilter it does not mark any email as spam. One has to train it, i.e. mark emails as spam AND ham in order to make it work. This means that if you do not have this problem with bogofilter and you do not get any bogosity header in new emails, bogofilter is simply not adding any header and thus the bug does not appear.

If spamassasin does not need any training it is better suited for re-producing.

Could be related to https://bugs.kde.org/show_bug.cgi?id=286043 :D

I can confirm this bug after updating to KMail 4.8.0, KDE 4.8.1 and Akonadi 1.7.1 this morning (packages retrieved through official openSUSE repo for openSUSE 11.4 64bit).

All my IMAP accounts (GMail and own server) were affected. The content of newly received emails was empty. As far as I can tell, the headers of the emails were not affected. After opening an affected email (and thus marking as read), its content was also deleted on the server (one downside of using IMAP).

After disabling the four default spam-filters in KMail, everything is fine. I'm using spamassassin 3.3.1.

Luckily, I only lost three emails of minor importance due to this bug. Though, my client-side spam filtering is not functional any more.

I tried to reproduce the bug with today's master, but with no luck. The filter code in master should be the same as in 4.8.1 and also the akonadi server should be relatively the same as 1.7.1 (with some nepomuk related changes). 
What I tried:
- create rules for spamassasin and bogofilter with the anti-spam wizard. I changed them to move the spam to a specific folder instead of Local Mail/trash.
- check that rules apply both to imap (disconnected) and POP3 accounts (local mail folder)
- send spam to these addresses: spam was caught, marked and move to the spam folder. The mails were not destroyed, only some headers added.
- send not-spam (including html mail and mails with attachment) to the accounts. I checked that extra headers were added by the spam filter, but still, the mail structure was not destroyed, I could read it and access the attachment.

The only problem I noticed was the spam mails multiplying in the spam folder after they were moved there, but that is another bug.

So what did I do wrong? How can I really reproduce the bug?
For the reporters, try to enable the folter log and post what filters are running on the mails that get destroyed. You could also start akonadi from a console after enabling the debug info for the filter agent and akonadi with kdebugdialog (enable for libakonadi, akonadi_mailfilter_agent and restart with "akonadictl restart").
The akonadiconsole's debugger for the mailfilter agent could also help to see what is going on, just don't forget to disable the debugger when not needed, as it slows down the operations.

Hi Andreas,

i've attached a akonadiconsole log. I've tested it with 2 mails. The mail just contains:
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on saigkill.homelinux.net
X-Spam-Level: ***
X-Spam-Status: No, score=3.2 required=5.0 tests=BAYES_95,NO_RECEIVED,NO_RELAYS autolearn=no version=3.3.2
From: bugzilla_noreply@novell.com
To: opensuse-bugs@opensuse.org
Subject: [Bug 752768] New: Cannot sort repositories by priority
Message-ID: <bug-752768-21960@http.bugzilla.novell.com/>
Date: Sun, 18 Mar 2012 11:37:28 +0000
X-Bogosity: Unsure, tests=bogofilter, spamicity=0.486134, version=1.2.1
MIME-Version: 1.0

The rtf is the produced log from that 2 mails.

Created attachment 69707 [details]
debug log

+1
After upgrade von Kubuntu 11.10 with KDE SC 4.8.0 to 4.8.1 from kubuntu backports ppa.
Lost some content when akonadi was first to check emails. When message was read email content was there.

After disabling filter "bogofilter Check" everything seams ok.

offending filter rule from akonadi mailfilter agent

[Filter #18]
Applicability=0
AutomaticName=true
ConfigureShortcut=false
ConfigureToolbar=false
Enabled=false
Icon=system-run
StopProcessingHere=false
ToolbarName=Bogofilter Check
action-args-0=bogofilter -p -e
action-name-0=filter app
actions=1
apply-on=check-mail,manual-filtering
contentsA=256000
fieldA=<size>
funcA=less-or-equal
identifier=8VCBDTpWk6CxFOXq
name=Bogofilter Check
operator=and
rules=1

(In reply to comment #10)
András, can you not reproduce bug 292283 either? Probably they are related, i.e. somehow akonadi does download the whole email for you while for us it does not and hence only pipes the headers thus losing the rest.

Hi,

any improvements here? I can reproduce this behaviour here with KDE 4.8 and spamassassin.

Greetings Steffen

I think I have the same issue. I have no idea how to reproduce it reliably, but it seems the issue only happens at KMail startup for me. KMail is configured to pipe all mails from my IMAP inbox through bogofilter and then move spam to a different IMAP folder.

In other words, if I start KMail when Akonadi was already running and there are new mails in my inbox, then there is a good chance I will get problems: empty mails or header-only mails in my inbox.
Today I also found spam (also tagged as such by bogofilter in the header) in my inbox that contained the complete mail (header and body), but was shown as "(No Subject) Unknown Unknown" in the message list.

(KMail 4.8.2, Akonadi 1.7.2)

I've got the same issue with Kubuntu 12.04

Hi,

i´ve just changed my Opensuse Installation to KDE 4.8.4. The issue is still here. I installed from the Release:/48/Opensuse_12.1 Repository.

Git commit d64d61b470c56b92294f6adee6d74305a217628d by Andras Mantia.
Committed on 04/07/2012 at 17:35.
Pushed by amantia into branch 'master'.

A biggish refactoring of the filter functionality:

1) Do not take the items from the cache only, as they might be incomplete (especially in online imap case, but not only).
This should fix the body loss bug on spam filtering.(Bug 287752 and similar ones + countless of reports on user list)
Might fix 293768.

2) Download only what is really needed for the filter and add safety checks that
the mail is not modified if the requested part is missing. So far the distinction
was only betwee the body and rest. This should speed up filtering.
Also related to 287752, but should fix also 292283, 288109.

3) Rename qDebug->kDebug + fix some includes

4) The code should also fix the duplicate mails appearing after filtering (some of the duplicates
with empty content), eg. 275233

All bug reporters: in case you still see the bug, reopen the corresponding one.
For the reports that were CC'd, but not closed, please try to reproduce and tell
the result.

So far the bug is fixed on master, but might be backported later to 4.9.
Related: bug 287752, bug 292283, bug 288109, bug 286043, bug 302337, bug 295000, bug 295684, bug 293768, bug 275233, bug 293918, bug 284310, bug 286364, bug 283682

M  +7    -11   kmail/kmcommands.cpp
M  +10   -7    kmail/kmcommands.h
M  +1    -1    kmail/kmmainwidget.cpp
M  +1    -5    mailcommon/filter/filteraction.cpp
M  +6    -5    mailcommon/filter/filteraction.h
M  +6    -0    mailcommon/filter/filteractionaddheader.cpp
M  +2    -0    mailcommon/filter/filteractionaddheader.h
M  +2    -2    mailcommon/filter/filteractionaddtag.cpp
M  +1    -1    mailcommon/filter/filteractionaddtag.h
M  +9    -3    mailcommon/filter/filteractionaddtoaddressbook.cpp
M  +2    -0    mailcommon/filter/filteractionaddtoaddressbook.h
M  +7    -0    mailcommon/filter/filteractionbeep.cpp
M  +1    -0    mailcommon/filter/filteractionbeep.h
M  +18   -1    mailcommon/filter/filteractioncopy.cpp
M  +7    -0    mailcommon/filter/filteractioncopy.h
M  +6    -0    mailcommon/filter/filteractiondelete.cpp
M  +1    -0    mailcommon/filter/filteractiondelete.h
M  +5    -0    mailcommon/filter/filteractionexec.cpp
M  +1    -0    mailcommon/filter/filteractionexec.h
M  +5    -0    mailcommon/filter/filteractionforward.cpp
M  +1    -0    mailcommon/filter/filteractionforward.h
M  +6    -0    mailcommon/filter/filteractionmove.cpp
M  +2    -0    mailcommon/filter/filteractionmove.h
M  +6    -0    mailcommon/filter/filteractionpipethrough.cpp
M  +1    -0    mailcommon/filter/filteractionpipethrough.h
M  +6    -0    mailcommon/filter/filteractionplaysound.cpp
M  +2    -0    mailcommon/filter/filteractionplaysound.h
M  +5    -0    mailcommon/filter/filteractionredirect.cpp
M  +1    -0    mailcommon/filter/filteractionredirect.h
M  +6    -1    mailcommon/filter/filteractionremoveheader.cpp
M  +1    -0    mailcommon/filter/filteractionremoveheader.h
M  +6    -0    mailcommon/filter/filteractionreplyto.cpp
M  +1    -0    mailcommon/filter/filteractionreplyto.h
M  +6    -0    mailcommon/filter/filteractionrewriteheader.cpp
M  +1    -0    mailcommon/filter/filteractionrewriteheader.h
M  +6    -0    mailcommon/filter/filteractionsendfakedisposition.cpp
M  +1    -0    mailcommon/filter/filteractionsendfakedisposition.h
M  +8    -2    mailcommon/filter/filteractionsendreceipt.cpp
M  +1    -0    mailcommon/filter/filteractionsendreceipt.h
M  +6    -0    mailcommon/filter/filteractionsetidentity.cpp
M  +1    -0    mailcommon/filter/filteractionsetidentity.h
M  +6    -0    mailcommon/filter/filteractionsetstatus.cpp
M  +1    -0    mailcommon/filter/filteractionsetstatus.h
M  +6    -0    mailcommon/filter/filteractionsettransport.cpp
M  +1    -0    mailcommon/filter/filteractionsettransport.h
M  +5    -0    mailcommon/filter/filteractionstatus.cpp
M  +2    -0    mailcommon/filter/filteractionstatus.h
M  +5    -2    mailcommon/filter/filteractionunsetstatus.cpp
M  +2    -1    mailcommon/filter/filteractionunsetstatus.h
M  +4    -4    mailcommon/filter/filterimporter/filterimporterevolution.cpp
M  +1    -1    mailcommon/filter/filterimporter/filterimporterprocmail.cpp
M  +1    -1    mailcommon/filter/filterimporter/filterimportersylpheed.cpp
M  +2    -2    mailcommon/filter/filtermanager.cpp
M  +2    -8    mailcommon/filter/filtermanager.h
M  +8    -2    mailcommon/filter/itemcontext.cpp
M  +8    -1    mailcommon/filter/itemcontext.h
M  +10   -17   mailcommon/filter/kmfilterdialog.cpp
M  +2    -1    mailcommon/filter/kmfilterdialog.h
M  +25   -8    mailcommon/filter/mailfilter.cpp
M  +4    -3    mailcommon/filter/mailfilter.h
M  +48   -20   mailcommon/searchpattern.cpp
M  +34   -11   mailcommon/searchpattern.h
M  +1    -1    mailcommon/tests/searchpatterntest.cpp
M  +78   -47   mailfilteragent/filtermanager.cpp
M  +10   -8    mailfilteragent/filtermanager.h
M  +8    -6    mailfilteragent/mailfilteragent.cpp
M  +3    -0    mailfilteragent/mailfilteragent.h

http://commits.kde.org/kdepim/d64d61b470c56b92294f6adee6d74305a217628d

Thanks a lot for fixing it. A backport to 4.9 would be appreciated. If not too much work, release a patched 4.9 tarball separately and call for a mini-betatest via Planet KDE. If no problems caused by this turn up, it could be released with 4.9.1 or so.

Thanks again, even it the patch is too large fro 4.9.x.

Git commit e3a03f1d704838a9ced4ff41358785392774ae69 by Andras Mantia.
Committed on 04/07/2012 at 17:35.
Pushed by amantia into branch 'KDE/4.9'.

(Backport filter fix from master)

A biggish refactoring of the filter functionality:

1) Do not take the items from the cache only, as they might be incomplete (especially in online imap case, but not only).
This should fix the body loss bug on spam filtering.(Bug 287752 and similar ones + countless of reports on user list)
Might fix 293768.

2) Download only what is really needed for the filter and add safety checks that
the mail is not modified if the requested part is missing. So far the distinction
was only betwee the body and rest. This should speed up filtering.
Also related to 287752, but should fix also 292283, 288109.

3) Rename qDebug->kDebug + fix some includes

4) The code should also fix the duplicate mails appearing after filtering (some of the duplicates
with empty content), eg. 275233

All bug reporters: in case you still see the bug, reopen the corresponding one.
For the reports that were CC'd, but not closed, please try to reproduce and tell
the result.

So far the bug is fixed on master, but might be backported later to 4.9.
Related: bug 287752, bug 292283, bug 288109, bug 286043, bug 302337, bug 295000, bug 295684, bug 293768, bug 275233, bug 293918, bug 284310, bug 286364, bug 283682
(cherry picked from commit d64d61b470c56b92294f6adee6d74305a217628d)

FIXED-IN: 4.9

M  +7    -11   kmail/kmcommands.cpp
M  +10   -7    kmail/kmcommands.h
M  +1    -1    kmail/kmmainwidget.cpp
M  +1    -5    mailcommon/filter/filteraction.cpp
M  +6    -5    mailcommon/filter/filteraction.h
M  +6    -0    mailcommon/filter/filteractionaddheader.cpp
M  +2    -0    mailcommon/filter/filteractionaddheader.h
M  +2    -2    mailcommon/filter/filteractionaddtag.cpp
M  +1    -1    mailcommon/filter/filteractionaddtag.h
M  +9    -3    mailcommon/filter/filteractionaddtoaddressbook.cpp
M  +2    -0    mailcommon/filter/filteractionaddtoaddressbook.h
M  +7    -0    mailcommon/filter/filteractionbeep.cpp
M  +1    -0    mailcommon/filter/filteractionbeep.h
M  +18   -1    mailcommon/filter/filteractioncopy.cpp
M  +7    -0    mailcommon/filter/filteractioncopy.h
M  +6    -0    mailcommon/filter/filteractiondelete.cpp
M  +1    -0    mailcommon/filter/filteractiondelete.h
M  +5    -0    mailcommon/filter/filteractionexec.cpp
M  +1    -0    mailcommon/filter/filteractionexec.h
M  +5    -0    mailcommon/filter/filteractionforward.cpp
M  +1    -0    mailcommon/filter/filteractionforward.h
M  +6    -0    mailcommon/filter/filteractionmove.cpp
M  +2    -0    mailcommon/filter/filteractionmove.h
M  +6    -0    mailcommon/filter/filteractionpipethrough.cpp
M  +1    -0    mailcommon/filter/filteractionpipethrough.h
M  +6    -0    mailcommon/filter/filteractionplaysound.cpp
M  +2    -0    mailcommon/filter/filteractionplaysound.h
M  +5    -0    mailcommon/filter/filteractionredirect.cpp
M  +1    -0    mailcommon/filter/filteractionredirect.h
M  +6    -1    mailcommon/filter/filteractionremoveheader.cpp
M  +1    -0    mailcommon/filter/filteractionremoveheader.h
M  +6    -0    mailcommon/filter/filteractionreplyto.cpp
M  +1    -0    mailcommon/filter/filteractionreplyto.h
M  +6    -0    mailcommon/filter/filteractionrewriteheader.cpp
M  +1    -0    mailcommon/filter/filteractionrewriteheader.h
M  +6    -0    mailcommon/filter/filteractionsendfakedisposition.cpp
M  +1    -0    mailcommon/filter/filteractionsendfakedisposition.h
M  +8    -2    mailcommon/filter/filteractionsendreceipt.cpp
M  +1    -0    mailcommon/filter/filteractionsendreceipt.h
M  +6    -0    mailcommon/filter/filteractionsetidentity.cpp
M  +1    -0    mailcommon/filter/filteractionsetidentity.h
M  +6    -0    mailcommon/filter/filteractionsetstatus.cpp
M  +1    -0    mailcommon/filter/filteractionsetstatus.h
M  +6    -0    mailcommon/filter/filteractionsettransport.cpp
M  +1    -0    mailcommon/filter/filteractionsettransport.h
M  +5    -0    mailcommon/filter/filteractionstatus.cpp
M  +2    -0    mailcommon/filter/filteractionstatus.h
M  +5    -2    mailcommon/filter/filteractionunsetstatus.cpp
M  +2    -1    mailcommon/filter/filteractionunsetstatus.h
M  +4    -4    mailcommon/filter/filterimporter/filterimporterevolution.cpp
M  +1    -1    mailcommon/filter/filterimporter/filterimporterprocmail.cpp
M  +1    -1    mailcommon/filter/filterimporter/filterimportersylpheed.cpp
M  +2    -2    mailcommon/filter/filtermanager.cpp
M  +2    -8    mailcommon/filter/filtermanager.h
M  +8    -2    mailcommon/filter/itemcontext.cpp
M  +8    -1    mailcommon/filter/itemcontext.h
M  +10   -17   mailcommon/filter/kmfilterdialog.cpp
M  +2    -1    mailcommon/filter/kmfilterdialog.h
M  +25   -8    mailcommon/filter/mailfilter.cpp
M  +4    -3    mailcommon/filter/mailfilter.h
M  +48   -20   mailcommon/searchpattern.cpp
M  +34   -11   mailcommon/searchpattern.h
M  +1    -1    mailcommon/tests/searchpatterntest.cpp
M  +78   -47   mailfilteragent/filtermanager.cpp
M  +10   -8    mailfilteragent/filtermanager.h
M  +8    -6    mailfilteragent/mailfilteragent.cpp
M  +3    -0    mailfilteragent/mailfilteragent.h

http://commits.kde.org/kdepim/e3a03f1d704838a9ced4ff41358785392774ae69

Git commit 8846d44b217108351c35f8afd883f72ee88388ec by Andras Mantia.
Committed on 06/07/2012 at 11:12.
Pushed by amantia into branch 'KDE/4.8'.

Backport part of the filter fix: this is the actual fix, the rest were optimizations and safety checks, but it is difficult to backport.
An unwanted side-effect might be that more data is downloaded than before,
based on the type of filters used.

FIXED-IN: 4.8.5
Related: bug 287752, bug 292283, bug 288109, bug 286043, bug 302337, bug 295000, bug 295684, bug 293768, bug 275233, bug 293918, bug 284310, bug 286364, bug 283682

M  +0    -1    mailfilteragent/mailfilteragent.cpp

http://commits.kde.org/kdepim/8846d44b217108351c35f8afd883f72ee88388ec

The 4.8.5 patch does not fix this issue.

The content of the emails is still removed.

I might have found another piece of information though. After kmail received the email it showed an akonadi conflicting message dialogue which seems to show that either spamassasin eats the email's content, which I think is unlikely, or that kmail/akonadi does already feed the external app with an empty email.

On the left:


Flags:
\SEEN, $JUNK
Data:
Received: from localhost by linux-ly0d.site with SpamAssassin (version 3.3.2); Sat, 07 Jul 2012 12:56:08 +0200
Subject: ****SPAM(6.4)**** 
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on linux-ly0d.site
X-Spam-Flag: YES
X-Spam-Level: ******
X-Spam-Status: Yes, score=6.4 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE, MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED, NO_RELAYS autolearn=no version=3.3.2
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"

On the right the full email without flags, i.e.:



Received: from mx-out-1.zzz ([134.130.5.186])
 by mta-1.ms.rz.RWTH-Aachen.de
 (Sun Java(tm) System Messaging Server 6.3-7.04 (built Sep 26 2008))
 with ESMTP id <0M6S009R8ECSUPC0@mta-1.ms.rz.RWTH-Aachen.de> for
 xyz@zzz; Sat, 07 Jul 2012 12:55:48 +0200 (CEST)
Received: from mx0.gmx.net ([213.165.64.100]) by mx-1.rz.zzz with
 SMTP; Sat, 07 Jul 2012 12:55:49 +0200
Received: (qmail 16818 invoked by alias); Sat, 07 Jul 2012 10:55:48 +0000
Received: (qmail invoked by alias); Sat, 07 Jul 2012 10:55:48 +0000
Received: from lists.opensuse.org (EHLO lists4.suse.de)
 [195.135.221.135] by mx0.gmx.net (mx079) with SMTP; Sat,
 07 Jul 2012 12:55:48 +0200
Received: from lists4.suse.de (localhost [127.0.0.1])
 by lists4.suse.de (Postfix) with SMTP id B7E0C833DFB; Sat,
 07 Jul 2012 10:55:45 +0000 (GMT)
Received: from relay2.suse.de (unknown [149.44.160.134])
 by lists4.suse.de (Postfix) with ESMTP id 4F871833CED  for
 <opensuse-de@lists4.opensuse.org>; Sat, 07 Jul 2012 10:55:43 +0000 (GMT)
Received: from localhost (localhost [127.0.0.1])        by relay2.suse.de (Postfix)
 with ESMTP id AD3BD18552D2     for <opensuse-de@lists4.opensuse.org>; Sat,
 07 Jul 2012 12:55:41 +0200 (CEST)
Received: from relay2.suse.de ([127.0.0.1])
 by localhost (localhost [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id 09585-07 for <opensuse-de@lists4.opensuse.org>; Sat,
 07 Jul 2012 12:55:41 +0200 (CEST)
Received: from mx2.suse.de (cantor2.suse.de [195.135.220.15])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)      by relay2.suse.de (Postfix)
 with ESMTPS id 6252318552D1    for <opensuse-de@lists4.opensuse.org>; Sat,
 07 Jul 2012 12:55:41 +0200 (CEST)
Received: from mailout.cboltz.de (mailout.cboltz.de [78.46.208.170])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)      by mx2.suse.de (Postfix)
 with ESMTP id 4F47AA2C8B       for <opensuse-de@opensuse.org>; Sat,
 07 Jul 2012 12:55:40 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
 by server.sprachakt.com (Postfix) with ESMTP id 2E8A8264327    for
 <opensuse-de@opensuse.org>; Sat, 07 Jul 2012 12:55:40 +0200 (CEST)
Received: from mail.cboltz.de ([127.0.0.1])
 by localhost (mail.sprachakt.com [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id 7hqd-Z046WwQ for <opensuse-de@opensuse.org>; Sat,
 07 Jul 2012 12:55:37 +0200 (CEST)
Received: from home.cboltz.de (unknown [10.8.0.10])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate) by mail.cboltz.de (Postfix)
 with ESMTPSA   for <opensuse-de@opensuse.org>; Sat,
 07 Jul 2012 12:55:37 +0200 (CEST)
Date: Sat, 07 Jul 2012 12:55:35 +0200
From: Christian Boltz <suse@cboltz.de>
X-Face: #?nL0}JpqNtLQy@q#bRm?B?pGS8[mx6r.6[91zp@*2DZ?18)haWs5wgvi,,wF/JWMTUh+6x
 ,b7_`pW3)m~0606sDW0&'EKA}_-W+)Bz~d]k>4E9TyU}k@b&1=%yk\
Subject: Re: kein ./script
In-reply-to: <wolfgang-1120707101501.A0328929@amadeus3.local>
X-Resent-to: xyz@zzz
To: opensuse-de@opensuse.org
Message-id: <8889098.rqCireFBYN@tux.boltz.de.vu>
MIME-version: 1.0
Content-type: text/plain; charset=iso-8859-15
Content-transfer-encoding: quoted-printable
Precedence: bulk
Delivered-to: GMX delivery to xyz@gmx.net
Delivered-to: opensuse-de@lists4.opensuse.org
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result:
 AkYCAHgU+E/VpUBklGdsb2JhbABEuAABAQEBCQsJCRQDJIIZAQEBAwEBAUMIMgEBAQUBCyElAwEBCgEhJhUEAQQYBIdqBAecC51zi0AaglaDHAOWSJFUgV0
X-IronPort-AV: E=Sophos;i="4.77,542,1336341600";   d="scan'208";a="183795025"
X-Original-To: opensuse-de@lists4.opensuse.org
X-Mailinglist: opensuse-de
X-MIME-Notice: attachments may have been removed from this message
X-Virus-Scanned: by amavisd-new at localhost
X-Spam-Score: -3.599
X-GMX-Antispam: 0 (BackTrace mail analyze);
 Detail=5D7Q89H36p4L00VTXC6D4q0N+AH0PUCnGL2vqOgpaBYL16oitsMrgDt/NQNpSCZFFjDOy
 97xb7Zpf+wZnd5ZXNcvLDXR3Wg3wRjdQbwEMh8=V1;
X-Resent-By: Forwarder <forwarder@gmx.net>
X-Resent-For: xyz@gmx.net
References: <wolfgang-1120707101501.A0328929@amadeus3.local>
Mailing-List: contact opensuse-de+help@opensuse.org; run by mlmmj
List-Owner: <mailto:opensuse-de+owner@opensuse.org>
List-Post: <mailto:opensuse-de@opensuse.org>
List-Subscribe: <mailto:opensuse-de+subscribe@opensuse.org>
List-Unsubscribe: <mailto:opensuse-de+unsubscribe@opensuse.org>
List-Archive: <http://lists.opensuse.org/opensuse-de/>
List-Help: <mailto:opensuse-de+help@opensuse.org>
User-Agent: KMail/4.8.4 (Linux/3.4.2-1-desktop; KDE/4.8.4; x86_64; ; )
X-Spam-Status: No, score=-3.599 tagged_above=-20 required=5
 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Hallo Wolfgang, hallo Leute,
Am Samstag, 7. Juli 2012 schrieb hamann.w@t-online.de:
> auf einem Rechner habe ich seit dem Upgrade auf 12.1 das Problem, das=
s
> die Befehlsausf=FChrung mit ./script (oder auch Hinzuf=FCgen von . zu=
m
> Pfad und Aufruf mit Skriptname) jeweils "keine Berechtigung" meldet.
> Skripte in anderen Verzeichnissen, sowie der Aufruf mit sh skript (bz=
w
> dem jeweils passenden Interpreter) funktionieren normal
>=20
> Wer hat eine Idee?
Klassische Kandidaten sind:
- chmod +x vergessen
- Partition ist als "noexec" gemountet (wird u. a. durch "user"=20
  impliziert)


Gru=DF
Christian Boltz
--=20
Sitzstreik
    analoger Denial of Service-Angriff gegen ein Geb=E4ude
[Jacob Appelbaum,
http://netzpolitik.org/2012/jacob-appelbaum-ruft-zu-mehr-digitalem-
aktivismus-auf-wir-sind-alle-anonymous/]
-- 
Um die Liste abzubestellen, schicken Sie eine Mail an:
    opensuse-de+unsubscribe@opensuse.org
Um den Listen Administrator zu erreichen, schicken
Sie eine Mail an: opensuse-de+owner@opensuse.org

Argh, would it be possible to try master? And with master's akonadi.

I can test 4.9 as soon as the unstable packages for openSUSE are updated. The last update was 10 days ago hence they do not yet include the fix.

Or is 4.9 not helpful either?

4.9 is ok, it has the full patch backported.

I applied the patch to kdepim. akonadi is akonadi-4.8.95_20120629-5.1.i586 though.

The bug does still exist.

Packages for openSUSE's KUSC repo with the patch applied can be found here:
http://download.opensuse.org/repositories/home:/rabauke:/branches:/KDE:/Unstable:/SC/

https://build.opensuse.org/package/view_file?file=changeset_re3a03f1d704838a9ced4ff41358785392774ae69.diff&package=kdepim4&project=home%3Arabauke%3Abranches%3AKDE%3AUnstable%3ASC&rev=304cbba7521fbe61a400d4cdf6b95123 is the patch applied.

Does it happen if the imap is not an online one, but a disconnected one? Set it up completely from scratch to be a disconnected imap, as on-the-fly switching is not doing a full-conversion.

I only use online imap, never switched. Necer tried disconnected imap.

Unfortunately there is no 4.9 live CD yet but I guess your patch improved filtering a lot, but did not hit the cause of the bug(s).

Hence 4.8.x might still be valid to find the actual cause.

To reproduce:
1. pick a live CD from http://download.opensuse.org/repositories/KDE:/Medias/images/iso/
(other distros should work as well)
2. create a gmail account and enable imap (bug does not only happen with gmail but it's free and easy to register)
3. boot the live CD and do sudo zypper in spamassasin
(3a. you can also install bogofilrer but the latter needs some training first, i.e. marking emails as spam and marking emails as ham)
4. set-up an online imap account within kmail/akonadi
5. run the spam assistant from kmail's extras (tools in English?) menu
6. send some emails to that account and retrieve them via kmail
result: only email headers + added spam header

for the other bugs:
8. send a huge email to the account
9. move that email via drag and drop to the local trash
10. use the networkmanagement plasmoid to disconnect the connection while kmail/akonadi are still downloading the huge email to move it to the local trash

result: empty email in trash, original email removed from inbox.

ah, sorry. I misunderstood. I will try to reproduce on disconnected imap in the next few days.

I found the anti-bug to the mail loss.

I was using kdepim 4.8 with the patch, selected a lot of emails and clicked on "apply all filters". This usually downloads all emails selected before it starts to move them according to the filters.

However, this time it did not move all mails but just those I received today, i.e. in the current KDE session. Applying filters to "old" email just does not do anything.

Akonadiconsole shows:
akonadi_mailfilter_agent (0x7e8170) 54 UID FETCH 64835 FULLPAYLOAD ANCESTORS 1 EXTERNALPAYLOAD (UID REMOTEID REMOTEREVISION COLLECTIONID FLAGS SIZE DATETIME) 
akonadi_mailfilter_agent (0x7e8170) * 64835 FETCH (UID 64835 REV 3 REMOTEID "385076" MIMETYPE "message/rfc822" COLLECTIONID 10 SIZE 10662 DATETIME "13-Jul-2012 10:39:05 +0000" FLAGS (\SEEN) ANCESTORS ((10 "/INBOX")) PLD:ENVELOPE[1] {459} ("Thu, 12 Jul 2012 14:00:43 +0000" "[Kde-pim] Review Request: [Fix bug] Disable focus proxy of Article List Viewer" (("Phương Lê Hoàng" NIL "herophuong93" "gmail.com")) ((NIL NIL "kde-pim-bounces" "kde.org")) (("Phương Lê Hoàng" NIL "herophuong93" "gmail.com") ("KDE PIM" NIL "kde-pim" "kde.org")) (("Phương Lê Hoàng" NIL "herophuong93" "gmail.com") ("KDEPIM" NIL "kde-pim" "kde.org")) NIL NIL NIL "<20120712140043.12945.44907@vidsolbach.de>" NIL) PLD:HEAD[1] [FILE] {57} /home/username/.local/share/akonadi/file_db_data/154699_r0 PLD:RFC822[1] [FILE] {57} /home/username/.local/share/akonadi/file_db_data/155479_r0) 
akonadi_mailfilter_agent (0x7e8170) 54 OK UID FETCH completed 

That's it.

Doing the same for an email that arrived today, no matter whether its status is read or unread works.

Now comes the interesting bit. If I enable the filter that pipes emails through bogofilter and apply it to new and unread emails the content gets removed (this bug). Yet if I "apply all filters" to an old and unread email which does not get moved the email gets the bogofilter header and does not lose its content, i.e. the spam filtering bit works as it should.

I tried to clean the akonadi cache via akonadiconsole in order to make the old emails move again but that does not work. Switching back to kdepim 4.8 without the patch does not work either.

I'll attach the akonadiconsole log which includes at the beginning:

right-click on an email which arrived today and select "apply all filters" (working)

and at the bottom the bit which is right-clicking on an email from yesterday and selecting "apply all filters" which does not work.

Created attachment 72495 [details]
RTF akonadiconsole log

The last few lines show what happens if one clicks on an "old" email and "apply all filters" which makes the bogofilter filter not remove the email's content.

Everything else is what happens to a new email if "apply all filters" is used.

Setting the inbox folder's properties to "always retrieve full email" seems to help for kdepim 4.8.4. Maybe this gives a hint what's wrong if it is set to "on demand".

That workaround seems to confirm my theories about kmail piping an empty email through the external app and replacing the original with the result.

According to Volker my patch should fix exactly this case. As it apparently doesn't (you said the problem is still there with it), we need to investigate more.

Have you tried with a disconnected imap account?

No.

Is it enough to create a dimap account within a kmail which already has an imap account for the same email account?

Yes, it is enough.

Correction: it is enough, but of course you must have filters that operate on that account as well.
Also be sure to use a version that has the fixes (4.9 recommended, 4.8.5 should also be ok).

Ok, I'll try as soon as packages for 4.9 are available from the openSUSE 4.9 repo.

There might still be data loss:

I filtered emails and did not see that there was a "conflict resolution dialog" behind kmail. The filtering obviously stopped and I wondered why. So I quit kamil and restarted, then answered the conflict dialog. A few more came up with some notification windows in-between:

akonadi_mailfilter_agent(2093): Error while modifying items.  103 "Unknown error. (NO Only resources can modify remote identifiers
)" 

That resulted in empty messages, i.e. spam headers added but content gone.

From the logs:

Is the 4.9 backport of the fix already released? I am running 4.9 beta2 (4.8.90) and still see that problem.

The commit was made shortly after 4.8.97 was released (so, no release includes the fix... yet)

OK, thanks.

I'm running 4.9, now. So I turned spam-filtering on again and sent myself an email. Unfortunately, the message body has been removed again.

I'm sad it is still reproducible.

Dirk, do you use an online imap account? Can you set up a disconnected resource for the same account and try there? You can even do it as a different KDE user, just to not mess up your current configuration. The only thing you need to do is that when you add the IMAP account to KMail, select Enable Disconnected Mode on the Advanced tab.

Yes, it is an online account. I tried with another and I can confirm that with this user filtering works fine with an offline account and does not with an online account. However, with my other account, after enabling offline mode and disabling it again, I can't get the filtering to work anymore (that is: filter's are not applied although they're activated).

Maybe I'll recreate that account completely and see what happens...

Transforming an online account to offline does not work as expected, that is "normal":https://bugs.kde.org/show_bug.cgi?id=255578

Transforming an online account to offline does not work as expected, that is "normal":https://bugs.kde.org/show_bug.cgi?id=255578. Thanks for comfirming that it is a problem with online accounts only.

(In reply to comment #45)
> Can you set up a disconnected
> resource for the same account and try there?

How? Disconnected IMAP has been removed as separate account type in KMail 4.6 and newer.

(In reply to comment #49)
> (In reply to comment #45)
> > Can you set up a disconnected
> > resource for the same account and try there?
> 
> How? Disconnected IMAP has been removed as separate account type in KMail
> 4.6 and newer.

So Dirk is a magiician if he could set it up.

Hint:
(In reply to comment #45)
> I'm sad it is still reproducible.
> 
> Dirk, do you use an online imap account? Can you set up a disconnected
> resource for the same account and try there? You can even do it as a
> different KDE user, just to not mess up your current configuration. The only
> thing you need to do is that when you add the IMAP account to KMail, select
> Enable Disconnected Mode on the Advanced tab.

Please add bug 304741 as a probable duplicate of this bug.

*** Bug 304741 has been marked as a duplicate of this bug. ***

This occurs when using bogofilter and spamassasin in KMail 4.9.0 "release 558" created with the Anti-Spam-Wizard under OpenSUSE 12.1. I trained it with >600 spam mails (a lot of CPU and HD usage for 1 h). It did not detect anything afterwards, but removes the content of every mail. Switching back to Thunderbird again.

Git commit c691cb5d9cc7bb337c149e0279e21397f168c314 by Andras Mantia.
Committed on 25/08/2012 at 10:46.
Pushed by amantia into branch 'master'.

Now fix for real the mail loss on spam filtering. The previous fix's idea was good, just that unfortunately the default value is "setCacheOnly(true)" set in AgentBasePrivate, so we have to explicitely set the cacheonly to false, otherwise mail content is not fetched from the mail server when passing through a filter that *needs* a content.

FIXED-IN: 4.10

M  +1    -0    mailfilteragent/mailfilteragent.cpp

http://commits.kde.org/kdepim/c691cb5d9cc7bb337c149e0279e21397f168c314

Git commit e98937475a7f03e69a3a3c6a4ee67a8d41f027b5 by Andras Mantia.
Committed on 25/08/2012 at 10:47.
Pushed by amantia into branch 'KDE/4.9'.

Now fix for real the mail loss on spam filtering. The previous fix's idea was good, just that unfortunately the default value is "setCacheOnly(true)" set in AgentBasePrivate, so we have to explicitely set the cacheonly to false, otherwise mail content is not fetched from the mail server when passing through a filter that *needs* a content.

FIXED-IN: 4.9.1
(cherry picked from commit c691cb5d9cc7bb337c149e0279e21397f168c314)

M  +1    -0    mailfilteragent/mailfilteragent.cpp

http://commits.kde.org/kdepim/e98937475a7f03e69a3a3c6a4ee67a8d41f027b5

Git commit 628faec2651f6f20a438f61fc9c861c0d5eab0f8 by Andras Mantia.
Committed on 25/08/2012 at 10:52.
Pushed by amantia into branch 'KDE/4.8'.

Now fix for real the mail loss on spam filtering. The previous fix's idea was good, just that unfortunately the default value is "setCacheOnly(true)" set in AgentBasePrivate, so we have to explicitely set the cacheonly to false, otherwise mail content is not fetched from the mail server when passing through a filter that needs a content.

FIXED-IN: 4.8.6

M  +1    -0    mailfilteragent/mailfilteragent.cpp

http://commits.kde.org/kdepim/628faec2651f6f20a438f61fc9c861c0d5eab0f8

Thank you, that is a good news.

4.8.6 ?????? :)

Whatever comes, if there will be another 4.8.x release.

Does that mean that this bug is also fixed in KDE 4.9 (4.9.1/2)?

Yes, it should be fixed. If not, please tell me how to reproduce it.

If your connection does not fail it is fixed. The data loss in case the transfer is aborted, e.g. because of internet connection loss, was only fixed for 4.9.3 if I understood correctly.

You say something with the data loss when connection goes down. Although I was not thinking about filtering (only copy/move when the resoruce is offline), indeed that might affect filtering as well, and that fix was committed only recently.

The -p (passthrough) option removes content only on imap accounts.

To login into hotmail account kindly follow the tutorial.
https://tangent.com/engl/hotmail-login/