Bug 128014

Summary: the last received email is duplicated each time there's a new mail check
Product: [Unmaintained] kmail Reporter: Nicolas Le Guen <goldenear>
Component: IMAPAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED DUPLICATE    
Severity: normal CC: fd0man, lucius1
Priority: NOR    
Version: 1.9.1   
Target Milestone: ---   
Platform: unspecified   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: Plaintext transcript of communication between KMail<==>IMAP Server
Ethereal debug file -- edited
Ethereal debug file -- edited w/ corrected cksum

Description Nicolas Le Guen 2006-05-25 15:34:39 UTC
Version:           1.9.1 (using KDE 3.5.2, Kubuntu Package 4:3.5.2-0ubuntu18 dapper)
Compiler:          Target: i486-linux-gnu
OS:                Linux (i686) release 2.6.15-23-686

I have an imap account (normal imap, NOT disconnected imap) configured in kmail/kontact. I've configure kmail to always display the little icon in the systray.
So kmail runs in the background and checks new email on the imap every 5 minutes. The problem is that, for each mail check, the latest email on the imap server is duplicated and appears as a new mail in kmail.
Comment 1 Magnus Holmgren 2006-05-25 15:57:49 UTC
Do you mean it's duplicated on every check, so that you end up with a large amount of copies? Does it matter whether the most recent message is read or unread? What IMAP server is it? Does it always happen or just sometimes?

Is it the same behaviour as in bug 95064, bug 118074, or bug 119149 (do you use filtering?).
Comment 2 Nicolas Le Guen 2006-05-25 18:47:43 UTC
> ------- Additional Comments From holmgren lysator liu se  2006-05-25 15:57
> ------- Do you mean it's duplicated on every check, so that you end up with
> a large amount of copies? Does it matter whether the most recent message is

Yes it's exactly this. I end up with a large amount of copies. In fact it 
doesn't only seems to happen on the latest email.
> read or unread? What IMAP server is it? Does it always happen or just

the mail is duplicated with its current state (eg: if I read it, it's 
duplicated as read, not as a new/unread one)
I use imap.free.fr
> sometimes?

It's only happens on some emails, so not always but very often. When a mail 
started to be duplicated it always happens if it's not deleted.
>
> Is it the same behaviour as in bug 95064, bug 118074, or bug 119149 (do you
> use filtering?).

I'm not sure. I don't use filtering exptect the integrated spam filter 
(bogofilter).

Please note that the duplication only happens during background/automatic new 
email check. The dupications doesn't occure when the check mail button is 
used (manual check).
Comment 3 Carsten Burghardt 2006-05-26 10:55:54 UTC
Do you see this behaviour without filtering (bogofilter)?
Comment 4 Nicolas Le Guen 2006-05-26 15:21:45 UTC
Le Vendredi 26 Mai 2006 10:55, Carsten Burghardt a écrit :
[bugs.kde.org quoted mail]

unlike what I said in a previous post, the duplication also happends with 
manual mail check (cliking icon). Each time I click to check new mails, the 
last mail was duplicated. So I did the test: I disabled bogofilter... and no 
more duplication :)

The duplication problem has indeed something to do with filters or bogofilter.
Comment 5 Sergei Zuyev 2006-06-02 16:47:43 UTC
Yes, I am also having this error kmail in KDE3.5.2 under Mandriva2006 Official. 
Comment 6 Sergei Zuyev 2006-06-02 19:29:31 UTC
I observed that each next duplicated email contains an additional (with resp. to the previous mail) line:
X-Original-X-UID: <some number>
So whatever adds this line is responsible.
Comment 7 Sergei Zuyev 2006-06-09 02:03:41 UTC
I also observed the following. kmail was upgraded from KDE3.4 with Mandriva's kiosk and it already had spam filters defined. After upgrade there are duplicates of the filters appeared. I cleaned the old ones and now I do not observe this bug anymore.
Comment 8 warcomp 2006-06-25 03:20:53 UTC
This bug still exists in KDE-3.5.3 !

I happens much less, but it happens
Comment 9 warcomp 2006-06-25 03:29:47 UTC
I've to add one strange thing I noticed...

KMail is downloading new mail, and then later after/while looking for 
new mail KMail is uploading duplicates.
(Btw. it is not always duplicating mails, since the first fix for this 
problem, kmail moves mails around. For example it downloads a bunch of mails,
seems to delete them from the imap-server and uploads them again to the
imap-server. So sometimes it seems to get confused while moving mail around,
and it uploads some mails more often then others.)

Strangely this does not happen for every mail so I added the header of 
one mail that duplicated itself:

X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on muehle.muehle.at
X-Spam-Level: 
X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY 
	autolearn=unavailable version=3.1.1
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
X-Virus-Flag: no
Return-Path: <Johann.Groszschaedl@iaik.tugraz.at>
Received: from sbox.tugraz.at ([unix socket])
	 by sbox.tugraz.at (Cyrus v2.2.13) with LMTPA;
	 Fri, 23 Jun 2006 15:47:15 +0200
X-Sieve: CMU Sieve 2.2
Received: from mailrelay2.tu-graz.ac.at (mailrelay.tu-graz.ac.at [129.27.2.202])
	by sbox.tugraz.at (8.13.7/8.13.7) with ESMTP id k5NDlFlT004350
	for <armin.krieg@student.tugraz.at>; Fri, 23 Jun 2006 15:47:15 +0200 (CEST)
Received: from [129.27.152.57] (t41 [129.27.152.57] (may be forged))
	(authenticated bits=0)
	by mailrelay2.tu-graz.ac.at (8.13.7/8.13.7) with ESMTP id k5NDlBAM006363
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
	for <armin.krieg@student.tugraz.at>; Fri, 23 Jun 2006 15:47:12 +0200 (CEST)
Message-ID: <449BF0DE.30808@iaik.tugraz.at>
Date: Fri, 23 Jun 2006 15:47:10 +0200
From: =?ISO-8859-1?Q?Johann_Gro=DFsch=E4dl?=
 <Johann.Groszschaedl@iaik.tugraz.at>
Organization: IAIK at Graz University of Technology, Austria
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT; rv:1.7.1) Gecko/20040707
X-Accept-Language: de, en
MIME-Version: 1.0
To: Armin Krieg <armin.krieg@student.tugraz.at>
Subject: Re: Telematik - Magisterprojekt
References: <44954E54.5070403@iaik.tugraz.at> <4497DE82.1040400@iaik.tugraz.at> <4497E033.3030900@student.tugraz.at> <4497E073.5010304@iaik.tugraz.at> <4497E096.2090804@student.tugraz.at> <449B0114.9000703@iaik.tugraz.at> <449BC280.5080005@student.tugraz.at>
In-Reply-To: <449BC280.5080005@student.tugraz.at>
Content-Type: text/plain;
  charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Scanned-By: MIMEDefang 2.56 on 129.27.3.24
X-Scanned-By: milter-sender/1.10.904 (sbox.tugraz.at [129.27.3.24]); Fri, 23 Jun 2006 15:47:15 +0200
X-Scanned-By: MIMEDefang 2.56 on 129.27.10.19
X-Length: 3037
X-UID: 42112
Comment 10 Carsten Burghardt 2006-06-27 09:16:22 UTC
*** Bug 65354 has been marked as a duplicate of this bug. ***
Comment 11 Michael Trausch 2006-06-27 16:25:32 UTC
I experience this bug, too.  I use to bogofilter to remove mail that I have marked as spam on a regular basis, since I receive quite a bit of it.  I am using KDE 3.5.3, and the KMail/Kontact versions that came with it, via Ubuntu packages.

Also, sometimes instead of completely duplicating the message, only part of the message makes it into the inbox, without headers or with partial headers.
Comment 12 Michael Trausch 2006-06-27 16:29:14 UTC
Created attachment 16806 [details]
Plaintext transcript of communication between KMail<==>IMAP Server

This is the plain-text transcript of the conversation that KMail has with the
IMAP server, this is just for a short session.	Can create a larger file with a
longer session if necessary.  Created using the "Follow TCP stream" option of
Ethereal, and the password is edited out.
Comment 13 Michael Trausch 2006-06-27 16:31:49 UTC
Created attachment 16807 [details]
Ethereal debug file -- edited

This is like attachment 16806 [details], but directly from ethereal w/o annotation. 
Password is edited out, so frame 11's checksum shows as bad.
Comment 14 Michael Trausch 2006-06-27 16:34:13 UTC
Created attachment 16808 [details]
Ethereal debug file -- edited w/ corrected cksum

This is similar to attachment 16807 [details], but with Frame 11's checksum updated. 
However, some data appeared to have been lost in the process, so I am attaching
both just to be safe.
Comment 15 Robert Brugman 2006-06-28 20:17:23 UTC
Confirmed for me as well.  Im running the latest KDE and Kmail, as well as F-Prot and Spamassasin on Kubuntu 6.06 Kernel 2.6.15-23.
Comment 16 ivan 2007-01-24 23:14:28 UTC
Hi,

i can confirm that too. (kmail 1.9.5)
it is surely related to spam filtering.
it's likely a kmail issue since i have the same problem with bogofilter as well as spamassassin. 
in header x-uid is changed in each duplicate mail.

ciao
Comment 17 Sergei Zuyev 2007-02-08 16:36:48 UTC
I just installed KDE3.5.6 and the bug is still there. Kmail has just crashed on me after I decided to delete the first out of 40or so duplicates of a message I sent to myself. Here's the backtrace it generated:

(no debugging symbols found)
Using host libthread_db library "/lib/i686/libthread_db.so.1".
(no debugging symbols found)
`shared object read from target memory' has disappeared; keeping its symbols.
(no debugging symbols found)
(no debugging symbols found)
....
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread -1248412800 (LWP 27483)]
[New Thread -1292895328 (LWP 27768)]
[New Thread -1284502624 (LWP 27767)]
[New Thread -1276109920 (LWP 27766)]
[New Thread -1267717216 (LWP 27765)]
(no debugging symbols found)
(no debugging symbols found)
....
[KCrash handler]
#6  0x00000001 in ?? ()
#7  0xb4c069eb in KMMoveCommand::execute () from /usr/lib/libkmailprivate.so
#8  0xb4bbabe6 in KMCommand::slotPostTransfer ()
   from /usr/lib/libkmailprivate.so
#9  0xb4c103e2 in KMCommand::qt_invoke () from /usr/lib/libkmailprivate.so
#10 0xb4c1082b in KMMenuCommand::qt_invoke () from /usr/lib/libkmailprivate.so
#11 0xb4c10908 in KMMoveCommand::qt_invoke () from /usr/lib/libkmailprivate.so
#12 0xb4c1099b in KMDeleteMsgCommand::qt_invoke ()
   from /usr/lib/libkmailprivate.so
#13 0xb63006df in QObject::activate_signal ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#14 0xb4b80149 in KMCommand::messagesTransfered ()
   from /usr/lib/libkmailprivate.so
#15 0xb4c1157e in KMCommand::transferSelectedMsgs ()
   from /usr/lib/libkmailprivate.so
#16 0xb4c11756 in KMCommand::slotStart () from /usr/lib/libkmailprivate.so
#17 0xb4c103f8 in KMCommand::qt_invoke () from /usr/lib/libkmailprivate.so
#18 0xb4c1082b in KMMenuCommand::qt_invoke () from /usr/lib/libkmailprivate.so
#19 0xb4c10908 in KMMoveCommand::qt_invoke () from /usr/lib/libkmailprivate.so
#20 0xb4c1099b in KMDeleteMsgCommand::qt_invoke ()
   from /usr/lib/libkmailprivate.so
#21 0xb63006df in QObject::activate_signal ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#22 0xb666a598 in QSignal::signal () from /usr/lib/qt3/lib/libqt-mt.so.3
#23 0xb63206ea in QSignal::activate () from /usr/lib/qt3/lib/libqt-mt.so.3
#24 0xb632813c in QSingleShotTimer::event ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#25 0xb6298e20 in QApplication::internalNotify ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#26 0xb629ac49 in QApplication::notify () from /usr/lib/qt3/lib/libqt-mt.so.3
#27 0xb6a5b58b in KApplication::notify () from /usr/lib/libkdecore.so.4
#28 0xb76cc5ad in QApplication::sendEvent () from /usr/lib/libkhtml.so.4
#29 0xb628b7ab in QEventLoop::activateTimers ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#30 0xb6242e48 in QEventLoop::processEvents ()
   from /usr/lib/qt3/lib/libqt-mt.so.3
#31 0xb62b2552 in QEventLoop::enterLoop () from /usr/lib/qt3/lib/libqt-mt.so.3
#32 0xb62b2362 in QEventLoop::exec () from /usr/lib/qt3/lib/libqt-mt.so.3
#33 0xb629a9c9 in QApplication::exec () from /usr/lib/qt3/lib/libqt-mt.so.3
#34 0x08065edb in ?? ()
#35 0xb598575c in __libc_start_main () from /lib/i686/libc.so.6
#36 0x0805be71 in ?? ()
Comment 18 Sergei Zuyev 2007-02-08 16:43:22 UTC
*** This bug has been confirmed by popular vote. ***
Comment 19 Petr Kulhavy 2007-03-11 20:45:15 UTC
I can confirm that on Kmail 3.5.6 (Gentoo), the same bug. But it seems to appear ONLY if the spamfilter is on. 

I'm using IMAP, my inbox is of type mailbox (not maildir) on the server. On the server I can observe the last message to be copied many times and subsequently deleted. Why??? Apparently this mechanism doesn't work correctly and sometimes the message is forgotten to be deleted. 

This behaviour is pretty annoying, is there a workaround at least?
Comment 20 Petr Kulhavy 2007-03-12 09:49:11 UTC
An update: it's not the LAST message which is duplicated, it's the CURRENTLY OPEN message! Which is quite often the last one :-)
Comment 21 Antonio Batovanja 2007-03-23 00:38:36 UTC
I can confirm comments #19 and #20 (KMail 3.5.5 and 3.5.6, KUbuntu).
Bug 95064 and bug 119149 are duplicates of this one (or rather the other way around).
Comment 22 Thomas McGuire 2007-03-27 20:30:11 UTC
>The duplication problem has indeed something to do with filters or bogofilter. 
OK, closing as dupe of bug 95064

*** This bug has been marked as a duplicate of 95064 ***