Bug 119149

Summary: Filtering in IMAP can duplicate emails
Product: [Unmaintained] kmail Reporter: Iñaki Baz Castillo <ibc>
Component: IMAPAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED DUPLICATE    
Severity: normal CC: bss, gustavo.boiko, joerg, warcomp
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: unspecified   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Iñaki Baz Castillo 2005-12-29 00:44:18 UTC
Version:           desconocido (using KDE 3.5.0, Debian Package 4:3.5.0-1 (testing/unstable))
Compiler:          Target: i486-linux-gnu
OS:                Linux (i686) release 2.6.14

I have a Postfix + Cyrus IMAP server and Kmail with an IMAP account ("my_imap_account@lalala.net").

I've detected a problem, I'll try to reproduce it:

- I have a "From filter" configured. This mark as important all the mails from "pepe@pepe.com" (example). This filter is applied to IMAP account too (by default it's not, but I've changed enabled it in:
  "Configure Filters..."
      "Advanced"
            "Apply this filter in incoming messages:"
                  [*] from all accounts

- Now I compose an email with FROM "pepe@pepe.com" and TO "my_imap_account@lalala.net", and send it.

- I press "Check Mail" and I receive the email marked as important (filters work, ok).

- But now, quickly, I press again and again in "Check Mail", and the same email appears as duplicate or more times if I press many times "Check Mail". Those emails appear too in "/var/spool/cyrus/mail/m/user/my_imap_account".

- The only difference in the duplicated emails is "X-UID: ****"

- Note that this only occurs with filtered emails, if I disable de filter for IMAP accounts or receive an email that is not filtered, this bug doesn't appear.
Comment 1 Andreas Gungl 2005-12-29 11:31:45 UTC
Please check KMail 1.9.1 (in KDE 3.5.1) when it's out.
There have been some adjustments in the IMAP filtering code which may have 
corrected this problem too.
Comment 2 Iñaki Baz Castillo 2005-12-29 18:55:35 UTC
OK, I'll wait until KDE 3.5.1 is available for Debian (Alioth sources).
Comment 3 Guido Barzini 2006-01-13 18:49:11 UTC
 I've seen this running kmail 1.9.1 ( KDE 3.5.0-4.2.fc3.kde as packaged by kde-redhat.sourceforge.net).
Comment 4 Andreas Gungl 2006-01-17 21:04:52 UTC
Well, I've tried it and couldn't reproduce. Perhaps my machine and / or the network connection to the IMAP server is faster.
I was not able to get duplicates of the messages.

Do you perhaps have a longer list of filters which are applied to incoming messages from the IMAP account?
Comment 5 Iñaki Baz Castillo 2006-01-17 23:01:37 UTC
I've now tried with just ONE filter for my IMAP account. This filter just marks as important the incomming messages from "userA@domain.com", nothing else.

I send an email from that account to my IMAP account so the email appears, but if I receive again the mail, there appears a duplicate mail. And if I receive again my mail another duplicate mail appears. Exactly the same email. The only difference is the following:

  root@machine /var/spool/cyrus/mail/m/user/myuser $ ls -l
  total 32K
  -rw-r-----  1 cyrus mail  734 2006-01-17 22:20 1131.
  -rw-r-----  1 cyrus mail  727 2006-01-17 22:29 1140.
  -rw-r-----  1 cyrus mail  727 2006-01-17 22:29 1141.
  -rw-r-----  1 cyrus mail  727 2006-01-17 22:29 1142.
  -rw-r-----  1 cyrus mail 2,3K 2006-01-17 22:29 cyrus.cache
  -rw-r-----  1 cyrus mail  149 2005-12-28 19:38 cyrus.header
  -rw-r-----  1 cyrus mail  284 2006-01-17 22:29 cyrus.index

1131. is a previous email that doesn't match any filter. The bug doesn't occur with this mail.

1140. is the first email that matches the filter rule.

1141. is the first duplicate mail generated when receiving again the mail.

1142. is the second duplicate mail.

  root@machine /var/spool/cyrus/mail/m/user/myuser $ diff 1140. 1141.
  20c20
  < X-UID: 1136
  ---
  > X-UID: 1137


I do "tail -f /var/log/mail.log" but nothing occur when receiving again the mail and the duplicates messages appear. The same in "mail.err", "mail.log" and "syslog".


My IMAP server is in MY COMPUTER, and I receive the mail in THE SAME computer, so it should be faster than any network.   ;)
My computer is an AMD 2200 Mhz with 757MB RAM. I think it's enough.


The following is about the same bug, but I can't understand:

- I have my IMAP box empty.

- I press "Receive" and a mail from "userA@domain.com" appear in Kmail. And the cyrus maildir shows this:

  -rw-r-----  1 cyrus mail  702 2006-01-17 22:52 1246.     <-- This is the email.
  -rw-r-----  1 cyrus mail 1,2K 2006-01-17 22:52 cyrus.cache
  -rw-r-----  1 cyrus mail  149 2005-12-28 19:38 cyrus.header
  -rw-r-----  1 cyrus mail  180 2006-01-17 22:52 cyrus.index

- I receive again the mail and see the original email duplicated in Kmail. But in my cyrus maildir I see the following:

  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1247.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1248.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1249.
  -rw-r-----  1 cyrus mail 2,3K 2006-01-17 22:53 cyrus.cache
  -rw-r-----  1 cyrus mail  149 2005-12-28 19:38 cyrus.header
  -rw-r-----  1 cyrus mail  284 2006-01-17 22:53 cyrus.index

   In Kmail I see just 2 email, but the cyrus maildir shows 3, and the original "1246." has dissapeared, look at the "22:53" of the new 3 emails.

- I receive again the mail, I see 3 times the email in Kmail, but...

  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1247.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1248.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1249.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1250.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1251.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1252.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1253.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1254.
  -rw-r-----  1 cyrus mail  730 2006-01-17 22:53 1255.
  -rw-r-----  1 cyrus mail 5,8K 2006-01-17 22:53 cyrus.cache
  -rw-r-----  1 cyrus mail  149 2005-12-28 19:38 cyrus.header
  -rw-r-----  1 cyrus mail  596 2006-01-17 22:53 cyrus.index

   ¿?¿?¿?¿?¿?¿

- Each time I receive my email, Kmail shows one more message but cyrus maildir shows a lot of them, more and more.

- Now I close Kmail and reopen. I see exactly the same number of emails in my IMAP account, and the same (a lot of them) in cyrus maildir.

- Bur now if I receive again the mail the bug doesn't occur.

- But if I send another email that mathes the filter to my IMAP account, the same problem occurs but just with the new mail. Maybe this is not because reopening Kmail, maybe after some time this problem dissapears, I'm not sure, sorry.


  
Comment 6 Tommi Tervo 2006-01-25 15:03:20 UTC
*** Bug 120726 has been marked as a duplicate of this bug. ***
Comment 7 Andy 2006-02-03 21:22:40 UTC
*** This bug has been confirmed by popular vote. ***
Comment 8 Iñaki Baz Castillo 2006-04-07 00:05:34 UTC
I think this bug doesn't occur more since Kmail 1.9.1 in KDE 3.5.2, but I'm not sure at all.
Comment 9 Volker Holthaus 2006-05-02 16:26:32 UTC
I have the same problem with Kmail 1.9.1 and KDE 3.5.2 an Exchange Server as IMAP and Spamassasin to filter spam mails.
Comment 10 Aaron Williams 2006-06-08 05:44:17 UTC
I can confirm this same problem when talking to an Exchange server with duplicate emails, even though I do not have any filters in place.  I am running KDE 3.5.2 on Solaris.
Comment 11 Grant Shearer 2006-07-17 17:31:37 UTC
I can confirm this on Kontact 1.2, Kmail 1.9.1, KDE 3.5.2 on Kubuntu (Dapper).

Talking with an Exchange server via IMAP...everything was working perfectly until I ran through the Anti-Spam Wizard, which set up filters for Bogofilter.  Afterward it looked like the latest new message would get duplicated everytime Kmail checked the IMAP server.  So, over the course of a half hour I was getting quite a few dupes.

Removing the filters fixed the problem, but it would be nice to not have to.
Comment 12 Gustavo Pichorim Boiko 2006-07-21 18:57:34 UTC
This is still happening on KMail 1.9.3
Comment 13 Jörg Noebels 2007-02-16 22:38:05 UTC
Same here with KMail 1.9.6. After disabling all the filters within KMail the problem doesn't occur.

I don't have an Exchange server, just a simple IMAP server
Comment 14 Andreas 2007-03-06 14:16:42 UTC
Same here with version 1.9.6. Problem occurs with different mailservers (Courier and hmailserver (Windows) )
Comment 15 Thomas McGuire 2007-03-27 20:25:30 UTC

*** This bug has been marked as a duplicate of 95064 ***
Comment 16 Daniel Oberhoff 2007-09-26 15:32:48 UTC
I think I just had the same problem with kmail 1.9.6. I had checkinterval to 1 minute. I got on average 20 dupes for each mail (spam or not)