Bug 95064 - Duplicates messages in imap folder after filtering
Summary: Duplicates messages in imap folder after filtering
Status: RESOLVED UNMAINTAINED
Alias: None
Product: kmail
Classification: Applications
Component: IMAP (show other bugs)
Version: 1.10.0
Platform: Gentoo Packages Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords: triaged
: 119149 128014 137736 143584 147680 149725 151980 167801 179385 357031 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-12-13 11:53 UTC by Pau Capdevila
Modified: 2015-12-22 14:10 UTC (History)
43 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
KMail with detailed progress window (269.71 KB, image/png)
2011-03-12 11:11 UTC, Eggert Ehmke
Details
Screenshot of mass email duplication (84.72 KB, image/png)
2015-10-31 19:58 UTC, Rick Timmis
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pau Capdevila 2004-12-13 11:53:03 UTC
Version:            (using KDE KDE 3.3.1)
Installed from:    Debian testing/unstable Packages

My mails seem to be reproducing asexually by bipartition!

I do not know how but some mails are appearing duplicated (somehow they are copied, the number of instance doubles every when I see them or I don't know)

IMAP server is a Lotus Domino server.

If I find how to reproduce it I'll post it later.

Thank you.
Comment 1 Till Adam 2004-12-13 12:23:38 UTC
On Monday 13 December 2004 11:53, pau.capdevila@upcnet.es wrote:

> My mails seem to be reproducing asexually by bipartition!

Must be all the viagra mails one gets these days ...

> I do not know how but some mails are appearing duplicated (somehow they are
> copied, the number of instance doubles every when I see them or I don't
> know)
>
> IMAP server is a Lotus Domino server.
>
> If I find how to reproduce it I'll post it later.

Is this an online imap or a disconnected imap account?

Till

Comment 2 Pau Capdevila 2004-12-13 19:18:26 UTC
By now I can only say it is (normal) imap.
Tomorrow I'll try to reproduce it at work and send more information.

Thank you!

On Monday 13 December 2004 12:23, Till Adam wrote:
> ------- You are receiving this mail because: -------
> You reported the bug, or are watching the reporter.
>
> http://bugs.kde.org/show_bug.cgi?id=95064
>
>
>
>
> ------- Additional Comments From adam kde org  2004-12-13 12:23 -------
>
> On Monday 13 December 2004 11:53, pau.capdevila upcnet es wrote:
> > My mails seem to be reproducing asexually by bipartition!
>
> Must be all the viagra mails one gets these days ...
>
> > I do not know how but some mails are appearing duplicated (somehow they
> > are copied, the number of instance doubles every when I see them or I
> > don't know)
> >
> > IMAP server is a Lotus Domino server.
> >
> > If I find how to reproduce it I'll post it later.
>
> Is this an online imap or a disconnected imap account?
>
> Till

Comment 3 Tiago Freire 2005-01-25 11:48:58 UTC
I am having a similar problem, but on disconnected imap. Things that might make a difference are that this is my FIRST sync with a M$ exchange server, and I have almost 1 GB of emails. The connection has broken before completion, and on each retry it starts from the beginning, creating duplicates. 
Another thing which might influence it is that I have some filters moving some messages to other folders. 
Maybe it is related (at least in my case) to a 'first-sync' scenario? I think it should continue from where it stopped when the connection was broken. 

I am using Gentoo on an athlon xp , kde 3.3.2.
Comment 4 Alexandre Perera 2006-03-24 11:10:55 UTC
Identical Problems in here. Interestingly it's is just some messages that get replicated without control. I left kontact open overnight and it filled the IMAP account with the same message repeated some hudreds of times. 

The point in here is that I just flagged this message before leaving yesterday (Marked as Important). however it also occurs with the message I just opened, so the mark is not the culprit. 

Alex
Using Gentoo on a x86, 2.6.something, server is a lotus notes server
Comment 5 Akos Polster 2006-08-21 10:17:44 UTC
Same problem here, using an MS Exchange IMAP server (on-line). KDE is 3.5.4-1.1.el4.kde. The filters are the standard SpamAssassin ones configured by the KMail spam filtering wizard.
Comment 6 Ingo Frommholz 2006-09-01 18:42:28 UTC
I also had the problem with email duplication. I am using kubuntu and kmail 4:3.5.4-0ubuntu1~dapper1 (1.9.4). IMAP server is Cyrus IMAP4 v2.1.18-IPv6-Debian-2.1.18-1+sarge2 on a Debian sarge machine. Precisely, I experienced the following problems:

On abovementioned IMAP account, when I mark a message as "important", it gets duplicated two or three times in kmail. Having a look at the IMAP folder with mutt, I found such a message nearly ten times(!) in my Inbox, seven times marked as "deleted". It seems this message gets duplicated every time I access my Inbox!

This also happens with another IMAP server. My wife and I share an account using the German provider web.de; she fetches messages with POP3 (leaves on server), I use IMAP to access the mailbox. It often happens now that my wife finds messages she previously deleted, and duplicate messages. I do not know which IMAP server is used by web.de, but as far as I know it is not Cyrus.

I also discovered that kmail often seems to upload emails; this might be related to this bug. I discovered it when opening an email with a 3 MB attachment which took some time because the whole email was uploaded again.

I am now switching back to kmail 1.9.1 (4:3.5.2-0ubuntu6) - let's see if the problem appears here. If so, I have to use thunderbird...:-(
Comment 7 Ingo Frommholz 2006-09-01 19:52:38 UTC
I could witness the procss now with a longer email which takes me approx. 2 minutes to download (3 MB, my connection at home is not the best). I opened kmail and watched the IMAP Inbox with mutt. kmail seems to to the following

1. Download the message
2. Mark the message as "delete" on the IMAP server
3. Upload the message again

I do not understand any of these steps since the message we are talking about was a new one, but it wasn't selected and I did not open it. kmail did the three steps after I started it, without selecting anything.

Another problem is that if within step 3 something happens (for example you accidently close kmail), the message gets purged! I had this case once. So we don't have only a duplication problem here, but it seems that emails can be deleted accidentaly, which is even more serious.
Comment 8 Ingo Klöcker 2006-09-09 09:41:59 UTC
@Ingo: Most likely KMail downloads the message and reuploads it because you apply filters to new messages for this IMAP account. Since with IMAP the only way to change a message is to download the message, delete the old copy and then upload the changed message there's no other solution for this. If you don't want to apply any filters to IMAP messages then you should disable the application of all filters for IMAP accounts.
Comment 9 Bruno 2006-09-12 19:09:45 UTC
*** This bug has been confirmed by popular vote. ***
Comment 10 Ingo Frommholz 2006-09-13 12:28:49 UTC
@IngoK: After a few days of testing I find that this seems to solve the problem. If you deactivate any filtering in IMAP folder, the duplicate message problem disappears. Many thanks! It seems client-side IMAP filtering is evil ;-).

But for me it sounds strange that an old email is going to be deleted and then uploaded again. In some cases this can lead to email loss! I am really not into the IMAP protocol nor in the kmail implementation, but shouldn't it be possible to mark an email as "delete" after the filtering process outputs that the mail should be moved elsewhere, but otherwise not?

Anyway, good work, many thanks again!
Comment 11 Frank Aune 2006-10-20 16:09:50 UTC
Same on Kubuntu Dapper - disabling filters for IMAP solves the duplication problem.
Comment 12 Ramon Sunye 2006-10-23 21:14:52 UTC
Same problem with gentoo with kmail 3.5.5 (and earlier) I'll try disabling the filters for IMAP.
If that is the problem, maybe that should be the default option for any filter, including the ones created using the anti-spam and anti-viruses wizards, as I haven't changed those options, nor a typical user will do.

Comment 13 Jean-Damien Durand 2007-01-27 07:33:27 UTC
The default should be that filters just work.
Annoying bug - voting and hoping that filters could be applied again on (d)IMAP accounts in the near future without this side-effect.
Comment 14 Jonathan Marten 2007-01-27 10:56:07 UTC
*** Bug 137736 has been marked as a duplicate of this bug. ***
Comment 15 Antonio Batovanja 2007-03-23 00:41:05 UTC
I can confirm this bug in KMail 1.9.5 and 1.9.6 using online IMAP (Courier) and client-side filtering.
There's a bit more information in bug 138811.

Bug 119149 seems to be a duplicate of this bug.
Comment 16 Thomas McGuire 2007-03-27 20:25:35 UTC
*** Bug 119149 has been marked as a duplicate of this bug. ***
Comment 17 Thomas McGuire 2007-03-27 20:30:14 UTC
*** Bug 128014 has been marked as a duplicate of this bug. ***
Comment 18 Tommi Tervo 2007-03-29 15:47:50 UTC
*** Bug 143584 has been marked as a duplicate of this bug. ***
Comment 19 Sergei Zuyev 2007-04-26 12:02:58 UTC
I agree that disabling filters is not a solution. Also, this bug seem to disappear and re-appear time to time. I am running spamassassin-3.0.4 with kdepim-kmail-3.5.4 on IMAP account under Mandriva 2006 Official without any problem. 

In contrast, spamassassin-3.1.8 with
kdepim-kmail-3.5.6 under Mandriva 2007.0 does have this bug.
Comment 20 dukat 2007-04-26 13:56:45 UTC
I can confirm this book with Kubuntu 7.04 (Kmail 1.9.6) after I ran the Spam wizard. Server is IMAP Exchange / Microsoft SMTPSVC(6.0.3790.1830); 
Comment 21 Richard Lewis 2007-06-14 15:20:33 UTC
I was just about to report this bug but, helpfully, the bug reporting system directed me to this page.

I'm on KMail 1.9.7, KDE 3.5.7 and using an MS Exchange server (don't know its details, except that it must be 3v1l).

I have Spamassassin running with slightly modified filters from those created by the wizard. I didn't get the duplication problem until I started using the spam filters.

I've also logged into my email account separately (using the equally 3v1l outlook web access, webmail-like system). And it confirms that there are hundreds of copies of one of my emails.

It seems this bug has been around for about 3 three years. Does anyone have any idea what causes it? Might someone be able to fix it?
Comment 22 Thomas McGuire 2007-07-09 17:54:24 UTC
*** Bug 147680 has been marked as a duplicate of this bug. ***
Comment 23 Thomas McGuire 2007-09-20 19:59:18 UTC
*** Bug 149725 has been marked as a duplicate of this bug. ***
Comment 24 John Andersen 2007-09-21 23:58:09 UTC
> It seems this bug has been around for about 3 three years.

Yes, at least that long.  Its clear none of the developers use IMAP.

It gets ignored, because as soon as someone hits this page via google they figure out that turning off Kmail spam filters seems to solve the problem, and
they go about their business and forget about it.

It takes a long time to track down this bug, but the end result is that KMAIL is duplicating the mail, and the duplicates REALLY do exist and are sent back to the imap server chewing up space, bandwidth, and time.

Comment 25 Michael Murphy 2007-10-01 01:25:43 UTC
It's happening for me with kubuntu 7.04, which I've just installed for the first time, but it *doesn't* happen with debian etch's kmail.  I used identical configuration files (I copied them over from debian).  Before I realized what was happening, kmail had generated thousands of duplicate messages.  
Comment 26 Marc Haisenko 2007-10-17 10:45:52 UTC
I'm seeing the same thing (right now it's three e-mails that get duplicated over and over again, all others not). It forces me to use POP3 instead of IMAP and do the backup myself. Because of all the mailing lists I need to communicate with I rely on filtering, so disabling filtering is just not an option: instead it's back to POP3 again for me.

Version 1.9.6 (enterprise 20070904.708012), KDE 3.5.7 (OpenSUSE 10.3) connected to an MS Exchange server (what a primitive thing, BTW... damn company policy).
Comment 27 Riskable 2007-10-23 16:17:26 UTC
I'm also seeing this bug in Kubuntu Gutsy (7.10).  Kmail version 3.5.7enterprise20070926-0ubuntu2 using MS Exchange as the IMAP server.
Comment 28 mi+kde 2007-12-15 18:47:03 UTC
Bug 153975 was deemed a duplicate of this bug.  So, in all likelihood, is the bug 118074.
Comment 29 Jean-Damien Durand 2007-12-15 18:52:59 UTC
Now, I don't know why (yes, I accessed the imap remote server using another client for one or two days - never concurrently with kmail - could not believe one would say me this is the cause - and in fact I am doing that since years, oh well), the /whole/ content of the inbox folder is explicitely downloaded, message per message, and uploaded, message per message of course, and this at each mail check. During this phase the folders shrinks up to the empty state, is progressively 'refilled'.

This makes now kmail now unusable - no to say impossible to promote to anybody but those that, like me, believe in the intrisic (and explicit) power of kmail.

Can please somebody do something with this bug ? I unfortunaly can't help, c++ nor kde are not at all in my computing background.

Thanks.
Comment 30 Jean-Damien Durand 2007-12-16 19:55:41 UTC
Followup of my previous port.
Just to inform that one day later, kmail now stopped to flash the full inbox.
There is something that looks out of control from user point of view in kmail+notcachedimap.
Anyway, it is usable again.
Comment 31 Baldrun Orv 2008-02-16 10:30:46 UTC
I have the same problem with disconnected imap and filtering (spamassasin & bogofilter) creating duplicate mails every time disconnected imap is synchronised. Pop3 is not affected. 

I am using kde 3.5.8, kmail 1.9.7, bogofilter 1.1.5 and spamassasin 3.2.1-r1 on a gentoo machine.

First occurance of this bug for me was after upgrading from kde 3.5.6 to 3.5.8 which upgraded kmail, too. 

This bug makes kmail pretty much unusable for business and forces me to switch to thunderbird. Please do something about it. Thanks a thousand tons in advance.
Comment 32 Thomas McGuire 2008-02-17 02:15:32 UTC
*** Bug 151980 has been marked as a duplicate of this bug. ***
Comment 33 luisfe 2008-03-16 20:29:39 UTC
The same problem using cachedimap in fedora 8 with kmail 1.9.9
It is just unusuable.
Comment 34 Sergei Zuyev 2008-04-10 11:40:50 UTC
The problem has re-appeared in Mandriva 2008.1rc2 (Spring)
It was present in the original 2008.0 until updates in Autumn ironed it out. And now it is back! It seems, it likes very much fresh distros :-(

spamassassin-3.2.4-3mdv2008.1
kdepim-kmail-3.5.9-8mdv2008.1
Comment 35 Hazel Victoria Campbell 2008-04-12 17:04:57 UTC
Try this: 

1. Disable all filtering on new mail
2. check the mail (you now see the messages that keep getting duplicated in your inbox)
3. and check again (they should not duplicate anymore)
4. move all messages out of your inbox
5. check again (they should not reappear)
6. copy a "dummy" message into your inbox
7. check again (only the "dummy" message you copied should be there)
--- Note: This step seems to be the important one; skip it and the dummy message will be treated as new and filtered/duplicated?  This step should upload the dummy message to the server.
8. check yet again, nothing should change
9. re-enable filtering on new messages
10. send yourself an email / recieve some new spam / etc.
11. check your email - the new message should appear in the folder that the filter moves it to (i.e. spam, ham, whatever) and the "dummy" should remain in your inbox
12. check your email again - the new message you sent yourself may not be duplicated!

Something like this worked for me -- I believe the problem happens when the local version of the inbox is empty AND the imap cache of the inbox is empty AND the remote imap folder is NOT empty AND filtering is enabled.

Does this work for anyone else?
Comment 36 mi+kde 2008-04-12 18:02:12 UTC
I, for one, am seeing this problem constantly with regular (rather than cached) IMAP. I don't use any filtering either...
Comment 37 Daniel LaLiberte 2008-07-22 21:23:57 UTC
Another bump...  same problem in Fedora Core 9,  KMail 1.9.9.  I had SpamBayes turned on, and over the weekend, duplicates filled up my inbox so bad that KMail hung.  I had to go clear them out with Thunderbird.  Only new messages get duplicated for me - once I have read them, even if I keep them, they don't get duplicated.

This is a very annoying problem, especially for people dealing with spam using filters, and should be delt with as very high priority.  Fortunately I can avoid the spam problem other ways.
Comment 38 Jean-Damien Durand 2008-07-22 21:41:01 UTC
It seems that nobody cares about what looks to me a showstopper for massive use of kdepim.
Bump.
Comment 39 Robert Marmorstein 2008-07-22 23:09:23 UTC
Actually, I read on a forum somewhere (or on IRC?) that this bug had been fixed in recent SVN versions of kmail (for kde 4).  I haven't yet built it on my box, so I can't confirm, but I think the problem is being worked on.
Comment 40 John Andersen 2008-07-22 23:15:56 UTC
But KDE 4 is still beta (in my not so humble opinion) and I can't switch my users to a half functioning system.  This fix needs to be back ported.  Until it is you really can't allow any client side filtering regardless of which IMAP server you use.
Comment 41 Thomas McGuire 2008-08-02 14:03:04 UTC
*** Bug 167801 has been marked as a duplicate of this bug. ***
Comment 42 Thomas McGuire 2008-08-02 14:06:24 UTC
I still had no luck reproducing this. It would nice if somebody could narrow that problem down so I can reproduce this myself and then fix it.
Comment 43 Rodrigo Fresneda 2008-08-02 20:59:51 UTC
Have you tried enabling spam filters through kmail's wizard? Together with imap, this is guaranteed to cause duplication of messages, at least for kmail v1.9.9 in debian sid.
Comment 44 Jonathan Traum 2008-08-04 15:53:31 UTC
Comment #43 seconded.  Using IMAP to connect to an Exchange server, I experienced the problem when I used the Anti-Spam wizard with Bogofilter.  Running Kmail 1.9.9, KDE 3.5.9, under Ubuntu 8.04.
Comment 45 Daniel LaLiberte 2008-08-05 16:25:06 UTC
Thomas, in Bug 167801, you mentioned that you tried reproducing this using gmail's IMAP feature.  Perhaps there is a problem with other IMAP servers that gmail has solved.  I don't know what IMAP server we are using locally, but it is almost certainly not the MS exchange server - I could find out if that helps.  

Also, I was using connected mode, not disconnected mode.  I see some people have the problem with disconnected mode as well.
Comment 46 mchugh19@yahoo.com 2008-08-06 04:48:50 UTC
My mail sever was just migrated to exchange, but I was seeing this behavior with UW's imap server.
Comment 47 Neal Becker 2008-08-09 03:57:31 UTC
There seems to be a strong correlation between duplicates and filters that have multiple 'or-ed' expressions, which both match.  I'm guessing that 2 matches of the or-ed terms result in 2 deliveries.

Like: match any of the following, with 2 conditions that both match, using 'move into folder'.
Comment 48 mi+kde 2008-08-11 17:21:24 UTC
I'm not using filters /at all/ yet routinely get some message duplicated. I'm afraid, filtering is not to blame here -- we may be barking up the wrong tree.
Comment 49 Nico Schlömer 2008-08-11 17:28:06 UTC
yep, that goes for me, too. imap runs on an ms exchange server (sigh), no filters here, but occasional message duplication.
Comment 50 John Andersen 2008-08-11 18:47:48 UTC
#48 and #49 :  Then I suggest you guys open yet another bug report, centered around exchange server, because I believe this bug is fairly focused around Kmail's internal filters causing duplication REGARDLESS of server type, and you
may be seeing something totally different. It might be an Exchange bug totally unrelated to Kmail, especially if you only get "occasional" dups.

Maybe you could fire up Thunderbird for a few days and see if the dupes persist?

But before you file a new bug report, double check that you have no manually created filters at all, and also that you have not turned on spam tools that are part of Kmail. (These spam extensions create filters).

Note also that not ALL filters cause problems.  For instance, filters that simply move messages to LOCAL FOLDERS (on your machine) do not seem to cause problems.  The problems arise when filters move messages to imap folders or flag messages in some way (mark them read or such).

Kmail when used with Dovecote, Curus, UW qmail have all shown problems, as well as MS Exchange. 

Comment 51 Neal Becker 2008-08-11 19:03:41 UTC
Setup here:

dimap
dovecot
All filters are to imap server (not local folders)
Lots of duplicates, seemingly random.
Comment 52 mi+kde 2008-08-11 21:29:39 UTC
John Andersen! I /had/ a separate bug on this issue as you suggest (Bug 153975). As you can see over there, Thomas McGuire then declared it a duplicate of this one.
Comment 53 John Andersen 2008-08-11 21:58:29 UTC
Yes, I see that. Also http://bugs.kde.org/show_bug.cgi?id=118074

153975 seems like it is being simply marked as invalid because it "Might" be a duplicate of bug 118074 or of bug 95064. 

Of those 95064 (this bug) has been outstanding the longest (since 2004!!!) so this bug should be the lead bug IMHO.

Comment 54 Gaël de Chalendar (aka Kleag) 2008-09-16 11:35:51 UTC
Duplication problems here too with kmail from SVN (trunk from yesterday).

Server: MS exchange
Acces: disconnected IMAP

Number of mails initially: ~3000

Now, each time the synchronization fails for a reason or another, mails are duplicated (on the server too), thus I have more than 12000 mails and my account size limit is over.

Also, I have no idea how I could remove all the duplicates (up to 8 by mails) otherwise than manually
Comment 55 S. Burmeister 2008-09-16 11:42:55 UTC
In the folder menu, you can find "remove duplicate messages". I never used it with dimap though.
Comment 56 tneumann 2008-10-10 13:11:26 UTC
I can reproduce this 100% reliable within a few seconds, and I am relatively sure that it is caused by a lack of locking (or other synchronization). When it happends some internal KMail data structures get currupted, as illustrated by the example below.

How to reproduce:
1. start KMail
2. connect to some online IMAP server (tested with dovecot and Lotus Notes)
3. run the Spam wizard (default settings are fine) if not already done to get some filters (not required if already installed)
4. enter the INBOX, select some message
5. send yourself a message. Do not open it!
6. quickly press ^L multiple times to check for mail. This creates a number of duplicates.

Interesting observation:
7. quit KMail. Now KMail crashes and produces the following error message

ASSERT: "item_exists()" in file /usr/include/qt4/QtCore/qhash.h, line 1012


So most likely some hash table got corrupted as a side affect of unsynchonized access.

I tested the SVN trunk as of today, the bug is still there. I tried to run valgrind to get some insight, but unfortunately KMail crashes with some apparently unrelated problems then.

If a KMail developer has some ideas about debugging I am willing to help, but I am unsure where to start. The debug output shows no obvious problems at a first glance.
Comment 57 Ward 2008-10-15 20:56:21 UTC
I can confirm this bug using kde 4.1.2 and kmail 1.10.1,
I don't use filters on the client-side and i'm using ms exchange imap online.

For me this is a really annoying bug as i'm using kmail constantly
Comment 58 Thomas McGuire 2008-11-02 22:29:40 UTC
SVN commit 879276 by tmcguire:

When a message is already in the metadata map (either by UID or by MD5), don't
filter it, as it (hopefully) means we have already seen and therefore filtered
the message.
This fixes filters for online IMAP running multiple times for the same message,
and might even fix the problem of duplicate messages after filtering.

CCBUG: 95064


 M  +16 -1     kmfolderimap.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=879276
Comment 59 Thomas McGuire 2008-11-13 19:37:30 UTC
SVN commit 883868 by tmcguire:

Backport r879276 by tmcguire from trunk to the 4.1 branch:

When a message is already in the metadata map (either by UID or by MD5), don't
filter it, as it (hopefully) means we have already seen and therefore filtered
the message.
This fixes filters for online IMAP running multiple times for the same message,
and might even fix the problem of duplicate messages after filtering.

CCBUG: 95064



 M  +16 -1     kmfolderimap.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=883868
Comment 60 mi+kde 2008-11-13 19:39:20 UTC
How about a backport to the 3.x branch as well? Thanks!
Comment 61 S. Burmeister 2008-11-13 19:50:24 UTC
Please backport to KDE3 only, if it takes little to none resources. KDE4 is the present and future and should get all resources available, especially since they are scarce.
Comment 62 CW West 2008-11-27 04:27:07 UTC
I'm also seeing continued duplicates using KMail 1.10.3 with KDE 4.1.3 and Dovecot when using the "bogofilter check" filter.  This filter is tagging the incoming messages which seems to be confusing kmail.  I disabled that filter and it seems to fix the problem, and I made another filter that honors the existing Spam Assassin tags as a work around.  It looks like the fix has been implemented upstream, so hopefully it'll show up soon in a kubuntu-members-kde4 build.  
Comment 63 Thomas McGuire 2008-11-27 12:03:29 UTC
> I'm also seeing continued duplicates using KMail 1.10.3 with KDE 4.1.3

I think the fix is not in KMail 1.10.3 (it was backported after tagging IIRC), you'll have to wait for 1.10.4.
Comment 64 Thomas McGuire 2009-01-04 14:13:33 UTC
*** Bug 179385 has been marked as a duplicate of this bug. ***
Comment 65 Sven Wünschmann 2009-03-28 18:58:10 UTC
That bug found yet another victim. And this time it looks pretty weird. I encountered it on Gentoo when I migrated from KMail 1.9.9 (stable, using KDE 3.5.9) to KMail 1.11.1 (testing, using KDE 4.2.1).

Reproducibility: always

Setup_1:

  - Two accounts
      - POP3 account (GMX)
      - Online IMAP account (web.de)
  - Filters (applied to _all_ incoming messages from both accounts)
      - F_1: FROM: <e-mail address_1> MOVE TO: <local folder>
      - F_2: FROM: <e-mail address_2> MOVE TO: <IMAP folder>

KMail-1.9.9:

Bug does not show up. Mails are properly stored.

KMail-1.11.1:

Mails to the IMAP-account are reproduced asexually by bipartition. One instance is left in the inbox IMAP folder and a copy is stored at the filter's move destination IMAP folder. The copy has additional X-KMail headers, e.g:

----------------------------
    X-KMail-Filtered: 87
    Status: R
    X-Status: N
    X-KMail-EncryptionState:
    X-KMail-SignatureState:
    X-KMail-MDN-Sent:
----------------------------

Setup_2:

As Setup_1, but F_2 MOVE TO <local folder>

    => Mails to the IMAP account are not replicated!

Can anybody explain the different behaviour between KMail-1.9.9 and KMail-1.11.1?
Comment 66 Sven Wünschmann 2009-03-28 19:20:32 UTC
(In reply to comment #65)
> Setup_1:
> 
>   - Two accounts
>       - POP3 account (GMX)
>       - Online IMAP account (web.de)
>   - Filters (applied to _all_ incoming messages from both accounts)
>       - F_1: FROM: <e-mail address_1> MOVE TO: <local folder>
>       - F_2: FROM: <e-mail address_2> MOVE TO: <IMAP folder>

Asexual bipartition does not occur in KMail-1.11.1 when the IMAP account is of type "disconnected IMAP" instead of "online IMAP".
Comment 67 Neal Becker 2009-03-28 19:46:18 UTC
kmail 1.11.1
Using KDE 4.2.1 (KDE 4.2.1)

Haven't seen this problem in a long time.
Comment 68 Salvo "LtWorf" Tomaselli 2009-03-28 21:51:04 UTC
Well i see it everyday...
Comment 69 Nico Schlömer 2009-03-29 21:13:41 UTC
Me too.

Despite the various efforts to fix it, and the numerous patches that fixed things around this odd phenomenon, it has never been fixed for me either.

Cheers,
Nico
Comment 70 Sérgio Durigan Júnior 2009-06-10 23:06:19 UTC
Hello,

I'm still seeing this bug happening.  I'm using Gentoo, Kmail 1.11.4 on KDE 4.2.4.  Any news about it??

Thanks.
Comment 71 Nico Schlömer 2009-06-10 23:24:10 UTC
Confirmed here.

I was silently assuming that there will be massive architectural changes with the advent of Akonadi into Kmail, and thus many bugs disappearing. Does anyone know to what extent Kmail uses Akonadi right now, and how it will change in the upcoming  KDE 4.3?
Comment 72 Salvo "LtWorf" Tomaselli 2009-07-02 08:51:44 UTC
Hey i am not experiencing the problem anymore..
Comment 73 Bruno Bigras 2009-07-17 02:32:21 UTC
I had this bug while testing a patch to add a new filter type but I can't reproduce it on demand. (KDE 4.3 RC2, Qt 4.5.1 with KMail from trunk)

I think Thomas and Marton may be right that this bug have something to do with Bug 125084
Comment 74 David Klasinc 2009-08-13 23:57:12 UTC
KDE 4.3.0 with KMail 1.12.0 still does that. I've been using it for a week or so and today mails started to duplicate. Not every mail tho, but some of them are duplicated.
Comment 75 Markus Straub 2009-10-09 19:27:13 UTC
This happens since a long time for me as well (as long as I am using KDE4), right now I am using KDE 4.3.2 on Kubuntu 9.04 and see this bug every other day. It occurs especially often when big mails are downloaded over IMAP.
Comment 76 Tim Eberhardt 2010-03-21 10:03:30 UTC
Same over here... kmail 4.4.1, bogofilter 1.2.1.
It seems that this problem only occurs to me when kmail checks the default account on startup. Never had this problem with my 2nd or 3rd account or if an email arrives when kmail is already started. I'm using KDE for ~1 month now and I believe ths bug occured less then a week ago for the first time.
I'll try to give some debug info later if I'm able to catch it.
Comment 77 Tim Eberhardt 2011-01-16 12:46:52 UTC
Just to give a small update:

I wasn't able to catch/debug the issue but since some time things got even worse (kmail 4.4.9 atm). Now every mail I get will be duplicated. But still only on my first account (I manage 2 IMAP accounts).
Comment 78 Eggert Ehmke 2011-03-12 11:11:33 UTC
Created attachment 57895 [details]
KMail with detailed progress window

shows how the same mails are fetched ever again
Comment 79 Eggert Ehmke 2011-03-12 11:17:21 UTC
It happens on a Gentoo system with KDE 4.4.5 / KMail 1.13.5 (IMAP) too. If you open the detailed progress window (see screenshot in other comment) you can view the same mails transfered ever again and again.
Comment 80 Thomas Tanghus 2012-12-12 14:39:08 UTC
Just wanted to join the choir ;)
I only now got struck by this bug, and from reading https://blogs.kde.org/2012/10/17/three-days-berlin-or-improving-pim-user-experience I thought a fix or workaround would be backported to 4.9.3. I am now on 4.9.4 and get this:

I two filters set up to catch mails from a github repository. 

1. List-Id contains '<apps.owncloud.github.com>' and CC contains <my-email-address> => mark as important. The filter is set to keep processing if it matches.

2. List-Id contains '<apps.owncloud.github.com>' => Move into folder 'Notifications/Github'.

After filtering is applied 'Notifications/Github' contains two copies of the same mail.

For now I'm going to disable the first filter until this is solved; KMail is still the best mail client around, and KDEPIM is really starting to shine now most Akonadi problems seems to have been addresses.
Comment 81 Eugene Sobolev 2013-10-03 06:50:59 UTC
I can confirm this bug in KMail 4.8.3 and using POP3 (!!!) with filtering on ROSA/Mandriva.
After execute filter for move to folder "X", comes infinitely duplicate email in "inbox".
Comment 82 Laurent Montel 2013-10-03 07:33:52 UTC
For me confirm on version < 4.10 is not a confirm.
We are fixed a lot of bugs in 4.9/4.10/4.11 so before to confirm on very old version try with new one please.
Comment 83 Salvo "LtWorf" Tomaselli 2013-10-03 07:43:47 UTC
Oh I have 4.10 and I get duplicated emails all the time...
Comment 84 Tobias Leupold 2013-12-01 14:58:48 UTC
Probably the same filter issue described in Bug 316419?
Comment 85 eh.courriel2 2014-01-07 10:40:01 UTC
Unfortunately, I can confirm this bug with KMail version 4.12 using bogofilter filter rules
Comment 86 Leszek Lesner 2014-06-10 20:34:11 UTC
Confirming with 4.13.1
Comment 87 Laurent Montel 2015-04-12 09:54:14 UTC
Thank you for taking the time to file a bug report.

KMail2 was released in 2011, and the entire code base went through significant changes. We are currently in the process of porting to Qt5 and KF5. It is unlikely that these bugs are still valid in KMail2.

We welcome you to try out KMail 2 with the KDE 4.14 release and give your feedback.
Comment 88 Tobias Leupold 2015-04-12 10:04:32 UTC
The whole duplication problem (cf. Bug #323980, Bug# 322126, Bug #322676, Bug #283682, and Bug #333035 in which I describe my opinion on what causes this) is definitely there in 4.14. At least, I see it every day, only workarounded by the sleep filter descibed in Bug #333035.
Comment 89 Rick Timmis 2015-10-31 19:58:48 UTC
Created attachment 95240 [details]
Screenshot of mass email duplication

Hi
This problem still exists.. please see Screenshot
I have Kmail 5.0.2 using through Kontacts 5.0.2 on Kubuntu 15.10

I have Kmail configured to an IMAP Server, running Debian and Courier IMAP, running above Exim4.
Kmail has Server side subscriptions is disabled for this IMAP account.
I have set up Filters that Move messages based upon Subject, To, or Reply To. The system is behaving almost as though it is in a loop. I checks the mail, reports a 100+ new message, and states so many In Trash, and some to my mailing lists. 
Kmail seems to loop round on interval check and creates more and more duplicates and then moves them according to the Filter Rules.

Kmail has Rinsed and repeated, over night which generated 27000+ email duplicates.

Please advise, if you want more diagnostic data, as I have full access to the Server and the Client
Comment 90 Rick Timmis 2015-10-31 20:36:17 UTC
Interestingly, I have now configured Thunderbird for my Email. Whilst KMail reports lots of Emails, in reality these duplicates are only notional entries relative to Kmail. The emails are NOT duplicated on the IMAP server.
Comment 91 Wolfgang Mader 2015-12-22 13:00:18 UTC
*** Bug 357031 has been marked as a duplicate of this bug. ***
Comment 92 Wolfgang Mader 2015-12-22 13:03:28 UTC
I filed https://bugs.kde.org/show_bug.cgi?id=357031 and after seeing this one marked is as douplicate.

By the way, what does status "RESOLVED UNMAINTAINED" mean?
Comment 93 Graeme Hewson 2015-12-22 14:10:41 UTC
See comment 87. This bug report was filed against Kmail version 1, which is unmaintained, so it's been closed. "UNRESOLVED UNMAINTAINED" would be more accurate. :-(

I suggest you find a more recent report for the problem (there are several), e.g., bug 283682. Note this was filed against Akonadi, which is the component which does the filtering.