Bug 362398 - duplicate mails but different size shown
Summary: duplicate mails but different size shown
Status: RESOLVED DUPLICATE of bug 283682
Alias: None
Product: kmail2
Classification: Applications
Component: message list (show other bugs)
Version: unspecified
Platform: openSUSE Linux
: NOR normal (vote)
Target Milestone: ---
Assignee: kdepim bugs
Depends on:
Reported: 2016-04-28 08:34 UTC by Martin Koller
Modified: 2017-03-22 00:06 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:

mail as mbox file (5.44 KB, application/mbox)
2016-04-28 08:34 UTC, Martin Koller
screenshot kmail (15.87 KB, image/png)
2016-04-28 08:35 UTC, Martin Koller

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Koller 2016-04-28 08:34:20 UTC
Version 5.2.0

I often receive mails from mailing lists (e.g. in this example opensuse KDE) which get duplicated.
What I found today is yet another duplicate email (see attached screenshot) but the interesting thing is that kmail shows the same mails with DIFFERENT sizes.
I did save both of them via RMB -> save as (attached) and compared both with "cmp".
They are exactly the same

Reproducible: Sometimes
Comment 1 Martin Koller 2016-04-28 08:34:53 UTC
Created attachment 98657 [details]
mail as mbox file
Comment 2 Martin Koller 2016-04-28 08:35:20 UTC
Created attachment 98658 [details]
screenshot kmail
Comment 3 Laurent Montel 2016-04-28 09:26:27 UTC
Indeed I see same problem.
Could you show source (in txt).
save source from 2 emails
and make a diff ?

So we can see what is the difference.
I want to see if it's the same diff as me
Comment 4 Martin Koller 2016-04-28 09:59:29 UTC
The mail I attached here is now shown with the same size(!). I don't know why, but some magic changed the display (I did set the mail to "important" and I switched folders in the meantime).

However I have other duplicate emails which show the same problem.
Saving one of these as .txt shows also no difference!
So for whatever reason, the mails are exactly the same just the sizes are wrong.

I have now also for the second mail the same sizes...
I just switched to the inbox, back to the opensuse mailing list folder, and now the 2 mails show the same size.

ok, third example. Let's see ...
Just switching folders does not change the size.
Selecting the second one, switching folders, does not change the size.
Selecting the first one, switching folders, does not change the size.
Hmmm ... these two never show the same size, although saves as mbox or as txt
just shows they are both exactly the same.

Comparing the .txt version against the .mbox version just shows the additional
"From" start line, wich the mbox format requires.
Comment 5 Martin Koller 2016-04-28 10:03:50 UTC
BTW: I once tried to find the root cause of the duplicate emails and came up with the following:

I'm hunting the "duplicate mail bug" and I think I've found a bug in the Akonadi Server.
What happens here is the following:
- a resource adds a new mail item (id=3, collection=17)
- Akonadi sends "Item added" to the filter agent
- filter agent sends a "move" command to move item id=3 from collection=17 to collection=19
- Akonadi sends "item added" to the maildir resource (this is the first "added" message from the creation, so the
  item still has id=3 collection=17)
- I'd say in the Akonadi DB the collectionId of item=3 is now 19 (after the move)
- Now the maildir resource gets triggered by its internal file watcher and triggers a sync
- maildir gets called with retrieveItems and lists what it has on disc, which is the item (id=3) in the collection=17
- Akonadi receives this and inside Merge::parseStream() there the bad thing happens:
  It hits this "if" branch (server/src/handler/merge.cpp Line 301)
      // No item with such GID/RID exists, so call AkAppend::insert() and behave
      // like if this was a new item
 and INSERTS the item AGAIN into collection=17 with a new id=4
 Now I have the same mail item twice in the DB
 Since the maildir "itemsRetrieved" answer contains for the first time the RID, the newly inserted item gets this RID,
 and the previously intserted one id=3 does not and is left alone with a NULL RID

Someone more knowledgeable about the Akonadi internals should have a look, please.
Comment 6 Laurent Montel 2016-04-28 14:10:28 UTC
Dan can know it better than me.
Comment 7 Tristan Miller 2016-10-03 14:57:09 UTC
I sometimes experience this problem too (KMail 5.3.0) -- I can help troubleshoot if necessary.
Comment 8 Thomas 2016-10-07 16:21:42 UTC
Didn't notice size changed so checked again and some duplicated mail is different in size but not all. 

Some folders I can use remove duplicate messages ctrl+* and it removes duplicates, other folders I get error message "Unable to fetch item from backend (collection 18) : Unable to retrieve item from resource: Invalid item retrieved" twice and nothing gets removed. 

Some duplicated messages can't be marked as read nor opened. 

I got a push mail from my router one mail per 24 hours, moved to a folder for just that, it contains 24 mails from yesterday and 32 mails from the day before and 81 mails before that. 

Funny I copied the duplicates from one day to trash to count them, trash total messages:1 I counted 81 in trash
Comment 9 Daniel Vrátil 2017-03-22 00:06:38 UTC

*** This bug has been marked as a duplicate of bug 283682 ***