Bug 362398

Created attachment 98657 [details]
mail as mbox file

Created attachment 98658 [details]
screenshot kmail

Indeed I see same problem.
Could you show source (in txt).
save source from 2 emails
and make a diff ?

So we can see what is the difference.
I want to see if it's the same diff as me

The mail I attached here is now shown with the same size(!). I don't know why, but some magic changed the display (I did set the mail to "important" and I switched folders in the meantime).

However I have other duplicate emails which show the same problem.
Saving one of these as .txt shows also no difference!
So for whatever reason, the mails are exactly the same just the sizes are wrong.

I have now also for the second mail the same sizes...
I just switched to the inbox, back to the opensuse mailing list folder, and now the 2 mails show the same size.

ok, third example. Let's see ...
Just switching folders does not change the size.
Selecting the second one, switching folders, does not change the size.
Selecting the first one, switching folders, does not change the size.
Hmmm ... these two never show the same size, although saves as mbox or as txt
just shows they are both exactly the same.

Comparing the .txt version against the .mbox version just shows the additional
"From" start line, wich the mbox format requires.

BTW: I once tried to find the root cause of the duplicate emails and came up with the following:

I'm hunting the "duplicate mail bug" and I think I've found a bug in the Akonadi Server.
What happens here is the following:
- a resource adds a new mail item (id=3, collection=17)
- Akonadi sends "Item added" to the filter agent
- filter agent sends a "move" command to move item id=3 from collection=17 to collection=19
- Akonadi sends "item added" to the maildir resource (this is the first "added" message from the creation, so the
  item still has id=3 collection=17)
- I'd say in the Akonadi DB the collectionId of item=3 is now 19 (after the move)
- Now the maildir resource gets triggered by its internal file watcher and triggers a sync
- maildir gets called with retrieveItems and lists what it has on disc, which is the item (id=3) in the collection=17
- Akonadi receives this and inside Merge::parseStream() there the bad thing happens:
  It hits this "if" branch (server/src/handler/merge.cpp Line 301)
      // No item with such GID/RID exists, so call AkAppend::insert() and behave
      // like if this was a new item
 and INSERTS the item AGAIN into collection=17 with a new id=4
 Now I have the same mail item twice in the DB
 Since the maildir "itemsRetrieved" answer contains for the first time the RID, the newly inserted item gets this RID,
 and the previously intserted one id=3 does not and is left alone with a NULL RID


Someone more knowledgeable about the Akonadi internals should have a look, please.

Dan can know it better than me.

I sometimes experience this problem too (KMail 5.3.0) -- I can help troubleshoot if necessary.

Didn't notice size changed so checked again and some duplicated mail is different in size but not all. 

Some folders I can use remove duplicate messages ctrl+* and it removes duplicates, other folders I get error message "Unable to fetch item from backend (collection 18) : Unable to retrieve item from resource: Invalid item retrieved" twice and nothing gets removed. 

Some duplicated messages can't be marked as read nor opened. 

I got a push mail from my router one mail per 24 hours, moved to a folder for just that, it contains 24 mails from yesterday and 32 mails from the day before and 81 mails before that. 

Funny I copied the duplicates from one day to trash to count them, trash total messages:1 I counted 81 in trash

*** This bug has been marked as a duplicate of bug 283682 ***