Bug 389608 - akonadictl fsck: offer to fix duplicate items
Summary: akonadictl fsck: offer to fix duplicate items
Status: RESOLVED WORKSFORME
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-01-29 16:44 UTC by Martin Steigerwald
Modified: 2022-11-16 09:15 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Steigerwald 2018-01-29 16:44:41 UTC
Offer to fix up duplicate items.

For example:

Checking qt-kde-ml
Found duplicates 1424447870.R861.merkaba
Found duplicates 1424453278.R177.merkaba
Found duplicates 1424453279.R44.merkaba
Found duplicates 1424455081.R806.merkaba
Found duplicates 1424455081.R860.merkaba
Found duplicates 1435497488.R146.merkaba
Found duplicates 1435497488.R303.merkaba:2,S
Found duplicates 1435499290.R326.merkaba:2,S
Found duplicates 1435499291.R292.merkaba
Found duplicates 1435499291.R683.merkaba
Found duplicates 1435499291.R865.merkaba
Found duplicates 1435499291.R886.merkaba:2,S


Currently user has to figure it out by herself. And from a simple ls -l it does not appear to be obvious what to do:

~/.local/share/local-mail/[…]/.Debian.directory/qt-kde-ml/new> ls -l  1424447870.R861.merkaba 1424453278.R177.merkaba 1424453279.R44.merkaba 1424455081.R806.merkaba 1424455081.R806.merkaba 1424455081.R806.merkaba 1424455081.R860.merkaba 1435497488.R146.merkaba 1435497488.R303.merkaba:2,S 1435499290.R326.merkaba:2,S 1435499291.R292.merkaba 1435499291.R683.merkaba 1435499291.R865.merkaba 1435499291.R886.merkaba:2,S
-rw-r--r-- 1 martin martin 6560 Feb 20  2015 1424447870.R861.merkaba
-rw-r--r-- 1 martin martin 6085 Feb 20  2015 1424453278.R177.merkaba
-rw-r--r-- 1 martin martin 4232 Feb 20  2015 1424453279.R44.merkaba
-rw-r--r-- 1 martin martin 4175 Feb 20  2015 1424455081.R806.merkaba
-rw-r--r-- 1 martin martin 4175 Feb 20  2015 1424455081.R806.merkaba
-rw-r--r-- 1 martin martin 4175 Feb 20  2015 1424455081.R806.merkaba
-rw-r--r-- 1 martin martin 4348 Feb 20  2015 1424455081.R860.merkaba
-rw-r--r-- 1 martin martin 4309 Jun 28  2015 1435497488.R146.merkaba
-rw-r--r-- 1 martin martin 4327 Jun 28  2015 1435497488.R303.merkaba:2,S
-rw-r--r-- 1 martin martin 4351 Jun 28  2015 1435499290.R326.merkaba:2,S
-rw-r--r-- 1 martin martin 4314 Jun 28  2015 1435499291.R292.merkaba
-rw-r--r-- 1 martin martin 8027 Jun 28  2015 1435499291.R683.merkaba
-rw-r--r-- 1 martin martin 4349 Jun 28  2015 1435499291.R865.merkaba
-rw-r--r-- 1 martin martin 7910 Jun 28  2015 1435499291.R886.merkaba:2,S


Currently only three files are obvious:

~/.local/share/local-mail/.Lichtvoll.directory/.Debian.directory/qt-kde-ml/new> sha512sum  1424447870.R861.merkaba 1424453278.R177.merkaba 1424453279.R44.merkaba 1424455081.R806.merkaba 1424455081.R806.merkaba 1424455081.R806.merkaba 1424455081.R860.merkaba 1435497488.R146.merkaba 1435497488.R303.merkaba:2,S 1435499290.R326.merkaba:2,S 1435499291.R292.merkaba 1435499291.R683.merkaba 1435499291.R865.merkaba 1435499291.R886.merkaba:2,S | sort
[…]
bcd41316ed95949d8c211c38df21f818687d568c408db53bcaee9d70798c09c0408bb64bdd0286bad2bf798efe7af15568ada43b3dcf8ab2907a9d19feb7bb3d  1424455081.R806.merkaba
bcd41316ed95949d8c211c38df21f818687d568c408db53bcaee9d70798c09c0408bb64bdd0286bad2bf798efe7af15568ada43b3dcf8ab2907a9d19feb7bb3d  1424455081.R806.merkaba
bcd41316ed95949d8c211c38df21f818687d568c408db53bcaee9d70798c09c0408bb64bdd0286bad2bf798efe7af15568ada43b3dcf8ab2907a9d19feb7bb3d  1424455081.R806.merkaba

and appear to show a consistency issue on my BTRFS filesystem as they have the same name.


Also related:

Bug 389607 - akonadictl fsck: Show more helpful messages
Comment 1 Martin Steigerwald 2018-01-29 16:47:44 UTC
Nope: BTRFS is fine. It just appears that Akonadi sees the same file mutiple times:

~/.local/share/local-mail/.Lichtvoll.directory/.Debian.directory/qt-kde-ml/new> ls -l 1424455081.R806.merkaba*
-rw-r--r-- 1 martin martin 4175 Feb 20  2015 1424455081.R806.merkaba


~/.local/share/local-mail/.Lichtvoll.directory/.Debian.directory/qt-kde-ml/new#130> sha512sum * | grep bcd41316ed95949d8c211c38df21f818687d568c408db53bcaee9d70798c09c0408bb64bdd0286bad2bf798efe7af15568ada43b3dcf8ab2907a9d19feb7bb3d

bcd41316ed95949d8c211c38df21f818687d568c408db53bcaee9d70798c09c0408bb64bdd0286bad2bf798efe7af15568ada43b3dcf8ab2907a9d19feb7bb3d  1424455081.R806.merkaba
Comment 2 Martin Steigerwald 2018-01-29 16:54:29 UTC
Some test with the other files showed that files with the same SHA512 sum appear only once in the folder

~/.local/share/local-mail/.Lichtvoll.directory/.Debian.directory/qt-kde-ml> sha512sum */* | grep 7eb122521a7e154bfc7752f3487c965926471c51f22bb76932ff0177976acf89d78e4e91e06ad30db99f248cf56937c168c266b8b7ec9880c32c7d10572abfa3
7eb122521a7e154bfc7752f3487c965926471c51f22bb76932ff0177976acf89d78e4e91e06ad30db99f248cf56937c168c266b8b7ec9880c32c7d10572abfa3  new/1435499291.R292.merkaba

~/.local/share/local-mail/.Lichtvoll.directory/.Debian.directory/qt-kde-ml> sha512sum */* | grep f48518033c959d7a17eb6ef452b93f226a2f9409ae3300d0826cda6ac33116ad9524c43d8b8a59a16d35f89d7a2981eb440f8ddcb8be9ef50b0ba91d45281bc1
f48518033c959d7a17eb6ef452b93f226a2f9409ae3300d0826cda6ac33116ad9524c43d8b8a59a16d35f89d7a2981eb440f8ddcb8be9ef50b0ba91d45281bc1  new/1435497488.R146.merkaba

But also by Message-Id the appear only once:

~/.local/share/local-mail/.Lichtvoll.directory/.Debian.directory/qt-kde-ml> fgrep "<E1Z9C8z-0005hk-5f@franck.debian.org>" */*
new/1435497488.R146.merkaba:Message-Id: <E1Z9C8z-0005hk-5f@franck.debian.org>

Its for sure that an ordinary user would usually not go as far in order to fix up issues, but now I give up as well. I honestly have no idea what to do about these "duplicates" messages.
Comment 3 Daniel Vrátil 2018-01-29 20:19:53 UTC
See my comment on bug 389607. The "duplicates" refer to Remote IDs in the database (which in case of maildir happen to be filenames), not to actual messages in the backend.

If the situation happens that there are two records in the Akonadi database that have identical RID, the only case when we could automatically delete one or the other would be if we could be sure that both records are absolutely identical (not just in the metadata but also in the payload). In most situations I've seen there actually was a difference between the items with the same RID and in that case, which one to keep?

I'll keep this opened for the "Should fix it" part, but note that most of what you describe below is wrong (based on wrong assumption of what the message implies - which goes back to bug 389607 of course :-)
Comment 4 Martin Steigerwald 2018-01-29 20:38:57 UTC
Ok, so I completely misunderstood your explanation on the mailing list. However, akonadictl fsck reports an issue, it cannot do anything about it, the user can´t either. So what is the point? How how do these duplicates happen? I do think there is something to fix here as the current situation is not suitable to make the ordinary user. In the end for the user it is still just a mail program, a calender application or an address book. And it has an fsck… I am pretty sure I could not really explain this to my father easily.
Comment 5 Justin Zobel 2022-11-15 22:58:30 UTC
Thank you for reporting this issue in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the issue with a recent software version?

If you can reproduce the issue, please change the status to "REPORTED" when replying. Thank you!
Comment 6 Martin Steigerwald 2022-11-16 09:15:38 UTC
"akonadictl fsck" still does not offer to repair duplicate RIDs or help the user to fix it themselves.

However I did not see this issue in the recent years. So works for me.