Bug 436550

Summary: Sporadic error when moving messages to trash.
Product: [Frameworks and Libraries] Akonadi Reporter: David C. Bryant <davidbryant>
Component: Maildir ResourceAssignee: kdepim bugs <kdepim-bugs>
Status: REPORTED ---    
Severity: normal CC: davidbryant
Priority: NOR    
Version: 5.16.3   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: Terminal output when I run "akonadictl fsck"

Description David C. Bryant 2021-05-03 20:23:46 UTC
Created attachment 138122 [details]
Terminal output when I run "akonadictl fsck"

Occasionally when I attempt to move a message to trash, the operation fails with an error message about "resource conflict". The message then becomes inaccessible in KMail, although it's still on the hard disk.

1. Select a message in the message list
2. Click on "Move to trash"

org.kde.pim.akonadiserver: Handler exception when handling command FetchItems on connection MessageViewer-94667889766560 (0x557dcf696430) : Unable to fetch item from backend (collection -1) : Unable to retrieve item from resource: [LRCONFLICT] Resource akonadi_maildir_resource_0 tries to modify item 101543 (1618061094817.R217.localhost:2,S) (in collection 11) with dirty payload, aborting STORE.
org.kde.pim.akonadiserver: Error while handling command ModifyItems on connection akonadi_maildir_resource_0 (0x557dcf6831b0)
org.kde.pim.akonadiserver: ItemRetrievalJob for request 1412 finished with error: "Unable to retrieve item from resource: [LRCONFLICT] Resource akonadi_maildir_resource_0 tries to modify item 101543 (1618061094817.R217.localhost:2,S) (in collection 11) with dirty payload, aborting STORE."

KMail / Akonadi should move the message to Trash

Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 5.20.5
KDE Frameworks Version: 5.80.0
Qt Version: 5.15.2

I have reported tis to the Gentoo developers. But I suspect it's really a KDE buug, because they're not making any changes to Akonadi / KMail, just repackaging it for "Portage".

Also, when I run "akonadictl fsck" I get a long list of "dirty" items. That output is attached. I would clean up the "dirty" items, if only I knew how to do that.
Comment 1 David C. Bryant 2021-05-03 22:07:31 UTC
I've done some more poking around. The messages that I can't access are asociated with "dirty" items reported by akonadictl fsck. Here's some terminal output.

david@localhost ~ $ akonadictl fsck 2>&1|grep ^Found
Found 13 external files.
Found 13 external parts.
Found no unreferenced external files.
Found 0 parts to be moved to external files
Found 0 parts to be moved to database
Found 5 collections without RID.
Found 0 items without RID.
Found 3001 dirty items.

I've posted inquiries in the KDE forum and in some chat rooms. I'll report back here if I get a response. I did do some spot checking, and the "dirty" items are definitely causing the problem. Every message that can't be accessed is on the list of "dirty" items. 

For about a year, akonadictl routinely reported 0 dirty items. Then, a couple of weeks ago, there were 3,003 of them. I have no idea how or why that happened. I think I can probably fix it by backing up all my emails and clearing all the data files out of .local/share/local-mail, then loading all the data back in there. But that would be a lot of work. I'm hopeful there's an easier way to clean the dirt out of Akonadi.
Comment 2 David C. Bryant 2021-05-03 22:44:38 UTC
Sam James (sam@gentoo.org) asked me to post the output from "emerge --info" as part of this bug report. Here it is.

david@localhost ~ $ emerge --info
Portage 3.0.18 (python 3.8.8-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-10.2.0, glibc-2.32-r7, 5.10.27-gentoo-x86_64 x86_64)
System uname: Linux-5.10.27-gentoo-x86_64-x86_64-Intel-R-_Core-TM-_i7-9700_CPU_@_3.00GHz-with-glibc2.2.5
KiB Mem:    16087792 total,  10926892 free
KiB Swap:    7999484 total,   7999484 free
Timestamp of repository gentoo: Mon, 03 May 2021 12:00:01 +0000
Head commit of repository gentoo: ebccfebd1e9a5999357933ffda08b3e11fce375a
sh bash 5.0_p18
ld GNU ld (Gentoo 2.35.2 p1) 2.35.2
app-shells/bash:          5.0_p18::gentoo
dev-lang/perl:            5.30.3::gentoo
dev-lang/python:          2.7.18_p8::gentoo, 3.8.8_p1::gentoo, 3.9.2_p1::gentoo
dev-lang/rust:            1.51.0-r2::gentoo
dev-util/cmake:           3.18.5::gentoo
sys-apps/baselayout:      2.7::gentoo
sys-apps/openrc:          0.42.1-r1::gentoo
sys-apps/sandbox:         2.21::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r5::gentoo
sys-devel/automake:       1.13.4-r2::gentoo, 1.16.2-r1::gentoo
sys-devel/binutils:       2.35.2::gentoo
sys-devel/gcc:            10.2.0-r5::gentoo
sys-devel/gcc-config:     2.4::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.3::gentoo
sys-kernel/linux-headers: 5.10::gentoo (virtual/os-headers)
sys-libs/glibc:           2.32-r7::gentoo

    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-max-age: 24

CFLAGS="-march=native -O2 -pipe"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -O2 -pipe"
FCFLAGS="-march=native -O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-march=native -O2 -pipe"
GENTOO_MIRRORS="https://gentoo.osuosl.org/ ftp://mirrors.rit.edu/gentoo/ https://mirror.sjc02.svwh.net/gentoo/"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
USE="X a52 aac acl acpi activities alsa amd64 berkdb bluetooth branding bzip2 cairo cdda cdr cli crypt cups dbus declarative dri dts dvd dvdr elogind emboss encode exif flac fortran gdbm gif gpm gtk gui iconv icu ipv6 jpeg kde kdesu kipi kwallet lcms libglvnd libnotify libtirpc mad mng mp3 mp4 mpeg multilib ncurses nls nptl ogg opengl openmp pam pango pcre pdf pdfimport phonon plasma png policykit ppds qml qt5 readline scanner sdl seccomp semantic-desktop spell split-usr ssl startup-notification svg tcpd tiff truetype udev udisks uefi unicode upower usb vorbis widgets wxwidgets x264 xattr xcb xml xv xvid zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_8" PYTHON_TARGETS="python3_8" RUBY_TARGETS="ruby26" USERLAND="GNU" VIDEO_CARDS="intel i915 nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Comment 3 David C. Bryant 2021-05-04 00:21:55 UTC
I guess I have a work around. I copied all the contents of ./local/share/local-mail/inbox to a new location. Then I closed KMail, stopped Akonadi, and deleted all the messages in ./local/share/local-mail/inbox. When I restarted KMail and ran "akonadictl fsck" I saw that 236 "dirty" items had been eliminated. Here is some terminal output.

david@localhost ~ $ akonadictl stop
david@localhost ~ $ akonadictl fsck 2>&1|grep ^Found
Found 13 external files.
Found 13 external parts.
Found no unreferenced external files.
Found 0 parts to be moved to external files
Found 0 parts to be moved to database
Found 5 collections without RID.
Found 0 items without RID.
Found 2765 dirty items.

Notice the last line ... that used to say 3,001. 3,001 - 2,765 = 236. So there were 236 "dirty" items in my inbox folder.

I killed KMail again, then stopped akonadi and copied the messages from the backup copy back where they had come from. I then restarted KMail, and all the inbox messages re-appeared. And when I reran akonadictl fsck, things were stil (partially) copacetic:

david@localhost ~ $ akonadictl fsck 2>&1|grep ^Found
Found 398 external files.
Found 398 external parts.
Found no unreferenced external files.
Found 0 parts to be moved to external files
Found 0 parts to be moved to database
Found 5 collections without RID.
Found 0 items without RID.
Found 2765 dirty items.

So I guess I can cure my KMail error messages by repeating this procedure for all the destinations in my folder tree. But I'd like to understand what created the "dirty" items in the first place. And it would also be nice to have a less labor-intensive method to correct this problem if and when it does arise.

(I'm not real hot with "C++". But I used to write database driver software in assembly language on IBM mainframes. It's pretty clear to me that a bunch of the pointers in Akonadi's relational database were corrupted somehow, leading to a lot of "dirty" items. When I removed all the data from one -- well, actually, two -- folders, a whole slew of pointers were cleared from the relational database. Then, when I put the data back and restarted Akonadi, the previously corrupted pointers were rebuilt correctly. So now a bunch of "dirty" items are gone. If some clever C++ programmer could figure out how to make Akonadi reconstruct all its relational database pointers without actually moving a lot of data around, he could at least construct a recovery tool that would probably be useful. Reading a bunch of forum posts, it's obvious a lot of people are unhappy with Akonadi.)