I have about 3GB of mails stored in ~/.local/share/local-mail (and .directory). Now akonadi grows and grows and already reached 4.5GB. (3.7GB is spent in akonadi/file_db_data where about 62k files are stored.) Some files are older than 3 years! It should be possible to limit the amount of space used for "cache". (As they seem be just copies of the mails I dont know what the benefit is after all. Both are just simple files in ~/.local/ which normaly resides under the same mount-point.) Reproducible: Always
You can specify cache expiration policy for each folder (or for entire folder tree) by right-clicking a folder, and opening Folder Properties -> Retrieval. Then check "Retrieve message bodies on demand" and set "Keep message bodies locally for" to 1 minute. This will cause that every email in that folder and all it's subfolders will be removed from Akonadi cache after 1 minute and it will be loaded again on-demand from ~/.local/share/local-mail when you try to open it. Note that after you set this option, the initial "cleanup" will take some time (and disk activity). Because the cache expiration policy pretty much covers your original request, I'm closing this as WORKSFORME. However if the approach above does not work (disk space is not reclaimed within ~30 minutes), feel free to reopen (or open a new bug about cache expiration being broken) and provide Akonadi server version (akonadictl --version).
I already have that option set as you described it. So if that should help, it seems to be broken. (Akonadi 1.12.1) (And "keep locally"... they are always locally in ~/.local/share/local-mail!) I already made a complete backup, stopped akonadi and removed akonadis files and dbs... it was rebuilding it completly for every mail! (I did not open any mail in kmail!) The cache was nearly as big as before. And did not shrink after some time (hours without hdd activity).
To clarify: - "keep locally" is relative to "Akonadi server", not the actual storage. - data are always initially imported into Akonadi (necessary for various reasons), but if the cache expiration policy is activated, they should be removed from Akonadi and ~/.local/share/akonadi/file_db_data again after some period of time. Just to make sure: please check that in ~/.config/akonadi/akonadiserverrc, EnableCleaner is set to True (or is not set at all). If it is set to False, stop Akonadi, remove the line and start Akonadi again.
EnableCleaner is not set at all in ~/.config/akonadi/akonadiserverrc . Is there a way to start this cleaner manually?
I confirm that problem. Folder has currently a size of over 1 GB and 5100+ files. Problem started after upgrading to kde-4.13 and the introduction of the baloo indexer. After the initial indexing the folder grew to this size. Sometimes the files vanish after a few days (not 1 minute as stated in the folder properties) but come back after any search. I would like to get rid of it.
Ok, that sounds like a problem caused by our indexer workaround. Simply put: the indexer will force-load the data every time into Akonadi, so that it can index them (it's hard to index something you don't have ;-)). It shouldn't be that aggressive though.
Thank you for your response, Daniel. Is it at least safe to delete the content of the folder after indexing without the risk to lose mails etc?
You can delete content of the folder, but then you must run "akonadictl fsck" so that Akonadi can adjust it's internal database.
Currently I have ~6GiB of e-mail, ~10GiB of Akonadi cache, ~4GiB of Baloo cache. I have *MORE THAN TWICE* as much disk space used for cache as for e-mail itself. This is absurd.
Git commit 3150efbb267c75163b4399ae97504e80d02e7cd7 by Dan Vrátil. Committed on 09/07/2014 at 20:04. Pushed by dvratil into branch '1.13'. Fix CollectionScheduler's timer not resuming when uninhibited Since CacheCleaner is inhibited during various operations after start, the timer was never correctly resumed when the inhibitor was removed, and so payload parts were never expired. M +3 -2 server/src/collectionscheduler.cpp http://commits.kde.org/akonadi/3150efbb267c75163b4399ae97504e80d02e7cd7
Git commit 6cd8ffa5ce7a09c6103d87eff68c791ebee819c3 by Dan Vrátil. Committed on 09/07/2014 at 20:12. Pushed by dvratil into branch '1.13'. Optimize the cacheOnly fetch scope override hack With Baloo we introduced a hack that ignores CACHEONLY parameter to FETCH command when all requested items are local (provided by maildir/mbox resources) so that Baloo was able to index local mails despite cache expiration policy. This (together with the broken CollectionScheduler) lead to local mails never expiring from Akonadi Cache. This change permits the CACHEONLY override only to the Baloo Indexer agent. Unlike some other agents/clients, the Baloo agent is smart and does only incremental checks, so it will not cause the entire maildirs to be loaded into Akonadi cache. FIXED-IN: 1.13.0 M +5 -0 server/src/handler/fetchhelper.cpp http://commits.kde.org/akonadi/6cd8ffa5ce7a09c6103d87eff68c791ebee819c3
Even though this bug is marked fixed I want to come back to it, since my file_db_data still keeps growing fast. And it is not only mail that is stored there and never gets erased, it's a whole bunch of different files. There are two VCARD's which are held in three different versions for weeks (in total six files). If I delete them manually the respective contacts disappear from KAddressbook and are not fetched again from my ownCloud account. BTW, both contacts hold a profile picture and are the only ones I added a picture to. There is one note from KNotes (a HTML document) already sitting in file_db_data for several days. Of this file also exist three different versions. There are six contactGroup files (XML files) sitting in the directory for about two weeks. When I delete them the respective groups disappear from KAddressbook as the afore mentioned contacts. All of the above mentioned items were not changes by me for a long time. There are dozens of strange looking plain text documents referring to old mail stuff in my mail archive hold in "Local Folders". These date back to the day when I re-imported the whole archive after finding a workaround for this bug: https://bugs.kde.org/show_bug.cgi?id=344242 And there are still hundreds of mail files in the directory, some dating back to the day I had completely emptied it - see above mentioned bug report for a detailed account of the whole procedure. Many, if not all, of the mails still residing in the directory have been deleted from the IMAP server yet or were moved to the local archive. I got the impression that Akonadi has a problem with were to store data and how to cache it. And this does not only hold true for mails but all data managed by Akonadi. And as one can see from the above mentioned contact files and contactGroup files and Bug 344242 this can cause the loss of data. Thus it would be great if the root of these problems could be found and eliminated. Right now I'm running KDE and Kontact 4.14.5-1 as well as Akonadi 1.13.0-3 under Chakra Linux. My system is always kept up-to-date. The last update of KDE-Pim and Akonadii was on March 4th 2015, a day before I deleted ~/.local/share/akonadi/file_db_data.
I can confirm this bug and it not seems to be fixed at all. I have 3-4GB of mail (IMAP) in 2 different accounts. Akonadi was storing over 100GB under file_db_data when I cleared it the first time (was filling up my disk). Yesterday (around one week later) I had again almost 40GB of data there. Until now (akonadi_imap_resource is still running with relative high load) it is again over 4GB in size.
Current status: folder is still growing and now 2 days later at 63GB! I think I need a cronjob that deletes the folder every 1-2 days.
(In reply to Tim Eberhardt from comment #14) > Current status: folder is still growing and now 2 days later at 63GB! I > think I need a cronjob that deletes the folder every 1-2 days. be careful when deleting. May can disappear. See my above post.
Just checked my ~/.local/share/akonadi/file_db_data/. It holds now more than 3.000 files and about 350 MB. I've got three IMAP accounts which together hold less than 1.500 mails that sum up to about 75 MB. Yesterday Akonadi had stored 111 new files in the directory, all of them mails. Some of these mails were definitely deleted from the accounts, like all spam. And quite a few of them - I'm not sure if all - exist in various versions, like one with the whole body of the message and one without. Moreover, of these messages exists something that looks like revisions with almost the same file name but marked _r0 _r1 _r2. Event though these files differ slightly in size in KMail-View they look the same. A closer look in Kate revealed that they hold different contents. Hope these comments help track down the bug(s) that make ~/.local/share/akonadi/file_db_data/ grow so fast and limitless.
Back again. Just deleted more than 6.000 files - most of them emails - from ~/.local/share/akonadi/file_db_data/ which had let it grow to almost 1 GB within less than 3 months. Thus I assume the bug still prevails. Is Akonadi still actively maintained? Nothing seems to have happened about this and related bugs for almost a year.
note: mysql database size generally does not shrink when removing data.
(In reply to Rex Dieter from comment #18) > note: mysql database size generally does not shrink when removing data. I was not talking about the size of the mysql data base. It's ~/.local/share/akonadi/file_db_data that is growing rapidly. Since my last post yesterday more thas 100 files were stored there. If it keeps growing at that rate in three months it will be about 10.000.
I am also seeing this issue KDE 4.14.2. It is making files_db_data is 4.8G and over 243,000 files. Should we open a new bug?
I do think this bug has been fixed. Please use at least kdepim 4.14.10 and Akonadi 1.13 + the necessary patches or Akonadi 15.08+. At least for Maildir items. Please file a different bug if you see it for other items than Maildir items. I have only 357 items with a total of 14 MiB in there after a long time, with SizeThreshold=16384 in [%General] section of ~/.config/akonadi/akonadiserverrc, this reduces the amount of files stored externally. The kdepim packages from Debian Unstable/Testing I think have this fix. It may has been backported for kdepim for Jessie, check Debian changelog.
Thanks for the command. After further resarch (and talking to dvratil on #akonadi) it seems I may be hitting this bug https://bugs.kde.org/show_bug.cgi?id=341884 I'm on Kubuntu, so I'll see what I can do to get the latest packages.