Bug 494807 - some local folders hidden
Summary: some local folders hidden
Status: RESOLVED FIXED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: Maildir Resource (other bugs)
Version First Reported In: 6.2.1
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 492233 (view as bug list)
Depends on:
Blocks:
 
Reported: 2024-10-15 11:53 UTC by kavol
Modified: 2025-05-19 10:31 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In: 25.04.2
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description kavol 2024-10-15 11:53:21 UTC
***
If you're not sure this is actually a bug, instead post about it at https://discuss.kde.org

If you're reporting a crash, attach a backtrace with debug symbols; see https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports

Please remove this comment after reading and before submitting - thanks!
***

After upgrade to 24.08.1, I cannot access some local folders in kmail, it pretends they don't exist.
These are visible in akonadiconsole, but grayed.
Trying to synchronise, clear caches, akonadictl fsck ... nothing helps.

STEPS TO REPRODUCE
1. have a big collection of mail over the years
2. use kmail regularly
3. have bad luck with akonadi ...

OBSERVED RESULT
some local folders are not visible in kmail

EXPECTED RESULT
all the contents of the configured maildir is visible and accessible in kmail

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
(available in the Info Center app, or by running `kinfo` in a terminal window)
$ kinfo
Operating System: Gentoo Linux 2.15
KDE Plasma Version: 6.1.5
KDE Frameworks Version: 6.6.0
Qt Version: 6.7.2
Kernel Version: 6.6.52-gentoo (64-bit)
Graphics Platform: X11
Processors: 8 × Intel® Core™ i7-4810MQ CPU @ 2.80GHz
Memory: 15.3 GiB of RAM
Graphics Processor: llvmpipe

ADDITIONAL INFORMATION
Comment 1 kavol 2024-10-15 11:55:54 UTC
(In reply to kavol from comment #0)
> Please remove this comment after reading and before submitting - thanks!

ups, sorry, I've scrolled down, got interrupted, then forgot :-)
Comment 2 kavol 2024-10-15 12:37:10 UTC
Looking closer, it seems that all of the affected folders were created within last two years, the older ones are visible all (but it doesn't apply vice versa, it is not that all the new ones would be invisible, just some of them).
Comment 3 kavol 2024-10-15 13:21:19 UTC
(In reply to kavol from comment #2)
> Looking closer, it seems that all of the affected folders were created
> within last two years, the older ones are visible all (but it doesn't apply
> vice versa, it is not that all the new ones would be invisible, just some of
> them).

seems that I was mistaken, scratch the note in parentheses, I haven't realised the real age of something

to be precise - inspecting collectiontable, I have found that the problem starts with id 1006, everything from this number up is hidden (or grayed out in akonadiconsole)

(now that's interesting - I'd expect 1000 or 1024 ... 1006 seems kinda random ...)

also note that when looking at these folders in akonadiconsole, I can browse only some of them, the others' contents are not shown despite it shows nonzero numbers for the count and size of the contents
Comment 4 kavol 2024-10-31 14:31:40 UTC
now this is getting interesting ... I've created a subfolder in one of the hidden folders manually, and it appeared in kmail immediately - only the one subfolder, not the others within the same parent folder, and the parent folder was grayed in kmail

after restart, both the parent and the new subfolder were hidden again, but after moving an email manually to the new subfolder, it became visible again in kmail (with the parent grayed)
Comment 5 whatifgodwasoneofus 2024-11-27 22:24:14 UTC
Check here: https://bugs.kde.org/show_bug.cgi?id=492233
It's the same bug!
Comment 6 kavol 2024-11-29 10:31:32 UTC
(In reply to whatifgodwasoneofus from comment #5)
> Check here: https://bugs.kde.org/show_bug.cgi?id=492233
> It's the same bug!

I don't think this is the same -

1) this is about local folders, bug 492233 is about IMAP

2) the folders are hidden no matter whether they contain new messages or not

well, I cannot rule out that there may be some common cause but at the first glance, it doesn't look the same (and I have no problem with IMAP, all folders are displayed)
Comment 7 whatifgodwasoneofus 2025-01-26 21:11:22 UTC
@kavol could you please check this:

1. go in the akonadi DB (it does not matter if it is postgres or mariadb)
2. SELECT * FROM public.collectiontable ORDER BY id ASC; and find the id of the LAST GOOD collection - that is: the folder that is "visible" in kmail (and not grey in akonadiconsole)
3. update collectiontable set remoterevision=null  where id > last_good_collection_id;
4. akonadi should immediately start a reindex on the maildir agent. if it is not the case, use the akonadiconsole or kmail (CTRL+F5) to trigger it.

You should now see all folders again, may they be maildir ones on IMAP ones. BUT: if you close kmail, and then reopen it, those folders are gone again.
If you close akonadiconsole, and then reopen it, those folders are grey again.

Can you confirm this? If so, then your bug is exactly the same as mine ([#492233](https://bugs.kde.org/show_bug.cgi?id=492233)).
Comment 8 whatifgodwasoneofus 2025-01-26 21:13:23 UTC
Also, another thing:

Once you identify the "last good collection id", could you please run this:

select count(*)
  from collectiontable
 where id <= last_good_collection_id
;

I get 1002. I am starting to think that somewhere in akonadi (server) there could be a forgotten hard-coded limit of 1000 collections that can be handled at the same time... 

If you also get 1000, then maybe it could be true...
Comment 9 Martin Koller 2025-05-18 16:30:15 UTC
I see the same issue (using SQLITE) and it really seems to be an issue with the number of "good" collections being 1000
Comment 10 Martin Koller 2025-05-18 21:18:30 UTC
I can easily reproduce this problem using kmail 6.4.1 (25.0.4.1) on openSuse Tumbleweed.
Simply in a shell create a new start folder, e.g. MailTest
Then inside it create more than 1000 maildir folders:
for i in `seq 1 1010`; do mkdir -p $i/new $i/cur $i/tmp; done

Then create a new receiving account of type maildir, selecting the start-dir "MailTest".
You will see that you will not find e.g. the folders 982 .. 999 in kmails folder tree.
With akonadiconsole however you find these folders but they are shown greyed out.
Comment 11 Martin Koller 2025-05-19 10:27:05 UTC
Git commit 3416562fbac8184ee881c81cc6710c1effcaa982 by Martin Koller.
Committed on 19/05/2025 at 10:14.
Pushed by mkoller into branch 'release/25.04'.

fix batch query operation really completing full set

Fixes a porting error which led to only doing a query for the first
batch of ids instead of continuing the whole id set
FIXED-IN: 25.04.2

M  +2    -2    src/server/handler/collectionfetchhandler.cpp

https://invent.kde.org/pim/akonadi/-/commit/3416562fbac8184ee881c81cc6710c1effcaa982
Comment 12 Martin Koller 2025-05-19 10:28:43 UTC
Git commit 8455ec1165e2bad4019cf344b8cc1831161597da by Martin Koller.
Committed on 19/05/2025 at 10:28.
Pushed by mkoller into branch 'master'.

fix batch query operation really completing full set

Fixes a porting error which led to only doing a query for the first
batch of ids instead of continuing the whole id set
FIXED-IN: 25.04.2
(cherry picked from commit 3416562fbac8184ee881c81cc6710c1effcaa982)

M  +2    -2    src/server/handler/collectionfetchhandler.cpp

https://invent.kde.org/pim/akonadi/-/commit/8455ec1165e2bad4019cf344b8cc1831161597da
Comment 13 Martin Koller 2025-05-19 10:31:36 UTC
*** Bug 492233 has been marked as a duplicate of this bug. ***