Right now the loading time for digikam is very high. I use digikam with something like 16k Albums and 150k pictures. Loading digikam can take up to 3-4 minutes with a high end CPU. Passing the fact that digikam only uses 1 core to load, it should still be much faster. I have two patches that improves loading time tremendously. Reproducible: Always Steps to Reproduce: 1. Start digikam 2. 3.
Created attachment 78989 [details] Change loading sequence. This patch change the loading sequence, so digikam scan albums before loading the GUI. This way, GUI Objects don't get all first loading SIGNAL that take a lot of time to handle for nothing...
Created attachment 78990 [details] Patch that fixes initialisation of ImageAlbumFilterModel This patch removes two useless loops. Indeed the loop call a method with an Album in parameter, but that method doesn't really depend on the Album but more on the AlbumManager. So I splitting the method in two other methods to avoid code redundance, and call them appropriately.
Thanks for your patches... Gilles Caulier
Unforseen consequence of the patches, digikam doesn't load the last album shown when previously quit. I'll take a look at this.
Created attachment 78993 [details] Restore the main view when restoreSession in called by main. Ok, so the view was restored when the AlbumManger emited allAlbumLoaded. Which is due to my patch never received by the main view. I joined a patch that put the restore view mechanism when restoreSession is called by main. This is becoming more intrusive that I would like...
Ok, with 16000 albums I never tested. I concede that some relevant stuff scales linearly. With you patches, you cut your loading time by...an order of magnitude? I mean, this was the problem? Then it's not that intrusive but a very relevant fix. We only need to test thoroughly for regressions.
Created attachment 79013 [details] Cleaner patch to change the loading sequence and keep the session restore. I joined a cleaner patch for the loading sequence. On a clean branch, with digikam compiled in debug mode. I get an average loading time of 33.36s (with 10 launches) With the patches I get an average loading time of 2.85s Almost 12x faster :) It's near 10x faster in release mode. I'm pretty sure it was the main issue. The fact that you try to reload 16k times all the filter models is quite a performance killer. Not to mention the AbstractAlbumModel that also took something like 32k events (AboutToBeAdded + Added)... Without the patches, profiling on the loading of digikam can show up to 103 million calls to Album::next() and a fair amount of Album::title()...
Hi I tested it on ~10 albums, and obviously I am not getting that big a speedup. @Marcel, @Gilles, do we need more testing, or shall I apply this to git? I have tested this. It doesn't seem to break anything. Smit
Please do not apply. This patch is not fixing everything, I did more testing, the scanDalbum with the delay via kioslave etc is still loading after the graphic elements and thus still slowing down everthing. I have some ideas for a more long term, elegant solution to this issue.
I applied this patch and found that when loading 1000+ pictures, digiKam loads pictures even after the loading window closes, which slows down digiKam considerably. This does not happen in the clean one. Is this the same bug you are facing Yann Le Hir? I saw that much time is spent on waiting for SQLite( it's in locked state). I think loading would be faster, if, when loading >1000 files, ~5-10 images are committed to the DB at a time, rather than one at a time.
Yann Can you give a feedback about comment #10 from Gowtham, please ? Marcel, There is any work from you about to review/test this entry recently ? Best Gilles Caulier
Marcel, Whats new about this file ? Gilles Caulier
To Yann le Hir, I would to see more investigations about your patches. Can you update these file to be compatible with current implementation from git/master based on Qt5 ? Thanks in advance Gilles Caulier
I have ported and tested the patches to git/master. However, we now use a delayed timer that gives even better results. I now close the bug report. Maik