Bug 318516

Created attachment 78989 [details]
Change loading sequence.

This patch change the loading sequence, so digikam scan albums before loading the GUI. This way, GUI Objects don't get all first loading SIGNAL that take a lot of time to handle for nothing...

Created attachment 78990 [details]
Patch that fixes initialisation of ImageAlbumFilterModel

This patch removes two useless loops.
Indeed the loop call a method with an Album in parameter, but that method doesn't really depend on the Album but more on the AlbumManager.
So I splitting the method in two other methods to avoid code redundance, and call them appropriately.

Thanks for your patches...

Gilles Caulier

Unforseen consequence of the patches, 
digikam doesn't load the last album shown when previously quit.
I'll take a look at this.

Created attachment 78993 [details]
Restore the main view when restoreSession in called by main.

Ok, so the view was restored when the AlbumManger emited allAlbumLoaded.
Which is due to my patch never received by the main view.
I joined a patch that put the restore view mechanism when restoreSession is called by main.

This is becoming more intrusive that I would like...

Ok, with 16000 albums I never tested. I concede that some relevant stuff scales linearly.
With you patches, you cut your loading time by...an order of magnitude? I mean, this was the problem?
Then it's not that intrusive but a very relevant fix. We only need to test thoroughly for regressions.

Created attachment 79013 [details]
Cleaner patch  to change the loading sequence and keep the session restore.

I joined a cleaner patch for the loading sequence.

On a clean branch, with digikam compiled in debug mode.
I get an average loading time of 33.36s (with 10 launches)
With the patches I get an average loading time of 2.85s
Almost 12x faster :)
It's near 10x faster in release mode.

I'm pretty sure it was the main issue. The fact that you try to reload 16k times all the filter models is quite a performance killer. Not to mention the AbstractAlbumModel that also took something like 32k events (AboutToBeAdded + Added)...
Without the patches, profiling on the loading of digikam can show up to 103 million calls to Album::next() and a fair amount of Album::title()...

Hi

I tested it on ~10 albums, and obviously I am not getting that big a speedup. @Marcel, @Gilles, do we need more testing, or shall I apply this to git? I have tested this. It doesn't seem to break anything.

Smit

Please do not apply.
This patch is not fixing everything, I did more testing, the scanDalbum with the delay via kioslave etc is still loading after the graphic elements and thus still slowing down everthing.
I have some ideas for a more long term, elegant solution to this issue.

I applied this patch and found that when loading 1000+ pictures, digiKam loads pictures even after the loading window closes, which slows down digiKam considerably. This does not happen in the clean one. Is this the same bug you are facing Yann Le Hir? 

I saw that much time is spent on waiting for SQLite( it's in locked state). I think loading would be faster, if, when loading >1000 files, ~5-10 images are committed to the DB at a time, rather than one at a time.

Yann

Can you give a feedback about comment #10 from Gowtham, please ?

Marcel,

There is any work from you about to review/test this entry recently ?

Best

Gilles Caulier

Marcel,

Whats new about this file ?

Gilles Caulier

To Yann le Hir,

I would to see more investigations about your patches. Can you update these file to be compatible with current implementation from git/master based on Qt5 ?

Thanks in advance

Gilles Caulier

I have ported and tested the patches to git/master. However, we now use a delayed timer that gives even better results. I now close the bug report.

Maik