Bug 315574 - CORE : bad performance when Tag-related Sidebars (Tag management / Filter) are open [patch]
Summary: CORE : bad performance when Tag-related Sidebars (Tag management / Filter) ar...
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Thumbs (show other bugs)
Version: 3.0.0
Platform: Other Linux
: NOR major
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-02-21 12:13 UTC by Lemke
Modified: 2017-07-26 07:02 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In: 4.5.0


Attachments
Difference between db-dumps with bad and good performance (8.67 KB, text/plain)
2013-02-24 13:53 UTC, Lemke
Details
Reformatted Diff (2.19 KB, text/plain)
2013-02-25 16:15 UTC, Lemke
Details
Added an icon cache to AlbumThumbnailLoader to reduce the calls to iconLoader->loadIcon (1.42 KB, patch)
2014-10-12 12:26 UTC, Antonio Larrosa
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Lemke 2013-02-21 12:13:32 UTC
DigiKam becomes very slow and doesn't react fluently on
mouse clicks, if a sidebar related to Tags is opened (filter or management).
When selecting a Tag or opening a Tag-tree, a delay of 3-5 seconds happens.

It also seems to me, that the whole client slows down.

Reproducible: Always

Steps to Reproduce:
1. Open Sidebar related to Tags
2. Try to select Tag (to filter with or to assign)
Actual Results:  
Delay of at least 3-5 seconds until reaction of client (visual feedback, checkbox is selcted).

Expected Results:  
Less delay, even with opened tag related sidebar.

I am running DigiKam 3.0.0 with the MySql-Backend. 
My Tag-Structure is complex, but not huge (about 30 Tags in different Tag-trees).
I had similar problems with DigiKam 2.8 - but less severe.

For me, the Tag feature is not useable at the moment.

Do not hesitate to contact me for further information.
Comment 1 Lemke 2013-02-21 21:36:13 UTC
I was able to improve performance by removing all icons from my Tag tree.

Maybe related to https://bugs.kde.org/show_bug.cgi?id=124688?
Comment 2 Lemke 2013-02-21 21:42:05 UTC
When starting DigiKam, I got the following Messages within the console:

QSqlDatabasePrivate::removeDatabase: connection 'ConnectionTest' is still in use, all queries will cease to work.
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Need Tagging History Graph"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Need Resolving History"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label None"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label Red"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label Orange"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label Yellow"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label Green"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label Blue"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label Magenta"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label Gray"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label Black"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Color Label White"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Pick Label None"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Pick Label Rejected"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Pick Label Pending"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Pick Label Accepted"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Original Version"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Intermediate Version"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Current Version"  with pid  64 
digikam(6817)/digikam (core) Digikam::AlbumManager::scanTAlbums: Failed to find parent tag for tag  "Unbekannt"  with pid  95
Comment 3 Marcel Wiesweg 2013-02-23 11:16:15 UTC
Is the performance merely better, or is the problem gone when removing icons?

The error messages indicate a problem; The parent tag of the listed, internal tags needs to be a tag _Digikam_Internal_Tags_ which itself needs to have no parent.
Do pick and color labels work for you?
Comment 4 Lemke 2013-02-24 13:53:14 UTC
Created attachment 77551 [details]
Difference between db-dumps with bad and good performance

I created two database dumps:
- The first one (mysql_dump_130221.sql) causes the bad performance issue.
- The second one (mysql_dump_130223.sql; based on the first dump) contains a slightly modified tag structure and improves the performance. It is possible to work fluently with this database and the performance is very good. 

The diff contains the tag-related differences of these two dumps described above.
Comment 5 Lemke 2013-02-24 13:54:33 UTC
Yes, color labels and pick work for me with both new and old tag structure.

Some more background information:
I created two database dumps:
- The first one causes the bad performance issue.
- The second one (based on the first dump) contains a slightly modified tag structure and improves the performance. It is possible to work fluently with this database and the performance is very good. 

But:
I get the posted error messages with both database dumps - so it seems to me, that theese messages are not related to the performance issue.

I attached a diff of the two database dumps to this bug report.
Comment 6 Marcel Wiesweg 2013-02-25 15:25:59 UTC
The diff is not really a diff because it's all in a single line, which differs then.
What has actually been changed?
Comment 7 Lemke 2013-02-25 16:15:25 UTC
Created attachment 77570 [details]
Reformatted Diff

The first attachment was a simple diff of two db dumps - therefore the diff seems to be senseless at first glance...  

I reformatted my posted diff result a little bit and applied another diff to make the low level differences more clearly.

The changes I made knowingly:
I removed the icons from some of the tags and renamed a tag - that improved the performance for me.
Comment 8 Marcel Wiesweg 2013-03-01 20:21:13 UTC
Ok, as far as I see the only difference really is in the icons. Mostly "gnome-control-center" and a few other icons have been removed. That directs us back to the icons being at the root of the problem. I suspect some problem with kdelibs taking a long time to load icons, maybe not existing icons or some other problem.
Is CPU consumed by digikam or another process when the delay occurs?
Comment 9 Lemke 2013-03-10 21:10:30 UTC
Yes, there is a significant CPU consumption caused by the digikam process.
It occurs, when the tag filter dialog is opened, and tags are selected / unselected.

In this case, the digikam process occupies 25% on my 4-core CPU (KDE system monitor)- "top" shows up to 100% - so it seems to me that the digikam process is using a core completely.

After the delay, the cpu utilisation falls to aprox. 0 %.
Comment 10 Marcel Wiesweg 2013-03-13 18:07:36 UTC
Just one more step: To prove that the icons are the problem, please take your "normal working" database, assign "gnome-control-center" to three or four tags, and check if the delay reappears then.
Comment 11 Lemke 2013-03-13 20:04:00 UTC
Maybe we found the cause of my problem:
I was not able to assign "gnome-control-center" to some of my tags - I was not able to find that icon within the dialog anymore.... I also tried a filesystem search - no results...

But I cannot remember missing icons within the gui in the past - is there a cache-mechanism implemented which buffers the displayed icon without the physical file?
Maybe the delay was caused by timeouts reading the (missing) icons from filesystem....

Cross Check:
I assigned 4 different (existing) icons to some tags - to impact on performace.
Comment 12 Marcel Wiesweg 2013-03-17 12:34:18 UTC
We use KDE libraries to load these system icons. Either there's a KDE bug causing these repeated delays (not caching the "not found" result) or we use methods synchronously which should be called asynchronously.
Comment 13 caulier.gilles 2014-08-07 06:19:09 UTC
As Marcel said in comment #12, a bug is present in KDELibs.

Which KDELibs do you use ?

Do you try to use digiKam in this context with a more recent KDE version ?

Gilles Caulier
Comment 14 Kristian 2014-08-17 09:41:23 UTC
Could be related or duplicate of #328124.
Comment 15 caulier.gilles 2014-10-07 10:57:12 UTC
*** Bug 339720 has been marked as a duplicate of this bug. ***
Comment 16 Antonio Larrosa 2014-10-12 12:24:15 UTC
I'm using digikam 4.4.0 and noticed this problem too. For me (with over 2300 tags in my database) it took over 40 seconds to regenerate the tag tree view (and my intention was to reorganize the tags, so it was quite unacceptable to wait around 40 seconds each time I renamed or deleted a tag).

I looked into the source code and debugged digikam to find that every time I stopped it, it was inside the icon loader, although most of my tags have the same icon and in total, I think there are 3 or 4 different icons in use. So I implemented an icon cache and now I can't even measure how long it takes since it's instantaneous.

I'm attaching the patch below.
Comment 17 Antonio Larrosa 2014-10-12 12:26:46 UTC
Created attachment 89100 [details]
Added an icon cache to AlbumThumbnailLoader to reduce the calls to iconLoader->loadIcon

QCache uses a default of 100 entries, which I think is enough for most uses
Comment 18 caulier.gilles 2014-10-12 12:51:08 UTC
Git commit 8f7f1f80e596b5205bd06c2900e07e05dd23e46b by Gilles Caulier.
Committed on 12/10/2014 at 12:49.
Pushed by cgilles into branch 'master'.

Apply patch #89100 from Antonio Larrosa to add Tags thumbnails tags cache for album tree-view
FIXED-IN: 4.5.0

M  +30   -17   app/album/albumthumbnailloader.cpp
M  +7    -6    app/album/albumthumbnailloader.h

http://commits.kde.org/digikam/8f7f1f80e596b5205bd06c2900e07e05dd23e46b