Bug 446616 - Digikam lags for about 2 - 3 seconds when I assign a Tag
Summary: Digikam lags for about 2 - 3 seconds when I assign a Tag
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Tags-Engine (show other bugs)
Version: 7.3.0
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-12-07 11:32 UTC by Bravo Baavo
Modified: 2022-10-22 13:50 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 8.0.0
Sentry Crash Report:


Attachments
Recording of the issue (352.55 KB, video/mp4)
2021-12-07 13:09 UTC, Bravo Baavo
Details
Log File (68.13 KB, text/plain)
2021-12-07 18:17 UTC, Bravo Baavo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Bravo Baavo 2021-12-07 11:32:17 UTC
SUMMARY
***
Digikam freezes for about 2 - 3 seconds when I assign a Tag. It doesn't do this for assigning ratings.
***

STEPS TO REPRODUCE
1. Select any image in an album.
2. Assign a tag, either throug a hotkey, right-click > Assign Tag, or open the Tag Manager, tick a tag and click apply.

OBSERVED RESULT
After about 2 seconds, Digikam will freeze for a further 2 - 3 seconds.

EXPECTED RESULT
Digikam should seamlessly apply the tag without freezing.

SOFTWARE/OS VERSIONS
Linux: Ubuntu 20.04.3 LTS
KDE Frameworks Version: 5.87.0
Qt Version: 5.15.3

ADDITIONAL INFORMATION
Using Digikam installed through Snappy.
At first I thought it might be writing to the disk and causing a bottleneck, so I moved everything onto an M.2 Pcie4 SSD but the issue remains.
I know this seems miner, but I'm assigning tags via hotkeys to individual images so when it freezes after each one it makes the process much longer, assigning a rating doesn't cause the same issue.
Comment 1 Maik Qualmann 2021-12-07 12:46:15 UTC
Is your digiKam version the Ubuntu Snap package? This uses a sandbox and does not work properly. It may also be related to this bug 444969, which has already been fixed. Please test whether the problem can be reproduced with our pre-release AppImage digiKam-7.4.0.

https://files.kde.org/digikam/

Maik
Comment 2 Maik Qualmann 2021-12-07 12:54:16 UTC
Ok, now read that it's a snap package. As I said, we have had a lot of negative feedback on this snap package. There is a repository for Ubuntu with a native digiKam package, I'm looking for the thread tonight.

Maik
Comment 3 Bravo Baavo 2021-12-07 13:09:51 UTC
Created attachment 144302 [details]
Recording of the issue
Comment 4 Bravo Baavo 2021-12-07 13:10:42 UTC
(In reply to Maik Qualmann from comment #2)
> Ok, now read that it's a snap package. As I said, we have had a lot of
> negative feedback on this snap package. There is a repository for Ubuntu
> with a native digiKam package, I'm looking for the thread tonight.
> 
> Maik

Hi Maik,
Thanks for yours ideas, unfortunately I've just tested the 7.4.0 Appimage and found the same issue, I've attached a recording.
Comment 5 Maik Qualmann 2021-12-07 16:19:52 UTC
Maybe there are hints in the debug output of digiKam in the terminal. Set the Qt Debug environment variable and post the output from the terminal if you assign tags, as described here: 

https://www.digikam.org/contribute/

Maik
Comment 6 Bravo Baavo 2021-12-07 16:23:15 UTC
(In reply to Maik Qualmann from comment #5)
> Maybe there are hints in the debug output of digiKam in the terminal. Set
> the Qt Debug environment variable and post the output from the terminal if
> you assign tags, as described here: 
> 
> https://www.digikam.org/contribute/
> 
> Maik

Hi Maik,
Thanks again, the terminal outputs the following that correlates exactly to when it starts and stops hanging:

```
Digikam::ActionThreadBase::setMaximumNumberOfThreads: Using  12  CPU core to run threads
Digikam::ActionThreadBase::run: Action Thread run  1  new jobs
Digikam::ActionThreadBase::slotJobFinished: One job is done
Digikam::ActionThreadBase::cancel: Cancel Main Thread
```
Comment 7 Maik Qualmann 2021-12-07 17:39:04 UTC
These messages are normal, many more from the before and after output, please. Do you write the tags in the images?

Maik
Comment 8 Bravo Baavo 2021-12-07 18:17:01 UTC
(In reply to Maik Qualmann from comment #7)
> These messages are normal, many more from the before and after output,
> please. Do you write the tags in the images?
> 
> Maik
Hi Maik,
Metadata is saved to the database which is SQLite
I started a new database and added just 5 images and a single tag and the issue did not occur.
I then added all of my previous tags and the issue returned, maybe it only occurs if you have a significant number of tags. I'm not sure how many there are perhaps over 1000, I can't find a counter anywhere.

See attached, entire log file for the following steps:

1 - Open Digikam
2 - Click Image inside the Album "Test"
3 - Assign the tag "Test"
4 - Close Digikam
Comment 9 Bravo Baavo 2021-12-07 18:17:29 UTC
Created attachment 144315 [details]
Log File
Comment 10 Maik Qualmann 2021-12-07 18:49:23 UTC
The log shows no errors, except that you have not yet downloaded the face model data. Is the log from your test database? We definitely need the log if the problem occurs. Because the initial scan of the image collection only takes 1ms.

Even 1000 tags would not be much and would not play a role when assigning tags to images.

Maik
Comment 11 Bravo Baavo 2021-12-07 18:55:16 UTC
(In reply to Maik Qualmann from comment #10)
> The log shows no errors, except that you have not yet downloaded the face
> model data. Is the log from your test database? We definitely need the log
> if the problem occurs. Because the initial scan of the image collection only
> takes 1ms.
> 
> Even 1000 tags would not be much and would not play a role when assigning
> tags to images.
> 
> Maik

Hi Maik,
This is the log where the issue occurs, the process was this:

1 - Launch Digikam with fresh Database
2 - Add 5 images, create one tag
3 - Assign one tag
Issue does not occur
4 - Add collection of 45,000 images (These images have .XMP sidecar files which imports the tags.)
5 - After import, remove the album (This leaves the imported tags but not the images.)
6 - Try to assign a tag, issue occurs and Digikam hangs for a few seconds.
7 - Close Digikam
8 - Re-launch but with logging piped into a text file
9 - Re-add a tag to an image
10 - Close Digikam
11 - Attach log file here.

Just want to clarify that although many tags are inside .XML sidecar files, I am not writing to these files only reading from them.
Comment 12 Maik Qualmann 2021-12-07 19:12:19 UTC
Hmm, 12 CPU cores, is it an AMD Ryzen? Can you possibly also reproduce the bug 444056 with you? 

Maik
Comment 13 Bravo Baavo 2021-12-07 22:41:38 UTC
(In reply to Maik Qualmann from comment #12)
> Hmm, 12 CPU cores, is it an AMD Ryzen? Can you possibly also reproduce the
> bug 444056 with you? 
> 
> Maik

I can't re-create that issue, I selected 10 images and they rotated immediately, I do have a Ryzen though full specs below:

System:    Host: Tiles-PC Kernel: 5.11.0-41-generic x86_64 bits: 64 Desktop: Gnome 3.36.9 
           Distro: Ubuntu 20.04.3 LTS (Focal Fossa) 

Machine:   Type: Desktop Mobo: Micro-Star model: MPG X570 GAMING PLUS (MS-7C37) v: 2.0 serial: <superuser/root required> 
           UEFI: American Megatrends LLC. v: A.D1 date: 03/03/2021 

CPU:       Topology: 6-Core model: AMD Ryzen 5 5600X bits: 64 type: MT MCP L2 cache: 3072 KiB 
           Speed: 2199 MHz min/max: 2200/3700 MHz Core speeds (MHz): 1: 2199 2: 2226 3: 2215 4: 2196 5: 2199 6: 2214 7: 2199 
           8: 2200 9: 2199 10: 2199 11: 2199 12: 2200 

Graphics:  Device-1: NVIDIA GP104 [GeForce GTX 1070] driver: nvidia v: 495.44 
           Display: x11 server: X.Org 1.20.11 driver: nvidia unloaded: fbdev,modesetting,nouveau,vesa 
           resolution: 1920x1080~60Hz, 2560x1440~144Hz, 1920x1080~60Hz 
           OpenGL: renderer: NVIDIA GeForce GTX 1070/PCIe/SSE2 v: 4.6.0 NVIDIA 495.44 

Audio:     Device-1: NVIDIA GP104 High Definition Audio driver: snd_hda_intel 
           Device-2: Texas Instruments PCM2902 Audio Codec type: USB driver: hid-generic,snd-usb-audio,usbhid 
           Sound Server: ALSA v: k5.11.0-41-generic 

Network:   Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8169 
           IF: enp39s0 state: up speed: 100 Mbps duplex: full mac: 2c:f0:5d:a3:90:0d 
           IF-ID-1: tun0 state: unknown speed: 10 Mbps duplex: full mac: N/A 

Drives:    Local Storage: total: 6.60 TiB used: 3.60 TiB (54.6%) 
           ID-1: /dev/nvme0n1 vendor: Crucial model: CT1000P5PSSD8 size: 931.51 GiB 
           ID-2: /dev/sda vendor: Western Digital model: WD20EZRX-00DC0B0 size: 1.82 TiB 
           ID-3: /dev/sdb vendor: Samsung model: SSD 850 EVO 250GB size: 232.89 GiB 
           ID-4: /dev/sdc vendor: Crucial model: CT500MX500SSD1 size: 465.76 GiB 
           ID-5: /dev/sdd vendor: HGST (Hitachi) model: HTS725050A7E630 size: 465.76 GiB 
           ID-6: /dev/sde vendor: Seagate model: ST2000DM008-2FR102 size: 1.82 TiB 
           ID-7: /dev/sdg type: USB vendor: Seagate model: ST1000LM024 HN-M101MBB size: 931.51 GiB
Comment 14 Bravo Baavo 2021-12-07 22:55:45 UTC
(In reply to Maik Qualmann from comment #12)
> Hmm, 12 CPU cores, is it an AMD Ryzen? Can you possibly also reproduce the
> bug 444056 with you? 
> 
> Maik

Just tested on 7.3.0 as well but no delay on rotation for me
Comment 15 Maik Qualmann 2021-12-08 20:02:58 UTC
At the moment I have no idea what the cause of the delay can be for you.

Maik
Comment 16 Maik Qualmann 2022-10-22 06:41:36 UTC
Git commit 0e0a3bc734e6841ee7c0b40084cd45b7e82ad2c7 by Maik Qualmann.
Committed on 22/10/2022 at 06:36.
Pushed by mqualmann into branch 'master'.

default tag models not sort on face tag counter update
Related: bug 460786, bug 438429, bug 422208

M  +9    -6    core/libs/tags/widgets/tagfolderview.cpp

https://invent.kde.org/graphics/digikam/commit/0e0a3bc734e6841ee7c0b40084cd45b7e82ad2c7
Comment 17 Maik Qualmann 2022-10-22 13:14:23 UTC
Git commit f48d78ecc04ba234f2010792cc679c9ce0c69992 by Maik Qualmann.
Committed on 22/10/2022 at 13:13.
Pushed by mqualmann into branch 'master'.

use a QHash for album and tag counters
Related: bug 460786, bug 438429, bug 422208

M  +2    -1    core/libs/album/manager/albummanager.cpp
M  +12   -12   core/libs/album/manager/albummanager.h
M  +6    -6    core/libs/album/manager/albummanager_album.cpp
M  +11   -11   core/libs/album/manager/albummanager_falbum.cpp
M  +4    -4    core/libs/album/manager/albummanager_p.h
M  +1    -1    core/libs/album/manager/albummanager_palbum.cpp
M  +7    -7    core/libs/album/manager/albummanager_talbum.cpp
M  +14   -14   core/libs/database/coredb/coredb.cpp
M  +6    -6    core/libs/database/coredb/coredb.h
M  +7    -7    core/libs/database/dbjobs/dbjob.cpp
M  +3    -3    core/libs/database/dbjobs/dbjob.h
M  +6    -6    core/libs/database/dbjobs/dbjobsthread.cpp
M  +4    -4    core/libs/database/dbjobs/dbjobsthread.h
M  +2    -2    core/libs/models/abstractalbummodel.h
M  +10   -10   core/libs/models/abstractalbummodel_counting.cpp
M  +14   -3    core/libs/models/albumfiltermodel.cpp
M  +3    -3    core/libs/models/albummodel.cpp
M  +1    -1    core/libs/models/albummodel.h
M  +5    -5    core/libs/models/albummodel_date.cpp
M  +9    -9    core/libs/models/albummodel_tag.cpp
M  +11   -11   core/tests/albummodel/albummodel_utest.cpp
M  +3    -3    core/tests/albummodel/albummodel_utest.h
M  +9    -9    core/tests/database/databasetags_utest.cpp
M  +2    -2    core/tests/database/databasetags_utest.h
M  +2    -2    core/utilities/maintenance/facesdetector.cpp

https://invent.kde.org/graphics/digikam/commit/f48d78ecc04ba234f2010792cc679c9ce0c69992
Comment 18 Maik Qualmann 2022-10-22 13:39:54 UTC
Git commit 447c28157c556387dd9044169b3e882fe6b82c4a by Maik Qualmann.
Committed on 22/10/2022 at 13:36.
Pushed by mqualmann into branch 'master'.

optimize tag counter update in the model
Related: bug 460786, bug 438429, bug 422208
FIXED-IN: 8.0.0

M  +2    -2    NEWS
M  +1    -1    core/libs/models/abstractalbummodel.h
M  +9    -11   core/libs/models/abstractalbummodel_counting.cpp

https://invent.kde.org/graphics/digikam/commit/447c28157c556387dd9044169b3e882fe6b82c4a
Comment 19 Maik Qualmann 2022-10-22 13:50:00 UTC
Git commit 94c6c851bb1608ae7fd16ea2a60c72b21fc91faa by Maik Qualmann.
Committed on 22/10/2022 at 13:49.
Pushed by mqualmann into branch 'qt5-maintenance'.

backport from master the main tag counter fix
Related: bug 460786, bug 438429, bug 422208

M  +9    -11   core/libs/models/abstractalbummodel.cpp
M  +1    -1    core/libs/models/abstractalbummodel.h
M  +9    -6    core/libs/tags/widgets/tagfolderview.cpp

https://invent.kde.org/graphics/digikam/commit/94c6c851bb1608ae7fd16ea2a60c72b21fc91faa