Bug 444056

Summary: Rotating 10 images takes 4 minutes
Product: [Applications] digikam Reporter: Jan van Bekkum <Jan.vanBekkum>
Component: Plugin-Bqm-RotateAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, Jan.vanBekkum, metzpinguin
Priority: NOR    
Version First Reported In: 7.3.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed/Implemented In: 8.1.0
Sentry Crash Report:
Attachments: Rotate settings
DebugView log
Log of an entire run
Debug log with environment variable set
Environment variable
DebugView log with system variable set

Description Jan van Bekkum 2021-10-19 16:43:30 UTC
SUMMARY
Rotating 10 images 180 degrees takes almost 4 minutes

STEPS TO REPRODUCE
1. Select a number of images (>1) to rotate in thumbnail view
2. Select rotate 180 degrees (2x <CTRL><SHIFT><Right arrow>)

OBSERVED RESULT
For rotating of 10 images 3'50" was needed. Subsequently rotating 11 images over 90 degrees took 1'12". Rotating a single image happens instantly.

EXPECTED RESULT
Instant effect

SOFTWARE/OS VERSIONS
Windows: 10
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION
I performed a database cleanup with all options selected and a rebuild of changed and non-cataloged fingerprints before executing the reported rotations. I use a fast computer (AMD Ryzen 9 5900X, GeForce RTX 3070, 32 GB RAM and SSD with I/F PCI Express 4.0). The used database is SQLite.
Comment 1 Maik Qualmann 2021-10-19 17:44:34 UTC
What kind of images are they, JPG, TIFF, PNG, etc.? Which setting for rotation do you use exactly in the digiKam settings?

Maik
Comment 2 Maik Qualmann 2021-10-19 17:48:09 UTC
My computer is far from the performance of yours, I need about 4 seconds for 10 JPG images with active lossless rotation.

Maik
Comment 3 Jan van Bekkum 2021-10-19 17:55:34 UTC
Created attachment 142633 [details]
Rotate settings

The images are 5,782x3,946 sized jpg images. Rotation settings are attached.
Comment 4 Maik Qualmann 2021-10-19 18:13:25 UTC
Well, my images are also 6000x4000 px. We need a DebugView (Microsoft program) log with an active debug variable, if you rotate images, as described here:

https://www.digikam.org/contribute/

Otherwise we won't be able to help, because a short test under Windows 11 in a VM shows no abnormalities either. Candidates are always anti-virus programs that can block digiKam.

Maik
Comment 5 Jan van Bekkum 2021-10-19 19:12:38 UTC
Created attachment 142635 [details]
DebugView log

Hi Maik,

Please find attached the DebugView log.

Regards,

Jan
Comment 6 Maik Qualmann 2021-10-19 19:23:45 UTC
Hmm, I tested it with MySQL, it seems to be a SQLite lock problem as several processes want to write to the DB at the same time. With SQLite, however, only one write process can take place at a time. Please make a new DebugView log with more from the beginning and the end.

Maik
Comment 7 Jan van Bekkum 2021-10-19 20:02:52 UTC
Created attachment 142636 [details]
Log of an entire run

Please find attached a more complete log. DebugView was started prior to Digikam and closed after Digikam was closed.
Comment 8 Maik Qualmann 2021-10-19 20:20:57 UTC
You did not set the debug environment variable in the Windows environment editor. We only see warnings in the log, not debug messages.
A test here with SQLite shows no problems for me either. We may have to reduce the number of threads because your processor has a lot.
Gilles, can you reproduce the problem?

Maik
Comment 9 Jan van Bekkum 2021-10-19 20:37:27 UTC
What debug environment variable should I set?
Comment 10 caulier.gilles 2021-10-19 20:46:16 UTC
Look at this page : https://www.digikam.org/contribute/
Comment 11 Jan van Bekkum 2021-10-19 21:07:53 UTC
That is what I did, but I couldn't find info about the environment variable.
Comment 12 caulier.gilles 2021-10-19 21:10:30 UTC
On Windows, application text output is not sent to the terminal. You need to install DebugView tool to capture text traces generated by digiKam.

Before to start digiKam and DebugView, from your Windows System Info panel, add a new user variable with these criteria:

    name: "QT_LOGGING_RULES"
    value: "digikam*=true"
Comment 13 Jan van Bekkum 2021-10-20 06:02:01 UTC
Created attachment 142652 [details]
Debug log with environment variable set

Now with user environment set as requested. I noticed that the first time I did a rotate it happened in a few seconds, rotating an new group of images after that gave much delay.
Comment 14 Jan van Bekkum 2021-10-20 06:03:10 UTC
Created attachment 142653 [details]
Environment variable

I see no difference with the previous logs. To be sure: environment variable as attached.
Comment 15 Maik Qualmann 2021-10-20 07:02:35 UTC
Your variable value is wrong, the star * is missing.

digikam*=true

Maik
Comment 16 caulier.gilles 2021-10-20 09:10:39 UTC
Hi Maik,

Few days ago, i explored the XNView forums and i discovered an entry explaining a lock conflict between the application and Windows file explorer. Typically, depending of codec installed on the system (to support video thumbnails for ex), a file lock is never free by explorer and client application is blocked a while.

This is typically a low level lock don by the system, which cannot be fixed in client application.

I will try to found this entry...

Gilles
Comment 17 caulier.gilles 2021-10-20 09:12:31 UTC
The Xnview MP is based on Qt internally. Download the binary tarball, and look the contents : qt*, qtav, ffmpeg, etc...
Comment 18 Jan van Bekkum 2021-10-20 10:41:27 UTC
Created attachment 142665 [details]
DebugView log with system variable set

Now a lot bigger log file.
Comment 19 Maik Qualmann 2021-10-20 11:10:29 UTC
First, do you actually need the monitoring of the albums to external change at runtime of digiKam? If no, disable the option in the Setup Collection page in the digiKam Setup.
Then it looks like the first scan of a collection is not yet completed. You should wait, no longer runs a process in the status bar, this can take on big collections hours. During these scans, the SQLite database is very busy and can hardly serve other tasks.

Maik
Comment 20 Maik Qualmann 2021-10-20 11:20:03 UTC
The next problem are 24 CPU cores leading to a traffic jam in the SQLite DB.

Maik
Comment 21 Jan van Bekkum 2021-10-20 11:47:43 UTC
I disabled monitoring of the albums to external change at runtime of digiKam and restarted. I then flipped 3 images in thumbnail view. This took a minute or so.
Comment 22 Jan van Bekkum 2021-11-10 21:23:51 UTC
Would it make sense to change databases to solve this issue?
Comment 23 Jan van Bekkum 2021-11-11 06:06:04 UTC
If I do the same operations on images that are on my NAS, which has a much lower access speed than the computer memory, the operation works as expected.
Comment 24 Jan van Bekkum 2021-11-12 21:27:57 UTC
I have run Digikam with one core active only, but the behaviour stayed the same.
Comment 25 Jan van Bekkum 2021-11-12 22:01:52 UTC
I also tried to change to "Rotate by only setting a flag" instead of changing the content. Although the thumbnails now rotate immediately, Digikam still freezes for quite some time.
Comment 26 Maik Qualmann 2021-11-13 07:22:09 UTC
The problem cannot be reproduced here and has not yet been reported again. At the moment I have no idea why the rotation leads to a database lock problem for you. Again the question, which antivirus software do you use?

Maik
Comment 27 Jan van Bekkum 2021-11-13 14:33:39 UTC
I use Windows Defender only. Switching it off makes no difference.
Comment 28 Jan van Bekkum 2021-11-14 20:19:16 UTC
I have migrated to MySQL and that looks like solving the issue. However, I encountered a different (much smaller) thing: if I rotate 6 images the thumbnails in thumbnail view of 5 or so images will rotate as well correctly, but one will not. After some switching between differnet views and forlders eventually it will rotate as well. The thumbnails above the image view are all correct immediately.
Comment 29 caulier.gilles 2023-05-01 07:32:42 UTC
Hi Jan,

I suspect that refresh thumbnail can be perturbed by the locking file under Windows. This problem have been already seen previously.

Just press F5 to refresh album thumbnail manually.

I close this file now

Gilles Caulier