Bug 487459 - Default exe priority set too high for find duplicates process.
Summary: Default exe priority set too high for find duplicates process.
Status: REPORTED
Alias: None
Product: digikam
Classification: Applications
Component: Searches-Similarity (other bugs)
Version First Reported In: 8.4.0
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-05-24 05:19 UTC by Roland
Modified: 2025-04-11 18:13 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed/Implemented In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Roland 2024-05-24 05:19:14 UTC
***
If you're not sure this is actually a bug, instead post about it at https://discuss.kde.org

If you're reporting a crash, attach a backtrace with debug symbols; see https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports
***

SUMMARY
At least in Windows, via code the app priority is set. In this case, the setup effectively locks Windows because even at 'normal' priority, the app's priority does not play well with core system processes. Yes, you can manually set the .exe priority to low, and that allows for more normal operation, but this is not persistent, and it is difficult to do with the OS nearly frozen.

STEPS TO REPRODUCE
1. Generically, run the Find Duplicates function via the GUI on a large dataset- something that may take minutes+

OBSERVED RESULT
Windows10/11 becomes effectively inoperative, and is very slow to respond to requests (i.e. 10 seconds from right click on task bar to a menu presentation, on a 4ghz 8-core 64GB machine).

EXPECTED RESULT
App should not block core OS response to low-level calls (i.e. right click on menu bar to allow access to task manager)

SOFTWARE/OS VERSIONS
Windows: 10
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION
The system works fine if the exe is set to low priority- it will still use all the CPU that is free, but still allow other apps to be opened etc, where things are a bit slower, but still very usable. It is difficult in Windows to script out a change in exe priority, and its possible the process in question is spawned as a new process/thread (i.e. maybe this is actually SQLLite, but the process reporting the utilization is digiKam.exe). So, probably best to set the exe priority in code. Setting default priority to low doesnt impact digiKam except to the extent the user is running another application that is heavy on resources. In that case, a slowdown is expected. In no case should an application effectively stop the OS from responding to user requests and control inputs, where that can be effectively the same as an OS freeze- at least for the duration of the operation.
Comment 1 Maik Qualmann 2024-05-24 09:17:26 UTC
The jobs are processed in a QThreadPool; since Qt-6.2 it has been possible to switch from "Normal" priority to a lower one or not to start threads with all CPU cores available.
However, we then immediately have another bug report that digiKam would not use all CPU cores.

Problem is, I can't reproduce it here in a quick test with my 4-core laptop. Windows can be used normally.

Maik
Comment 2 caulier.gilles 2024-05-24 10:09:30 UTC
Same behavior than Maik here with my i7 8 cores laptop under Windows 10 with all last updates installed...

Gilles
Comment 3 Roland 2024-05-24 10:56:47 UTC
(In reply to Maik Qualmann from comment #1)
> The jobs are processed in a QThreadPool; since Qt-6.2 it has been possible
> to switch from "Normal" priority to a lower one or not to start threads with
> all CPU cores available.
> However, we then immediately have another bug report that digiKam would not
> use all CPU cores.
> 
> Problem is, I can't reproduce it here in a quick test with my 4-core laptop.
> Windows can be used normally.
> 
> Maik

Hi Maik-

I dont think I understand. What Im seeing is starvation avoidance- which is to say, Windows has a built-in fail safe for lower priority threads where they see 10-50ms of CPU time per 5 seconds or so. And depending on how complex the task is (i.e. a right click to bring up a menu), that may require several of those cycles- because it is creating new processes which themselves are running lower priority. So, this suggests that the engine is running in a config where it isnt sharing time with baseline tasks. During Duplicate finding, Im seeing 59 threads associated with the .exe, and all processors running at 100%. So, even at low priority, the app isnt being limited to some fraction of the cores or core cycles- it just defines how the scheduler prioritizes them. 

I wonder if this could be a function of how the CPU itself is setup- if for example, VT-x/IOMMU may play a role in how the OS is scheduling.

But would it be possible to provide as a setting a value for Priority (Low vs Normal), so that people who are seeing this behavior dont have to run batch scripts or manually alter this? That might be an easier solution (to pass that as part of the procedure call) vs determining what is going on with the threading? If this is 100% repeatable here, and on another user's machine, and not on yours, its fair to say this impacts a fair number of users. I suspect that for many, they just assume that the process is slow and they accept that, or they are running smaller datasets (I have something like 300,000 images in play) where a 30 second 'blocking' process they see only infrequently just isnt impactful on their work.

Best
R
Comment 4 caulier.gilles 2025-04-11 18:13:57 UTC
Hi,

The 8.7.0 pre-release Windows installer from today have been rebuilt from
scratch with Qt 6.8.3, KDE 6.12, OpenCV 4.11 + CUDA support, Exiv2 0.28.5, ExifTool 13.27, ffmpeg 7, all image codecs updated to last version (jxl, avif, heif, aom, etc.).

Please try with this version to see if your problem still reproducible...

https://files.kde.org/digikam/

Thanks in advance
Best regards

Gilles Caulier