SUMMARY While going through the DebugView for another issue, I noticed the following aspects. It may be necessary for the internal workflow, but it seems to me that the face recognition workflow could have a potential for optimization the runtime. 1) It seems that for new images, the image is read in, then the image is read in again for face search/recognition, and then finally read in again to create thumbnails of the recognised image. It seems to me that this leads to unnecessary reads and that the image could be held in memory so that the three options can be run in turn when new images are to be searched for faces. Otherwise, you could at least combine recognition and thumbnail creation. 2) When assigning names to identified faces, the image seems to be reloaded completely, even if nothing is written to the metadata or sidecars are used. This loading process could be omitted if necessary. 3) Thumbnail creation should also be categorised as an active process, so you can see that digiKam is doing something in the background. 4) When assigning keywords, processes of varying complexity seem to interfere with each other. However, I need to delve a little deeper into the matter to be able to make a meaningful suggestion. Commands to the SQLite database with a duration of 10-20 (without unit, I assume it is seconds, as the milliseconds are mentioned as ms) seem to be far too long. ADDITIONAL INFORMATION
(In reply to fuerallesg from comment #0) > SUMMARY > While going through the DebugView for another issue, I noticed the following > aspects. It may be necessary for the internal workflow, but it seems to me > that the face recognition workflow could have a potential for optimization > the runtime. > > 1) It seems that for new images, the image is read in, then the image is > read in again for face search/recognition, and then finally read in again to > create thumbnails of the recognised image. It seems to me that this leads to > unnecessary reads and that the image could be held in memory so that the > three options can be run in turn when new images are to be searched for > faces. Otherwise, you could at least combine recognition and thumbnail > creation. > > 2) When assigning names to identified faces, the image seems to be reloaded > completely, even if nothing is written to the metadata or sidecars are used. > This loading process could be omitted if necessary. > > 3) Thumbnail creation should also be categorised as an active process, so > you can see that digiKam is doing something in the background. > > 4) When assigning keywords, processes of varying complexity seem to > interfere with each other. However, I need to delve a little deeper into the > matter to be able to make a meaningful suggestion. Commands to the SQLite > database with a duration of 10-20 (without unit, I assume it is seconds, as > the milliseconds are mentioned as ms) seem to be far too long. > > ADDITIONAL INFORMATION Hi fuerallesg, The face pipeline reads the image only once and does face detection, recognition, thumbnail creation all with the same image from memory. We have seen there are occasions where the face thumbnail process does recreate the thumbnail, and in doing so, re-reads the image. We're looking into it. When doing a recognition-only scan, the main image isn't read into memory. Only the unknown thumbnails are re-read from the thumbnail DB. Can you share your logs or traces so we can look at this more? Cheers, Mike
Hi Mike, Thank you for checking the process again. I did a small test run and added 20 images to a test folder: https://drive.google.com/file/d/1ddQVKUrV1enS8E5mlUur0_wl7ElwEiFb/view?usp=sharing The import process was started by pressing F5 while clicking on the folder in the album view (line 380). Although the option "Detect faces in newly added images" is enabled, I could not find any tags (maybe I did not wait long enough). So I started "Detect faces" by right-clicking on the folder (line 1258). After the process bar showed 100% and no active process was indicated, I switched to the people view (line 1840). A few thumbnails were already created. The rest started to be created until everything was fine (maybe I did not wait long enough). On previous imports with a larger collection, the result was similar, some thumbnails were created, but not all. There were several hours of waiting in between. Perhaps the problem is that the process works fine in general, but changing the view starts the thumbnailing process all over again. Last time I also could identify the network bandwith as the limit for the process. This time the load was quite low (cpu/gpu as well) although it took much longer for each image (maybe because of heic instead of raw). Feel free to use more resources for a higher speed in case it does not depend on an external library. https://drive.google.com/file/d/1HlXRG37pR-aY3iGpkINeXQOK0cYXQ3mp/view?usp=sharing I will run an additional test with more images and increase the waiting time to several hours in between so that I can check again if the whole process runs as one pipeline. Additionally, I will avoid changing views for each step. BR Mik
(In reply to fuerallesg from comment #2) > Hi Mike, > > Thank you for checking the process again. > > I did a small test run and added 20 images to a test folder: > https://drive.google.com/file/d/1ddQVKUrV1enS8E5mlUur0_wl7ElwEiFb/ > view?usp=sharing > The import process was started by pressing F5 while clicking on the folder > in the album view (line 380). Although the option "Detect faces in newly > added images" is enabled, I could not find any tags (maybe I did not wait > long enough). So I started "Detect faces" by right-clicking on the folder > (line 1258). After the process bar showed 100% and no active process was > indicated, I switched to the people view (line 1840). A few thumbnails were > already created. The rest started to be created until everything was fine > (maybe I did not wait long enough). > On previous imports with a larger collection, the result was similar, some > thumbnails were created, but not all. There were several hours of waiting in > between. > Perhaps the problem is that the process works fine in general, but changing > the view starts the thumbnailing process all over again. > > Last time I also could identify the network bandwith as the limit for the > process. This time the load was quite low (cpu/gpu as well) although it took > much longer for each image (maybe because of heic instead of raw). Feel free > to use more resources for a higher speed in case it does not depend on an > external library. > https://drive.google.com/file/d/1HlXRG37pR-aY3iGpkINeXQOK0cYXQ3mp/ > view?usp=sharing > > I will run an additional test with more images and increase the waiting time > to several hours in between so that I can check again if the whole process > runs as one pipeline. Additionally, I will avoid changing views for each > step. > > BR Mik Hi Mik, You're correct that the initial import does it's own load of the image, and the face pipeline does another load. I rewrote the face pipeline in 8.6.0, but I think you're right we should have a fully integrated single-read pipeline for importing images. It should read the image once, and then do image thumbnail creation, face pipeline, Image Quality Sort, autotagging, and all other functions that require loading the image. I like this. I'll take it to the rest of the team and see what we can come up with for 8.7.0 or 8.8.0. Cheers, Mike
Hi Mike, Happy you understand my idea. Take it as a suggestion and discuss it with the team. Unfortunately, I am quite busy at the moment and my C++ knowledge is quite old so that I cannot make more precise suggestions for the workflow at the moment. Best regards, Mik
Hi, The digiKam 8.7.0 pre-release for Windows have been rebuild from scratch today with many improvements and updates especially with the face management workflow. Please test with this version to see if the problem is reproducible. Link to download: https://files.kde.org/digikam/ Thanks in advance Gilles Caulier
(In reply to caulier.gilles from comment #5) Last month I tried using digiKam for the first time for two weeks and was amazed by the richness of its capabilities and how unoptimized it was. According to the logs, the application processes image metadata at a speed of ~20 pcs/second, which in the case of large collections (more than half a million files) turns into 30-40 hours of waiting for each mass operation. And this is in a situation where the application interface is often completely blocked for many hours due to background processes, the execution of which is not displayed anywhere, as well as their ETA. For this reason, I refused to use this application. But if you say that the new version of the application has significantly improved performance, I will install it again, this time I will choose MySQL instead of SQLite, wait a few days until the end of indexing, check if there are any differences, and write here.
(It's a pity that there is no way to supplement my previous message here instead of adding a new one. Also, please forgive me for describing my experience with tags here, given that the topic primarily concerns file processing after a single read versus multiple reads of the same file for different processing operations. However, I cannot delete my previous message, nor leave it without the promised addition.) After three days of testing version 8.7.0 on Windows 10, I can say that it is noticeably more pleasant when working with tags. According to my observations, if there are no more than ~8-10 thousand files in the queue, the application freezes completely for the duration of the task for only a few hours. If the number of files in the queue is more than ~10 thousand, the application stops responding forever (at least for 15 hours). At least such irreversible freezes occur only 2-3 times a day, which is bearable. After a couple of hours of waiting, I kill the application processes, launch it again and repeat the last tasks in smaller portions. This is much better than it was a month ago! I would have guessed that it was less about application optimization and more about the move from SQLite to (embedded) MySQL, but https://bugs.kde.org/show_bug.cgi?id=431951#c16 claimed that with about half a million files, SQLite actually unexpectedly gives better performance. Sorry, I didn't give any specifics.
Thanks korwin for the feedback. I'm sure that Michael in this room will ask you about details to optimize the current implementation. Best Gilles Caulier