When selecting a person on the "People" panel on the right, the faces are loaded on demand. Unlike normal thumbnails, faces are only loaded when they have the windows focus (that is, when you are scrolling through them). A face won't load until you scroll to its position. In addition to that, faces load quite slowly, around 3 faces per second (pictures stored in a NAS). I have some persons with more than 3000 faces, and it's quite inconvenient to browse through their faces, since it means scrolling to a position and wait a few minutes until faces appear. 3000 faces at 3faces/second means more than 15 minutes of actively and manual scrolling for the faces to load. The workaround of leaving that person selected and waiting a few minutes does not work, since only the faces which are currently "seen" will be loaded. That works for normal thumbnails, but not for faces. It would be convenient if faces would load in the background when there's no activity. Or even generate the faces the first time they are detected (maybe that already exists, my face regions were saved from another software). That way, once you select a person, the faces would be already pre-generated. I hope you understand what I mean. I am using digiKam-6.0.0-git-20180603T215351-Win64 on Windows 10, by the way.
Where is the database, also on the NAS? If it is local, MySQL or SQLite? If the face thumbnails were created all for one person, will it take just as long next time? Maik
Yes, I've temporarily moved the database (SQLite) to the NAS as well, for testing purposes, and that probably exacerbates the issue. Once the face thumbnails have been generated, loading times are generally much better. But for that, you must have spent a substantial amount of time scrolling through the person's faces in order to pre-load them.
7.0.0-beta1 is out with new Face Recognition algorithm based on Deep Learning/Neural Network API from OpenCV https://download.kde.org/unstable/digikam/ Please test and give us a feedback Thanks in advance Gilles Caulier
This problem is fully reproducible here, and not only under Windows. Amount of people faces detected and not assigned is around 8000 and loading virtual album contents take age... Gilles Caulier
digiKam 7.0.0 stable release is now published: https://www.digikam.org/news/2020-07-19-7.0.0_release_announcement/ We need a fresh feedback on this file using this version. Best Regards Gilles Caulier
This behavior is still present.
This problem only arises for new faces when face detection is running. The second time they are loaded from the thumbnail database. The problem is that with face detection, CPU, file system, database or network drives are all under full load. It doesn't get any faster if we preloaded the thumbnails in this situation. Maik
After running "Detect Faces", "Recognize faces", and "Rebuild Thumbnails" processes the face thumbnails are slow to load. They seem to be generated "on the fly" as I scroll down the Face tag view. When revisiting the Face tag again, it loads much faster. OS: Windows 10 DB: MySQL (data is on an SSD) Version: 7.2.0-rc (6f650d59) Processor: Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz
I don't know why you keep doing "Rebuild Thumbnails", but it doesn't make any sense. Only the normal thumbnails are currently being recreated. Face thumbnails are not recreated. And that makes sense in order to immediately get a correct thumbnail again when changing the size of the face in manual drawing mode. Maik
Ok thanks. I ran "Rebuild Thumbnails" in the off chance it also rebuilds the Face Thumbnails.
Hi all, What' the status of this file now ? Best Gilles Caulier
Well, my initial report is still valid. Faces in the people panel load as you scroll by them, not in the background. This is inconvenient in slow networks because they can take a while to generate. The rest of the thumbnails in digikam, in contrast, seem to be generated in the background when an album is accessed, so it feels quicker. So, for instance, if a person has 200 face regions, when you click on that person, only the "focused" thumbnails will load (around 20, depending on your screen size and zoom), and once it finishes loading, scroll down and wait for the next batch to be generated. So you have to scroll 10 times until the whole face set thumbnail is generated.
I created a new sqlite thubmnail database and ran the "Rebuild Thumbnails" option from Tools -> Maintenance. Full image thumbnails were generated, but once completed I went to the "Faces" view and those had not been generated. The face thumbnails were generated "on the fly" as I scrolled through the faces - I could see the sqlite file increase in size as I did this. Could "Rebuild Thumbnails" also include the face thumbnails as well?
digikam 7.5 windows. I can confirm that face thumbnails are built and added to the thumbnail database only when first required for display. Mousewheel scrolling through thousands of blank face thumbnails will queue them all for generation in the background. This used to be much more annoying because any database maintenance used to trash all the face thumbnails and that seems to be fixed. It doesn't bother me much now, but, when you detect a face you already have the file open to do it. Why wouldn't you generate a face thumbnail at the same time?
Hi all, digiKam 8.0.0 is out. This report still valid ? Best regards
Yes, this is still valid for v8.0.
Here the wish is expressed that the face thumbnail is already created during face detection. In fact, that is already the case. The image in the face pipeline that is also used for detection is used for the thumbnail - it is not reloaded. I debugged this and it works perfectly. If I detected a large set of faces but don't have the people view open and after the face detection process switch to Unknown people, the face thumbnails are there immediately and no more are generated. Maik
Hi, This is not my experience. After new pictures are found, they are scanned for faces and recognized (I have checked the option to scan them automatically on startup on the settings). Then, I usually go to Unconfirmed o Unrecognized sections in the People panel to tag them. At that point I think the face thumbnails for the new faces area already generated. But then, when you confirm any of these new faces, all the face thumbnails in that picture will need to be recreated, so you have to go to each one of the persons in that picture and scroll through that specific picture to have it generated. My network is rather slow, so that process is very noticeable because it takes several seconds per face. Ideally, the face thumbnails created when the faces are first detected shouldn't need to be recreated unless the image is edited (just modifying the metadata should not trigger the regeneration of the thumbnail).
Git commit a15e63f394fb8e2df283e960f8881456e446fa7c by Maik Qualmann. Committed on 05/09/2023 at 22:15. Pushed by mqualmann into branch 'master'. for a test add a thumbnail thread that only loads from storage Related: bug 474105 M +51 -2 core/libs/database/models/itemthumbnailmodel.cpp M +2 -0 core/libs/database/models/itemthumbnailmodel.h M +5 -0 core/libs/threadimageio/fileio/loadingdescription.cpp M +3 -1 core/libs/threadimageio/fileio/loadingdescription.h M +8 -6 core/libs/threadimageio/thumb/thumbnailcreator.cpp M +5 -3 core/libs/threadimageio/thumb/thumbnailcreator.h M +13 -5 core/libs/threadimageio/thumb/thumbnailloadthread.cpp M +6 -3 core/libs/threadimageio/thumb/thumbnailloadthread.h M +4 -2 core/libs/threadimageio/thumb/thumbnailtask.cpp https://invent.kde.org/graphics/digikam/-/commit/a15e63f394fb8e2df283e960f8881456e446fa7c
@MarcP, This problem still reproducible with the new digiKam 8.2.0 pre-release Windows installer available at usual place: https://files.kde.org/digikam/ This new bundle is based on last Qt framework 5.15.11 and KDE framework 5.110. Thanks in advance Gilles Caulier
digiKam 8.5.0 is out with many improvements in face detection and recognition. Please update these entry accordingly with this version. Thanks in advance... https://www.digikam.org/news/2024-11-16-8.5.0_release_announcement/
Hi. I can confirm this behavior is still the same on version 8.5.
I can also confirm that faces load slowly while scrolling on 8.5. This email on the user mailing list says that face thumbnails are created on the fly and says it is a known issue. I guess this is the ticket. I wanted to put some numbers to the issue. https://mail.kde.org/pipermail/digikam-users/2024-December/036821.html ---- ** My setup when scroling faces ** I am using MySQL, with the MySQL stored on a local SSD, and now have my photos on a local SSD as well. I am on the People tab, with one person selected. My sort order is: * Sort Album: By Folder * Sort Item: By Path * Item Sort Order: Descending * Separate Items: By Faces * Item Separation Order: Ascending I have 77 face thumbnails visible in an 11x7 grid, no filters applied. There is an additional row of 11 thumbnails partially visible, which I can see have loaded, but which I can't see enough of to use. There are 4,336 unconfirmed faces for this person. I am paging down by clicking in the scrollbar area below the scroll pill/handle. As I page down, here are the load times until all thumbnail are visible: * 29.35 seconds * 30.08 seconds * 31.85 seconds * 24.08 seconds * 22.58 seconds ---- When scrolling my full album with 263,000 items, it takes about a minute for the initial page load, but subsequent page loads as I scroll load the thumbnails too quickly to measure.
Hi all, Perhaps a timer settings to delay the thumbnails reconstruction can be a first approach to not stress the GUI while users review the faces. Typically while the edit mode is active, the timer is restarted. When i talk about timer, i want mean something like 10 seconds for example. Another idea is to let's the user to decide when thumbnails must be shown, typically with a button. A timer will automatically process the refresh if the user interaction go away. What do you think about. Gilles
Hi Gilles, For me the UI remains responsive. I can change away from the current People tab at any time without any lag. I'm not sure I understand how the timer helps. For me, I think generating face thumbnails when a face is recognized, or maybe as a new option within Rebuild Thumbnails seems like it would help. The load time is not problematic because the UI freezes, it is just problematic because of the workflow. I have 4000+ unconfirmed faces for this person, but the accuracy isn't 100%, so I have to see them before confirming. So I confirm up to 77 photos, page down, wait 30 seconds, confirm up to 77 more, page down, confirm up to 77 more, and so on. That's like 25 minutes of just waiting for thumbnails to be generated/load.
(In reply to Michael Moore from comment #25) > Hi Gilles, > > For me the UI remains responsive. I can change away from the current People > tab at any time without any lag. > > I'm not sure I understand how the timer helps. For me, I think generating > face thumbnails when a face is recognized, or maybe as a new option within > Rebuild Thumbnails seems like it would help. > > > > The load time is not problematic because the UI freezes, it is just > problematic because of the workflow. > > I have 4000+ unconfirmed faces for this person, but the accuracy isn't 100%, > so I have to see them before confirming. So I confirm up to 77 photos, page > down, wait 30 seconds, confirm up to 77 more, page down, confirm up to 77 > more, and so on. That's like 25 minutes of just waiting for thumbnails to be > generated/load. Hi Michael, The face thumbnail is generated from the original image when the face is detected in the scan. The thumbnail image is then saved to the thumbnail DB. We also have a thumbnail caching service in digikam that keeps a portion of the thumbnails in memory. I'm not sure, but I think Gilles might be referring to refreshing (rebuilding) the thumbnail cache, and not the thumbnail itself since the thumbnail shouldn't change once it's been written to the DB. The thumbnail will get rewritten to the DB if there is a change to the face region or editing done to the image that changes the dimensions. Can you provide the specs of your system? How fast is the CPU, what kind of storage (local SSD, network share, etc...), and probably most importantly, how much memory do you have? Cheers, Mike
>I'm not sure, but I think Gilles might be referring to refreshing (rebuilding) the thumbnail cache, and not the thumbnail itself since the thumbnail >shouldn't change once it's been written to the DB. This is exactly what i mean, thanks to have clarified this point (:=)))... Gilles
> Can you provide the specs of your system? How fast is the CPU, what kind of storage (local SSD, network share, etc...), and probably most importantly, how much memory do you have? * 8 core AMD Ryzen 7 5700G, max speed 4.6 GHz * 64GB RAM * Photos are on a local NVME SSD (WD Blue SN580 2TB) * MariaDB is on a different local NVME SSD, along with the rest of the system install (Intel SSDPEKNW010T8) I am running Debian Trixie/Testing, and using the digiKam 8.5 AppImage. > The face thumbnail is generated from the original image when the face is detected in the scan. The thumbnail image is then saved to the thumbnail DB. We also have a thumbnail caching service in digikam that keeps a portion of the thumbnails in memory. I don't think I'm seeing this behavior. I made a screen recording to show it. You may have to watch in 4k to see all the text clearly. https://www.youtube.com/watch?v=MwQCRO-9ujc In the video I have 4 terminals open: 1 with a MariaDB connection to the digikam_thumbs database. 1 with iotop sorted by disk read activity, and one with iotop sorted by disk write activity. The last terminal has htop running to show RAM and CPU usage. I open digiKam. It starts on the People tab, with my face selected. All thumbnails load quickly I wait until it says "No active process" again and then run "SELECT COUNT(*) FROM Thumbnails" in the terminal. We start with 500854 thumbnails I page down by clicking in the scroll bar, below the scroll handle. Some thumbs load instantly, others take some seconds to load. During this time you can see that perl is running exiftool quite a lot and mariadb is doing quite a lot of writing. Both of these processes stop when all thumbnails have been loaded. I repeat the page down process a few times. On the last page down I rapidly re-run the "SELECT COUNT(*)" query and we see the number of thumbnails increasing. Once the thumbs are all loaded, the count stabilizes at 501006. Maybe I'm looking at the wrong things, but to me it looks like the thumbnails are being generated when they are first viewed, not when the faces were detected.
(In reply to Michael Moore from comment #28) > > Can you provide the specs of your system? How fast is the CPU, what kind of storage (local SSD, network share, etc...), and probably most importantly, how much memory do you have? > > * 8 core AMD Ryzen 7 5700G, max speed 4.6 GHz > * 64GB RAM > * Photos are on a local NVME SSD (WD Blue SN580 2TB) > * MariaDB is on a different local NVME SSD, along with the rest of the > system install (Intel SSDPEKNW010T8) > > I am running Debian Trixie/Testing, and using the digiKam 8.5 AppImage. > > > The face thumbnail is generated from the original image when the face is detected in the scan. The thumbnail image is then saved to the thumbnail DB. We also have a thumbnail caching service in digikam that keeps a portion of the thumbnails in memory. > > I don't think I'm seeing this behavior. > > I made a screen recording to show it. You may have to watch in 4k to see all > the text clearly. > > https://www.youtube.com/watch?v=MwQCRO-9ujc > > In the video I have 4 terminals open: > > 1 with a MariaDB connection to the digikam_thumbs database. > 1 with iotop sorted by disk read activity, and one with iotop sorted by disk > write activity. > The last terminal has htop running to show RAM and CPU usage. > > I open digiKam. > It starts on the People tab, with my face selected. All thumbnails load > quickly > I wait until it says "No active process" again and then run "SELECT COUNT(*) > FROM Thumbnails" in the terminal. > We start with 500854 thumbnails > > I page down by clicking in the scroll bar, below the scroll handle. > Some thumbs load instantly, others take some seconds to load. > During this time you can see that perl is running exiftool quite a lot and > mariadb is doing quite a lot of writing. Both of these processes stop when > all thumbnails have been loaded. > > I repeat the page down process a few times. > > On the last page down I rapidly re-run the "SELECT COUNT(*)" query and we > see the number of thumbnails increasing. Once the thumbs are all loaded, the > count stabilizes at 501006. > > > Maybe I'm looking at the wrong things, but to me it looks like the > thumbnails are being generated when they are first viewed, not when the > faces were detected. Hi Michael, That's definitely interesting. The number of thumbnails shouldn't go up. Is this behavior repeatable? In other words, if you restart digikam will the number of thumbnails go back down to 500854, or will it start at 501006? I'm wondering if there might be an issue with the thumbnail cache. The face pipelines coming in 8.6.0 have been completely rewritten from scratch. I can tell you with certainty that in the 8.6.0 pipelines the thumbnail is written to the DB when it is first extracted from the original image during face detection. If you're willing to try very unstable code, grab the latest AppImage and let's see if we can reproduce the issue. If we can, then it has to be something in the thumbnail cache. You can find it here: https://files.kde.org/digikam/ ** BACK UP ALL YOUR DATABASES BEFORE USING THE 8.6.0 NIGHTLY BUILD ** Also, I see from the video (thank you!) that you have "Remove obsolete core database objects" selected. While this shouldn't affect the thumbnail DB, it might be good to turn it off for testing. Finally, I think you are delegating all metadata functions to exiftool, correct? Can you please turn this off, too. Exiftool can really slow things down. Cheers, Mike
We had an error in < digiKam-8.5.0 which, together with MySQL, meant that thumbnails were created again or files were scanned again, depending on the time zone. The problem was related to the modification date. This should no longer occur, but a scan or thumbnail process can take place again, after which this should no longer take place for "old" files. Note that we also basically create the normal thumbnails "on the fly". The problem with face detection is that the machine is already at full capacity, which makes the process even slower. Incorporating the creation of thumbnails in the face scan process is also not a good idea, because we might then create the thumbnails twice. This is because the item view also starts a thumbnail process when a new item without a thumbnail is found. The digiKam image/thumbnail loader does check the running loader processes for this scenario, but it is not certain that it always works. Maik
I've just watched the video, but your thumbnails are really slow to create on a machine with these specifications. My computer is far from that, and yet the faces are created much faster. As Michael has already written, ExifTool makes the whole thing slower, but for some images it is the only way to get correct metadata. Maik
> [Mike] The number of thumbnails shouldn't go up. Is this behavior repeatable? In other words, if you restart digikam will the number of thumbnails go back down to 500854, or will it start at 501006? The number of thumbnails remains at whatever it was when Digikam quit. Scrolling to unconfirmed faces I haven't looked at yet continues to generate new thumbnails. Revisting previously viewed faces uses the existing thumbnail as expected. > [Mike] If you're willing to try very unstable code, grab the latest AppImage and let's see if we can reproduce the issue. If we can, then it has to be something in the thumbnail cache. You can find it here: https://files.kde.org/digikam/ I'm OK with unstable code. My photos are safely backed up. I have disabled unchecked both "Use ExifTool backend to read metadata from files" and "Delegate to ExifTool backend al loperations to write metadata to files". I have also unchecked "Remove obsolete core database objects". The performance is much better, not nearly as fast as album view, but better than it was. As for the thumbnail generation, I'm not 100% sure what to expect. > [Mike] I can tell you with certainty that in the 8.6.0 pipelines the thumbnail is written to the DB when it is first extracted from the original image during face detection. > [Maik] Incorporating the creation of thumbnails in the face scan process is also not a good idea, because we might then create the thumbnails twice. These two statements seem to contradict each other. It sounds like Mike is saying that thumbnails are supposed to be generated during first face detection, and Maik is saying that we shouldn't (and don't) do this. In any case, I guess that faces which are already detected won't have thumbnails automatically generated. If thumbnails are in fact supposed to be generated somewhere in the detection process, how can I test it and report back here? Do I need to do the Maintenance item "Reset and clear all faces and training" and then scan them again to test if they are generated at some point after detection? Maybe a new Maintenance item could be helpful under "Rebuild Thumbnails" titled "Build missing face thumbnails" or "Rebuild face thumbnails".
(In reply to Michael Moore from comment #32) > > [Mike] If you're willing to try very unstable code, grab the latest AppImage and let's see if we can reproduce the issue. If we can, then it has to be something in the thumbnail cache. You can find it here: https://files.kde.org/digikam/ > > I'm OK with unstable code. My photos are safely backed up. Awesome. Please let us know how it goes. > > [Mike] I can tell you with certainty that in the 8.6.0 pipelines the thumbnail is written to the DB when it is first extracted from the original image during face detection. > > [Maik] Incorporating the creation of thumbnails in the face scan process is also not a good idea, because we might then create the thumbnails twice. Maik, we can easily remove thumbnail creation from the 8.6.0 pipelines if you think that's best. My thoughts, though, are the most expensive operation is loading the original image (especially RAW images), which I need to do in the pipeline to detect the faces. Then, the face detector gives me the face region coordinates, so we already have them. The only step that's missing is saving the thumbnail to the DB, which is relatively quick in comparison. The 8.6.0 pipelines are the same speed as 8.5.0 for single threading, and much faster when using full CPU so there isn't a performance penalty for creating the thumbnails when we detect faces. If we compare the image modification timestamp with the thumbnail timestamp we can quickly determine if we need to rebuild the thumbnail from the original image. I'm thinking this could speed up loading face thumbnails. > In any case, I guess that faces which are already detected won't have > thumbnails automatically generated. > > If thumbnails are in fact supposed to be generated somewhere in the > detection process, how can I test it and report back here? Do I need to do > the Maintenance item "Reset and clear all faces and training" and then scan > them again to test if they are generated at some point after detection? Yes, clear all faces and rescan. The 8.6.0 face scanning has been simplified so that "Scan new images" and "Scan all images" also does recognition (matching). Cheers, Mike
@Michael, When creating thumbnails in the face pipeline, we need to make sure that we already have the thumbnail in the Thumbnail DB before saving the face to the Core DB. Otherwise, the item view model will be notified via the CoreDB notification that a new item is available and will also start creating a thumbnail. Maik
(In reply to Maik Qualmann from comment #34) > @Michael, When creating thumbnails in the face pipeline, we need to make > sure that we already have the thumbnail in the Thumbnail DB before saving > the face to the Core DB. Otherwise, the item view model will be notified via > the CoreDB notification that a new item is available and will also start > creating a thumbnail. > > Maik Hi Michael, Maik is correct (as always :) ) about when to create the thumbnail in the face pipeline. I updated the 8.6.0 face pipeline code to match Maik's recommendation. We do create the thumbnail when we detect the face in 8.6.0. Can you grab the latest 8.6.0 build, please? Turn on ExifTool and "Remove obsolete core database objects", so your configuration is like what you had when you originally submitted the issue. Finally, can you run the same test you did before where you were watching disk I/O and the size of the thumbail DB? Let's see if it's doing the same thing with 8.6.0. Cheers, Mike
Sorry for the delay, it was a busy weekend. https://www.youtube.com/watch?v=8UWhkx3GjoA I tried to simplify my testing a bit. This is using the latest appimage, but I had to modify the startup script to not load libgnutls.so.30 I deleted the database and made an empty database. I created a photo folder with three photos - one with a pre-tagged face, one with no face and one with three faces. After the images are found and added to the album there are a total of 10 thumbnails. Accessing People > Michael, viewing the pre-tagged face does not create a new thumbnail. Accessing People > Unknown creates two new thumbnails for a total of 12 thumbnails. Perhaps only the first face in a photo is getting its thumbnail pre-created? Or, looking at records 8 & 12 or 10 & 11, perhaps there is some rounding happening making the thumbnail identifiers not match. The "rect" parameter seems to only be off by 1 pixel in various dimensions.
> Or, looking at records 8 & 12 or 10 & 11, perhaps there is some rounding happening making the thumbnail identifiers not match. The "rect" parameter seems to only be off by 1 pixel in various dimensions. Sorry, that wasn't very clear. At timestamp 6:02 in the video you can see the output of "SELECT * FROM CustomIdentifiers;" Records 8 & 12, and 10 & 11 are very close to matching.
> https://www.youtube.com/watch?v=8UWhkx3GjoA > > After the images are found and added to the album there are a total of 10 > thumbnails. > > Accessing People > Michael, viewing the pre-tagged face does not create a > new thumbnail. > Accessing People > Unknown creates two new thumbnails for a total of 12 > thumbnails. > > Perhaps only the first face in a photo is getting its thumbnail pre-created? > > Or, looking at records 8 & 12 or 10 & 11, perhaps there is some rounding > happening making the thumbnail identifiers not match. The "rect" parameter > seems to only be off by 1 pixel in various dimensions. Gilles and Maik, If you haven't already watched the video, please watch it soon. It looks like there is a dysfunction comparing/creating new thumbnails. I think it's because we store the rect in pixels and Exif stores the rect as float percentages. My guess is when converting the Exif RectF to Rect pixels there's a chance floating point rounding will be off by a pixel. The other possible cause is we're padding the face rect somewhere, but I'm not adding the same padding when I create the thumbail in the face pipelines. Let's discuss this offline to come up with a fix. Michael, thank you so much for your time. This is incredibly helpful. Cheers, Mike