I'm trying face recognition (again) with version 8.5.0. Summary: it doesn't work at all. For years I re-try the face recognition in digikam. In the beginning it works. But when you tagged a few thousand images it becomes very quickly useless. The results are no better than random. You can find several descriptions by users with the same symptom. With the new SFace model not one face get's recognized, even with lowest accuracy of 1. Bug? When I switch back to OpenFace it will again recognize some faces but all random. I rebuild the training data - which seems to be done anyway when switching models. I can imagine that the training data becomes "random" in some way. One persons face is provided in so many variants that the trained face model becomes blur? Which results in useless results. So maybe the recognition by itself is not the problem but a filter for good training data is needed? Maybe there's a bug why SFace doesn't recognize anything for me, which is independent from bad training data. Related bug 432537, bug 475488 I respect the effort which went into this feature over the years. On the other hand, it was mentioned a few times how it will improve, but it didn't.
(In reply to René Fritz from comment #0) > I'm trying face recognition (again) with version 8.5.0. Summary: it doesn't > work at all. > > For years I re-try the face recognition in digikam. In the beginning it > works. But when you tagged a few thousand images it becomes very quickly > useless. The results are no better than random. You can find several > descriptions by users with the same symptom. > > With the new SFace model not one face get's recognized, even with lowest > accuracy of 1. Bug? > When I switch back to OpenFace it will again recognize some faces but all > random. > > I rebuild the training data - which seems to be done anyway when switching > models. > > I can imagine that the training data becomes "random" in some way. One > persons face is provided in so many variants that the trained face model > becomes blur? Which results in useless results. > > So maybe the recognition by itself is not the problem but a filter for good > training data is needed? > > Maybe there's a bug why SFace doesn't recognize anything for me, which is > independent from bad training data. > > Related bug 432537, bug 475488 > > I respect the effort which went into this feature over the years. On the > other hand, it was mentioned a few times how it will improve, but it didn't. Hi René, I'm sorry you're having trouble with face recognition. I'll do my best to get this working for you. The partial rewrite of the face engine in 8.5.0 is about 95% accurate based on our testing and multiple user reports. What you are experiencing is the first report of poor performance in 8.5.0. The face engine is completely rewritten for the next 8.6.0 release. Can you provide more information, please? How many different people have a confirmed face in your library? How many confirmed faces do you have in total? Did you remove ALL faces including confirmed, unconfirmed, and unknow and restart with face detection, or are you retraining with faces that were found with older versions of digiKam? To make sure you are using all the correct face models, I'd like you to delete the directory where we download the face AI models. If you are using the AppImage, look for a directory called facesengine and delete all the directories under it. The directories should be name yunet, dnnface, and shape-predictor. After the directories are deleted, please restart digiKam and it should download all the models again. Please let me know if you have any questions. Cheers, Mike
Thanks for your reply! I use the Neon package. V8.5.0. Build Date 10.12.24 > To make sure you are using all the correct face models, I'd like you to > delete the directory where we download the face AI models. If you are using > the AppImage, look for a directory called facesengine and delete all the > directories under it. The directories should be name yunet, dnnface, and > shape-predictor. After the directories are deleted, please restart digiKam > and it should download all the models again. I answer this first - more later. In .local/share/digikam/facesengine/ there are no subfolders but: 21675 Okt 31 14:48 classification_classes_ILSVRC2012.txt 625 Okt 31 14:48 coco.names 28092 Okt 31 14:11 deploy.prototxt 232589 Dez 15 13:54 face_detection_yunet_2023mar.onnx 38696353 Dez 15 13:54 face_recognition_sface_2021dec.onnx 31510785 Okt 31 14:11 openface_nn4.small2.v1.t7 5351047 Okt 31 14:11 res10_300x300_ssd_iter_140000_fp16.caffemodel 102146365 Okt 31 14:48 resnet50.onnx 67740572 Okt 31 12:03 shapepredictor.dat 88007527 Okt 31 14:11 weights_inceptionv3_299.pb 8334 Okt 31 12:03 yolov3-face.cfg 246305388 Okt 31 14:11 yolov3-wider_16000.weights 7616008 Okt 31 14:11 yolov5n_batch_16_s320.onnx 347010839 Okt 31 14:48 yolov5x_batch_16_s320.onnx After removing the files following got downloaded: 21675 Dez 29 18:48 classification_classes_ILSVRC2012.txt 625 Dez 29 18:48 coco.names 28092 Dez 29 18:47 deploy.prototxt 232589 Dez 29 18:48 face_detection_yunet_2023mar.onnx 38696353 Dez 29 18:48 face_recognition_sface_2021dec.onnx 31510785 Dez 29 18:48 openface_nn4.small2.v1.t7 5351047 Dez 29 18:47 res10_300x300_ssd_iter_140000_fp16.caffemodel 102146365 Dez 29 18:48 resnet50.onnx 67740572 Dez 29 18:47 shapepredictor.dat 88007527 Dez 29 18:48 weights_inceptionv3_299.pb 8334 Dez 29 18:48 yolov3-face.cfg 246305388 Dez 29 18:48 yolov3-wider_16000.weights 7616008 Dez 29 18:48 yolov5n_batch_16_s320.onnx 347010839 Dez 29 18:48 yolov5x_batch_16_s320.onnx A quick test showed no difference. SFace with rebuild model. I try to test with some faces removed and report my results and more details.
Hi Réné, Neon package is a Snap stuff or not ? If yes, stop immediately, Snap is just not suitable for a huge application as digiKam (it's the same with Flatpak). Considerate also Neon as development stuff, not enough stable for a production usage. To test a software, use a stable Linux. Kubuntu 24.04 LTS is a good choice. In all cases, i recommend to test with the AppImage Linux bundle that we provides. This one works perfectly. I tested the recognition workflow with 8.5.0 in-deep under Kubuntu 24.04, and it works like a charm. It's not comparable to previous digiKam releases, using SFace model. Best Gilles Caulier
> Neon package is a Snap stuff or not ? If yes, stop immediately, Snap is just > not suitable for a huge application as digiKam (it's the same with Flatpak). No Neon has nothing to do with snap. Neon use native deb packages. In fact they're trying to avoid snap. > Considerate also Neon as development stuff, not enough stable for a > production usage. To test a software, use a stable Linux. Kubuntu 24.04 LTS > is a good choice. I strongly disagree. A few years back Kubuntu was totally buggy. I heard it is better now. Neon has a stable user edition which is comparable to Kubuntu using Ubuntu LTS but often more up to date. There is a Neon dev edition which I don't use. > In all cases, i recommend to test with the AppImage Linux bundle that we > provides. This one works perfectly. I tested the recognition workflow with > 8.5.0 in-deep under Kubuntu 24.04, and it works like a charm. It's not > comparable to previous digiKam releases, using SFace model. I try the Appimage and report
A request has just arrived on the mailing list, which suggests that the OpenCV Debian packages are broken. I'll make a connection here. https://mail.kde.org/pipermail/digikam-users/2024-December/036839.html Maik
(In reply to Maik Qualmann from comment #5) > A request has just arrived on the mailing list, which suggests that the > OpenCV Debian packages are broken. I'll make a connection here. > > https://mail.kde.org/pipermail/digikam-users/2024-December/036839.html > > Maik Do we know what version of OpenCV Debian is using? I saw that OpenCV v5 has changes that will break parts of digiKam, but I didn't see any changes that should affect the face models. Cheers, Mike
... and we never validate OpenCV 5. 4.10 is validated. As i already said previously, use stable distribution to judge an application as digiKam. Use the AppImage which is compiled with the components validated by digiKam team... Gilles Caulier
In Ubuntu 24.04 libopencv packages have the version 4.6.0+dfsg-13.1ubuntu1 The Appimage digikam is currently building models, which takes way longer than before. Could be a packaging problem. Let's see how it works when the models are finished.
The Appimage version works with Yunet and SFace. The face recognition with SFace is indeed much better. Sorry to bother you and thanks for your time!! If you have an idea what might cause that problem, I would report that packaging bug. Interesting is that the old models still work and the new find nothing. Again, thank you
Hi all, Look to the OpenCV change log : https://github.com/opencv/opencv/wiki/OpenCV-Change-Logs - 4.11 is the last stable one just released. digiKam bundles are currently compiled with 4.10. - 5.0 is an alpha stage version, for TESTING only, not for production. Typically, as i explained before, all Linux distro which are based to alpha code and trying to use it in production must be ignored imediatly. If you lose data because you use alpha code, we cannot help you at all... As i already said, and as i already seen in the KDE forum, Neon is not recommended to use in production. Kubuntu is so far better. I use it in all Linux computers in my office (Control room from an experimental nuclear reactor to process big data and analysis), and it works perfectly... Q to Michael : accordingly to 4.11 revision changes, it's fine for you to upgrade 4.10 to 4.11 safety in the bundles ? My best Gilles Caulier
(In reply to caulier.gilles from comment #10) > Hi all, > > Look to the OpenCV change log : > > https://github.com/opencv/opencv/wiki/OpenCV-Change-Logs > > Q to Michael : accordingly to 4.11 revision changes, it's fine for you to > upgrade 4.10 to 4.11 safety in the bundles ? Hi Gilles, Yes, it's safe to upgrade to OpenCV 4.11.0. Cheers, Mike
*** Bug 498104 has been marked as a duplicate of this bug. ***
(In reply to Maik Qualmann from comment #12) > *** Bug 498104 has been marked as a duplicate of this bug. *** Maik and Gilles, I've confirmed this to be a bug in OpenCV v 4.6. I don't know in what version between 4.6 and 4.10 it was fixed, but it's fixed in 4.10. Cheers, Mike
Did you recommend to increase the OpenCV minimal version to 4.10 as dependencies, to prevent this kind of dysfunction in the future? Gilles
(In reply to caulier.gilles from comment #14) > Did you recommend to increase the OpenCV minimal version to 4.10 as > dependencies, to prevent this kind of dysfunction in the future? > > Gilles I will once I determine the correct minimum version of OpenCV. I'll look at the OpenCV release notes to see if I can find the fix. If not, I'll have to test every version from 4.7 to 4.9. We know 4.6 is has an issue and 4.10 is good. I don't know which version has the fix. Cheers, Mike
Perhaps asking to the OpenCV project forum is the simpler solution... https://forum.opencv.org/ Gilles
(In reply to caulier.gilles from comment #16) > Perhaps asking to the OpenCV project forum is the simpler solution... > > https://forum.opencv.org/ > > Gilles We need a minimum of OpenCV 4.8 to fix the bug. I'll update the code later today. Cheers, Mike
Hi Michael, I have generally set OpenCV-4.8.0 as the minimum version for digiKam. Or did you plan to do this individually for each model. Maik
Git commit cc682e43f986081b9df608e3f44e4b0b5744e430 by Gilles Caulier. Committed on 05/01/2025 at 16:02. Pushed by cgilles into branch 'master'. api doc : opencv must be 4.8.0 or later M +2 -2 Mainpage.dox https://invent.kde.org/graphics/digikam/-/commit/cc682e43f986081b9df608e3f44e4b0b5744e430
(In reply to Maik Qualmann from comment #18) > Hi Michael, > > I have generally set OpenCV-4.8.0 as the minimum version for digiKam. Or did > you plan to do this individually for each model. > > Maik Hi Maik and Gilles, I was just writing an email to you about this. Please check email and let's discuss. Cheers, Mike
Hello Mike, I am facing the same issue as René. First, SFace worked significantly better after training a few faces. The more faces I trained though, the worse face recognition became. Here are my answers to your questions: > How many different people have a confirmed face in your library? 652 > How many confirmed faces do you have in total? 26,264 > Did you remove ALL faces including confirmed, unconfirmed, and unknow and > restart with face detection, or are you retraining with faces that were > found with older versions of digiKam? I retained confirmed faces and retrained with faces that were newly detected by YuNet as well as faces that were detected by older digiKam versions.
(In reply to haselnuss87 from comment #21) > Hello Mike, I am facing the same issue as René. First, SFace worked > significantly better after training a few faces. The more faces I trained > though, the worse face recognition became. > > Here are my answers to your questions: > > > How many different people have a confirmed face in your library? > 652 > > How many confirmed faces do you have in total? > 26,264 > > Did you remove ALL faces including confirmed, unconfirmed, and unknow and > > restart with face detection, or are you retraining with faces that were > > found with older versions of digiKam? > I retained confirmed faces and retrained with faces that were newly detected > by YuNet as well as faces that were detected by older digiKam versions. Hello, Thank you for the information. Face recognition will never be 100%. There will always be a margin of error. I'm trying to determine if the results are due to a bug, a mislabeled face, or if it's within the margin of error. For version 8.5.0, the margin of error is about 5%. In version 8.6.0 the margin of error is deceased to about 2%-3%. With over 26,000 confirmed faces, there could be over 1,300 errors and still be within the expected margin of error. For the erroneously identified faces, is there a common pattern you can see? For example, faces for the same unidentified person are matching to the same confirmed person? This is usually an indicator of an incorrectly confirmed face. I think
Hi Michael, I had been thinking a little about Bug 498320. An idea occurred to me: what if users assign face names twice? We don't use a flat list of people, because of our hierarchy, for example, a Tom can be under "father" and a Tom under "uncle". This means that we have two "Toms" in the face database with different faces. That would undoubtedly lead to errors, or am I wrong? Maik
Hi Maik, Tom1 and Tom2 will have 2 fingerprints in the database if Tom1 != Tom2. Of course if Tom1 == Tom2, what's will be the expected result ? Gilles
(In reply to caulier.gilles from comment #24) > Hi Maik, > > Tom1 and Tom2 will have 2 fingerprints in the database if Tom1 != Tom2. > > Of course if Tom1 == Tom2, what's will be the expected result ? > > Gilles Hi everyone, I tested this scenario when I was first working with YuNet and SFace. Each face is defined by 128 unique floating-point vectors. The only way for Tom1 == Tom2 is if there are 2 exact copies of the same image in the library and image1 is assigned Tom1 and image2 is assigned Tom2. Even a slightly resized version of the same image will result in slightly different values for the face vectors. The classifier selects the face that is the closest match. If there are 2 sets of vectors with exactly the same values, the classifier will select the first one it finds. There won't be an error. An issue I've been researching is sometimes when confirming a face, an extra face is added to the list of confirmed faces. The extra face seems to be random, but is being passed from the UI to the faces engine. I've verified the extra confirmation is not in the faces engine. It happens infrequently, so it wasn't my top priority to research. In a library of over 26,000 confirmed faces, there is a chance this dysfunction has included "extra" faces for a person that shouldn't be there, which would lead to "wrong" faces being presented as suggestions. There's also a good chance that small, pixelated faces have been confirmed, and the face vectors aren't good for matching. Once I'm done with the autotagging rewrite, I'm going to revisit the idea of applying a quality metric to the face thumbnail and excluding low quality faces from training. Cheers, Mike
Face Image Quality Assessment (FIQA) has been added in the latest build. It filters out too small, blurry, and noisy images from being used for face training. You can download the latest nightly build from https://files.kde.org/digikam. You'll need to retrain your face library after you install the new version. This should help improve face matching quality. Cheers, Mike
(In reply to Michael Miller from comment #26) > Face Image Quality Assessment (FIQA) has been added in the latest build. It > filters out too small, blurry, and noisy images from being used for face > training. You can download the latest nightly build from > https://files.kde.org/digikam. You'll need to retrain your face library > after you install the new version. This should help improve face matching > quality. > > Cheers, > Mike What is the correct method to retrain the face system?
> What is the correct method to retrain the face system? To retrain the face DB, go to Tools->Maintenance->Detect and Recognize Faces. Select "Rebuild all training data" only. Cheers, Mike
> To retrain the face DB, go to Tools->Maintenance->Detect and Recognize > Faces. Select "Rebuild all training data" only. Thanks Mike. Not sure if you are aware that on 8.5.0, when this task is initiated there is no feedback from Digikam that this process is being done. The only way I could tell was through Windows Task Manager, and eventually the percentage bar in Digikam showed up. I would assume if there aren't many faces in the system then it would have shown up sooner.
(In reply to Craig from comment #29) > > To retrain the face DB, go to Tools->Maintenance->Detect and Recognize > > Faces. Select "Rebuild all training data" only. > > Thanks Mike. Not sure if you are aware that on 8.5.0, when this task is > initiated there is no feedback from Digikam that this process is being done. > The only way I could tell was through Windows Task Manager, and eventually > the percentage bar in Digikam showed up. I would assume if there aren't many > faces in the system then it would have shown up sooner. Hi Craig, The face pipelines were completely rewritten in 8.6.0. This issue was fixed in the rewrite. Cheers, Mike