Bug 271679 - Digikam detects but does not recognize faces.
Summary: Digikam detects but does not recognize faces.
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Faces-Recognition (show other bugs)
Version: 5.0.0
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-04-25 11:39 UTC by Flavio
Modified: 2019-12-25 15:17 UTC (History)
40 users (show)

See Also:
Latest Commit:
Version Fixed In: 7.0.0


Attachments
Log Digikam not recognizing faces (90.53 KB, text/plain)
2011-09-30 00:57 UTC, BartOtten
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Flavio 2011-04-25 11:39:15 UTC
Version:           2.0.0 (using KDE 4.6.2) 
OS:                Linux

Digikam does a decent job at detecting faces (which is excellent), but once it does, it is absolutely incapable of recognizing and sorting them. 

Even after manually tagging a large number of the faces it detected, digikam cannot automatically tag any of the faces in the "unknown" folder.

Also the processing speed for face recognition is suspiciously fast: the job is completed in a matter of a few seconds, even though the catalog has many hundreds faces in it. Just detecting faces takes ages and much computing power (understandably), but it's strange that face recognition is so much faster (but non functional!). 

Reproducible: Always

Steps to Reproduce:
open "scan for faces dialog"
select "recognize faces"
click "scan"

Actual Results:  
no faces tagged as "unknown" are recognized, no matter how many times the people in those images have been tagged already (manually). 

Expected Results:  
"mike's" face tagged as unknown should be automatically tagged as "mike" and put in the "mike" folder, provided the user already manually tagged "mike" enough times for the software to "know" his face.

Also it would be helpful it digikam sorted all faces in different tags e.g. unknown#1, unknown#2, etc, so it would be easier to perform batch tagging.
Comment 1 Thomas Eschenbacher 2011-04-26 19:27:26 UTC
I can confirm all points mentioned above too!
Here: Gentoo Linux, media-gfx/digikam-2.0.0_beta4, media-libs/opencv-2.1.0

Any clues why this does not work?
Is it a "bug" for which I could help debugging, 
or is it just "not (yet) implemented" functionality?
Comment 2 Flavio 2011-04-26 20:52:37 UTC
Ok just repeated the procedure to get the bugrunning Digikam in a terminal.

This is the kind of error message I get:

A few hundred of these messages

OpenCV Error: Assertion failed (ssize.area() > 0) in resize, file /build/buildd/opencv2.2-2.2.0/modules/imgproc/src/imgwarp.cpp, line 1391
digikam(6608)/KFACE: cv::Exception: /build/buildd/opencv2.2-2.2.0/modules/imgproc/src/imgwarp.cpp:1391: error: (-215) ssize.area() > 0 in function resize

then some of these

QImage::scaled: Image is a null image
QImage::scaled: Image is a null image
QImage::scaled: Image is a null image

and so on....

The two kind of messages alternate in a seemingly random way, but there are way more OpenCV errors than QImage errors. I think each of these errors correspond to a face it is trying to recognize.

So yes, it appears to be a bug, because digikam is surely trying do do something and but gets OpenCV errors. This feature is already implemented, it's just buggy, at least in my case.

Does this feature work for any of you guys out there or is it buggy for everyone?

Thanks
Comment 3 Thomas Eschenbacher 2011-05-01 11:14:42 UTC
just some update on this:
I ugraded opencv to version 2.2.0, as this was mentioned on the project homepage, and rebuilt kipi-plugins + digikam, but nothing changed, the recognition still does not work at all.
I do not see any error messages coming from opencv, but always the same debug output when I scan my test album, which contains only one image with 2 faces.

-------------- snip --------------

digikam(20292)/digikam (core) Digikam::FaceScanDialog::doLoadState: "Face Detection Dialog"
digikam(20292)/digikam (core) Digikam::FaceScanDialog::doSaveState: "Face Detection Dialog"
digikam(20292)/digikam (core) Digikam::BatchFaceDetector::startAlbumListing: Total is 1
digikam(20292)/digikam (core) Digikam::BatchFaceDetector::continueAlbumListing: false false
digikam(20292)/digikam (core) Digikam::BatchFaceDetector::continueAlbumListing: Album "__TEST"
Object::disconnect: Unexpected null parameter
Object::disconnect: Unexpected null parameter
digikam(20292)/digikam (core) Digikam::BatchFaceDetector::continueAlbumListing: false true
digikam(20292)/digikam (core) Digikam::DImg::load: "/var/tmp/test-album/__TEST/12280009.jpg"  : JPEG file identified
digikam(20292)/digikam (core) Digikam::DMetadata::getIccProfile: Exif color-space tag is sRGB. Using default sRGB ICC profile.
Cascade directory located as : /usr/share/opencv/haarcascades
digikam(20292)/digikam (core) Digikam::DetectionWorker::process: Found 2 faces in "12280009.jpg" QSize(2048, 1536) QSize(2048, 1536)
Cascade directory located as : /usr/share/opencv/haarcascades
digikam(20292)/digikam (core) Digikam::FaceIface::databaseFaces: rect found as  QRect(594,333 325x325) ""QVariant(QRect, QRect(594,333 325x325) ) for attribute "autodetectedFace" tag 30
digikam(20292)/digikam (core) Digikam::FaceIface::databaseFaces: rect found as  QRect(1083,379 312x312) ""QVariant(QRect, QRect(1083,379 312x312) ) for attribute "autodetectedFace" tag 30
digikam(20292)/digikam (core) Digikam::FaceIface::writeUnconfirmedResults: Entry QRect(1083,379 312x312) ""QVariant(QRect, QRect(1083,379 312x312) ) 30 overlaps QRect(1083,379 312x312) ""QVariant(QRect, QRect(1083,379 312x312) ) 30 , skipping
digikam(20292)/digikam (core) Digikam::FaceIface::writeUnconfirmedResults: Entry QRect(594,333 325x325) ""QVariant(QRect, QRect(594,333 325x325) ) 30 overlaps QRect(594,333 325x325) ""QVariant(QRect, QRect(594,333 325x325) ) 30 , skipping
digikam(20292)/digikam (core) Digikam::FacePipeline::FacePipelinePriv::checkFinished: Check for finish:  0 packages, 0 infos to filter, hasFinished() true
digikam(20292)/digikam (core) Digikam::BatchFaceDetector::continueAlbumListing: false false
-------------- snap --------------

Can anyone of the developers see what goes wrong here?
How could I help debugging this?
Comment 4 terri.bugs 2011-05-30 11:51:27 UTC
I have the same problem (faces detected, but never recognized) with beta5. Installation was done using ppa (philip5) with opencv 2.2.
Comment 5 againsttcpa84 2011-06-10 10:59:24 UTC
*** This bug has been confirmed by popular vote. ***
Comment 6 Gareth 2011-06-15 17:20:59 UTC
Still suspiciously quick on performing a scan of 80 'unknown' marks (1000+ images) from a GIT pull 1800 hrs GMT 14 Jun 2011
I am not seeing the CV errors reported though, only the QImage
Comment 7 Leif Huhn 2011-07-09 03:48:01 UTC
I am having the same problem "QImage::scaled: Image is a null image".

I was searching for where digikam keeps the face DB.  On some forum, they suggested the DB is pointed to be the following file:

~/.kde/share/apps/libkface/database/libface-config.xml

I do not have that file.  Could that be related?
Comment 8 J Appel 2011-08-06 22:44:33 UTC
I can confirm the behavior on archlinux, x64

digikam 2.0
opencv 2.3

haarcascades in
/usr/share/opencv/haarcascades/

Face detection works, recognition does not, message in console is:

QImage::scaled: Image is a null image
Comment 9 Michel Lafon-Puyo 2011-08-10 20:37:38 UTC
Same as #8.

Tested on arch 32bits too and the problem is exactly the same.
Comment 10 christian 2011-08-21 10:48:11 UTC
The same with digikam 2.0 and opencv 2.3.1 (32b on Fedora).
Comment 11 felix_mauch 2011-09-11 08:41:53 UTC
Yep, same here.

Archlinux: Digikam 2.1.0, opencv 2.3.1

I also get errors like this:
digikam(16439)/digikam (core) Digikam::FaceTagsEditor::databaseFaces: rect found as  QRect(584,448 415x415) for attribute "autodetectedFace" tag 84
digikam(16439)/digikam (core) Digikam::FacePipeline::FacePipelinePriv::checkFinished: Check for finish:  0 packages, 3 infos to filter, hasFinished() false
QImage::scaled: Image is a null image
Comment 12 BartOtten 2011-09-30 00:41:46 UTC
Here the same. No other related messages then:

QImage::scaled: Image is a null image
QImage::scaled: Image is a null image
QImage::scaled: Image is a null image
etc etc
Comment 13 BartOtten 2011-09-30 00:57:32 UTC
Created attachment 64081 [details]
Log Digikam not recognizing faces

Here is a full log from Digikam about not recognizing people/faces.
Comment 14 Thomas Eschenbacher 2011-10-31 18:14:41 UTC
even after upgrading to a newer opencv version this still does not work on my Gentoo system:

digikam-2.2.0
media-plugins/kipi-plugins-2.2.0
media-libs/opencv-2.3.0-r1

(I get practically the same output as above in #13, even with a new created test user account)

Even on a new Kbuntu 11.10 installation it behaves the same, and the same on two other machines I had access to.

I more and more get the impression that this feature has never worked for anyone - or is there some "reference system" around there where this feature really works?

INFO REQUEST:
dear author, please tell us how your system looks like: which distribution and version of kipi, kde, opencv and so on did you use when developing that feature?
Did you also test it with a new user account?
Comment 15 Zack Evans 2011-12-05 23:38:50 UTC
So over in bug 269720 the author himself explains that in libkface, the face detection works but face recognition is not implemented.

So "is this a face, or a building or a tree" can detect the faces... but it cannot recognise which face this is at all. So we should not expect it to work yet.

Perhaps we should have a bug for the documentation, as everything you read seems to say that it should work...
Comment 16 caulier.gilles 2011-12-14 13:45:17 UTC
*** Bug 262212 has been marked as a duplicate of this bug. ***
Comment 17 BartOtten 2011-12-14 20:50:45 UTC
Maybe it should be removed from the GUI?
Comment 18 bubukind 2011-12-14 21:18:15 UTC
+1 on that.
This most definitely is an unfinished feature and as such should not be present in the production GUI and/or feature list.
Comment 19 caulier.gilles 2012-01-23 07:53:27 UTC
*** Bug 292248 has been marked as a duplicate of this bug. ***
Comment 20 caulier.gilles 2013-02-09 08:25:43 UTC
*** Bug 314744 has been marked as a duplicate of this bug. ***
Comment 21 caulier.gilles 2013-03-05 12:04:41 UTC
*** Bug 316161 has been marked as a duplicate of this bug. ***
Comment 22 Lyall 2013-05-06 11:14:03 UTC
I can confirm that face location is working very well, but recognition does not work at all.
Digikam version 3.1.0 on KDE platform 4.10.2 on Gentoo amd64 system.

Setting face recognition accuracy anywhere between 70% and 95% makes no difference.

When running digikam from an xterm, it appears, during face recognition, digikam generates the following text, once for each image.

QImage::scaled: Image is a null image

No additional diagnostics are generated on the terminal, not even a source module or line number.
Comment 23 caulier.gilles 2013-05-06 11:21:59 UTC
Good news. 

Face Recognition is now implemented in dedicated git branches :

libkface : https://projects.kde.org/projects/extragear/libs/libkface/repository/show?rev=opentld

digiKam : https://projects.kde.org/projects/extragear/graphics/digikam/repository/show?rev=libkface

Code still alpha and we plan to merge these branch to master for next 3.3.0 release.

But if you want, you can test to checkout code and recompile whole digiKam...

Gilles Caulier
Comment 24 caulier.gilles 2014-06-24 13:47:57 UTC
I recommend to test next version 4.1.0 where a lots of fixes and improvements have been introduced around face recognition feature

Gilles Caulier
Comment 25 caulier.gilles 2015-06-29 17:47:22 UTC
New digiKam 4.11.0 is available with official PKG installer for OSX.

https://www.digikam.org/node/740

Can you reproduce the problem with this release ?

Gilles Caulier
Comment 26 Samuel Gilbert 2015-06-30 03:52:22 UTC
Following Gilles' post, I tried to get face detection to work again.

What I did :
- Went to the Face tab
- Clicked on Scan collection for faces
- Selected "Skip images already scanned" and "Detect faces"
- I selected a new album for which I had never ran face detection
- I clicked scan

After a while, the faces started showing up for the new album. When the process was complete, here is what I did :

- Clicked on Scan collection for faces
- Selected "Recognize faces"
- Clicked scan

There was no visible result even after the operation was complete.  I went back to the album tab, selected another album then came back to the face tab and scrolled to the album I had just scanned. I then then manually identified about one third of the face of each person in the album.

I relaunched the scan with the "Recognize faces" option.  Still no change.

I then scanned again with the "Clear unconfirmed results and rescan" option and "Detect and recognize faces".

At some point, Digikam crashed and since I am using the ArchLinux packages, the stack trace wasn't really useful.

I have done a few more scans, and now I get a suggestion in bellow some of the thumbnails in the face tab. The suggestion is pretty bad, and quite a few pictures don't even have a suggestion. Usually the suggestion controls (textbox + 2 buttons) follow the mouse cursor, but sometimes it gets stuck on a thumbnail.

I had already manually identified a few people in the album in other albums before I launched the tests. I don't know if the identification performance just needs more training to improve.

I don't know if I should consider this a win or fail considering the issues I encountered.

Is there a way to clear all the face identification data _without_ launching a scan of the entire collection? I would like to start from scratch for the detection, training and recognition.
Comment 27 caulier.gilles 2015-06-30 07:55:00 UTC
yes. remove the database file. It's located in your home directodry, as explained here :

http://api.kde.org/4.x-api/kdegraphics-apidocs/libs/libkface/libkface/html/index.html

Gilles Caulier
Comment 28 caulier.gilles 2015-08-22 06:39:13 UTC
digiKam 4.12.0 is out :

https://www.digikam.org/node/741

We need a fresh feedback using this release please...
Thanks in advance.
Comment 29 Jan Schneider 2016-02-26 11:25:34 UTC
Still doesn't work at all with 4.12.0.

Face detection works, face recognition doesn't. It runs, and scans, and does its magic, but without any results, not even on almost identical images.
Comment 30 caulier.gilles 2016-02-26 11:35:30 UTC
Try with last 4.14.0...

Gilles Caulier
Comment 31 Jan Schneider 2016-02-26 14:27:57 UTC
Is there some backport to Kubuntu? The official backports on launchpad are outdated badly.
Comment 32 caulier.gilles 2016-07-14 06:02:43 UTC
This file still valid using last stable 5.0.0 release ?
Gilles Caulier
Comment 33 caulier.gilles 2016-07-14 09:02:28 UTC

*** This bug has been marked as a duplicate of bug 365321 ***
Comment 34 Jmz D3an 2016-07-19 11:30:10 UTC
Still valid in 5.0.0

Faces of people tagged before are not recognised as the person but instead if the face is detected it is shown as unknown.

To have a List of recognised faces with suggested Name-Tags to be confirmed by the user (like in picasa or KPhotoAlbum) would be the best solution I guess.
Comment 35 caulier.gilles 2019-12-23 15:00:01 UTC
7.0.0-beta1 is out with new Face Recognition algorithm based on Deep
Learning/Neural Network API from OpenCV

https://download.kde.org/unstable/digikam/

Please test and give us a feedback

Thanks in advance
Gilles Caulier
Comment 36 caulier.gilles 2019-12-23 15:15:56 UTC
7.0.0-beta1 is out with new Face Recognition algorithm based on Deep
Learning/Neural Network API from OpenCV

https://download.kde.org/unstable/digikam/

Please test and give us a feedback

Thanks in advance
Gilles Caulier
Comment 37 caulier.gilles 2019-12-25 15:17:44 UTC
See the Neural Network example from digiKam 7.0.0-beta1 performing faces recognition with 2 items face-tagged in first.

https://i.imgur.com/HHF4T9X.png

1 new item is recognized, name-tagged, highlighted with green frame, and linked in "unconfirmed" virtual tag for confirmation.

Recognition work well using Deep Learning. I close this file now.

Merry Christmas and Happy new year.

Gilles Caulier