Bug 375375 - Various bugs detecting and tagging faces in current 5.4.0 Digikam appimage
Summary: Various bugs detecting and tagging faces in current 5.4.0 Digikam appimage
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Faces-Detection (show other bugs)
Version: 5.4.0
Platform: Appimage Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-01-21 14:05 UTC by Jens
Modified: 2019-03-28 21:03 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In: 6.1.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jens 2017-01-21 14:05:49 UTC
Using the current appimage:

1. Cannot use mouse to select a name from dropdown menu.
I want to add a face rectangle to an existing image. so I click and drag a rectangle and a combobox with name tags appears below it. I can open the menu and scroll to a name, but I cannot select the name using the mouse. Nothing happens.
I have to use the keyboard (ENTER) to select the name and then again the mouse to click OK.
I would like to be able to use the mouse for the whole face tagging process (for existing names).

2. Autocompletion of names does not work when typing inside the names combobox.
In the faces detection view, when hovering over a photo and typing into the combobox that then appears, I can only type 1-2 letters before the input field disappears again.
In the combobox that appears below existing faces rectangles in image previews, I can type into the combobox, but the tag selection list does not filter according to my entries. 

3. Many faces are not detected - e.g. side portraits or upside-down faces.
Is this a known issue with the current faces detection library?

4. In the "Search faces" dialog which appears when opening the left side "People" tag and clicking on "Search collection for faces" (approx.), only the radio button option "Detect faces" works (ie. has any result). The second option ("Detect and recognize faces") and third ("Recognize faces") always yield 0 results at once.
I would like Digikam to detect and recognize faces. I have a corpus large enough for it to learn. How do I do this?

Thank you!
Comment 1 Jens 2017-01-21 14:07:27 UTC
Update for #2: Autocomplete works in this case, but is case sensitive. I didn't realize this, can you make it case insensitive (but case retaining, ie. use the case of the word in the list, not the word that was typed)?
Comment 2 Jens 2017-01-21 14:13:59 UTC
5: when browsing images and face tags are active, the first "unknown" face rectangle's bottom combobox (to enter a name) grabs the cursor focus so you cannot navigate using the cursor keys any more.
Possible solution: Do not let the face rectangle combobox grab the keys assigned to "next/previous/..." images, or generally, any non-character key events.
Comment 3 caulier.gilles 2017-01-21 14:39:07 UTC
Face Management recieve a lots of fix recently from Mario.

Please use 5.5.0 pre-release AppImage Bundle located here :

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

...and update this report...

Gilles Caulier
Comment 4 Jens 2017-01-22 10:54:06 UTC
Done. Here is my feedback:

1: fixed
2: works, but typing a "b" opens the batch processing window. New bug. :)
   Also I cannot switch between face rectangles using the mouse. How about "tab"?
3: no difference detected yet.
4: still does not do anything.
5: still not working
Comment 5 caulier.gilles 2017-01-22 11:04:27 UTC
About point 2 :

"b" is the shorcut to open BQM.
"l" is the shortcut to open Light table.

This can be customized in config shortcuts of course, but i think we must disable this kind of top level shortcuts when focus is in tags enter field.

Gilles Caulier
Comment 6 Maik Qualmann 2017-01-22 11:31:22 UTC
2: When you hover a rectangle has the input filed the focus and you can type "b" or "i". This is not reproducible here. After every 2. input the popup disappears is known. It is hard to fix because the hovered graphic element has no valid parent object. The position of the popup is already a hack, but better than the inline completer. And yes, the combobox is not filtererd. The popup is case insensitive filtererd.

Maik
Comment 7 Maik Qualmann 2017-01-22 18:28:43 UTC
*** Bug 375418 has been marked as a duplicate of this bug. ***
Comment 8 Mario Frank 2017-01-24 14:42:53 UTC
Git commit dee3647b19a3e8c44292da19821fa0221400bb04 by Mario Frank.
Committed on 24/01/2017 at 14:37.
Pushed by mfrank into branch 'master'.

Made the detect and recognise pipeline for face detection and recognition work again.
There is some design problem I think. So this is just a workaround.
The faces count in people sidebar should now be refreshed properly when confirming and rejecting faces.
Selecting specific tags to scan for faces included not only the selected tags but the tag paths.
So, faces from the super tags of the selected ones were scanned, too.

M  +7    -0    libs/album/albummanager.cpp
M  +6    -0    utilities/facemanagement/faceutils.cpp
M  +1    -1    utilities/maintenance/facesdetector.cpp
M  +3    -1    utilities/maintenance/imageinfojob.cpp

https://commits.kde.org/digikam/dee3647b19a3e8c44292da19821fa0221400bb04
Comment 9 Mario Frank 2017-01-24 14:47:18 UTC
Hey, had some time to search for the bugs yesterday evening.
The option detect and recognise should work now. There was a bug in fact.
I introduced a workaround until we can take care of the faces management in detail.

I currently see no elegant way to fix the completer problem in line edit.

Concerning not detected faces: the detection algorithms usually need a fitting orientation. So, upside-down faces may be problematic.

Traininig the face recognition is possible by explicitely setting face regions and assigning the identities. And then rebuild the training data. The more images you have, the better the recognition should become.
Comment 10 Jens 2017-01-24 21:33:43 UTC
Great, I'd be happy to test an updated appimage. :) So:

1: fixed
2: no easy workaround for face tag dropdown not working on each second keypress
3: waiting for appimage to test
4: waiting for appimage to test
5: ?

Thank you!
Comment 11 caulier.gilles 2017-01-24 21:40:48 UTC
Jens,

32 and 64 bits AppImage are just updated at usual place to GDrive repository :

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

Gilles Caulier
Comment 12 Jens 2017-01-26 11:14:47 UTC
Perfect, thank you!
With the new appimage, the menubar integration in Ubuntu also works again :-D

Also "detect and recognize" now does detect faces.
But no faces are recognized (although I already have a siginificant face corpus with confirmed faces).
Also with recognized (unconfirmed and confirmed) photos a "group by face" option for the main view would be great, instead of "group by album" etc. Then it becomes much easier to confirm and reorder photos.

Thank you!
Comment 13 caulier.gilles 2017-01-26 11:18:36 UTC
Jens, 

As i know, the Face Recognition algorithm needs at least 4 faces already tagged to be able to recognize automatically a new one.

Gilles Caulier
Comment 14 Sven 2017-01-27 16:55:18 UTC
I was testing the new appimage from Googledrive and must say, you did it. No more crashes when tagging faces.

I did not test detecting faces, but the crashes where really annoying...

Thx
Comment 15 caulier.gilles 2017-01-27 16:58:17 UTC
So, which point still valid for this report ?

Gilles Caulier
Comment 16 Sven 2017-01-27 17:07:55 UTC
Point 1 is working now.
Comment 17 caulier.gilles 2017-01-27 17:14:44 UTC
And 2/, 3/, 4/, 5/ ?

Gilles Caulier
Comment 18 Sven 2017-01-27 17:17:10 UTC
Points 2 - 5 are still valid.

Sorry i got carried away with joy because of no more crashes during tagging..
Comment 19 Stefano 2017-01-27 20:13:28 UTC
Note that point 2 works fine within the People view, but NOT in the album view (last AppImage)
Comment 20 Maik Qualmann 2017-02-18 20:25:30 UTC
*** Bug 376627 has been marked as a duplicate of this bug. ***
Comment 21 Jens 2017-02-19 11:13:01 UTC
Update as of today:

1. Selecting face names with the mouse now works fine.
2. Autocompletion of names when typing works but the dropdown disappears on every second letter. Usable but a little buggy.
3. Still a little buggy. Many faces (e.g. slightly rotated, lying or upside down) are not detected at all. Adjusting the detection sensitivity ("fast" vs "precise") does not improve detection. Most of these faces were detected fine by iPhoto. Is Digikam's OpenCV interface using eye detection first and using the angle between eyes to compute the face rotation before finding the actual face? This was suggested on StackOverflow.
4. fixed
5. Unknown faces still grab the cursor focus when browsing images using preview mode, so they stop the browsing (with keyboard). Maybe the input field should not grab non-alphanumeric key events.

Thank you!
Comment 22 Stefano 2017-02-21 08:11:15 UTC
For 2), keep in mind that there is also a focus problem, because:

1. You create the face region by click and drag
2. The face select appears
3. You start typing, but the select disappears at the first keypress because it seems to detect a "mouse out" event (the mouse position is always outside the select): you have to position the mouse on the select BEFORE typing
Comment 23 Maik Qualmann 2017-04-02 10:37:37 UTC
Git commit fbb0d6387b415cbcb10dab092bbe629b8316d8ae by Maik Qualmann.
Committed on 02/04/2017 at 10:35.
Pushed by mqualmann into branch 'master'.

ugly hack for the tags completer to fix QTBUG-20531

M  +39   -37   libs/tags/addtagscombobox.cpp
M  +0    -10   utilities/facemanagement/assignnamewidget.cpp
M  +0    -2    utilities/facemanagement/assignnamewidget.h
M  +0    -8    utilities/facemanagement/facegroup.cpp

https://commits.kde.org/digikam/fbb0d6387b415cbcb10dab092bbe629b8316d8ae
Comment 24 caulier.gilles 2018-08-28 13:48:57 UTC
Problem still reproducible with AppImage bundle 6.0.0-beta1 available here :

https://files.kde.org/digikam/

Gilles Caulier
Comment 25 caulier.gilles 2018-08-29 07:50:03 UTC
The AppImage Linux Bundle have been recompiled with last changes from source code and is available here for testing :

https://files.kde.org/digikam/

Take a care : 6.0.0 is a beta release for the moment. Make a database backup before to test.

Problem still here ?

Gilles Caulier
Comment 26 Jens 2018-09-02 21:43:26 UTC
6.0-beta1 appimage:

1: (selecting face names from dropdown with mouse) fixed
2: (autocompletion of names when creating face rectangle) fixed
3: (detection of rotated faces) not fixed
4: (only "detect faces" works, "Recognize" doesn't) fixed
5: ("unknown face" rectangle grabs cursor focus when browsing images) not fixed

additional note: when selecting multiple face images in the "People" tab, it is sometimes difficult to get the "right" face to have a name suggestion box below it. If no images are selected, the name suggestion box changes with the mouse cursor position, but if some are selected (using CTRL+click), sometimes this focus "sticks" with a single image, and you have to unselect and reselect some images to get it to focus the correct image.
Comment 27 caulier.gilles 2019-03-20 15:16:11 UTC
After 3 weeks of work, i finally completed the compilation of AppImage using Qt
5.11.3 + QWebkit 5.212.

New 6.1.0 pre-release AppImage bundle can be found here (64 bits only for the
moment) :

https://files.kde.org/digikam/

Please check if this bugzilla entry still valid.

Thanks in advance

Gilles Caulier
Comment 28 Maik Qualmann 2019-03-28 20:35:24 UTC
Point 3 is now also fixed, all possible rotations and flip operations are now possible. It does not matter if you only change the metadata flag or if the image data is rotated.

Point 5 is also fixed, with cursor up/down or image up/down it is possible to scroll over images with unknown face rectangles. I close the bug now, other bug reports cover still existing problems.

Maik
Comment 29 Jens 2019-03-28 21:03:11 UTC
Yes agreed. Thank you!