Bug 269373 - adding face tag to large amount of pictures fails
Summary: adding face tag to large amount of pictures fails
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Faces-Engine (show other bugs)
Version: 2.0.0
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-03-25 10:24 UTC by Rinus Bakker
Modified: 2012-06-27 10:20 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In: 2.1.0
Sentry Crash Report:


Attachments
to illustrate the extremety of the situation (671.92 KB, image/png)
2011-04-28 09:51 UTC, Rinus Bakker
Details
Facepipeline patch (2.70 KB, patch)
2011-05-21 15:50 UTC, Marcel Wiesweg
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rinus Bakker 2011-03-25 10:24:32 UTC
Version:           2.0.0 (using KDE 4.6.1) 
OS:                Linux

In a huge art collection several thousends faces detected. After selecting all added the tag "face". Stops after 1178, second try stops at about 2400, third trial crashes,  4th trial (after relaunch) succeeded to tag all items

Reproducible: Didn't try

Steps to Reproduce:
as described in details

Actual Results:  
as described in details


Will show up only in extreme heavy use
Comment 1 Marcel Wiesweg 2011-03-26 14:44:27 UTC
Please provide the backtrace of the crash.
What does "stop" mean: Stops + you can try again, or freezes and you need to kill?
Comment 2 Rinus Bakker 2011-03-26 15:13:33 UTC
It just stopped doing the job as if it was done, second try the same, 
third try crash. sorry no crash backtrace. Looks very much memory issue 
to me. 1178 done first time, about the same amount second time, crash 
third time. After reboot (memory cleaned up?) succeedes.
Op 26-3-2011 14:44, Marcel Wiesweg schreef:
> https://bugs.kde.org/show_bug.cgi?id=269373
>
>
>
>
>
> --- Comment #1 from Marcel Wiesweg<marcel wiesweg gmx de>   2011-03-26 14:44:27 ---
> Please provide the backtrace of the crash.
> What does "stop" mean: Stops + you can try again, or freezes and you need to
> kill?
>
Comment 3 Marcel Wiesweg 2011-04-24 21:33:39 UTC
I've been thinking about possible problems causing this bug. It's not clear. There is one crash, and two hangs. We'd need backtraces of either situation to clarify things.
Comment 4 Rinus Bakker 2011-04-26 11:24:23 UTC
Op 24-04-11 21:33, Marcel Wiesweg schreef:
> https://bugs.kde.org/show_bug.cgi?id=269373
>
>
>
>
>
> --- Comment #3 from Marcel Wiesweg<marcel wiesweg gmx de>   2011-04-24 21:33:39 ---
> I've been thinking about possible problems causing this bug. It's not clear.
> There is one crash, and two hangs. We'd need backtraces of either situation to
> clarify things.
>
As far as I remenber I was working on a very huge database of art at 
that time. I wonder if such a situation to tag thousends of pictures 
with just the tag ¨face¨ happens ever again in normal life.

If you think it´s worth the effort I can try to reproduce. But I do need 
to install a special debug version of dk, don´t I? Will it not mess up 
my stuff doing so?
Let me know what you think.
Comment 5 Marcel Wiesweg 2011-04-26 22:15:02 UTC
Can you give a rough number how many photos were affected?
There are a couple of thing that scale with O(n).

Maybe I can do some tests myself in the future. I can understand your reluctance to do a special setup.
Comment 6 Rinus Bakker 2011-04-26 22:19:59 UTC
Op 26-04-11 22:15, Marcel Wiesweg schreef:
> https://bugs.kde.org/show_bug.cgi?id=269373
>
>
>
>
>
> --- Comment #5 from Marcel Wiesweg<marcel wiesweg gmx de>   2011-04-26 22:15:02 ---
> Can you give a rough number how many photos were affected?

affected? sorry, do you mean involved?
> There are a couple of thing that scale with O(n).
>
> Maybe I can do some tests myself in the future. I can understand your
> reluctance to do a special setup.
>
Comment 7 Marcel Wiesweg 2011-04-27 21:43:28 UTC
Yes, how many faces were roughly selected when you batch-applied a tag
Comment 8 Rinus Bakker 2011-04-27 22:53:37 UTC
The database holds 136043 pictures  and in those were 36033 faces (of 
course not all real faces) found. You must take in account that the pc 
had been computing for about 8 hours and than without restart I tried 
all 36033 found pictures to give the tag ¨face¨, and than happened as 
described in the bug report.

Op 27-04-11 21:43, Marcel Wiesweg schreef:
> https://bugs.kde.org/show_bug.cgi?id=269373
>
>
>
>
>
> --- Comment #7 from Marcel Wiesweg<marcel wiesweg gmx de>   2011-04-27 21:43:28 ---
> Yes, how many faces were roughly selected when you batch-applied a tag
>
Comment 9 Rinus Bakker 2011-04-28 09:51:22 UTC
Created attachment 59385 [details]
to illustrate the extremety of the situation

I selected them all and added the tag ¨testtag¨, all went fine. I wonder if there is a way to reverse the situation (without computing 8 hours) to as it was just after detection, all the found faces as ¨people/unknown¨  and give them all the same real face tag at once to see what happens.

note the different item counts on the screen which I mentioned in another bug report.
Comment 10 Marcel Wiesweg 2011-05-21 15:50:21 UTC
Created attachment 60204 [details]
Facepipeline patch

Patch to add a flow control at the beginning of the pipeline to limit the amount of inter-thread signals.
I dont commit right now because the code is error-prone and should receive some testing first.
Comment 11 Marcel Wiesweg 2011-08-09 19:58:50 UTC
Git commit f6d5b4c6b900cec754eaee6ff7f58695eb3f35a1 by Marcel Wiesweg.
Committed on 09/08/2011 at 21:56.
Pushed by mwiesweg into branch 'master'.

Limit number of packages in the pipeline at a point of time.

If a large number of packages is added, they are queued to avoid
running out of memory.

As there is no backtrace and no good setup to reproduce, feel free
to reopen if the bug still occurs.

BUG: 269373

M  +26   -2    utilities/facedetection/facepipeline.cpp
M  +5    -0    utilities/facedetection/facepipeline_p.h

http://commits.kde.org/digikam/f6d5b4c6b900cec754eaee6ff7f58695eb3f35a1