Bug 427380

Summary: Crashes when running face detection
Product: [Applications] digikam Reporter: Dominic <register_kde_bugtracking>
Component: Faces-DetectionAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: crash CC: metzpinguin
Priority: NOR    
Version: 7.1.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed In: 7.2.0
Sentry Crash Report:
Attachments: DbgView log
Screenshot of Application Error

Description Dominic 2020-10-06 10:24:23 UTC 
Created attachment 132145 [details]
DbgView log

SUMMARY

I've got around 82K pictures hosted on a local disk.
They're all imported into digiKam and I'm trying to run face detection on them.
Every time I start it, it runs for a while (maybe detecting anywhere between 100 and 1000 photos) and then it crashes.  No message, window just vanishes.

It leaves the database in a bad state so next start up is slow and errors, which clearly triggers some sort of clean up and then it starts cleanly.

SOFTWARE/OS VERSIONS
Windows: Windows 10 Build 19041.508 

ADDITIONAL INFORMATION

I've tried the latest 7.2 beta release (I see bug 426236 that is hopefully included in that build).  It still crashes.

I've attached the output from DbgView but it doesn't look very helpful.
It mentions the location of a crashlog file but there is no file there.
Comment 1 Maik Qualmann 2020-10-06 10:58:48 UTC 
Have you really tested the last available digiKam-7.2.0-beta1?

digiKam-7.2.0-beta1-20201005T233421-Win64.exe

Maik
Comment 2 Dominic 2020-10-06 12:32:19 UTC 
I downloaded and installed this:

digiKam-7.2.0-beta1-20201005T233421-Win64.exe
Comment 3 Dominic 2020-10-06 16:26:52 UTC 
Just as an update.  The beta build is crashing far less often than 7.1.0.
I was getting a crash every 5-15 minutes I would say.
The 7.2.0 beta has only crash twice in the last 3 or 4 hours.
Comment 4 Maik Qualmann 2020-10-06 16:35:54 UTC 
Did you attention to the memory usage, how high it was? At the moment we are holding up to 50 image objects in the queue for the face engine, which can be many gigabytes depending on the image size. I will reduce this number in the queue for testing purposes.

Maik
Comment 5 Dominic 2020-10-06 16:50:44 UTC 
Memory wasn't bad (the images are mostly 2-10MB).
I watched it intermittently and it never went over about 1.2Gb of RAM (the machine has at least 6Gb free with this running so there's no pressure).
Comment 6 Maik Qualmann 2020-10-06 18:39:02 UTC 
Git commit 99dbe70e85912aca5f0d9dd7f5e36cd1343aac4e by Maik Qualmann.
Committed on 06/10/2020 at 18:38.
Pushed by mqualmann into branch 'master'.

reduce the packages in the face pipeline to save memory

M  +1    -1    core/utilities/facemanagement/threads/facepipeline_p.cpp

https://invent.kde.org/graphics/digikam/commit/99dbe70e85912aca5f0d9dd7f5e36cd1343aac4e
Comment 7 Maik Qualmann 2020-10-08 11:46:45 UTC 
A new build of digiKam-7.2.0-beta1 is available. Can you please test whether the problem can still be reproduced?

https://files.kde.org/digikam/digiKam-7.2.0-beta1-20201008T123957-Win64.exe

Maik
Comment 8 Dominic 2020-10-08 13:16:42 UTC 
Yup, still crashes, but in a different way.
I get a standard windows Application Error dialog appearing (I'll attach a screenshot).

I'll also run again to see if it fails in the previous way sometimes.
Comment 9 Dominic 2020-10-08 13:17:52 UTC 
Created attachment 132209 [details]
Screenshot of Application Error
Comment 10 Dominic 2020-10-08 13:36:41 UTC 
It's still crashing in the other way as well.  Windows just vanished with no error dialog or anything like that appearing.
Comment 11 Dominic 2020-10-08 13:37:44 UTC 
Sorry, disregard that last comment (2020-10-08 13:36:41 UTC).  I don't know that to true just now - my mistake.
Comment 12 Dominic 2020-10-08 13:57:23 UTC 
OK.  I can confirm it's still crashing by totally vanishing.

In the windows event log I found this:

Log Name:      Application
Source:        Application Error
Date:          08/10/2020 14:43:58
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      badger
Description:
Faulting application name: digikam.exe, version: 0.0.0.0, time stamp: 0x00000000
Faulting module name: libexiv2.dll, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x0000000000029db9
Faulting process ID: 0x1aa8
Faulting application start time: 0x01d69d77fb8bd15e
Faulting application path: C:\Program Files\digiKam\digikam.exe
Faulting module path: C:\Program Files\digiKam\libexiv2.dll
Report ID: d359cd6f-0f37-42d3-ba01-b08e39d5d9e6
Faulting package full name: 
Faulting package-relative application ID: 
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>100</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2020-10-08T13:43:58.3616891Z" />
    <EventRecordID>10857</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>Application</Channel>
    <Computer>badger</Computer>
    <Security />
  </System>
  <EventData>
    <Data>digikam.exe</Data>
    <Data>0.0.0.0</Data>
    <Data>00000000</Data>
    <Data>libexiv2.dll</Data>
    <Data>0.0.0.0</Data>
    <Data>00000000</Data>
    <Data>c0000005</Data>
    <Data>0000000000029db9</Data>
    <Data>1aa8</Data>
    <Data>01d69d77fb8bd15e</Data>
    <Data>C:\Program Files\digiKam\digikam.exe</Data>
    <Data>C:\Program Files\digiKam\libexiv2.dll</Data>
    <Data>d359cd6f-0f37-42d3-ba01-b08e39d5d9e6</Data>
    <Data>
    </Data>
    <Data>
    </Data>
  </EventData>
</Event>
Comment 13 Dominic 2020-10-08 14:20:40 UTC 
It feels like random.  
This time the crash is in a different library.


Log Name:      Application
Source:        Application Error
Date:          08/10/2020 15:14:28
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      badger
Description:
Faulting application name: digikam.exe, version: 0.0.0.0, time stamp: 0x00000000
Faulting module name: Qt5Core.dll, version: 5.15.1.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x000000000001b30d
Faulting process ID: 0x2a4c
Faulting application start time: 0x01d69d7b0ba4fbc2
Faulting application path: C:\Program Files\digiKam\digikam.exe
Faulting module path: C:\Program Files\digiKam\Qt5Core.dll
Report ID: 9369758b-b70c-4002-a197-ce2160eed74d
Faulting package full name: 
Faulting package-relative application ID: 
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>100</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2020-10-08T14:14:28.8816900Z" />
    <EventRecordID>10942</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>Application</Channel>
    <Computer>badger</Computer>
    <Security />
  </System>
  <EventData>
    <Data>digikam.exe</Data>
    <Data>0.0.0.0</Data>
    <Data>00000000</Data>
    <Data>Qt5Core.dll</Data>
    <Data>5.15.1.0</Data>
    <Data>00000000</Data>
    <Data>c0000005</Data>
    <Data>000000000001b30d</Data>
    <Data>2a4c</Data>
    <Data>01d69d7b0ba4fbc2</Data>
    <Data>C:\Program Files\digiKam\digikam.exe</Data>
    <Data>C:\Program Files\digiKam\Qt5Core.dll</Data>
    <Data>9369758b-b70c-4002-a197-ce2160eed74d</Data>
    <Data>
    </Data>
    <Data>
    </Data>
  </EventData>
</Event>
Comment 14 Maik Qualmann 2020-10-08 14:41:49 UTC 
You should find a crash log in AppData\Local that might help us a little better:

C:\Users\%USERNAME%\AppData\Local\digikam_crash.log

Maik
Comment 15 Dominic 2020-10-08 14:44:31 UTC 
I don't have a crash dump there but I do have some useful sounding files in AppData/Local/CrashDumps...

Attaching one of those - hopefully helpful.
Comment 16 Dominic 2020-10-08 14:50:58 UTC 
Ah, nope.  Those dumps are 50Mb (12Mb zipped) so can't attach them.
Comment 17 Dominic 2020-10-08 14:51:47 UTC 
Is there some setting required to trigger the creation of the crash log files?
Comment 18 Maik Qualmann 2020-10-08 14:58:23 UTC 
@Gilles, will the crash log only be created with the debug version of Windows?

Maik
Comment 19 Maik Qualmann 2020-10-08 15:01:33 UTC 
@Dominic, the Windows crash logs don't really help, although we can see that it crashes in libexiv2 or in Qt. Do you have special images in your collection, large TIFF files (gigapixel) or something similar?

Maik
Comment 20 Dominic 2020-10-08 15:07:20 UTC 
No, nothing special at all in my photo collection, and I suspect that whatever is making it crash, it subsequently gets past somehow as it's gradually making progress.  When I first reported this it had scanned about 20% of my collection and now it's at 80%.

I looked at a few of the debug logs when it was first happening and it was not the case that the same images kept cropping up.  It was different ones each time it crashed as far as I could tell.
Comment 21 Maik Qualmann 2021-01-05 11:43:59 UTC 
Git commit df6e4fcf0157246f6d1d646f6d768d2b27e48f89 by Maik Qualmann.
Committed on 05/01/2021 at 11:42.
Pushed by mqualmann into branch 'master'.

this could be a cause of the crash in the preview loader
Related: bug 429307, bug 421043, bug 427333

M  +4    -0    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +5    -0    core/libs/threadimageio/preview/previewtask.cpp
M  +2    -0    core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/graphics/digikam/commit/df6e4fcf0157246f6d1d646f6d768d2b27e48f89
Comment 22 Maik Qualmann 2021-01-14 06:35:31 UTC 
Git commit d62937a9e0c3a496f8ccbdbe72835d08899d6718 by Maik Qualmann.
Committed on 14/01/2021 at 06:33.
Pushed by mqualmann into branch 'master'.

the DynamicThread should not be a QRunnable
Otherwise it is a bit strange that a QRunnable
started a QRunnable, only the QThreadPool should do this.
Related: bug 429307, bug 421043, bug 427333

M  +2    -3    core/libs/threads/dynamicthread.cpp
M  +7    -3    core/libs/threads/dynamicthread.h

https://invent.kde.org/graphics/digikam/commit/d62937a9e0c3a496f8ccbdbe72835d08899d6718
Comment 23 Maik Qualmann 2021-01-14 06:50:17 UTC 
Git commit 46d4abd0cfcbdbdbab1f1189186e1737caec2839 by Maik Qualmann.
Committed on 14/01/2021 at 06:49.
Pushed by mqualmann into branch 'master'.

call wait() after stop() before call start() again
Otherwise a running task has not yet ended and and
is deleted when the QThreadPool is started again.
Related: bug 429307, bug 421043, bug 427333

M  +1    -0    core/utilities/facemanagement/threads/facepreviewloader.cpp

https://invent.kde.org/graphics/digikam/commit/46d4abd0cfcbdbdbab1f1189186e1737caec2839