Bug 425063

Summary:	Detect Faces crashes the App
Product:	[Applications] digikam	Reporter:	Andy Gavin <registrations>
Component:	Faces-Detection	Assignee:	Digikam Developers <digikam-bugs-null>
Status:	RESOLVED FIXED
Severity:	crash	CC:	caulier.gilles, metzpinguin
Priority:	NOR
Version:	7.0.0
Target Milestone:	---
Platform:	Microsoft Windows
OS:	Microsoft Windows
Latest Commit:		Version Fixed In:	7.2.0
Sentry Crash Report:
Attachments:	MS Debug Crash Log MS Debug capture with environment variables set Reproducing, another run, No Reboot MS Debug Crash Log for 32bit Memory

Description Andy Gavin 2020-08-06 08:25:09 UTC

SUMMARY

Its a new install, with a fresh Database of approx 40k photos added. Detect faces, with Skip already scanned select won't complete, it crashes every 1-2 %. There is no feedback, it just closes. I have rebooted a few times to check its a fresh system. I have reached 18% with about 10 crashes so far.

STEPS TO REPRODUCE
1. Open App
2. Wait for startup check to finish
3. Click Detect Faces (with skip images already scanned selected)

OBSERVED RESULT
App will either close almost immediately or try for a little while, achieve 1-2% more and then close. I am getting results, so its doing something.

EXPECTED RESULT
It to finish


SOFTWARE/OS VERSIONS
Windows: Windows 10 Pro 64bit
AMD Ryzen 7 2700X 16Gb RAM

ADDITIONAL INFORMATION

Comment 1 Maik Qualmann 2020-08-06 21:09:32 UTC

Set environment variables in Windows:

Variable: QT_LOGGING_RULES
Value: digikam.*=true

Download DebugView from Microsoft and start it. Post the DebugView messages from start of digiKam to crash.

Maik

Comment 2 Andy Gavin 2020-08-07 08:59:28 UTC

Created attachment 130702 [details]
MS Debug Crash Log

The crash happened at approx 9:38. It doesn't seem to capture anything about it.

This was a fresh boot, first run scenario. It ran face detection from 20% to 24% before closing itself.

Comment 3 Andy Gavin 2020-08-07 09:02:03 UTC

Sorry, forgot to set the env variables. I'll do it again.

Comment 4 Andy Gavin 2020-08-07 09:29:43 UTC

Created attachment 130705 [details]
MS Debug capture with environment variables set

Ignore the first attachment. I forgot to set the env variables.

Again, with a fresh boot. it ran for about 1 more percentage and closed itself.

Comment 5 Andy Gavin 2020-08-07 09:46:54 UTC

Created attachment 130708 [details]
Reproducing, another run, No Reboot

I ran it again without a reboot to see if it crashed on the same photo. Which it didn't.

As the previous logs last line was an issue with Logitech Keyboard software I update the Logitech software first to eliminate that as a cause.

It made no difference. Digikam closed itself again before completing another percent.

Comment 6 Maik Qualmann 2020-08-07 12:17:01 UTC

Some JPG files seem to be broken and throw an exception. We know that the exception jump does not currently work with the MinGW compiler we are using at 64 bit. Can you please install the 32 bit digiKam to test whether the problem does not occur there?

------------
Invalid SOS parameters for sequential JPEG  ( -1 )
------------

Maik

Comment 7 Maik Qualmann 2020-08-07 12:38:20 UTC

@Gilles
Is there a macro to recognize the MinGW compiler, or do I have to build something out of Q_OS_WIN and not MSVC? Or use DrMinGW macro?

Maik

Comment 8 Andy Gavin 2020-08-07 12:48:58 UTC

Created attachment 130710 [details]
MS Debug Crash Log for 32bit

Sure.

I Uninstalled 64bit, installed 32bit. Rebooted. Face detection bombed out again. Debug attached.

Comment 9 Maik Qualmann 2020-08-07 12:56:37 UTC

With 32 bit there seems to be a problem with the allocation of memory, that such a small value is not available, I can't imagine, no idea at the moment.

-----------
digikam.dimg: Failed to allocate chunk of memory of size 55987200	
digikam.dimg.jpeg: Cannot allocate memory!	
-----------

Maik

Comment 10 caulier.gilles 2020-08-07 12:59:45 UTC

Maik,

Look in libraw which support well Mingw, it use this low level preprocessor macro

https://invent.kde.org/graphics/digikam/-/blob/master/core/libs/rawengine/libraw/src/libraw_datastream.cpp#L19

This sound like this macro is defined in 32 and 64 bits version of cross-compiler.

Gilles

Comment 11 Andy Gavin 2020-08-07 13:10:49 UTC

Created attachment 130712 [details]
Memory

I can't imagine either. Plenty available.  I tried again, crashed with the same...

00020810	14:01:25	[24528] digikam.dimg: Failed to allocate chunk of memory of size 55987200	
00020811	14:01:25	[24528] digikam.dimg.jpeg: Cannot allocate memory!	
00020812	14:01:25	[24528] digikam.dimg: "F:/Pictures/Photos/Holiday Cuba 2012/P1000072.JPG" : Cannot load file !!!	
00020813	14:01:25	[24528] digikam.general: Cannot extract preview for "F:/Pictures/Photos/Holiday Cuba 2012/P1000072.JPG"

Comment 12 Maik Qualmann 2020-08-08 19:08:11 UTC

Git commit fc22d48c0b59828bbd1229a023c3edc5b50980b8 by Maik Qualmann.
Committed on 08/08/2020 at 19:06.
Pushed by mqualmann into branch 'master'.

try with built in setjmp/longjmp function for MinGW
Related: bug 425096

M  +8    -0    core/dplugins/dimg/jpeg/dimgjpegloader.cpp
M  +8    -0    core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp
M  +8    -0    core/dplugins/dimg/jpeg/dimgjpegloader_save.cpp
M  +16   -0    core/libs/jpegutils/jpegutils.cpp

https://invent.kde.org/graphics/digikam/commit/fc22d48c0b59828bbd1229a023c3edc5b50980b8

Comment 13 Andy Gavin 2020-08-10 08:00:47 UTC

Hi Mark,

can you tell me how to try that please ?

Comment 14 Andy Gavin 2020-08-10 08:03:20 UTC

Sorry, I meant Maik :-)

Comment 15 Maik Qualmann 2020-08-10 08:30:08 UTC

You have to wait until a new pre-release of digiKam-7.1.0 is published here:

https://files.kde.org/digikam/

Gilles build the bundles, he is currently on holiday. It will take some time. My build for Windows is out of date and it takes many hours to compile, maybe I'll create an up-to-date bundle. I'll post it here.

Maik

Comment 16 caulier.gilles 2020-08-10 10:03:33 UTC

Maik, Andy, 

I go back at home around 22 august. I will restart all bundles process at this time.

Here, i use a reduced network bandwidth and i cannot build all bundles.

Gilles

Comment 17 Andy Gavin 2020-08-10 10:24:07 UTC

ok thanks :-)

Comment 18 Andy Gavin 2020-09-10 08:36:08 UTC

Hi,
I installed the 64bit 7.1 update to see how that behaved with regards to this issue. I saw it had a few face recognition tickets in the change log.

It does seem to run longer than previously. It made it from 28% to 83% but again just closed. I have reproduced it twice.

The log was massive as it ran for hours, below are the last lines. The query it ran just before closing I tried myself with SQLite. it results in 0 records.

There was no obvious error message.

21369	08:24:05	[213316]	digikam.dimg: "F:/Pictures/Photos/Vickys Pictures/Another old Laptop/Samsung PC Studio 3/Photo/Photo-0083.jpg" : "JPEG" file identified
21370	08:24:05	[213316]	digikam.dimg.jpeg: Start of Image  ( 1 )
21371	08:24:05	[213316]	digikam.dimg.jpeg: Miscellaneous marker 0xe1, length 270  ( 1 )
21372	08:24:05	[213316]	digikam.dimg.jpeg: Define Quantization Table 0  precision 0  ( 1 )
21373	08:24:05	[213316]	digikam.dimg.jpeg: Define Quantization Table 1  precision 0  ( 1 )
21374	08:24:05	[213316]	digikam.dimg.jpeg: Start Of Frame 0xc0: width=800, height=600, components=3  ( 1 )
21375	08:24:05	[213316]	digikam.dimg.jpeg:     Component 1: 2hx1v q=0  ( 1 )
21376	08:24:05	[213316]	digikam.dimg.jpeg:     Component 2: 1hx1v q=1  ( 1 )
21377	08:24:05	[213316]	digikam.dimg.jpeg:     Component 3: 1hx1v q=1  ( 1 )
21378	08:24:05	[213316]	digikam.dimg.jpeg: Define Huffman Table 0x00  ( 1 )
21379	08:24:05	[213316]	digikam.dimg.jpeg:           0   1   5   1   1   1   1   1  ( 2 )
21380	08:24:05	[213316]	digikam.dimg.jpeg:           1   0   0   0   0   0   0   0  ( 2 )
21381	08:24:05	[213316]	digikam.dimg.jpeg: Define Huffman Table 0x01  ( 1 )
21382	08:24:05	[213316]	digikam.dimg.jpeg:           0   3   1   1   1   1   1   1  ( 2 )
21383	08:24:05	[213316]	digikam.dimg.jpeg:           1   1   1   0   0   0   0   0  ( 2 )
21384	08:24:05	[213316]	digikam.dimg.jpeg: Define Huffman Table 0x10  ( 1 )
21385	08:24:05	[213316]	digikam.dimg.jpeg:           0   2   1   3   3   2   4   3  ( 2 )
21386	08:24:05	[213316]	digikam.dimg.jpeg:           5   5   4   4   0   0   1 125  ( 2 )
21387	08:24:05	[213316]	digikam.dimg.jpeg: Define Huffman Table 0x11  ( 1 )
21388	08:24:05	[213316]	digikam.dimg.jpeg:           0   2   1   2   4   4   3   4  ( 2 )
21389	08:24:05	[213316]	digikam.dimg.jpeg:           7   5   4   4   0   1   2 119  ( 2 )
21390	08:24:05	[213316]	digikam.dimg.jpeg: Start Of Scan: 3 components  ( 1 )
21391	08:24:05	[213316]	digikam.dimg.jpeg:     Component 1: dc=0 ac=0  ( 1 )
21392	08:24:05	[213316]	digikam.dimg.jpeg:     Component 2: dc=1 ac=1  ( 1 )
21393	08:24:05	[213316]	digikam.dimg.jpeg:     Component 3: dc=1 ac=1  ( 1 )
21394	08:24:05	[213316]	digikam.dimg.jpeg:   Ss=0, Se=63, Ah=0, Al=0  ( 1 )
21395	08:24:05	[213316]	digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
21396	08:24:05	[213316]	digikam.dimg.jpeg: End Of Image  ( 1 )
21397	08:24:05	[213316]	digikam.general: Try to get preview from "F:/Pictures/Photos/Vickys Pictures/Another old Laptop/Samsung PC Studio 3/Photo/Photo-0084.jpg"
21398	08:24:05	[213316]	digikam.general: Preview quality:  1
21399	08:24:05	[213316]	digikam.dimg: "F:/Pictures/Photos/Vickys Pictures/Another old Laptop/Samsung PC Studio 3/Photo/Photo-0084.jpg" : "JPEG" file identified
21400	08:24:05	[213316]	digikam.dimg.jpeg: Start of Image  ( 1 )
21401	08:24:05	[213316]	digikam.dimg.jpeg: Miscellaneous marker 0xe1, length 270  ( 1 )
21402	08:24:05	[213316]	digikam.dimg.jpeg: Define Quantization Table 0  precision 0  ( 1 )
21403	08:24:05	[213316]	digikam.dimg.jpeg: Define Quantization Table 1  precision 0  ( 1 )
21404	08:24:05	[213316]	digikam.dimg.jpeg: Start Of Frame 0xc0: width=800, height=600, components=3  ( 1 )
21405	08:24:05	[213316]	digikam.dimg.jpeg:     Component 1: 2hx1v q=0  ( 1 )
21406	08:24:05	[213316]	digikam.dimg.jpeg:     Component 2: 1hx1v q=1  ( 1 )
21407	08:24:05	[213316]	digikam.dimg.jpeg:     Component 3: 1hx1v q=1  ( 1 )
21408	08:24:05	[213316]	digikam.dimg.jpeg: Define Huffman Table 0x00  ( 1 )
21409	08:24:05	[213316]	digikam.dimg.jpeg:           0   1   5   1   1   1   1   1  ( 2 )
21410	08:24:05	[213316]	digikam.dimg.jpeg:           1   0   0   0   0   0   0   0  ( 2 )
21411	08:24:05	[213316]	digikam.dimg.jpeg: Define Huffman Table 0x01  ( 1 )
21412	08:24:05	[213316]	digikam.dimg.jpeg:           0   3   1   1   1   1   1   1  ( 2 )
21413	08:24:05	[213316]	digikam.dimg.jpeg:           1   1   1   0   0   0   0   0  ( 2 )
21414	08:24:05	[213316]	digikam.dimg.jpeg: Define Huffman Table 0x10  ( 1 )
21415	08:24:05	[213316]	digikam.dimg.jpeg:           0   2   1   3   3   2   4   3  ( 2 )
21416	08:24:05	[213316]	digikam.dimg.jpeg:           5   5   4   4   0   0   1 125  ( 2 )
21417	08:24:05	[213316]	digikam.dimg.jpeg: Define Huffman Table 0x11  ( 1 )
21418	08:24:05	[213316]	digikam.dimg.jpeg:           0   2   1   2   4   4   3   4  ( 2 )
21419	08:24:05	[213316]	digikam.dimg.jpeg:           7   5   4   4   0   1   2 119  ( 2 )
21420	08:24:05	[213316]	digikam.dimg.jpeg: Start Of Scan: 3 components  ( 1 )
21421	08:24:05	[213316]	digikam.dimg.jpeg:     Component 1: dc=0 ac=0  ( 1 )
21422	08:24:05	[213316]	digikam.dimg.jpeg:     Component 2: dc=1 ac=1  ( 1 )
21423	08:24:05	[213316]	digikam.dimg.jpeg:     Component 3: dc=1 ac=1  ( 1 )
21424	08:24:05	[213316]	digikam.dimg.jpeg:   Ss=0, Se=63, Ah=0, Al=0  ( 1 )
21425	08:24:05	[213316]	digikam.general: One job is done
21426	08:24:05	[213316]	digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
21427	08:24:05	[213316]	digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
21428	08:24:05	[213316]	digikam.dimg.jpeg: End Of Image  ( 1 )
21429	08:24:05	[213316]	digikam.general: Using  16  CPU core to run threads
21430	08:24:05	[213316]	digikam.general: Action Thread run  1  new jobs
21431	08:24:05	[213316]	digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) ) "
21432	08:24:05	[213316]	digikam.database: Search query:
21433	08:24:05	[213316]	"SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images        INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );"
21434	08:24:05	[213316]	(QVariant(int, 1), QVariant(int, 1), QVariant(QString, "autodetectedPerson"), QVariant(int, 1), QVariant(int, 1), QVariant(QString, "autodetectedFace"), QVariant(int, 1), QVariant(int, 1), QVariant(QString, "tagRegion"))

Comment 19 Maik Qualmann 2020-10-03 19:38:55 UTC

Fixed with bug 426175.

Maik