I am not sure if the actual issue is related to face recognition but it crashes when I start it SUMMARY Local collection on Z:\ drive that is actually a permanent link to a SMB network attached storage collection contains several tenthousands of photos Database stored on local PC thumbnails-digikam.db 1.5GB similarity.db 36MB recognition.db 24 MB digikam4.db STEPS TO REPRODUCE 1. start face recognition on several thousand images 2. 3. OBSERVED RESULT digikam crashes without any error message EXPECTED RESULT digikam stops the operation with an meaningful error message SOFTWARE/OS VERSIONS Windows: 10 (1909) 64-bit digikam version 6.4.0 CPU cores: 4 Eigen: 3.3.7 Exiv2: 0.27.2 Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes HEIF encoding support: Yes ImageMagick codecs: 7.0.8 KF5: 5.61.0 LensFun: 0.3.95-0 LibCImg: 130 LibJPEG: 90 LibJasper: 2.0.16 LibLCMS: 2090 LibLqr support: No LibPGF: 7.19.03 LibPNG: 1.6.37 LibRaw: 0.19.5 LibTIFF: 4.0.10 Marble: 0.27.20 Parallelized demosaicing: No Qt: 5.13.1 Qt Webkit support: Yes VKontakte support: No AkonadiContact support: No Baloo support: No Calendar support: Yes DBus support: No Database backend: QSQLITE HTML Gallery support: Yes LibAVCodec: 58.54.100 LibAVFormat: 58.29.100 LibAVUtil: 56.31.100 LibGphoto2 support: No LibOpenCV: 3.4.7 LibQtAV: 1.13.0 Media player support: Yes Panorama support: Yes ADDITIONAL INFORMATION Dr.Mingw log digikam.exe caused an Access Violation at location 00007FFBC712594D in module libdigikamcore.dll Reading from location 000002201CDBEC8E. AddrPC Params 00007FFBC712594D 0000000000000000 0000021EB43AD010 0000021EB29D9C70 libdigikamcore.dll!dimgScaleAARGB [/home/gilles/dktemp/digikam-v6.4.0/core/libs/dimg/dimg_scale.cpp @ 1484] 00007FFBC712880E 0000000000000002 00007FFC277C083D 0000000000000002 libdigikamcore.dll!smoothScaleClipped [/home/gilles/dktemp/digikam-v6.4.0/core/libs/dimg/dimg_scale.cpp @ 255] 00007FFBC71288E2 00000072899FBB78 0000021EBFA26010 0000021ECD9C7010 libdigikamcore.dll!smoothScaleClipped [/home/gilles/dktemp/digikam-v6.4.0/core/libs/dimg/dimg_scale.cpp @ 187] 00007FFBC7128969 00000072899FBB80 0000021EA3CE0000 00000072899FBB80 libdigikamcore.dll!smoothScale [/home/gilles/dktemp/digikam-v6.4.0/core/libs/dimg/dimg_scale.cpp @ 176] 00007FFBC71289C1 00000072899FBBB0 0000021EBFA5D190 00000072899FBB98 libdigikamcore.dll!smoothScale [/home/gilles/dktemp/digikam-v6.4.0/core/libs/dimg/dimg_scale.cpp @ 181] 00007FFBADFC4A59 00000072899FBBA0 0000021EAE82BF48 00000072899FBBB0 libdigikamgui.dll!scaleForDetection [/home/gilles/dktemp/digikam-v6.4.0/core/utilities/facemanagement/faceworkers.cpp @ 61] 00007FFBADFC52C0 0000021EAE82BF30 000000006695084B 0000000000000072 libdigikamgui.dll!process [/home/gilles/dktemp/digikam-v6.4.0/core/utilities/facemanagement/faceworkers.cpp @ 43] 00007FFBADFD1AF3 0000021EA9DE7C70 0000021EACD68040 0000021EA9DE7480 libdigikamgui.dll!qt_static_metacall [/home/gilles/dktemp/digikam-v6.4.0/build.mxe/core/utilities/facemanagement/facemanagement_src_autogen/EWIEGA46WW/moc_faceworkers.cpp @ 88] 0000000066954A9A 0000007288FFF3A0 0000021EA591BF10 0000021EB3594F40 Qt5Core.dll!QObject::event 0000000001087ADC 0000021EA9DE75D0 0000021EADBF7F70 000000000000006D Qt5Widgets.dll!QApplicationPrivate::notify_helper 000000000108E9E3 0000021EAE2BA340 00007FFC277BFC11 0000049400000003 Qt5Widgets.dll!QApplication::notify 00000000669280DA 0000021EA9DE71C0 0000021EACD68040 0000021EA9DEAF30 Qt5Core.dll!QCoreApplication::notifyInternal2 000000006692F15C 0000000000000401 0000000000060BFC 0000000000000000 Qt5Core.dll!QCoreApplicationPrivate::sendPostedEvents 00000000669817B6 0000021EB1A83100 00007FFC272F2843 0000000000000001 Qt5Core.dll!QEventDispatcherWin32Private::sendTimerEvent 00007FFC272E63ED 0000021EA42393A0 0000000066981590 0000000000060BFC USER32.dll!UserCallWinProcCheckWow 00007FFC272E5DE2 0000000066981590 0000000000000000 00000072899FC5B0 USER32.dll!DispatchMessageWorker 0000000066980F7A 0000021EB0918110 0000007200000024 0000021EB30A9DF0 Qt5Core.dll!QEventDispatcherWin32::processEvents 0000000066926A34 0000021EAE82BF30 0000000000000000 0000000000000000 Qt5Core.dll!QEventLoop::exec 00007FFBC720E392 0000021EB11D1F90 000000006675E6FF 0000000000000010 libdigikamcore.dll!run [/home/gilles/dktemp/digikam-v6.4.0/core/libs/threads/threadmanager.cpp @ 193] 0000000066764E00 0000021EB0F8D370 0000000000000000 0000000000000000 Qt5Core.dll!QThreadPool::cancel 000000006675DC8D 0000000000000000 0000000000000000 0000000000000000 Qt5Core.dll!QThread::qt_metacall 00007FFC26297BD4 0000000000000000 0000000000000000 0000000000000000 KERNEL32.DLL!BaseThreadInitThunk 00007FFC277ECED1 0000000000000000 0000000000000000 0000000000000000 ntdll.dll!RtlUserThreadStart
Intel Core i3 2.26GHz 8 GB RAM
Can you please test the last digikam-7.0.0-Beta2 from here: https://files.kde.org/digikam/ Maik
same issue with digikam version 7.0.0beta2 - 20200115 CPU cores: 4 Eigen: 3.3.7 Exiv2: 0.27.2 Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes HEIF encoding support: Yes ImageMagick codecs: 7.0.8 KF5: 5.65.0 LensFun: 0.3.95-0 LibCImg: 130 LibJPEG: 90 LibJasper: 2.0.16 LibLCMS: 2090 LibLqr support: No LibPGF: 7.19.03 LibPNG: 1.6.37 LibRaw: 0.20.0 LibTIFF: 4.1.0 Marble: 0.27.20 Parallelized demosaicing: No Qt: 5.14.0 Qt Webkit support: Yes VKontakte support: No AkonadiContact support: No Baloo support: No Calendar support: Yes DBus support: No Database backend: QSQLITE HTML Gallery support: Yes LibAVCodec: 58.54.100 LibAVFormat: 58.29.100 LibAVUtil: 56.31.100 LibGphoto2 support: No LibOpenCV: 3.4.9 LibQtAV: 1.13.0 Media player support: Yes Panorama support: Yes digikam.exe caused an Access Violation at location 000000000362539E in module libopencv_imgproc349.dll Reading from location 00000207224B6001. AddrPC Params 000000000362539E 00000037129FB950 00000037129FB990 00000037129FBEE8 libopencv_imgproc349.dll!cv::UMat::operator() 000000000356C067 000002059C950000 0000000000000003 000000000000007C libopencv_imgproc349.dll!cv::hal::sepFilter2D 00000000035699D6 0000000000000000 0000000000000000 00000037129FBCE0 libopencv_imgproc349.dll!cv::hal::cvtBGRtoBGR 0000000003523A61 0000000000000000 000002059C950000 000002059C9502A4 libopencv_imgproc349.dll!cv::calcBackProject 000000000359F15C 0000000000000002 00000205A8005640 00000205A6A1DCF0 libopencv_imgproc349.dll!cv::cvtColor 00007FFBC054498B 03E88CB3C9484E2B 0000000066CB0FA0 0000000000000066 libdigikamcore.dll!prepareForDetection [/home/gilles/dktemp/digikam-master/core/libs/facesengine/detection/opencv-dnn/opencvdnnfacedetector.cpp @ 96] 00007FFBC053D23E 0000000000000064 00000205A8778A80 00000205A5F338C0 libdigikamcore.dll!detectFaces [/home/gilles/dktemp/digikam-master/core/libs/facesengine/detection/facedetector.cpp @ 265] 00007FFBA9BBD0B6 00000205B2CF0270 000000006695A7CB 00000205A5F335F0 libdigikamgui.dll!process [/home/gilles/dktemp/digikam-master/core/utilities/facemanagement/workers/detectionworker.cpp @ 55] 00007FFBA9BA70C3 00000205A6050590 000000006B446346 0000000000000061 libdigikamgui.dll!qt_static_metacall [/home/gilles/dktemp/digikam-master/build.mxe/core/utilities/facemanagement/facemanagement_src_autogen/WLPROMMMF6/moc_detectionworker.cpp @ 88] 000000006695F6D7 00000037105FFAE0 000002059E53E690 00000205A7306A70 Qt5Core.dll!QObject::event 0000000000D179D0 0000000000000000 00000037129FC548 0000000066CB0C80 Qt5Widgets.dll!QApplicationPrivate::notify_helper 0000000000D1E973 00000205AB39F9E0 00000037129FC728 0000000000000061 Qt5Widgets.dll!QApplication::notify 000000006693285A 0000000000000060 00000205A6050800 00000205A5F3A680 Qt5Core.dll!QCoreApplication::notifyInternal2 0000000066938AC5 00000205B7C0E4B0 0000020500000004 00000037129FC780 Qt5Core.dll!QCoreApplicationPrivate::sendPostedEvents 0000000066989FDE 00000205B7C0E4B0 00000037129FFA00 00000205B300C770 Qt5Core.dll!QEventDispatcherWin32::processEvents 0000000066931214 00000205B2CF0270 0000000000000003 00000037129FF940 Qt5Core.dll!QEventLoop::exec 00007FFBC04787C2 00000205B165A2D0 000000006675F83F 0000000000000010 libdigikamcore.dll!run [/home/gilles/dktemp/digikam-master/core/libs/threads/threadmanager.cpp @ 202] 0000000066765AC0 00000205A5618870 0000000000000000 0000000000000000 Qt5Core.dll!QThreadPool::cancel 000000006675EDBE 0000000000000000 0000000000000000 0000000000000000 Qt5Core.dll!QThread::qt_metacall 00007FFC26297BD4 0000000000000000 0000000000000000 0000000000000000 KERNEL32.DLL!BaseThreadInitThunk 00007FFC277ECED1 0000000000000000 0000000000000000 0000000000000000 ntdll.dll!RtlUserThreadStart
It is not the same problem. It crashes in OpenCV on an image with an alpha channel. Can you identify the image and possibly make it available? Maik
Thank you for the fast reply. How do I identify the last processed image after the crash? After a fresh start, all views as somehow sorted (by name, date, album,...) Can I enable some logging?
My pictures are stored on a NAS, so I was able to identify the picture by using Wireshark. It is a stitched panorama (using PTGui) with 32969x36403 pixel (1200.17Mpx) / 146 MByte. You can find the image at https://web.tresorit.com/l#GlHC02W2ld3zPL4nOJg9WA
There hasn't been any activity since a while. Are you planning to implement an error handling (?) or is it low priority and nothing will be changed? The previous link to the problematic file expired. This is the new link https://web.tresorit.com/l#gJR8oBuEvYYIoLPImTnNZw
The bug report was stored under the wrong product category, so we overlooked it a bit. I can reproduce the problem here with 8GB RAM. The memory consumption is enormous with this image. We have to keep several copies of the image data in the memory, conversion to OpenCV RGB order etc. We can reduce the image beforehand or exclude images with this image size from face recognition. Maik
Gilles, we have a problem in DImg::dimgScaleAARGB() with really big images. Depending on the page size, testing the sample image, no preview is possible, when scaling down it crashes. You can also reproduce it in the image editor. Scale an image to 18000-20000 pixels, now use the zoom slider. Before it crashes, you see a graphic error on the right edge. My guess is that a value range of a variable is exceeded here. I have already narrowed down a little where it crashes in function. Maik
yes, i seen. I already read some report by cppcheck around implementation from DImg::dimgScale*(). Let's me check in code... Gilles
I found it. It's in older Coverity Scan report, closed as well without any fix. Look reports: *** #986604: 1624 if (XAP > 0) 1625 { 1626 llong rr = 0, gg = 0, bb = 0; Test Coverage Test Policy Rule Exclusions Line Impact Function Impact 1627 1628 pix = ypoints[dyy + y] + xpoints[x]; CID 986604 (#9 of 15): Unintended sign extension (SIGN_EXTENSION) [select issue] 1629 r = R_VAL16(pix) * INV_XAP; CID 986604 (#3 of 15): Unintended sign extension (SIGN_EXTENSION) [select issue] 1630 g = G_VAL16(pix) * INV_XAP; CID 986604: Unintended sign extension (SIGN_EXTENSION) [select issue] 1631 b = B_VAL16(pix) * INV_XAP; 1632 ++pix; CID 986604: Unintended sign extension (SIGN_EXTENSION) [select issue] 1633 r += R_VAL16(pix) * XAP; CID 986604: Unintended sign extension (SIGN_EXTENSION) [select issue] 1634 g += G_VAL16(pix) * XAP; CID 986604: Unintended sign extension (SIGN_EXTENSION) [select issue] 1635 b += B_VAL16(pix) * XAP; 1636 pix += sow; CID 986604: Unintended sign extension (SIGN_EXTENSION) [select issue] 1637 rr = R_VAL16(pix) * XAP; CID 986604: Unintended sign extension (SIGN_EXTENSION) [select issue] 1638 gg = G_VAL16(pix) * XAP; CID 986604: Unintended sign extension (SIGN_EXTENSION) [select issue] 1639 bb = B_VAL16(pix) * XAP; 1640 pix--; CID 986604 (#15 of 15): Unintended sign extension (SIGN_EXTENSION)sign_extension: Suspicious implicit sign extension: (ushort *)pix[2] with type ushort (16 bits, unsigned) is promoted in (ushort *)pix[2] * (256 - xapoints[x]) to type int (32 bits, signed), then sign-extended to type long (64 bits, signed). If (ushort *)pix[2] * (256 - xapoints[x]) is greater than 0x7FFFFFFF, the upper bits of the result will all be 1. 1641 rr += R_VAL16(pix) * INV_XAP; CID 986604 (#11 of 15): Unintended sign extension (SIGN_EXTENSION) [select issue] 1642 gg += G_VAL16(pix) * INV_XAP; CID 986604: Unintended sign extension (SIGN_EXTENSION) [select issue] 1643 bb += B_VAL16(pix) * INV_XAP; 1644 r = ((rr * YAP) + (r * INV_YAP)) >> 16; 1645 g = ((gg * YAP) + (g * INV_YAP)) >> 16; 1646 b = ((bb * YAP) + (b * INV_YAP)) >> 16; 1647 1648 R_VAL16(dptr) = r; 1649 G_VAL16(dptr) = g; 1650 B_VAL16(dptr) = b; 1651 A_VAL16(dptr) = 0xFFFF; 1652 1653 ++dptr; 1654 } 1655 else *** and #986605 : 2266 int xap; 2267 2268 // go through every scanline in the output buffer 2269 2270 for (y = y_begin ; y < y_end ; ++y) 2271 { 2272 dptr = dest + (y - y_begin) * dow; 2273 2274 for (x = x_begin ; x < x_end ; ++x) 2275 { 2276 Cx = XAP >> 16; 2277 xap = XAP & 0xffff; 2278 pix = ypoints[dyy + y] + xpoints[x]; 2279 r = (R_VAL16(pix) * xap) >> 10; 2280 g = (G_VAL16(pix) * xap) >> 10; 2281 b = (B_VAL16(pix) * xap) >> 10; 2282 a = (A_VAL16(pix) * xap) >> 10; 2283 2284 for (j = (1 << 14) - xap ; j > Cx ; j -= Cx) 2285 { 2286 ++pix; 2287 r += (R_VAL16(pix) * Cx) >> 10; CID 986605: Unintended sign extension (SIGN_EXTENSION) [select issue] 2288 g += (G_VAL16(pix) * Cx) >> 10; CID 986605 (#15 of 15): Unintended sign extension (SIGN_EXTENSION)sign_extension: Suspicious implicit sign extension: (ushort *)pix[0] with type ushort (16 bits, unsigned) is promoted in (ushort *)pix[0] * Cx >> 10 to type int (32 bits, signed), then sign-extended to type long (64 bits, signed). If (ushort *)pix[0] * Cx >> 10 is greater than 0x7FFFFFFF, the upper bits of the result will all be 1. 2289 b += (B_VAL16(pix) * Cx) >> 10; CID 986605: Unintended sign extension (SIGN_EXTENSION) [select issue] 2290 a += (A_VAL16(pix) * Cx) >> 10; 2291 } 2292 2293 if (j > 0) 2294 { 2295 ++pix; CID 986605 (#7 of 15): Unintended sign extension (SIGN_EXTENSION) [select issue] 2296 r += (R_VAL16(pix) * j) >> 10; CID 986605: Unintended sign extension (SIGN_EXTENSION) [select issue] 2297 g += (G_VAL16(pix) * j) >> 10; CID 986605: Unintended sign extension (SIGN_EXTENSION) [select issue] 2298 b += (B_VAL16(pix) * j) >> 10; CID 986605: Unintended sign extension (SIGN_EXTENSION) [select issue] 2299 a += (A_VAL16(pix) * j) >> 10; 2300 } 2301 2302 if (YAP > 0) 2303 { 2304 pix = ypoints[dyy + y] + xpoints[x] + sow; Gilles
Coverity Scan marks an error in the scaling for 16 bit images. But it's roughly the same crash location as for 8 bits. I will try to identify the problem. Maik
Git commit 8b18fb20f5eda64f73f5e3867af95f448e8dbf40 by Maik Qualmann. Committed on 10/05/2020 at 19:49. Pushed by mqualmann into branch 'master'. fix memory request for very large images Related: bug 420868 M +3 -1 core/libs/dimg/dimg_data.cpp https://invent.kde.org/kde/digikam/commit/8b18fb20f5eda64f73f5e3867af95f448e8dbf40
Git commit b66eb0a0556388fc532fcc5bba7a40ed6eda2fc1 by Maik Qualmann. Committed on 12/07/2020 at 06:58. Pushed by mqualmann into branch 'master'. disable OpenCL for everyone An analysis with Heaptrack shows that we have a big memory leak with active OpenCL. Related: bug 423632, bug 421043, bug 420411 M +10 -0 core/app/main/main.cpp M +0 -5 project/bundles/appimage/data/AppRun https://invent.kde.org/graphics/digikam/commit/b66eb0a0556388fc532fcc5bba7a40ed6eda2fc1
Hi, Can you check if this crash still exist with last weekly AppImage build available here : https://files.kde.org/digikam/ Thanks in advance
digiKam 7.0.0 stable release is now published: https://www.digikam.org/news/2020-07-19-7.0.0_release_announcement/ We need a fresh feedback on this file using this version. Thanks in advance Gilles Caulier
Pardon for the late reply. Tested on the official release 7.0.0 and Digikam is not crashing. Thank you very much for the fix and the new release. It is working like a charm. I am looking forward to the new announced features!
Thanks for the feedback. I close this file now. Gilles Caulier