As a note at the moment, we have the same backtrace in this Bug 397919. I have never been able to reproduce this bug here. I see no problems even after a review of the code. Gilles, how well does face recognition work on OSX? OSX is a good catalyst for stack issues.

Maik

Git commit 807847709a3dfc4b8fb90391bc0729839af203ec by Maik Qualmann.
Committed on 19/10/2018 at 10:35.
Pushed by mqualmann into branch 'master'.

try to fix this problem, move image loaders to the heap
Related: bug 397919

M  +73   -57   core/libs/dimg/dimg.cpp

https://commits.kde.org/digikam/807847709a3dfc4b8fb90391bc0729839af203ec

I never tried Face Engine on OSX. I will do it while this week end.

Gilles

Thanks for the attempted fix :)

I built 807847709a3dfc4b8fb90391bc0729839af203ec and re-tried. First I had to delte and rebuild the database because digikam told me the database scheme had changed from 9 to 10. Then I unformunately got what apperas to be the same error, although the regular digikam log preceding it is a bit different:


> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/pictures/Telegram/Telegram Images/230130499_15576.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2017-12-27 19:52:17.000 CET Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2017-12-27 19:52:17.000 CET Qt::TimeSpec(LocalTime))
> digikam.facesengine: detectMultiScale gave  (QRect(21,365 48x48))
> digikam.facesengine: Verifying face  QRect(21,365 48x48)  using cascade  0
> digikam.facesengine: detectMultiScale: image size  128   125  searchIncrement  1.1  grouping  3  flags  0  min size  29   29 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.facesengine: Verifying face  QRect(21,365 48x48)  using cascade  1
> digikam.facesengine: detectMultiScale: image size  128   125  searchIncrement  1.1  grouping  3  flags  0  min size  29   29 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.facesengine: Verifying face  QRect(21,365 48x48)  using cascade  2
> digikam.facesengine: Verifying face  QRect(21,365 48x48)  using cascade  3
> digikam.facesengine: detectMultiScale: image size  128   125  searchIncrement  1.1  grouping  3  flags  0  min size  29   29 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.facesengine: Verifying face  QRect(21,365 48x48)  using cascade  4
> digikam.facesengine: detectMultiScale: image size  128   125  searchIncrement  1.1  grouping  3  flags  0  min size  29   29 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.facesengine: Verifying face  QRect(21,365 48x48)  using cascade  5
> digikam.facesengine: feature  QRectF(0,0 0.6x0.6) QRect(21,365 48x48) QRect(21,365 29x29)
> digikam.facesengine: detectMultiScale: image size  29   29  searchIncrement  1.1  grouping  2  flags  0  min size  5   5 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.facesengine: Verifying face  QRect(21,365 48x48)  using cascade  6
> digikam.facesengine: feature  QRectF(0.4,0 0.6x0.6) QRect(21,365 48x48) QRect(40,365 29x29)
> digikam.facesengine: detectMultiScale: image size  29   29  searchIncrement  1.1  grouping  2  flags  0  min size  5   5 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.facesengine: Verifying face  QRect(21,365 48x48)  using cascade  7
> digikam.facesengine: feature  QRectF(0.2,0.25 0.6x0.6) QRect(21,365 48x48) QRect(31,377 29x29)
> digikam.facesengine: detectMultiScale: image size  29   29  searchIncrement  1.1  grouping  2  flags  0  min size  5   5 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.facesengine: Verifying face  QRect(21,365 48x48)  using cascade  8
> digikam.facesengine: feature  QRectF(0.1,0.4 0.8x0.6) QRect(21,365 48x48) QRect(26,384 38x29)
> digikam.facesengine: detectMultiScale: image size  38   29  searchIncrement  1.1  grouping  2  flags  0  min size  5   5 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "223034758_1114063.jpg" QSize(1280, 720) QSize(1280, 720)
> digikam.facesengine: detectMultiScale: image size  800   450  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "223034787_1150106.jpg" QSize(1280, 720) QSize(1280, 720)
> digikam.facesengine: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  25   25 
> 
> 
> Thread 30 "Thread (pooled)" received signal SIGBUS, Bus error.
> [Switching to Thread 0x7fff26ffd700 (LWP 26159)]
> 0x00007ffff5fe5b33 in QMapNode<QString, QVariant>::copy(QMapData<QString, QVariant>*) const () from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> 
> 
> 
> 
> (gdb) bt
> #0  0x00007ffff5fe5b33 in QMapNode<QString, QVariant>::copy(QMapData<QString, QVariant>*) const () from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #1  0x00007ffff5fe5c82 in QMap<QString, QVariant>::insert(QString const&, QVariant const&) () from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #2  0x00007ffff5ff68f1 in Digikam::DImgLoader::uniqueHashV2(QString const&, Digikam::DImg const*) () from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #3  0x00007ffff5ff6bd6 in Digikam::DImgLoader::createHistoryImageId(QString const&, Digikam::DImg const&, Digikam::DMetadata const&) ()
>    from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #4  0x00007ffff5ff70e3 in Digikam::DImgLoader::readMetadata(QString const&, Digikam::DImg::FORMAT) () from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #5  0x00007ffff5ffef21 in Digikam::JPEGLoader::load(QString const&, Digikam::DImgLoaderObserver*) () from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #6  0x00007ffff617ad1e in Digikam::DImg::load(QString const&, int, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) ()
>    from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #7  0x00007ffff625786c in Digikam::PreviewLoadingTask::execute() () from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #8  0x00007ffff623ed6d in Digikam::LoadSaveThread::run() () from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #9  0x00007ffff6288694 in Digikam::DynamicThread::Private::run() () from /nix/store/rccjxs6680jfbylilyp4ria6ryad7mnk-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #10 0x00007fffe9326602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #11 0x00007fffe932f12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007ffff4f8f5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #13 0x00007fffe66c322f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

Git commit 83db20a5c25c31500b93ce8732ca6728ebd53b16 by Maik Qualmann.
Committed on 19/10/2018 at 16:50.
Pushed by mqualmann into branch 'master'.

Revert "try to fix this problem, move image loaders to the heap"
Related: bug 397919

M  +57   -73   core/libs/dimg/dimg.cpp

https://commits.kde.org/digikam/83db20a5c25c31500b93ce8732ca6728ebd53b16

We also know the last bugtrace. Simply attaching to a QMap crashes. It looks to me like there is no free memory left. Can you observe the memory usage during face detection?

Maik

Git commit f8d2d5f66532c3695211f06eb09f967ba1248986 by Maik Qualmann.
Committed on 19/10/2018 at 19:40.
Pushed by mqualmann into branch 'master'.

close the QFile manually is better
Now the memory usage for face recognition does
not increase above 1GB and stays at 600-800KB
Related: bug 397919

M  +4    -0    core/libs/dimg/loaders/dimgloader.cpp

https://commits.kde.org/digikam/f8d2d5f66532c3695211f06eb09f967ba1248986

I don't think running out of memory is the issue. I have 8G RAM and 8G swap. I built f8d2d5f66532c3695211f06eb09f967ba1248986 and ran `free` every .5s until the crash (it did crash). The RAM did get close to used up (in the 7Gs) some times, but there was always plenty of swap left. The highest usage I saw was 7G RAM and 3G swap. Memory usage went down towards the end though and at the time of the crash it looked like this:

>               total        used        free      shared  buff/cache   available
> Mem:        8057628     4580060      197368      178532     3280200     2994156
> Swap:       8388604     3202384     5186220

Do I understand correctly that digiKam consumes up to 7G of memory? Do you use face detection and recognition together?

Maik

Created attachment 115778 [details]
memory consumed during face detection until crash

No, there were other programs running too. I ran a scan ("Detect face", "Scan again and merge results", 80% accuracy, *not* "Work on all processor cores") under `memory_profiler` and it peaks at 3G (see attachment).
I was running with 100% accuracy before so it might have used more then.

Git commit b11a1a407f5069b7f47810a49ace08d1ed011e87 by Maik Qualmann.
Committed on 20/10/2018 at 16:09.
Pushed by mqualmann into branch 'master'.

try to fix the crash with undocumented method in Qt
Related: bug 397919

M  +3    -0    core/libs/dimg/dimg_p.h

https://commits.kde.org/digikam/b11a1a407f5069b7f47810a49ace08d1ed011e87

Unfortunately b11a1a407f5069b7f47810a49ace08d1ed011e87 still crashes, this time with the backtrace

> digikam.metaengine: Orientation => Exif.Image.Orientation => 1
> digikam.dimg: "/home/timo/pictures/Telegram/Telegram Images/246926317_18149.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2018-06-21 22:00:59.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2018-06-21 22:00:59.000 CEST Qt::TimeSpec(LocalTime))
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "246923134_270485.jpg" QSize(720, 1280) QSize(720, 1280)
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.dimg: "/home/timo/pictures/Pictures/Abitur 2015/Abiball/Professionell/Fotobox/dsc_0230.jpg"  : JPEG file identified
> 
> Thread 235 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff257fa700 (LWP 21997)]
> 0x00007ffff5fd80ba in QMap<QString, QVariant>::insert(QString const&, QVariant const&) ()
>    from /nix/store/8nnk56ip4xpjzdqfbvz0r8n7509yr2hn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> (gdb) bt
> #0  0x00007ffff5fd80ba in QMap<QString, QVariant>::insert(QString const&, QVariant const&) ()
>    from /nix/store/8nnk56ip4xpjzdqfbvz0r8n7509yr2hn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #1  0x00007ffff623ebf2 in Digikam::LoadSaveThread::exifRotate(Digikam::DImg&, QString const&) ()
>    from /nix/store/8nnk56ip4xpjzdqfbvz0r8n7509yr2hn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #2  0x00007ffff6256c9c in Digikam::PreviewLoadingTask::execute() ()
>    from /nix/store/8nnk56ip4xpjzdqfbvz0r8n7509yr2hn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #3  0x00007ffff623ddad in Digikam::LoadSaveThread::run() ()
>    from /nix/store/8nnk56ip4xpjzdqfbvz0r8n7509yr2hn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #4  0x00007ffff62876f4 in Digikam::DynamicThread::Private::run() ()
>    from /nix/store/8nnk56ip4xpjzdqfbvz0r8n7509yr2hn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #5  0x00007fffe9318602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #6  0x00007fffe932112b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #7  0x00007ffff4f815a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #8  0x00007fffe66b522f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

I tried to reproduce this with a public set of images[0] but I could not. That might be because the set is much smaller than my usual (586M vs 39G, although the 39G contain several videos) but the face detection in the public set still took ~3h to finish while it usually crashes within an hour when I run it on my private set.

Git commit 4b84ca31a2f1c86fd3e980f093e296807180d129 by Maik Qualmann.
Committed on 21/10/2018 at 08:07.
Pushed by mqualmann into branch 'master'.

try with QMutexLocker in DImg
Related: bug 397919

M  +36   -1    core/libs/dimg/dimg.cpp
M  +3    -3    core/libs/dimg/dimg_p.h

https://commits.kde.org/digikam/4b84ca31a2f1c86fd3e980f093e296807180d129

Unfortunately that didn't fix it either:

> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "2_5309997771762172030.png" QSize(1236, 781) QSize(1236, 781)
> digikam.facesengine: detectMultiScale: image size  475   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.dimg: Cannot open workspace color profile "/nix/store/s4ih9rqmb33lgavf2027jjg36sy9ydvs-digikam-5.9.0/share/digikam/profiles/srgb-d65.icm"
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/pictures/Telegram/Telegram Documents/2_5389028614572343387.gif"  : QIMAGE file identified
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "2_5312030137401737360.png" QSize(603, 1014) QSize(603, 1014)
> digikam.facesengine: detectMultiScale: image size  800   450  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "2_5343993975508304200.png" QSize(1600, 900) QSize(1600, 900)
> 
> Thread 87 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3d7af700 (LWP 1596)]
> 0x00000000035b8268 in ?? ()
> (gdb) bt
> #0  0x00000000035b8268 in ?? ()
> #1  0x00007ffff62506db in non-virtual thunk to Digikam::LoadingTask::progressInfo(Digikam::DImg const*, float) ()
>    from /nix/store/b8zcx52qmn06i991vxa57dkzdivlg11f-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #2  0x00007ffff5ff00c5 in Digikam::JPEGLoader::load(QString const&, Digikam::DImgLoaderObserver*) ()
>    from /nix/store/b8zcx52qmn06i991vxa57dkzdivlg11f-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #3  0x00007ffff616a250 in Digikam::DImg::load(QString const&, int, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) ()
>    from /nix/store/b8zcx52qmn06i991vxa57dkzdivlg11f-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #4  0x00007ffff625610c in Digikam::PreviewLoadingTask::execute() ()
>    from /nix/store/b8zcx52qmn06i991vxa57dkzdivlg11f-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #5  0x00007ffff623d4bd in Digikam::LoadSaveThread::run() ()
>    from /nix/store/b8zcx52qmn06i991vxa57dkzdivlg11f-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #6  0x00007ffff6286f54 in Digikam::DynamicThread::Private::run() ()
>    from /nix/store/b8zcx52qmn06i991vxa57dkzdivlg11f-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #7  0x00007fffe9314602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe931d12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff4f7d5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe66b122f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

The color profile error seems to be an unrelated issue: digikam saves the paths of `MonitorProfileFile` and `WorkProfileFile` in `digikamrc` but that old version does no longer exist and the paths aren't updated.

I think this problem is due to a race condition, undetectable with valgrind as speed is reduced a lots.

This can be relevant of thread serialization used in face management. For me this code that I review a split in dedicated source code is too complicated.

Marcel has written all this code with a student when face detection have been introduced and lets as well for a while

Another negative point is opencast. This library is a large puzzle and, as I can see unit test code for image quality sorter, with few line of code, valgrind report a lots of memory corruption

In project/script/ there is a bootstrap script to compile opencast with the minimum requirements for digikam. With this kind of env. digikam do not crash here less time with opencv but it,s not perfect.

So with this multiple error prone, face management can become instable as we don’t control how opencv is compiled on target computer.

Some way to progress in investigations:

1/ write one unit test to call face management with all options available in digikam
This will simplify the test but as low level code is complicated, this kind of test can only confirm the bug without to isolate the source of the problem.

2/ as all classes as well isolated in face management, rewrite the thread serialization can be better. Sure the performance will decrease, but the gain over code complexity will be better to understand and to maintain. Typically, an ActionJob  to parse image in a separated thread to detect faces, one other to perform identification, and one other to process database requests through a cache mechanism to not bloat with multiple queries.

Oups, in my previous comment : opencast —> opencv

Hmm... I think we have 2 problems here, once the crash in QMap. There are a few entries on the web that deal with it. QMap/QHash is not thread save and must be locked with QMutexLocker. The other crash is in LoadingTask::progressInfo (), if gdb does not take us the wrong way here, it's very strange. My thanks go to timokau for testing the latest git/master version.

Maik

Git commit 8e43d83ad9193f59551d95129b4b9b6183f39590 by Maik Qualmann.
Committed on 21/10/2018 at 19:35.
Pushed by mqualmann into branch 'master'.

try to fix a possible VTable problem
Related: bug 397919

M  +2    -1    core/libs/dimg/filters/icc/icctransformfilter.cpp
M  +1    -1    core/libs/dimg/filters/icc/icctransformfilter.h
M  +9    -2    core/libs/dimg/loaders/dimgloaderobserver.h
M  +16   -6    core/libs/threadimageio/loadsavetask.cpp
M  +6    -6    core/libs/threadimageio/loadsavetask.h
M  +2    -1    core/utilities/assistants/expoblending/manager/expoblendingthread.cpp
M  +2    -1    core/utilities/assistants/panorama/tasks/preprocesstask.cpp
M  +2    -1    core/utilities/queuemanager/manager/batchtool.cpp

https://commits.kde.org/digikam/8e43d83ad9193f59551d95129b4b9b6183f39590

The fact is that this function should never be called in this context:

Digikam::LoadingTask::progressInfo(Digikam::DImg const*, float)

Only this overloaded function:

Digikam::SharedLoadingTask::progressInfo(Digikam::DImg const*, float)

Maik

Experimenting a bit I can remove a lot of pictures while still reliably getting it to crash (in 5 minutes or less), but I can't isolate a set of pictures that definitely causes a crash.

@caulier.gilles I've re-build opencv with the same cmake flags the "bootstrap" uses just to be sure, that didn't fix it.

@Maik thank you for investing your time to fix this bug, the very least I can do is test the fixes. Unfortunately no luck with 8e43d83ad9193f59551d95129b4b9b6183f39590 either:

> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "IMG-20160721-WA0000.jpg" QSize(720, 1280) QSize(720, 1280)
> digikam.facesengine: detectMultiScale: image size  450   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/pictures/WhatsApp Images/IMG-20160724-WA0004.jpg"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2017-05-14 23:11:25.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2017-05-14 23:11:25.000 CEST Qt::TimeSpec(LocalTime))
> 
> Thread 44 "Thread (pooled)" received signal SIGBUS, Bus error.
> [Switching to Thread 0x7fff14a7b700 (LWP 9485)]
> 0x00007ffff5fd5063 in QMapNode<QString, QVariant>::copy(QMapData<QString, QVariant>*) const () from /nix/store/q3kr6shz92ybg9dkqxd9k79cm0mq5nfw-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> (gdb) bt
> #0  0x00007ffff5fd5063 in QMapNode<QString, QVariant>::copy(QMapData<QString, QVariant>*) const () from /nix/store/q3kr6shz92ybg9dkqxd9k79cm0mq5nfw-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #1  0x00007ffff5fd51b2 in QMap<QString, QVariant>::insert(QString const&, QVariant const&) () from /nix/store/q3kr6shz92ybg9dkqxd9k79cm0mq5nfw-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #2  0x00007ffff623e862 in Digikam::LoadSaveThread::exifRotate(Digikam::DImg&, QString const&) () from /nix/store/q3kr6shz92ybg9dkqxd9k79cm0mq5nfw-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #3  0x00007ffff6256a5c in Digikam::PreviewLoadingTask::execute() () from /nix/store/q3kr6shz92ybg9dkqxd9k79cm0mq5nfw-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #4  0x00007ffff623da1d in Digikam::LoadSaveThread::run() () from /nix/store/q3kr6shz92ybg9dkqxd9k79cm0mq5nfw-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #5  0x00007ffff62874b4 in Digikam::DynamicThread::Private::run() () from /nix/store/q3kr6shz92ybg9dkqxd9k79cm0mq5nfw-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #6  0x00007fffe9315602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #7  0x00007fffe931e12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007ffff4f7e5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #9  0x00007fffe67e022f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> (gdb) 


Is there anything else I can try to investigate this?

Ok, this is the crash in QMap again. I'll add the QMutexLocker patch again.

Maik

Git commit 985be20fd70563bb3989cbb6670880c27f3babba by Maik Qualmann.
Committed on 22/10/2018 at 05:56.
Pushed by mqualmann into branch 'master'.

add QMutexLocker patch again
Related: bug 397919

M  +36   -1    core/libs/dimg/dimg.cpp
M  +3    -0    core/libs/dimg/dimg_p.h

https://commits.kde.org/digikam/985be20fd70563bb3989cbb6670880c27f3babba

I tried to sample multiple segfaults this time, in case it helps. I feel like it does crash less often and I couldn't reproduce the QMap crash again, so maybe at least that is fixed by the Mutex. I still got these however:

> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/pictures/WhatsApp Images/IMG-20150924-WA0001.jpeg"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2015-09-24 20:13:57.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2015-09-24 20:13:57.000 CEST Qt::TimeSpec(LocalTime))
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "IMG-20150902-WA0001.jpg" QSize(1600, 1200) QSize(1600, 1200)
> digikam.facesengine: detectMultiScale: image size  600   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> 
> Thread 40 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff311fe700 (LWP 22008)]
> 0x00007fffe676e311 in free () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007fffe676e311 in free () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> #1  0x00007ffff6165415 in Digikam::DImg::~DImg() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #2  0x00007ffff6076adb in Digikam::IccManager::~IccManager() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #3  0x00007ffff62529ae in Digikam::SharedLoadingTask::postProcess() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #4  0x00007ffff6255d85 in Digikam::PreviewLoadingTask::execute() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #5  0x00007ffff623d4bd in Digikam::LoadSaveThread::run() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #6  0x00007ffff6286f54 in Digikam::DynamicThread::Private::run() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #7  0x00007fffe9314602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe931d12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff4f7d5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe67df22f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6


> [New Thread 0x7fff148f9700 (LWP 27765)]
> digikam.general: Found 0 faces in "IMG-20150724-WA0083.jpg" QSize(1032, 581) QSize(1032, 581)
> digikam.dimg: "/home/timo/pictures/WhatsApp Images/IMG-20150725-WA0008.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2017-05-14 23:09:27.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2017-05-14 23:09:27.000 CEST Qt::TimeSpec(LocalTime))
> digikam.facesengine: detectMultiScale: image size  450   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> 
> Thread 40 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff177fe700 (LWP 27673)]
> 0x00007ffff6255d82 in Digikam::PreviewLoadingTask::execute() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> (gdb) bt
> #0  0x00007ffff6255d82 in Digikam::PreviewLoadingTask::execute() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #1  0x00007ffff623d4bd in Digikam::LoadSaveThread::run() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #2  0x00007ffff6286f54 in Digikam::DynamicThread::Private::run() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #3  0x00007fffe9314602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #4  0x00007fffe931d12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #5  0x00007ffff4f7d5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #6  0x00007fffe67df22f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6


> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/Pictures/WhatsApp Images/IMG-20150814-WA0001.jpg"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2017-05-14 23:09:35.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2017-05-14 23:09:35.000 CEST Qt::TimeSpec(LocalTime))
> 
> Thread 29 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3c8bf700 (LWP 31673)]
> 0x00007fffe930ea72 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007fffe930ea72 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #1  0x00007ffff6166c67 in Digikam::DImg::setAttribute(QString const&, QVariant const&) () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #2  0x00007ffff623e302 in Digikam::LoadSaveThread::exifRotate(Digikam::DImg&, QString const&) () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #3  0x00007ffff62564fc in Digikam::PreviewLoadingTask::execute() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #4  0x00007ffff623d4bd in Digikam::LoadSaveThread::run() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #5  0x00007ffff6286f54 in Digikam::DynamicThread::Private::run() () from /nix/store/d046363bhpl5lf8rn7hi30hd9m59cw50-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #6  0x00007fffe9314602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #7  0x00007fffe931d12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007ffff4f7d5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #9  0x00007fffe67df22f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

All your last backtrace do not provide a suitable trace. Did you compile digiKam with debug symbols ? It's a option from cmake to pass at configuration stage. The bootstrap.sh as this option enabled at line #73.

Gilles Caulier

Yes I'm passing `-DCMAKE_BUILD_TYPE=debugfull` as I've seen in some instructions. The boostrap script instead only passes `-DCMAKE_BUILD_TYPE=debug`, is `debugfull` incorrect?

Why are the traces not suitable?

"debug" do the stuff here.

backtrace are not suitable because there is not source code lines where the debugger list the function calls until the crash, at least in libdigikam.so.

Gilles Caulier

`-DCMAKE_BUILD_TYPE=debug` has the same result. I got the QMap segfault again:

> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.dimg: "/home/timo/Pictures/Pictures/Abitur 2015/Abiball/Professionell/Fotobox/dsc_0347.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.metaengine: Orientation => Exif.Image.Orientation => 1
> digikam.dimg: "/home/timo/Pictures/Pictures/Private Bilder/Klassenfotos/Masken 2006 Klasse 5/IMG_1416.JPG"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2007-05-30 19:33:25.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Photo.DateTimeDigitized => QDateTime(2007-05-30 19:33:25.000 CEST Qt::TimeSpec(LocalTime))
> digikam.dimg: "/home/timo/Pictures/Pictures/Abitur 2015/Abiball/Professionell/Fotobox/dsc_0195.jpg"  : JPEG file identified
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "IMG_1411.JPG" QSize(1152, 1728) QSize(1152, 1728)
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.facesengine: detectMultiScale: image size  533   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "IMG_1412.JPG" QSize(1152, 1728) QSize(1152, 1728)
> digikam.facesengine: detectMultiScale: image size  533   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> 
> Thread 48 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff0d693700 (LWP 2250)]
> 0x00007fffe937f8d0 in QMapDataBase::rotateRight(QMapNodeBase*) ()
>    from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> (gdb) 
> (gdb) bt
> #0  0x00007fffe937f8d0 in QMapDataBase::rotateRight(QMapNodeBase*) ()
>    from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #1  0x00007fffe937fa47 in QMapDataBase::rebalance(QMapNodeBase*) ()
>    from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #2  0x00007fffe937ff40 in QMapDataBase::createNode(int, int, QMapNodeBase*, bool) ()
>    from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #3  0x00007ffff5fd42bc in QMap<QString, QVariant>::insert(QString const&, QVariant const&) ()
>    from /nix/store/n4bps1zsriih8kvnc8q0yqz0pckr1glz-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #4  0x00007ffff6166c7a in Digikam::DImg::setAttribute(QString const&, QVariant const&) ()
>    from /nix/store/n4bps1zsriih8kvnc8q0yqz0pckr1glz-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #5  0x00007ffff623e302 in Digikam::LoadSaveThread::exifRotate(Digikam::DImg&, QString const&) ()
>    from /nix/store/n4bps1zsriih8kvnc8q0yqz0pckr1glz-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #6  0x00007ffff62564fc in Digikam::PreviewLoadingTask::execute() ()
>    from /nix/store/n4bps1zsriih8kvnc8q0yqz0pckr1glz-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #7  0x00007ffff623d4bd in Digikam::LoadSaveThread::run() ()
>    from /nix/store/n4bps1zsriih8kvnc8q0yqz0pckr1glz-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #8  0x00007ffff6286f54 in Digikam::DynamicThread::Private::run() ()
>    from /nix/store/n4bps1zsriih8kvnc8q0yqz0pckr1glz-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #9  0x00007fffe9314602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #10 0x00007fffe931d12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #11 0x00007ffff4f7d5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #12 0x00007fffe67df22f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

Was this debug trace created with today's git/master version? If not, please repeat again.

Maik

And recompile all. remove build directory, reconfigure, and recompile all.

Typically, the bracktrace still uncomplete as not all source code information is available.

Look in this entry for ex :

https://bugs.kde.org/show_bug.cgi?id=400140#c2

As you can see, the internal source code path is given in the backtrace, because whole digiKam (and Exiv2 library) is compiled with all debug symbols.)

Gilles Caulier

(In reply to Maik Qualmann from comment #30)
> Was this debug trace created with today's git/master version? If not, please
> repeat again.
> 
> Maik

It was with the last version from this ticket, 985be20fd70563bb3989cbb6670880c27f3babba. I'll retry with the latest master.

(In reply to caulier.gilles from comment #31)
> And recompile all. remove build directory, reconfigure, and recompile all.
> 
> Typically, the bracktrace still uncomplete as not all source code
> information is available.
> 
> Look in this entry for ex :
> 
> https://bugs.kde.org/show_bug.cgi?id=400140#c2
> 
> As you can see, the internal source code path is given in the backtrace,
> because whole digiKam (and Exiv2 library) is compiled with all debug
> symbols.)
> 
> Gilles Caulier

Every build happens in a sandbox completely isolated from all the builds before (since I'm modifying my distro's build receipe and using my package manager to build), so everything is recompiled with `-DCMAKE_BUILD_TYPE=debug`. The exact cmake flags passed are

> -DENABLE_MYSQLSUPPORT=1 -DENABLE_INTERNALMYSQL=1 -DENABLE_MEDIAPLAYER=1 -DCMAKE_BUILD_TYPE=debug

I've found the issue with the debug symbols. My package manager is being a bit too helpful and inserting "-DCMAKE_BUILD_TYPE=Release" by default, overriding me setting it to debug. My bad, will fix.

With the version from today (c71842f1624c6d6518a20e071d36512f0250595b) I could reproduce the QMap error again. That was before I managed to get the debug working however:

> digikam.general: Check for finish:  22 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/Pictures/Pictures/Android/Pictures/Screenshots/Screenshot_2014-05-15-18-47-42.png"  : PNG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2014-05-15 18:47:44.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2014-05-15 18:47:44.000 CEST Qt::TimeSpec(LocalTime))
> 
> Thread 77 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7ffef3fff700 (LWP 26867)]
> 0x00007ffff5fd0063 in QMapNode<QString, QVariant>::copy(QMapData<QString, QVariant>*) const () from /nix/store/752mjcqc64a8v4q241kzrmsx3vrl2sjn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> (gdb) bg
> Undefined command: "bg".  Try "help".
> (gdb) bt
> #0  0x00007ffff5fd0063 in QMapNode<QString, QVariant>::copy(QMapData<QString, QVariant>*) const () from /nix/store/752mjcqc64a8v4q241kzrmsx3vrl2sjn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #1  0x00007ffff5fd01b2 in QMap<QString, QVariant>::insert(QString const&, QVariant const&) () from /nix/store/752mjcqc64a8v4q241kzrmsx3vrl2sjn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #2  0x00007ffff61618de in Digikam::DImg::setAttribute(QString const&, QVariant const&) () from /nix/store/752mjcqc64a8v4q241kzrmsx3vrl2sjn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #3  0x00007ffff6239922 in Digikam::LoadSaveThread::exifRotate(Digikam::DImg&, QString const&) () from /nix/store/752mjcqc64a8v4q241kzrmsx3vrl2sjn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #4  0x00007ffff6251b1c in Digikam::PreviewLoadingTask::execute() () from /nix/store/752mjcqc64a8v4q241kzrmsx3vrl2sjn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #5  0x00007ffff6238add in Digikam::LoadSaveThread::run() () from /nix/store/752mjcqc64a8v4q241kzrmsx3vrl2sjn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #6  0x00007ffff6282574 in Digikam::DynamicThread::Private::run() () from /nix/store/752mjcqc64a8v4q241kzrmsx3vrl2sjn-digikam-6.0.0-beta2/lib/libdigikamcore.so.6.0.0
> #7  0x00007fffe9310602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe931912b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff4f795a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe67db22f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

Stacktraces after I got debug working (all the other error):

> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/Pictures/Pictures/Android/Whatsapp/IMG-20130707-WA0000.jpg"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2014-04-17 14:44:29.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2014-04-17 14:44:29.000 CEST Qt::TimeSpec(LocalTime))
> digikam.dimg: "/home/timo/Pictures/Pictures/Android/Whatsapp/IMG-20130708-WA0000.jpg"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2013-07-08 16:44:29.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Photo.DateTimeDigitized => QDateTime(2013-07-08 16:44:29.000 CEST Qt::TimeSpec(LocalTime))
> double free or corruption (out)
> 
> Thread 34 "Thread (pooled)" received signal SIGABRT, Aborted.
> [Switching to Thread 0x7fff3dae2700 (LWP 31498)]
> 0x00007fffe699cbe0 in raise () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007fffe699cbe0 in raise () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> #1  0x00007fffe699ddc1 in abort () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> #2  0x00007fffe69de2ac in __libc_message () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> #3  0x00007fffe69e449a in malloc_printerr () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> #4  0x00007fffe69e53e0 in _int_free () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> #5  0x00007ffff6399d9d in Digikam::DImg::rotate (this=0x36095f8, angle=angle@entry=Digikam::DImg::ROT90)
>     at /home/timo/tmp/digikam-build/source/core/libs/dimg/dimg.cpp:2328
> #6  0x00007ffff6399ff7 in Digikam::DImg::rotateAndFlip (this=this@entry=0x36095f8, orientation=<optimized out>)
>     at /home/timo/tmp/digikam-build/source/core/libs/dimg/dimg.cpp:2652
> #7  0x00007ffff6448fba in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadsavethread.cpp:335
> #8  0x00007ffff645d8cd in Digikam::PreviewLoadingTask::execute (this=0x3609400)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/previewtask.cpp:384
> #9  0x00007ffff6448d3e in Digikam::LoadSaveThread::run (this=0x7fffbc00be80)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadsavethread.cpp:133
> #10 0x00007ffff6487a8e in Digikam::DynamicThread::Private::run (this=0x38c27f0)
>     at /home/timo/tmp/digikam-build/source/core/libs/threads/dynamicthread.cpp:183
> #11 0x00007fffe9573602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007fffe957c12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #13 0x00007ffff51dc5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #14 0x00007fffe6a5a22f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6


> digikam.facesengine: detectMultiScale gave  (QRect(94,54 92x56), QRect(61,112 135x81))
> digikam.general: Found 1 faces in "croppedImage.png" QSize(971, 1128) QSize(971, 1128)
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Entry QRect(318,176 506x506)6overlapsQRect(318,176 506x506)280, skipping
> digikam.dimg:  : return null image! ( false ,  0 ,  0 )
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/Pictures/DCIM/OpenCamera/IMG_20180505_020021.jpg"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2018-05-05 02:00:24.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2018-05-05 02:00:24.000 CEST Qt::TimeSpec(LocalTime))
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "IMG_20180427_002628.jpg" QSize(4160, 3120) QSize(4160, 3120)
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> 
> Thread 34 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3dae2700 (LWP 12534)]
> 0x00007fffe95f818a in operator==(QString const&, QString const&) ()
>    from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007fffe95f818a in operator==(QString const&, QString const&) ()
>    from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #1  0x00007ffff644d5d5 in Digikam::LoadingDescription::equalsIgnoreReducedVersion (this=<optimized out>, other=...)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadingdescription.cpp:313
> #2  0x00007ffff6458f93 in Digikam::SharedLoadingTask::notifyNewLoadingProcess (this=0x37fd640, process=<optimized out>, 
>     description=...) at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadsavetask.cpp:436
> #3  0x00007ffff644ed10 in Digikam::LoadingCache::notifyNewLoadingProcess (this=this@entry=0x111e870, 
>     process=process@entry=0x2b42540, description=...)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadingcache.cpp:243
> #4  0x00007ffff645d736 in Digikam::PreviewLoadingTask::execute (this=0x2b42370)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/previewtask.cpp:162
> #5  0x00007ffff6448d3e in Digikam::LoadSaveThread::run (this=0x2b40a00)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadsavethread.cpp:133
> #6  0x00007ffff6487a8e in Digikam::DynamicThread::Private::run (this=0x37ba7f0)
>     at /home/timo/tmp/digikam-build/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe9573602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe957c12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51dc5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe6a5a22f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6


> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.metaengine: Orientation => Exif.Image.Orientation => 1
> digikam.dimg: "/home/timo/Pictures/DCIM/OpenCamera/IMG_20180614_114558.jpg"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2018-06-14 11:45:58.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Image.DateTime => QDateTime(2018-06-14 11:45:58.000 CEST Qt::TimeSpec(LocalTime))
> 
> Thread 585 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff19ffb700 (LWP 23948)]
> 0x00007fffe69e9311 in free () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007fffe69e9311 in free () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> #1  0x00007ffff63991b5 in Digikam::DImg::Private::~Private (this=0x7fff3800c900, __in_chrg=<optimized out>)
>     at /home/timo/tmp/digikam-build/source/core/libs/dimg/dimg_p.h:89
> #2  Digikam::DSharedDataPointer<Digikam::DImg::Private>::~DSharedDataPointer (this=<optimized out>, __in_chrg=<optimized out>)
>     at /home/timo/tmp/digikam-build/source/core/libs/dimg/dshareddata.h:182
> #3  Digikam::DImg::~DImg (this=<optimized out>, __in_chrg=<optimized out>)
>     at /home/timo/tmp/digikam-build/source/core/libs/dimg/dimg.cpp:174
> #4  0x00007ffff62c385b in Digikam::IccManager::Private::~Private (this=0x7fff100acf20, __in_chrg=<optimized out>)
>     at /home/timo/tmp/digikam-build/source/core/libs/dimg/filters/icc/iccmanager.cpp:41
> #5  Digikam::IccManager::~IccManager (this=0x7fff19ffa8f8, __in_chrg=<optimized out>)
>     at /home/timo/tmp/digikam-build/source/core/libs/dimg/filters/icc/iccmanager.cpp:101
> #6  0x00007ffff6459558 in Digikam::SharedLoadingTask::postProcess (this=0x529b460)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadsavetask.cpp:340
> #7  0x00007ffff645d169 in Digikam::PreviewLoadingTask::execute (this=0x529b460)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/previewtask.cpp:389
> #8  0x00007ffff6448d3e in Digikam::LoadSaveThread::run (this=0x7fffbc00bb50)
>     at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadsavethread.cpp:133
> #9  0x00007ffff6487a8e in Digikam::DynamicThread::Private::run (this=0x4bd1880)
>     at /home/timo/tmp/digikam-build/source/core/libs/threads/dynamicthread.cpp:183
> #10 0x00007fffe9573602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #11 0x00007fffe957c12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007ffff51dc5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #13 0x00007fffe6a5a22f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

This crashes in so many different places in the code. What surprises me that I can not reproduce it, even after hours. Which CPU does your computer have? Intel or AMD?

Maik

Intel i5-3470. I can also reproduce the error on my laptop (also intel cpu but different one). The two factors laptop and desktop have in common are

- the exact dependencies (both the same package manager which exactly defines the environment)
- the photo collection

Is there some sizeable public collection available that I can try to reproduce it on?

I have collection of test images, not public, as photos was send by users since many years. And this is not only specific to Face management.

Starting to share data file is a good idea but we will have a problem with rights, as faces can be present.

This can be only done through a limited access over a cloud storage. 

To resume, it's complicated...

Gilles Caulier

I found a huge public image database with faces: https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/

I could reproduce the crash with their cropped 7G version (https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/static/imdb_crop.tar). If you cannot reproduce it with that, you could try to install the nix package manager (https://nixos.org/nix/) which runs under any linux distro and install the digikam version there (`nix build nixpkgs.digikam` will create a symlink `result` in the current folder pointing to a digikam installation).

The backtrace from the crash with the IMDB dataset:

> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 1 faces in "nm0224703_rm2961290496_1967-7-20_2011.jpg" QSize(500, 500) QSize(500, 500)
> digikam.facesengine: detectMultiScale: image size  356   355  searchIncrement  1.1  grouping  3  flags  1  min size  0   0 
> 
> 
> Thread 28 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff177fe700 (LWP 22023)]
> 0x00007fffe95f8248 in operator<(QString const&, QString const&) () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007fffe95f8248 in operator<(QString const&, QString const&) () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #1  0x00007ffff6232fd1 in qMapLessThanKey<QString> (key2=..., key1=...) at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:71
> #2  QMap<QString, QVariant>::insert (this=0x7ffeec197ac8, akey=..., avalue=...) at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:711
> #3  0x00007ffff639a1de in Digikam::DImg::Private::attributesInsert (value=..., key=..., this=0x7ffeec197aa0) at /home/timo/tmp/digikam-build/source/core/libs/dimg/dimg_p.h:135
> #4  Digikam::DImg::setAttribute (this=<optimized out>, key=..., value=...) at /home/timo/tmp/digikam-build/source/core/libs/dimg/dimg.cpp:1049
> #5  0x00007ffff6448ff2 in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...) at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadsavethread.cpp:336
> #6  0x00007ffff645d8cd in Digikam::PreviewLoadingTask::execute (this=0x8d2d470) at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/previewtask.cpp:384
> #7  0x00007ffff6448d3e in Digikam::LoadSaveThread::run (this=0xd72f240) at /home/timo/tmp/digikam-build/source/core/libs/threadimageio/loadsavethread.cpp:133
> #8  0x00007ffff6487a8e in Digikam::DynamicThread::Private::run (this=0x10bfc870) at /home/timo/tmp/digikam-build/source/core/libs/threads/dynamicthread.cpp:183
> #9  0x00007fffe9573602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #10 0x00007fffe957c12b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #11 0x00007ffff51dc5a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #12 0x00007fffe6a5a22f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

The crash happened for me after recognizing 10423 faces, but it can happen pretty much anytime.

I have an idea after a deeper review of the preview task.
I'll commit it tonight.

Maik

Git commit bede4dcc3829b24494b4e4733579455452c62775 by Maik Qualmann.
Committed on 24/10/2018 at 17:47.
Pushed by mqualmann into branch 'master'.

try to fix the crash in the preview task
Related: bug 397919

M  +26   -28   core/libs/threadimageio/previewtask.cpp

https://commits.kde.org/digikam/bede4dcc3829b24494b4e4733579455452c62775

No luck:

> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "235139171_69724.jpg" QSize(493, 495) QSize(493, 495)
> digikam.facesengine: detectMultiScale: image size  418   800  searchIncrement  1.1  grouping  3  flags  1  min size  25   25 
> 
> 
> Thread 101 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7ffef37fd700 (LWP 3731)]
> 0x00007ffff64522c5 in Digikam::LoadingTask::progressInfo (this=0x4b5d740, img=0x4b5d938, progress=0.100000001) at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavetask.cpp:65
> 65              if (m_thread->querySendNotifyEvent())
> (gdb) bt
> #0  0x00007ffff64522c5 in Digikam::LoadingTask::progressInfo (this=0x4b5d740, img=0x4b5d938, progress=0.100000001) at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavetask.cpp:65
> #1  0x00007ffff6244f2a in Digikam::JPEGLoader::load (this=this@entry=0x7ffef37fc890, filePath=..., observer=observer@entry=0x4b5d750) at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/jpegloader.cpp:374
> #2  0x00007ffff6398160 in Digikam::DImg::load (this=this@entry=0x4b5d938, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x4b5d750, rawDecodingSettings=...)
>     at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff6398760 in Digikam::DImg::load (this=this@entry=0x4b5d938, filePath=..., observer=observer@entry=0x4b5d750, rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff64574b1 in Digikam::PreviewLoadingTask::execute (this=0x4b5d740) at /home/timo/tmp/digikam/source/core/libs/threadimageio/previewtask.cpp:317
> #5  0x00007ffff6442efe in Digikam::LoadSaveThread::run (this=0x5595100) at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavethread.cpp:133
> #6  0x00007ffff6481bae in Digikam::DynamicThread::Private::run (this=0x4ef5860) at /home/timo/tmp/digikam/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe956d602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe957612b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51d65a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe6a5222f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

Git commit 71fe4107faa82bd992756392e53910897ab4fc09 by Maik Qualmann.
Committed on 25/10/2018 at 06:12.
Pushed by mqualmann into branch 'master'.

try to fix progressInfo crash
Related: bug 397919

M  +3    -3    core/libs/threadimageio/loadsavetask.cpp
M  +3    -0    core/libs/threadimageio/loadsavetask.h

https://commits.kde.org/digikam/71fe4107faa82bd992756392e53910897ab4fc09

Still segfaulting:

> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 1 faces in "IMG-20180705-WA0003.jpg" QSize(1024, 768) QSize(1024, 768)
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> 
> Thread 279 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff1d7fa700 (LWP 23866)]
> 0x00007ffff63bf22e in QSharedDataPointer<Digikam::MetaEngineData::Private>::operator= (o=..., this=0x7ffedc3e6390)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qshareddata.h:99
> 99                  if (old && !old->ref.deref())
> (gdb) bt
> #0  0x00007ffff63bf22e in QSharedDataPointer<Digikam::MetaEngineData::Private>::operator= (o=..., this=0x7ffedc3e6390)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qshareddata.h:99
> #1  Digikam::MetaEngineData::operator= (this=0x7ffedc3e6390, other=...)
>     at /home/timo/tmp/digikam/source/core/libs/metadataengine/engine/metaengine_data.cpp:51
> #2  0x00007ffff638eacc in Digikam::DImg::setMetadata (this=this@entry=0x42111a8, data=...)
>     at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:1015
> #3  0x00007ffff623c153 in Digikam::DImgLoader::readMetadata (this=0x7fff1d7f9890, filePath=...)
>     at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/dimgloader.cpp:219
> #4  0x00007ffff6243d11 in Digikam::JPEGLoader::load (this=this@entry=0x7fff1d7f9890, filePath=..., 
>     observer=observer@entry=0x4210fc0) at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/jpegloader.cpp:92
> #5  0x00007ffff6398160 in Digikam::DImg::load (this=this@entry=0x42111a8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, 
>     observer=observer@entry=0x4210fc0, rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:429
> #6  0x00007ffff6398760 in Digikam::DImg::load (this=this@entry=0x42111a8, filePath=..., observer=observer@entry=0x4210fc0, 
>     rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:373
> #7  0x00007ffff64574b1 in Digikam::PreviewLoadingTask::execute (this=0x4210fb0)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/previewtask.cpp:317
> #8  0x00007ffff6442efe in Digikam::LoadSaveThread::run (this=0x366f3d0)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavethread.cpp:133
> #9  0x00007ffff6481bae in Digikam::DynamicThread::Private::run (this=0x36d50a0)
>     at /home/timo/tmp/digikam/source/core/libs/threads/dynamicthread.cpp:183
> #10 0x00007fffe956d602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #11 0x00007fffe957612b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007ffff51d65a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #13 0x00007fffe6a5222f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

Git commit 132352e8668695fd3ab2718e6184a5e794c64179 by Maik Qualmann.
Committed on 27/10/2018 at 18:07.
Pushed by mqualmann into branch 'master'.

Reorganization DImg loading and caching from the PreviewLoadingTask
Now we cache an unprocessed DImg
Related: bug 397919

M  +167  -213  core/libs/threadimageio/previewtask.cpp

https://commits.kde.org/digikam/132352e8668695fd3ab2718e6184a5e794c64179

This time DImg is not involved:

> digikam.dimg: "/home/timo/Pictures/Pictures/Android/Whatsapp/IMG-20140419-WA0001.jpg"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2014-04-19 16:32:22.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Photo.DateTimeDigitized => QDateTime(2014-04-19 16:32:22.000 CEST Qt::TimeSpec(LocalTime))
> [New Thread 0x7fff103dc700 (LWP 12126)]
> 
> Thread 31 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff12be1700 (LWP 31519)]
> 0x00007ffff6458512 in Digikam::PreviewLoadingTask::execute (this=0x125bf30) at /home/timo/repos/nixpkgs/backport/source/core/libs/threadimageio/previewtask.cpp:404
> 404             if (needsPostProcessing())
> (gdb) bt
> #0  0x00007ffff6458512 in Digikam::PreviewLoadingTask::execute (this=0x125bf30) at /home/timo/repos/nixpkgs/backport/source/core/libs/threadimageio/previewtask.cpp:404
> #1  0x00007ffff6443eee in Digikam::LoadSaveThread::run (this=0x7fffbc00fad0) at /home/timo/repos/nixpkgs/backport/source/core/libs/threadimageio/loadsavethread.cpp:133
> #2  0x00007ffff648293e in Digikam::DynamicThread::Private::run (this=0x37aaf20) at /home/timo/repos/nixpkgs/backport/source/core/libs/threads/dynamicthread.cpp:183
> #3  0x00007fffe956e602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #4  0x00007fffe957712b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #5  0x00007ffff51d75a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #6  0x00007fffe6a4822f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

A new test with the latest git/master version would be good, maybe 2 or 3 backtraces.

Maik

Okay, tested with 848a76780f537f807f0be90d6e1ab00c451578e1 from today:

> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "IMG_20171220_173602.jpg" QSize(3120, 4208) QSize(3120, 4208)
> 
> Thread 480 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff849ef700 (LWP 10580)]
> 0x0000000003cbe610 in ?? ()
> (gdb) bt
> #0  0x0000000003cbe610 in ?? ()
> #1  0x00007ffff644e29d in Digikam::LoadingTask::progressInfo (this=0x3a03fe0, img=<optimized out>, progress=0.800000012)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavetask.cpp:65
> #2  0x00007ffff62412e0 in Digikam::JPEGLoader::load (this=this@entry=0x7fff849ee8d0, filePath=..., 
>     observer=observer@entry=0x3a03ff0) at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/jpegloader.cpp:470
> #3  0x00007ffff6394130 in Digikam::DImg::load (this=this@entry=0x3a041d8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, 
>     observer=observer@entry=0x3a03ff0, rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:429
> #4  0x00007ffff6394730 in Digikam::DImg::load (this=this@entry=0x3a041d8, filePath=..., observer=observer@entry=0x3a03ff0, 
>     rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:373
> #5  0x00007ffff6453759 in Digikam::PreviewLoadingTask::execute (this=0x3a03fe0)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/previewtask.cpp:292
> #6  0x00007ffff643eece in Digikam::LoadSaveThread::run (this=0x12aebe0)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavethread.cpp:133
> #7  0x00007ffff647d9be in Digikam::DynamicThread::Private::run (this=0x3acc060)
>     at /home/timo/tmp/digikam/source/core/libs/threads/dynamicthread.cpp:183
> #8  0x00007fffe9569602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007fffe957212b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #10 0x00007ffff51d25a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #11 0x00007fffe6a4322f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

> digikam.dimg: "/home/timo/Pictures/Pictures/Abitur/Zeugnisvergabe/P1030017.JPG"  : JPEG file identified
> digikam.metaengine: Loading image history  ""
> digikam.general: Check for finish:  6 packages, 0 infos to filter, hasFinished() false
> digikam.metaengine: DateTime => Exif.Photo.DateTimeOriginal => QDateTime(2015-06-17 19:51:06.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: DateTime => Exif.Photo.DateTimeDigitized => QDateTime(2015-06-17 19:51:06.000 CEST Qt::TimeSpec(LocalTime))
> digikam.metaengine: Orientation => Exif.Image.Orientation => 1
> digikam.general: Cancel Main Thread
> digikam.general: Check for finish:  5 packages, 0 infos to filter, hasFinished() false
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> [Thread 0x7fff0081b700 (LWP 1902) exited]
> 
> Thread 101 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff32ae6700 (LWP 14013)]
> 0x00007ffff7189598 in typeinfo for Digikam::LoadingTask () from /home/timo/tmp/digikam/source/core/app/libdigikamcore.so.6.0.0
> (gdb) bt
> #0  0x00007ffff7189598 in typeinfo for Digikam::LoadingTask () from /home/timo/tmp/digikam/source/core/app/libdigikamcore.so.6.0.0
> #1  0x00007ffff64536aa in Digikam::PreviewLoadingTask::execute (this=0x4a78360)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/previewtask.cpp:407
> #2  0x00007ffff643eece in Digikam::LoadSaveThread::run (this=0x7fffbc00dd00)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavethread.cpp:133
> #3  0x00007ffff647d9be in Digikam::DynamicThread::Private::run (this=0x3708f20)
>     at /home/timo/tmp/digikam/source/core/libs/threads/dynamicthread.cpp:183
> #4  0x00007fffe9569602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #5  0x00007fffe957212b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #6  0x00007ffff51d25a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #7  0x00007fffe6a4322f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

> digikam.facesengine: detectMultiScale gave  ()
> digikam.facesengine: Verifying face  QRect(186,312 48x48)  using cascade  4
> digikam.facesengine: detectMultiScale: image size  128   128  searchIncrement  1.1  grouping  3  flags  0  min size  29   29 
> 
> 
> Thread 588 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7ffefaffd700 (LWP 9145)]
> 0x00007fffe978d8b0 in QVariant::QVariant(QVariant const&) ()
>    from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007fffe978d8b0 in QVariant::QVariant(QVariant const&) ()
>    from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #1  0x00007ffff6228dcb in QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7ffee4232d40)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:230
> #2  QMapNode<QString, QVariant>::copy (this=0x7ffee42082e0, d=d@entry=0x7ffee4232d40)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:258
> #3  0x00007ffff6228e20 in QMapNode<QString, QVariant>::copy (this=0x7ffee4209880, d=d@entry=0x7ffee4232d40)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:261
> #4  0x00007ffff6228ef2 in QMap<QString, QVariant>::detach_helper (this=0x48af2d8)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:1006
> #5  QMap<QString, QVariant>::detach (this=0x48af2d8)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:364
> #6  QMap<QString, QVariant>::insert (this=0x48af2d8, akey=..., avalue=...)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:704
> #7  0x00007ffff6237a21 in Digikam::DImgLoader::uniqueHashV2 (filePath=..., img=img@entry=0x4cef698)
>     at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/dimgloader.cpp:364
> #8  0x00007ffff6237d07 in Digikam::DImgLoader::createHistoryImageId (filePath=..., image=..., metadata=...)
>     at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/dimgloader.cpp:276
> #9  0x00007ffff62381f3 in Digikam::DImgLoader::readMetadata (this=0x7ffefaffc8d0, filePath=...)
>     at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/dimgloader.cpp:224
> #10 0x00007ffff623fce1 in Digikam::JPEGLoader::load (this=this@entry=0x7ffefaffc8d0, filePath=..., 
>     observer=observer@entry=0x4cef4b0) at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/jpegloader.cpp:92
> #11 0x00007ffff6394130 in Digikam::DImg::load (this=this@entry=0x4cef698, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, 
>     observer=observer@entry=0x4cef4b0, rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:429
> #12 0x00007ffff6394730 in Digikam::DImg::load (this=this@entry=0x4cef698, filePath=..., observer=observer@entry=0x4cef4b0, 
>     rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:373
> #13 0x00007ffff6453759 in Digikam::PreviewLoadingTask::execute (this=0x4cef4a0)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/previewtask.cpp:292
> #14 0x00007ffff643eece in Digikam::LoadSaveThread::run (this=0x7fffbc007d70)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavethread.cpp:133
> #15 0x00007ffff647d9be in Digikam::DynamicThread::Private::run (this=0x4e828a0)
>     at /home/timo/tmp/digikam/source/core/libs/threads/dynamicthread.cpp:183
> #16 0x00007fffe9569602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #17 0x00007fffe957212b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #18 0x00007ffff51d25a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #19 0x00007fffe6a4322f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

Splitting the comment because the bug tracker tells me it is spam otherwise.
Machine was under pretty heavy load memory-wise during the next one, so it may be caused by different reasons:

> digikam.facesengine: detectMultiScale gave  (QRect(40,40 47x47))
> digikam.facesengine: Verifying face  QRect(549,123 47x47)  using cascade  4
> digikam.facesengine: detectMultiScale: image size  127   127  searchIncrement  1.1  grouping  3  flags  0  min size  28   28 
> 
> [Thread 0x7fff0ebe1700 (LWP 4402) exited]
> 
> Thread 1094 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3dae2700 (LWP 31431)]
> 0x00007fffe69ef626 in __memmove_sse2_unaligned_erms () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007fffe69ef626 in __memmove_sse2_unaligned_erms ()
>    from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6
> #1  0x00007ffff639189b in memcpy (__len=<optimized out>, __src=<optimized out>, __dest=<optimized out>)
>     at /nix/store/akak0rxhbi4n87z3nx78ipv76frvj841-glibc-2.27-dev/include/bits/string_fortified.h:34
> #2  Digikam::DImg::detach (this=this@entry=0x4916288) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:217
> #3  0x00007ffff645366c in Digikam::PreviewLoadingTask::execute (this=0x4916090)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/previewtask.cpp:376
> #4  0x00007ffff643eece in Digikam::LoadSaveThread::run (this=0x3746ed0)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavethread.cpp:133
> #5  0x00007ffff647d9be in Digikam::DynamicThread::Private::run (this=0x38b1700)
>     at /home/timo/tmp/digikam/source/core/libs/threads/dynamicthread.cpp:183
> #6  0x00007fffe9569602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #7  0x00007fffe957212b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007ffff51d25a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #9  0x00007fffe6a4322f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

> digikam.general: Entry QRect(1371,1021 218x218)280overlapsQRect(1371,1021 218x218)280, skipping
> 
> Thread 27 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff1ffff700 (LWP 16402)]
> QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff081d6280)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:228
> 228                 new (&n->key) Key(k);
> (gdb) bt
> #0  QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff081d6280)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:228
> #1  QMapNode<QString, QVariant>::copy (this=0xb1, d=d@entry=0x7fff081d6280)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:258
> #2  0x00007ffff6228e20 in QMapNode<QString, QVariant>::copy (this=0x7fff0807c180, d=d@entry=0x7fff081d6280)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:261
> #3  0x00007ffff6228e20 in QMapNode<QString, QVariant>::copy (this=0x7fff0802a9b0, d=d@entry=0x7fff081d6280)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:261
> #4  0x00007ffff6228ef2 in QMap<QString, QVariant>::detach_helper (this=0x4a578d8)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:1006
> #5  QMap<QString, QVariant>::detach (this=0x4a578d8)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:364
> #6  QMap<QString, QVariant>::insert (this=0x4a578d8, akey=..., avalue=...)
>     at /nix/store/zah3h9qlxs79452a3g0pyrxb77i9zrw6-qtbase-5.11.1-dev/include/QtCore/qmap.h:704
> #7  0x00007ffff62405c3 in Digikam::JPEGLoader::load (this=this@entry=0x7fff1fffe8d0, filePath=..., 
>     observer=observer@entry=0x431fc70) at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/jpegloader.cpp:663
> #8  0x00007ffff6394130 in Digikam::DImg::load (this=this@entry=0x431fe58, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, 
>     observer=observer@entry=0x431fc70, rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:429
> #9  0x00007ffff6394730 in Digikam::DImg::load (this=this@entry=0x431fe58, filePath=..., observer=observer@entry=0x431fc70, 
>     rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:373
> #10 0x00007ffff6453759 in Digikam::PreviewLoadingTask::execute (this=0x431fc60)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/previewtask.cpp:292
> #11 0x00007ffff643eece in Digikam::LoadSaveThread::run (this=0x7fffbc00bfd0)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavethread.cpp:133
> #12 0x00007ffff647d9be in Digikam::DynamicThread::Private::run (this=0x4482cb0)
>     at /home/timo/tmp/digikam/source/core/libs/threads/dynamicthread.cpp:183
> #13 0x00007fffe9569602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #14 0x00007fffe957212b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #15 0x00007ffff51d25a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #16 0x00007fffe6a4322f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

Git commit 09266d5d2c97fa5554f32b022d2d62230f0bb061 by Maik Qualmann.
Committed on 28/10/2018 at 19:24.
Pushed by mqualmann into branch 'master'.

only a QHash string key test

M  +2    -0    core/libs/threadimageio/loadingcache.cpp

https://commits.kde.org/digikam/09266d5d2c97fa5554f32b022d2d62230f0bb061

You can find the output in the console:

digikam.general: imageFilePathHash Size: XX

I'm interested in whether the value goes up or stays the same during facial recognition.

Maik

Git commit 72c8e61d48a3dd9b17eca6449cab9c7ba192ce3b by Maik Qualmann.
Committed on 28/10/2018 at 20:58.
Pushed by mqualmann into branch 'master'.

add Q_ASSERT to test if PreviewLoadingTask is running when destructor is called

M  +4    -1    core/libs/threadimageio/loadsavetask.h
M  +4    -0    core/libs/threadimageio/previewtask.cpp

https://commits.kde.org/digikam/72c8e61d48a3dd9b17eca6449cab9c7ba192ce3b

The size changes: https://gist.github.com/timokau/484001d2242c31a8daa0676befb06a20

This was on the 3rd try, the first two actually went through the whole test without crashing. On the first two I didn't have any previews visible, on the 3rd I only had them visible near the end (clicked on "People" to view all the face previews). So probably/maybe it doesn't crash when not displaying previews while scanning.

Did't test 72c8e61d48a3dd9b17eca6449cab9c7ba192ce3b yet.

With 72c8e61d48a3dd9b17eca6449cab9c7ba192ce3b:

> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "IMG-20130707-WA0000.jpg" QSize(800, 479) QSize(800, 479)
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> [Thread 0x7fff10805700 (LWP 4133) exited]
> 
> Thread 1227 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff037fe700 (LWP 13593)]
> 0x00007ffff6236738 in Digikam::DImgLoader::granularity (this=0x7fff037fd8d0, observer=0x4e9c9e8, 
>     total=1920, progressSlice=<optimized out>)
>     at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/dimgloader.cpp:81
> 81              granularity = (int)((total / (20 * progressSlice)) / observer->granularity());
> (gdb) bt
> #0  0x00007ffff6236738 in Digikam::DImgLoader::granularity (this=0x7fff037fd8d0, observer=0x4e9c9e8, total=1920, 
>     progressSlice=<optimized out>) at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/dimgloader.cpp:81
> #1  0x00007ffff62411ad in Digikam::JPEGLoader::load (this=this@entry=0x7fff037fd8d0, filePath=..., 
>     observer=observer@entry=0x4e9c9e8) at /home/timo/tmp/digikam/source/core/libs/dimg/loaders/jpegloader.cpp:460
> #2  0x00007ffff6394130 in Digikam::DImg::load (this=this@entry=0x4e9cbd0, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, 
>     observer=observer@entry=0x4e9c9e8, rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff6394730 in Digikam::DImg::load (this=this@entry=0x4e9cbd0, filePath=..., observer=observer@entry=0x4e9c9e8, 
>     rawDecodingSettings=...) at /home/timo/tmp/digikam/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff6453899 in Digikam::PreviewLoadingTask::execute (this=0x4e9c9d0)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/previewtask.cpp:294
> #5  0x00007ffff643eeee in Digikam::LoadSaveThread::run (this=0x49031c0)
>     at /home/timo/tmp/digikam/source/core/libs/threadimageio/loadsavethread.cpp:133
> #6  0x00007ffff647dafe in Digikam::DynamicThread::Private::run (this=0x40cbc40)
>     at /home/timo/tmp/digikam/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe9569602 in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe957212b in ?? () from /nix/store/br6yml05dzhccpaf4zi48vw3iw1xayzg-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51d25a7 in start_thread () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe6a4322f in clone () from /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

Ok, thanks first. It does not stop where I possibly expected it. But this image "IMG-20130707-WA0000.jpg" is already involved twice in a crash. Of course it can also be a image in front of or behind, because we pre-load images in the background. Maybe take a look.

Maik

Created attachment 115962 [details]
Suspicious image

What do you want me to test with that image?

I just checked and its nothing personal, just some random photo of a screen (because who takes screenshots anymore amiright?). I've attached it.

Is there something I can do to move this forward?

Can the problem still be reproduced with the current git/master version? Gilles has protected many Exiv2 calls with QMutexLocker because Exiv2 is not thread safe. A current bugtrace would be good.

Maik

For some reason I can't run the digikam I've build from source anymore. Not sure what I'm doing wrong. I'm working from a fresh digikam clone at revision 42f181311373002eb464b5dea67b714974a7dc7d". First I remove all leftover state from previous runs:

> rm -f ~/pictures/*.db
> rm -f ~/.config/digikamrc
> rm -rf ~/.local/share/digikam
> rm -rf ~/.cache/digikam

Then I cd into the digikam source folder and run

> cmake -DCMAKE_INSTALL_PREFIX=/home/timo/tmp/res -DCMAKE_BUILD_TYPE=debug
> make -j4
> core/app/digikam

Which first lets me select the folder my collection is in and then crashes with

> digikam.general: AlbumWatch use QFileSystemWatcher
> digikam.general: Database Parameters:
>    Type:                     "QSQLITE"
>    DB Core Name:             "/home/timo/pictures/digikam4.db"
>    DB Thumbs Name:           "/home/timo/pictures/thumbnails-digikam.db"
>    DB Face Name:             "/home/timo/pictures/recognition.db"
>    DB Similarity Name:      "/home/timo/pictures/similarity.db"
>    Connect Options:          ""
>    Host Name:                ""
>    Host port:                -1
>    Internal Server:          false
>    Internal Server Path:     ""
>    Internal Server Serv Cmd: ""
>    Internal Server Init Cmd: ""
>    Username:                 ""
>    Password:                 ""
> 
> digikam.dbengine: Loading SQL code from config file ""
> digikam.dbengine: "Could not open the configuration file <b></b>. This file is installed with the digikam application and is absolutely required to run digikam. Please check your installation."
> KMemoryInfo: Platform identified :  "LINUX"
> KMemoryInfo: TotalRam:  12467781632
> digikam.general: Allowing a cache size of 200 MB
> digikam.dimg: The bundled profile "digikam/profiles/srgb-d65.icm" cannot be found. Check your installation.

As far as I understand its looking for `dbconfig.xml`, which is present in `core/data/database/dbconfig.xml`. Any idea whats going wrong?

The dbconfig.xml can not be found in the installed system, mostly under /usr/share/digikam/database. We use QStandardPaths to find them in the system.

Maik

Okay for some reason it runs now, not sure why. It does still crash, but with a new error:

> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Android/Camera/IMG_20140623_173958.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> double free or corruption (!prev)
> ---Type <return> to continue, or q <return> to quit---
> 
> Thread 11 "Thread (pooled)" received signal SIGABRT, Aborted.
> [Switching to Thread 0x7fff5bfff700 (LWP 29681)]
> 0x00007fffe697dbe0 in raise () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007fffe697dbe0 in raise () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #1  0x00007fffe697edc1 in abort () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #2  0x00007fffe69bf2ac in __libc_message () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #3  0x00007fffe69c549a in malloc_printerr () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #4  0x00007fffe69c63ec in _int_free () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #5  0x00007ffff638ca6d in Digikam::DImg::rotate (this=0x532eab8, angle=angle@entry=Digikam::DImg::ROT90) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:2329
> #6  0x00007ffff638cca7 in Digikam::DImg::rotateAndFlip (this=this@entry=0x532eab8, orientation=<optimized out>) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:2653
> #7  0x00007ffff645c91a in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:335
> #8  0x00007ffff64452cd in Digikam::PreviewLoadingTask::execute (this=0x532e8c0) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:402
> #9  0x00007ffff645c69e in Digikam::LoadSaveThread::run (this=0x7fffbc014870) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #10 0x00007ffff647e1ce in Digikam::DynamicThread::Private::run (this=0x2c75790) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #11 0x00007fffe9560602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007fffe956912b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #13 0x00007ffff51cc5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #14 0x00007fffe6a3b22f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6

Another backtrace:

> digikam.general: Check for finish:  24 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Android/Pictures/EVERNOTE/IMG_20140206_173437.565.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Android/Whatsapp/IMG-20140510-WA0002.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Android/Whatsapp/IMG-20140510-WA0001.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Android/Whatsapp/IMG-20140510-WA0000.jpg"  : JPEG file identified
> 
> Thread 24 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3effd700 (LWP 3844)]
> 0x00007ffff6468a47 in Digikam::LoadingTask::progressInfo (this=0x3cdf000, img=0x3cdf1f8, progress=0.699999988)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) bt
> #0  0x00007ffff6468a47 in Digikam::LoadingTask::progressInfo (this=0x3cdf000, img=0x3cdf1f8, progress=0.699999988)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007ffff623d1e0 in Digikam::JPEGLoader::load (this=this@entry=0x7fff3effc8d0, filePath=..., observer=observer@entry=0x3cdf010)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:470
> #2  0x00007ffff63901a0 in Digikam::DImg::load (this=this@entry=0x3cdf1f8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x3cdf010, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff63907a0 in Digikam::DImg::load (this=this@entry=0x3cdf1f8, filePath=..., observer=observer@entry=0x3cdf010, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff6445095 in Digikam::PreviewLoadingTask::execute (this=0x3cdf000) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #5  0x00007ffff645c69e in Digikam::LoadSaveThread::run (this=0x39d1cc0) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff647e1ce in Digikam::DynamicThread::Private::run (this=0x4417d80) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe9560602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe956912b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51cc5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe6a3b22f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6

Git commit d3e91e5c150c61392c5edb8eb93c5bd30aa54e22 by Maik Qualmann.
Committed on 20/11/2018 at 20:25.
Pushed by mqualmann into branch 'master'.

try to fix a possible deletion of DImg through QCache

M  +9    -1    core/libs/threadimageio/fileio/loadingcache.cpp

https://commits.kde.org/digikam/d3e91e5c150c61392c5edb8eb93c5bd30aa54e22

Git commit 84b51860e7327094c574f41ae0f91ccbdb6b1e9a by Maik Qualmann.
Committed on 20/11/2018 at 21:38.
Pushed by mqualmann into branch 'master'.

use QSharedPointer for the QCache

M  +32   -19   core/libs/threadimageio/fileio/loadingcache.cpp

https://commits.kde.org/digikam/84b51860e7327094c574f41ae0f91ccbdb6b1e9a

No luck:

> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "2013-11-01 20.36.26-3.jpg" QSize(400, 225) QSize(400, 225)
> digikam.facesengine: detectMultiScale: image size  400   225  searchIncrement  1.1  grouping  3  flags  1  min size  0   0 
> 
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Removing old entries ()
> 
> Thread 77 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff1bffe700 (LWP 29247)]
> 0x00007ffff648d1f7 in Digikam::LoadingTask::progressInfo (this=0x412efb0, img=0x412f1a8, progress=0.300000012)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) bt
> #0  0x00007ffff648d1f7 in Digikam::LoadingTask::progressInfo (this=0x412efb0, img=0x412f1a8, progress=0.300000012)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007ffff6261900 in Digikam::JPEGLoader::load (this=this@entry=0x7fff1bffd8d0, filePath=..., observer=observer@entry=0x412efc0)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:470
> #2  0x00007ffff63b48c0 in Digikam::DImg::load (this=this@entry=0x412f1a8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x412efc0, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff63b4ec0 in Digikam::DImg::load (this=this@entry=0x412f1a8, filePath=..., observer=observer@entry=0x412efc0, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff64697b5 in Digikam::PreviewLoadingTask::execute (this=0x412efb0) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #5  0x00007ffff6480dbe in Digikam::LoadSaveThread::run (this=0x430f2b0) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff64a297e in Digikam::DynamicThread::Private::run (this=0x3e75ad0) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe958f602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe959812b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51fb5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe6a6a22f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6

Just now noticed that you pushed two commits, didn't test the second one yet.

Unfortunately that didn't solve it either:

> digikam.facesengine: detectMultiScale gave  (QRect(33,67 125x75))
> digikam.general: Found 1 faces in "IMG-20140822-WA0032.jpg" QSize(1456, 832) QSize(1456, 832)
> digikam.facesengine: detectMultiScale: image size  457   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.general: Removing old entries ()
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Abitur 2015/Bilder Mottowoche & Abisturm, Abitur 2015/Bilder von Judith Meding/Abisturm/IMG_7567.JPG"  : JPEG file identified
> [New Thread 0x7fff2cbdd700 (LWP 14968)]
> 
> Thread 24 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3ffff700 (LWP 7144)]
> 0x00007ffff648c867 in Digikam::LoadingTask::progressInfo (this=0x3937ab0, img=0x3937ca8, progress=0.395999998)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) 
> (gdb) bt
> #0  0x00007ffff648c867 in Digikam::LoadingTask::progressInfo (this=0x3937ab0, img=0x3937ca8, progress=0.395999998)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007ffff6260850 in Digikam::JPEGLoader::load (this=this@entry=0x7fff3fffe8d0, filePath=..., observer=observer@entry=0x3937ac0)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:470
> #2  0x00007ffff63b3810 in Digikam::DImg::load (this=this@entry=0x3937ca8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x3937ac0, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff63b3e10 in Digikam::DImg::load (this=this@entry=0x3937ca8, filePath=..., observer=observer@entry=0x3937ac0, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff6468705 in Digikam::PreviewLoadingTask::execute (this=0x3937ab0) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #5  0x00007ffff647fd0e in Digikam::LoadSaveThread::run (this=0x37a54f0) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff64a1fee in Digikam::DynamicThread::Private::run (this=0x3fbeb10) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe958e602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe959712b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51fa5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe6a6922f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6

Git commit 92403db2394a0acbc0e49a36db5bf0d119d36208 by Maik Qualmann.
Committed on 21/11/2018 at 19:44.
Pushed by mqualmann into branch 'master'.

try to fix wrong overloaded function call

M  +4    -2    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +1    -1    core/libs/threadimageio/fileio/loadsavetask.h
M  +8    -8    core/libs/threadimageio/preview/previewtask.cpp

https://commits.kde.org/digikam/92403db2394a0acbc0e49a36db5bf0d119d36208

You can also watch the messages on the console, you should not see this message:

!!!!!!!!!!!!!!!!!!!!!! Wrong function called !!!!!!!!!!!!!!!!!!!!!!

Maik

Looks like we're on the right track! At first I got the `double free` error again (unfortunately didn't save the backtrace). In that run "Wrong function called" did not occur in the log.

But then I tried again and got the usual segfault. This time Wrong function called *did* occur in the log exactly once immediately before the segfault:

> digikam.general: Found 0 faces in "Ostsee[...]065.JPG" QSize(3264, 2448) QSize(3264, 2448)
> digikam.database: Search result: 3309
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/[...]Ostsee[...]083.JPG"  : JPEG file identified
> digikam.general: !!!!!!!!!!!!!!!!!!!!!! Wrong function called !!!!!!!!!!!!!!!!!!!!!!
> digikam.general: Cancel Main Thread
> 
> Thread 476 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3deaf700 (LWP 13902)]
> 0xffffffffffffffff in ?? ()
> (gdb) bt
> #0  0xffffffffffffffff in ?? ()
> #1  0x00007ffff648c9bc in Digikam::LoadingTask::progressInfo (this=0x314b320, img=<optimized out>, progress=0.850000024)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:67
> #2  0x00007ffff6260850 in Digikam::JPEGLoader::load (this=this@entry=0x7fff3deae8d0, filePath=..., observer=observer@entry=0x314b330)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:470
> #3  0x00007ffff63b3810 in Digikam::DImg::load (this=this@entry=0x314b518, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x314b330, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #4  0x00007ffff63b3e10 in Digikam::DImg::load (this=this@entry=0x314b518, filePath=..., observer=observer@entry=0x314b330, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #5  0x00007ffff6468736 in Digikam::PreviewLoadingTask::execute (this=0x314b320) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #6  0x00007ffff647fd3e in Digikam::LoadSaveThread::run (this=0x3d58b60) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #7  0x00007ffff64a213e in Digikam::DynamicThread::Private::run (this=0x3d5ab60) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #8  0x00007fffe958e602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007fffe959712b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #10 0x00007ffff51fa5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #11 0x00007fffe6a6922f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb)

And this is exactly the point where I am lost. Because this function call should never, never absolutely. Either we have a memory corruption or some system problem with you. The LoadingTask::progressInfo() function gets overloaded and never should the overloaded function called at run time. I do not know if Gilles still has an idea. Otherwise I would try a dynamic cast of the "this" pointer.

Maik

Lets' me take a look 

Gilles

Maik,

The question is why the method LoadingTask::progressInfo() is called in this case. Right ?

Gilles

Git commit b62cd3fb48ae14498c10eb10ef7d06c277c54115 by Maik Qualmann.
Committed on 24/11/2018 at 16:11.
Pushed by mqualmann into branch 'master'.

try with a dynamic cast

M  +4    -2    core/libs/threadimageio/preview/previewtask.cpp

https://commits.kde.org/digikam/b62cd3fb48ae14498c10eb10ef7d06c277c54115

As i can see in the backtrace, the context of this method call is correct, at least...

Gilles

Maik,

Warning. A dynamic cast result must always be checked for null. cppcheck will certainly report a problem here.

So you need to check the validity pointer before to use it, or somewhere in called function.

Gilles

Yes, I know, I will remove the code if it does not help.

Maik

The progressInfo() function is called from the image loaders as DImgLoaderObserver when loading image progress. We have the simple LoadingTask::progressInfo() overloaded by the SharedLoadingTask::progressInfo(). The PreviewLoadingTask is derived from the SharedLoadingTask. Therefore, progressInfo() should always be executed by the SharedLoadingTask.

Maik

Right. 

The Q_DECL_OVERRIDE can perhaps help here if it applied to SharedLoadingTask::progressInfo() virtual method declaration...

But if yes, i don't understand why. The compiler must do the job properly as well...


Gilles Caulier

timokau,

Which compiler version do you use exactly ? GCC or Clang ?

Gilles Caulier

Hmm, the compiler does it right. It goes well thousands of times, it happens at runtime... I fantasize a bit, an Intel Spectra kernel patch problem...?

Maik

gcc 7.3.0.

(The latest attempt is still compiling)

Didn't help :/

> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Android/Whatsapp/IMG-20140322-WA0003.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: !!!!!!!!!!!!!!!!!!!!!! Wrong function called !!!!!!!!!!!!!!!!!!!!!!
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images        INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );" 
>  (QVariant(int, 275), QVariant(int, 275), QVariant(QString, "autodetectedPerson"), QVariant(int, 275), QVariant(int, 275), QVariant(QString, "autodetectedFace"), QVariant(int, 275), QVariant(int, 275), QVariant(QString, "tagRegion"))
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "IMG-20140304-WA0000.jpg" QSize(480, 854) QSize(480, 854)
> digikam.database: Search result: 4962
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> Thread 24 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff589e4700 (LWP 1327)]
> 0x00007ffff648c9b6 in Digikam::LoadingTask::progressInfo (this=0x3ec51b0, img=<optimized out>, progress=0.100000001)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:67
> 67              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) bt
> #0  0x00007ffff648c9b6 in Digikam::LoadingTask::progressInfo (this=0x3ec51b0, img=<optimized out>, progress=0.100000001)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:67
> #1  0x00007ffff626046a in Digikam::JPEGLoader::load (this=this@entry=0x7fff589e38d0, filePath=..., observer=observer@entry=0x3ec51c0)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:374
> #2  0x00007ffff63b3810 in Digikam::DImg::load (this=this@entry=0x3ec53a8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x3ec51c0, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff63b3e10 in Digikam::DImg::load (this=this@entry=0x3ec53a8, filePath=..., observer=observer@entry=0x3ec51c0, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff6468736 in Digikam::PreviewLoadingTask::execute (this=0x3ec51b0) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:293
> #5  0x00007ffff647fd3e in Digikam::LoadSaveThread::run (this=0x7fffbc00c400) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff64a213e in Digikam::DynamicThread::Private::run (this=0x3f41490) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe958e602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe959712b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51fa5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe6a6922f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit e2f5717e14f29b71832dd541faa27c64dedfd0c4 by Maik Qualmann.
Committed on 24/11/2018 at 20:27.
Pushed by mqualmann into branch 'master'.

remove dynamic cast and try with Q_DECL_OVERRIDE

M  +11   -11   core/libs/threadimageio/fileio/loadsavetask.h
M  +2    -4    core/libs/threadimageio/preview/previewtask.cpp
M  +1    -1    core/libs/threadimageio/preview/previewtask.h

https://commits.kde.org/digikam/e2f5717e14f29b71832dd541faa27c64dedfd0c4

"Bus error", thats a new one:

> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "2_5309997771762172030.png" QSize(1236, 781) QSize(1236, 781)
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size  475   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> 
> Thread 29 "Thread (pooled)" received signal SIGBUS, Bus error.
> [Switching to Thread 0x7fff3effd700 (LWP 29373)]
> QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff080024b0)
>     at /nix/store/yd5wgc2yaz8biynyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qmap.h:228
> 228                 new (&n->key) Key(k);
> (gdb) 
> (gdb) bt
> #0  QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff080024b0)
>     at /nix/store/yd5wgc2yaz8biynyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qmap.h:228
> #1  QMapNode<QString, QVariant>::copy (this=0x7243656d69542e32, d=d@entry=0x7fff080024b0)
>     at /nix/store/yd5wgc2yaz8biynyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qmap.h:258
> #2  0x00007ffff6248462 in QMap<QString, QVariant>::detach_helper (this=0x3f18a18) at /nix/store/yd5wgc2yaz8biynyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qmap.h:1006
> #3  QMap<QString, QVariant>::detach (this=0x3f18a18) at /nix/store/yd5wgc2yaz8biynyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qmap.h:364
> #4  QMap<QString, QVariant>::insert (this=0x3f18a18, akey=..., avalue=...) at /nix/store/yd5wgc2yaz8biynyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qmap.h:704
> #5  0x00007ffff6256f91 in Digikam::DImgLoader::uniqueHashV2 (filePath=..., img=img@entry=0x4b97c48) at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:364
> #6  0x00007ffff6257277 in Digikam::DImgLoader::createHistoryImageId (filePath=..., image=..., metadata=...)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:276
> #7  0x00007ffff6257763 in Digikam::DImgLoader::readMetadata (this=0x7fff3effc8d0, filePath=...) at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:224
> #8  0x00007ffff625f251 in Digikam::JPEGLoader::load (this=this@entry=0x7fff3effc8d0, filePath=..., observer=observer@entry=0x4b97a60)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:92
> #9  0x00007ffff63b3810 in Digikam::DImg::load (this=this@entry=0x4b97c48, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x4b97a60, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #10 0x00007ffff63b3e10 in Digikam::DImg::load (this=this@entry=0x4b97c48, filePath=..., observer=observer@entry=0x4b97a60, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #11 0x00007ffff6468736 in Digikam::PreviewLoadingTask::execute (this=0x4b97a50) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #12 0x00007ffff647fd3e in Digikam::LoadSaveThread::run (this=0x39aa620) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #13 0x00007ffff64a213e in Digikam::DynamicThread::Private::run (this=0x5971220) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #14 0x00007fffe958e602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #15 0x00007fffe959712b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #16 0x00007ffff51fa5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #17 0x00007fffe6a6922f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb)

Ran it again, got `SIGABRT` this time. No `Wrong function called` in the logs.

> digikam.general: Using  4  CPU core to run threads
> digikam.general: Action Thread run  1  new jobs
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/[...]/IMG_2038.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Removing old entries ()
> double free or corruption (out)
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images        INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );" 
>  (QVariant(int, 275), QVariant(int, 275), QVariant(QString, "autodetectedPerson"), QVariant(int, 275), QVariant(int, 275), QVariant(QString, "autodetectedFace"), QVariant(int, 275), QVariant(int, 275), QVariant(QString, "tagRegion"))
> 
> Thread 3775 "Thread (pooled)" received signal SIGABRT, Aborted.
> [Switching to Thread 0x7fff36042700 (LWP 7943)]
> 0x00007fffe69abbe0 in raise () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007fffe69abbe0 in raise () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #1  0x00007fffe69acdc1 in abort () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #2  0x00007fffe69ed2ac in __libc_message () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #3  0x00007fffe69f349a in malloc_printerr () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #4  0x00007fffe69f43e0 in _int_free () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #5  0x00007ffff63b00dd in Digikam::DImg::rotate (this=0x40d91f8, angle=angle@entry=Digikam::DImg::ROT90) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:2329
> #6  0x00007ffff63b0317 in Digikam::DImg::rotateAndFlip (this=this@entry=0x40d91f8, orientation=<optimized out>) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:2653
> #7  0x00007ffff647ffba in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:335
> #8  0x00007ffff646896d in Digikam::PreviewLoadingTask::execute (this=0x40d9000) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:402
> #9  0x00007ffff647fd3e in Digikam::LoadSaveThread::run (this=0x46f5830) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #10 0x00007ffff64a213e in Digikam::DynamicThread::Private::run (this=0x3fba940) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #11 0x00007fffe958e602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007fffe959712b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #13 0x00007ffff51fa5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #14 0x00007fffe6a6922f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6

And another try gives a segfault again, with "Wrong function called":

> digikam.general: Found 0 faces in "2013-10-21 20.52.47.jpg" QSize(2448, 3264) QSize(2448, 3264)
> digikam.general: Using  4  CPU core to run threads
> digikam.general: Action Thread run  1  new jobs
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images        INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );" 
>  (QVariant(int, 18), QVariant(int, 18), QVariant(QString, "autodetectedPerson"), QVariant(int, 18), QVariant(int, 18), QVariant(QString, "autodetectedFace"), QVariant(int, 18), QVariant(int, 18), QVariant(QString, "tagRegion"))
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Kamera-Uploads/2013-11-01 19.26.53.jpg"  : JPEG file identified
> digikam.general: !!!!!!!!!!!!!!!!!!!!!! Wrong function called !!!!!!!!!!!!!!!!!!!!!!
> digikam.general: !!!!!!!!!!!!!!!!!!!!!! Wrong function called !!!!!!!!!!!!!!!!!!!!!!
> digikam.facesengine: detectMultiScale: image size  600   800  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.general: !!!!!!!!!!!!!!!!!!!!!! Wrong function called !!!!!!!!!!!!!!!!!!!!!!
> 
> Thread 1435 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff349bd700 (LWP 8535)]
> 0x00007ffe00000002 in ?? ()
> (gdb) bt
> #0  0x00007ffe00000002 in ?? ()
> #1  0x00007ffff3e4f56d in Exiv2::ExifKey::clone() const () from /nix/store/8ikn9snq812jrhx74daix6siza40j01h-exiv2-0.26.2018.06.09/lib/libexiv2.so.26
> #2  0x00007ffff3de0772 in Exiv2::Exifdatum::Exifdatum(Exiv2::Exifdatum const&) () from /nix/store/8ikn9snq812jrhx74daix6siza40j01h-exiv2-0.26.2018.06.09/lib/libexiv2.so.26
> #3  0x00007ffff63de55d in __gnu_cxx::new_allocator<std::_List_node<Exiv2::Exifdatum> >::construct<Exiv2::Exifdatum, Exiv2::Exifdatum const&> (this=<optimized out>, 
>     __p=<optimized out>) at /nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/include/c++/7.3.0/ext/new_allocator.h:136
> #4  std::allocator_traits<std::allocator<std::_List_node<Exiv2::Exifdatum> > >::construct<Exiv2::Exifdatum, Exiv2::Exifdatum const&> (__a=..., __p=<optimized out>)
>     at /nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/include/c++/7.3.0/bits/alloc_traits.h:475
> #5  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::_M_create_node<Exiv2::Exifdatum const&> (this=0x7fff349bc060)
>     at /nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/include/c++/7.3.0/bits/stl_list.h:575
> #6  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::_M_insert<Exiv2::Exifdatum const&> (__position=..., this=0x7fff349bc060)
>     at /nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/include/c++/7.3.0/bits/stl_list.h:1801
> #7  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::emplace_back<Exiv2::Exifdatum const&> (this=0x7fff349bc060)
>     at /nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/include/c++/7.3.0/bits/stl_list.h:1133
> #8  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::_M_initialize_dispatch<std::_List_const_iterator<Exiv2::Exifdatum> > (__last=..., __first=..., 
>     this=0x7fff349bc060) at /nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/include/c++/7.3.0/bits/stl_list.h:1730
> #9  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::list (__x=..., this=0x7fff349bc060)
>     at /nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/include/c++/7.3.0/bits/stl_list.h:655
> #10 Exiv2::ExifData::ExifData (this=0x7fff349bc060) at /nix/store/g6j077w01akw84403iwag4smqv6vsgzv-exiv2-0.26.2018.06.09-dev/include/exiv2/exif.hpp:438
> #11 Digikam::MetaEngine::getExifTagData (this=this@entry=0x7fff349bc160, exifTagName=exifTagName@entry=0x7ffff6cddd55 "Exif.Image.InterColorProfile")
>     at /home/timo/tmp/source/core/libs/metadataengine/engine/metaengine_exif.cpp:682
> #12 0x00007ffff641b474 in Digikam::DMetadata::getIccProfile (this=this@entry=0x7fff349bc160)
>     at /home/timo/tmp/source/core/libs/metadataengine/dmetadata/dmetadata_exif.cpp:85
> #13 0x00007ffff6256aef in Digikam::DImgLoader::checkExifWorkingColorSpace (this=this@entry=0x7fff349bc8d0) at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:291
> #14 0x00007ffff6260296 in Digikam::JPEGLoader::load (this=this@entry=0x7fff349bc8d0, filePath=..., observer=observer@entry=0x3af2a10)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:637
> #15 0x00007ffff63b3810 in Digikam::DImg::load (this=this@entry=0x3af2bf8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x3af2a10, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #16 0x00007ffff63b3e10 in Digikam::DImg::load (this=this@entry=0x3af2bf8, filePath=..., observer=observer@entry=0x3af2a10, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #17 0x00007ffff6468736 in Digikam::PreviewLoadingTask::execute (this=0x3af2a00) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #18 0x00007ffff647fd3e in Digikam::LoadSaveThread::run (this=0x7fffbc00d590) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #19 0x00007ffff64a213e in Digikam::DynamicThread::Private::run (this=0x42a3de0) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #20 0x00007fffe958e602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #21 0x00007fffe959712b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #22 0x00007ffff51fa5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #23 0x00007fffe6a6922f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6

Git commit c67c7a92fc68bdbd2adee91d37e18310382f0d6b by Maik Qualmann.
Committed on 25/11/2018 at 10:51.
Pushed by mqualmann into branch 'master'.

add test debug message and Q_ASSERT

M  +4    -0    core/libs/threadimageio/preview/previewtask.cpp

https://commits.kde.org/digikam/c67c7a92fc68bdbd2adee91d37e18310382f0d6b

Maik,

Perhaps the best way to try to catch exactly the problem is to isolate the threadimageio code used by Face management in an unit test, and to run this code around a large set of images to parse.

This can reveal (or not) some problems. If you can reproduce the problem with an unit test, this want mean certainly that problem is more in-deep in face management level, as i suspect. The crash in preview code is just a side effect due to memory corruption. After an analyze of current code, i don't understand why the wrong virtual method is called in place.

The advantage to use QTest API is that lots of tools available from CLI to proof, benchmark, and hack the code while running. The integration with KDE CI  following all commit permit to check is somebody has broken something in core API.

Look my recent work with meta-engine unit tests, used to consolidate the Exiv2 interface.

Gilles

SIGABRT again, with `loadingTaskStatus` always `false true`.

> digikam.general: ------------> loadingTaskStatus: false true
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Android/Camera/IMG_20140623_173958.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> QPainter::begin: Paint device returned engine == 0, type: 2
> QPainter::setCompositionMode: Painter not active
> QPainter::end: Painter not active, aborted
> QPainter::begin: Paint device returned engine == 0, type: 2
> QPainter::setCompositionMode: Painter not active
> QPainter::end: Painter not active, aborted
> QPainter::begin: Paint device returned engine == 0, type: 2
> QPainter::setCompositionMode: Painter not active
> QPainter::end: Painter not active, aborted
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> QPainter::begin: Paint device returned engine == 0, type: 2
> QPainter::setCompositionMode: Painter not active
> QPainter::end: Painter not active, aborted
> QPainter::begin: Paint device returned engine == 0, type: 2
> QPainter::setCompositionMode: Painter not active
> QPainter::end: Painter not active, aborted
> double free or corruption (!prev)
> 
> Thread 27 "Thread (pooled)" received signal SIGABRT, Aborted.
> [Switching to Thread 0x7fff5a9e8700 (LWP 24238)]
> 0x00007fffe69abbe0 in raise () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007fffe69abbe0 in raise () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #1  0x00007fffe69acdc1 in abort () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #2  0x00007fffe69ed2ac in __libc_message () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #3  0x00007fffe69f349a in malloc_printerr () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #4  0x00007fffe69f43ec in _int_free () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #5  0x00007ffff63b00dd in Digikam::DImg::rotate (this=0x287b478, angle=angle@entry=Digikam::DImg::ROT90) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:2329
> #6  0x00007ffff63b0317 in Digikam::DImg::rotateAndFlip (this=this@entry=0x287b478, orientation=<optimized out>) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:2653
> #7  0x00007ffff648012a in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:335
> #8  0x00007ffff6468ad4 in Digikam::PreviewLoadingTask::execute (this=0x287b280) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:406
> #9  0x00007ffff647feae in Digikam::LoadSaveThread::run (this=0x3ae5b10) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #10 0x00007ffff64a22ae in Digikam::DynamicThread::Private::run (this=0x3aecc40) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #11 0x00007fffe958e602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007fffe959712b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #13 0x00007ffff51fa5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #14 0x00007fffe6a6922f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 45d25606fe419766678774259e5610501face598 by Maik Qualmann.
Committed on 25/11/2018 at 17:01.
Pushed by mqualmann into branch 'master'.

try with a debug check for deleted preview task

M  +4    -1    core/libs/threadimageio/fileio/loadsavetask.h
M  +8    -0    core/libs/threadimageio/preview/previewtask.cpp

https://commits.kde.org/digikam/45d25606fe419766678774259e5610501face598

`PreviewLoadingTaskIsDeleted` is always `false`, at all points.

> digikam.general: ------> Point 3 - PreviewLoadingTask is deleted: false
> digikam.general: Cannot extract preview for "/home/timo/Pictures/pictures/images/Telegram Documents/4_532581008470442788.mp4"
> digikam.facesengine: detectMultiScale gave  ()
> digikam.general: Found 0 faces in "2_5422666759779909661.jpg" QSize(540, 960) QSize(540, 960)
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size  800   533  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> Thread 25 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3f7fe700 (LWP 16867)]
> 0x00007ffff648be66 in Digikam::LoadingTask::progressInfo (this=0x386aa40, 
>     img=<optimized out>, progress=0.899999976)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:67
> 67              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) bt
> #0  0x00007ffff648be66 in Digikam::LoadingTask::progressInfo (this=0x386aa40, img=<optimized out>, progress=0.89999
> 9976)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:67
> #1  0x00007ffff6267999 in Digikam::QImageLoader::load (this=this@entry=0x7fff3f7fd8d0, filePath=..., observer=obser
> ver@entry=0x386aa58)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/qimageloader.cpp:62
> #2  0x00007ffff63b2074 in Digikam::DImg::load (this=this@entry=0x386ac40, filePath=..., loadFlagsInt=loadFlagsInt@e
> ntry=63, observer=observer@entry=0x386aa58, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:565
> #3  0x00007ffff63b2e10 in Digikam::DImg::load (this=this@entry=0x386ac40, filePath=..., observer=observer@entry=0x386aa58, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff64679c9 in Digikam::PreviewLoadingTask::execute (this=0x386aa40) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:296
> #5  0x00007ffff647f1ee in Digikam::LoadSaveThread::run (this=0x387b980) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff64a15ee in Digikam::DynamicThread::Private::run (this=0x3724c90) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe958d602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe959612b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51f95a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe6a6822f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 4f0e00648e3cab9d21fe0b34e7924a5912de5a25 by Maik Qualmann.
Committed on 27/11/2018 at 20:55.
Pushed by mqualmann into branch 'master'.

try with a QMutexLocker for the libjpeg

M  +2    -0    core/libs/dimg/loaders/dimgloader.cpp
M  +4    -1    core/libs/dimg/loaders/dimgloader.h
M  +2    -0    core/libs/dimg/loaders/jpegloader.cpp

https://commits.kde.org/digikam/4f0e00648e3cab9d21fe0b34e7924a5912de5a25

No luck:

> digikam.general: Found 0 faces in "IMG-20130708-WA0000.jpg" QSize(2560, 1920) QSize(2560, 1920)
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size  800   600  searchIncrement  1.1  grouping  3  flags  1  min size  26   26 
> 
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Android/Whatsapp/IMG-20130817-WA0000.jpg"  : JPEG file identified
> 
> Thread 25 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff599e6700 (LWP 6031)]
> 0x00007ffff62485da in std::__atomic_base<int>::load (__m=std::memory_order_relaxed, 
>     this=<optimized out>)
>     at /nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/include/c++/7.3.0/bits/at
> omic_base.h:396
> 396             return __atomic_load_n(&_M_i, __m);
> (gdb) bt
> #0  0x00007ffff62485da in std::__atomic_base<int>::load (__m=std::memory_order_relaxed
> , this=<optimized out>)
>     at /nix/store/6qmfmricr58bjivgr16y9sjisakdykag-gcc-7.3.0/include/c++/7.3.0/bits/at
> omic_base.h:396
> #1  QAtomicOps<int>::load<int> (_q_value=...) at /nix/store/yd5wgc2yaz8biynyjxsh743xyh
> 7bnr77-qtbase-5.11.1-dev/include/QtCore/qatomic_cxx11.h:227
> #2  QBasicAtomicInteger<int>::load (this=<optimized out>) at /nix/store/yd5wgc2yaz8biy
> nyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qbasicatomic.h:103
> #3  QtPrivate::RefCount::isShared (this=<optimized out>) at /nix/store/yd5wgc2yaz8biynyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qrefcount.h:101
> #4  QMap<QString, QVariant>::detach (this=0x3b0e4c8) at /nix/store/yd5wgc2yaz8biynyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qmap.h:364
> #5  QMap<QString, QVariant>::insert (this=0x3b0e4c8, akey=..., avalue=...) at /nix/store/yd5wgc2yaz8biynyjxsh743xyh7bnr77-qtbase-5.11.1-dev/include/QtCore/qmap.h:704
> #6  0x00007ffff6480282 in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:336
> #7  0x00007ffff6468bfd in Digikam::PreviewLoadingTask::execute (this=0x3f790c0) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:402
> #8  0x00007ffff647ffce in Digikam::LoadSaveThread::run (this=0x36c4990) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #9  0x00007ffff64a1afe in Digikam::DynamicThread::Private::run (this=0x3e43d50) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #10 0x00007fffe958e602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #11 0x00007fffe959712b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007ffff51fa5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #13 0x00007fffe6a6922f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb)

OK, not within the JpegLoader, it may be coincidence, we'll see. QMap is definitely not thread save, so we'll put it to the test with a global QMutexLocker.

Maik

Git commit 632d1a58d442b17a7f752f12a1e5d3be6e906d35 by Maik Qualmann.
Committed on 28/11/2018 at 18:49.
Pushed by mqualmann into branch 'master'.

try with a QMutexLocker for the QMap

M  +38   -0    core/libs/dimg/dimg.cpp
M  +3    -0    core/libs/dimg/dimg.h

https://commits.kde.org/digikam/632d1a58d442b17a7f752f12a1e5d3be6e906d35

SIGABRT again:

> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Android/Camera/IMG_20140910_121859.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> double free or corruption (!prev)
> 
> Thread 1833 "Thread (pooled)" received signal SIGABRT, Aborted.
> [Switching to Thread 0x7fff37fff700 (LWP 13905)]
> 0x00007fffe69abbe0 in raise () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007fffe69abbe0 in raise () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #1  0x00007fffe69acdc1 in abort () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> #2  0x00007fffe69ed2ac in __libc_message () from /nix/store/g2yk54hifqlsjiha3szr4q3ccm
> dzyrdv-glibc-2.27/lib/libc.so.6
> #3  0x00007fffe69f349a in malloc_printerr () from /nix/store/g2yk54hifqlsjiha3szr4q3cc
> mdzyrdv-glibc-2.27/lib/libc.so.6
> #4  0x00007fffe69f43ec in _int_free () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrd
> v-glibc-2.27/lib/libc.so.6
> #5  0x00007ffff63b054d in Digikam::DImg::rotate (this=0x2a39558, angle=angle@entry=Dig
> ikam::DImg::ROT90) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:2361
> #6  0x00007ffff63b0797 in Digikam::DImg::rotateAndFlip (this=this@entry=0x2a39558, ori
> entation=<optimized out>) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:2687
> #7  0x00007ffff648061a in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...
> ) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:335
> #8  0x00007ffff6468fcd in Digikam::PreviewLoadingTask::execute (this=0x2a39360) at /ho
> me/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:402
> #9  0x00007ffff648039e in Digikam::LoadSaveThread::run (this=0x3808470) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #10 0x00007ffff64a1ece in Digikam::DynamicThread::Private::run (this=0x3816060) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #11 0x00007fffe958e602 in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007fffe959712b in ?? () from /nix/store/r6n4b9crppc0rmrr2z46xkqa87agydqy-qtbase-5.11.1/lib/libQt5Core.so.5
> #13 0x00007ffff51fa5a7 in start_thread () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread.so.0
> #14 0x00007fffe6a6922f in clone () from /nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc.so.6
> (gdb)

*** Bug 402470 has been marked as a duplicate of this bug. ***

I just opened the now-confirmed duplicate bug regarding crashes during face detection.

Please see my following responses & updates to my bug:

CPU0: Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz (family: 0x6, model: 0x8e, stepping: 0x9)

On a Lenovo T470 Laptop.  Hyperthreading is enabled.

I have re-run Face Detection about 2 dozen times in the name of experiments.

It slowly grows in heap size until crashing around 2 GB.  This is on a machine that has 32GB physical RAM and not much else going on when the crash occurs.

Crash seems to occur quickly when I turn accuracy up (98-100%)

After about two dozen runs, I submitted a job at 80% accuracy (literally no changes from the default values) and it completed all the way with no crash.  Excited to find Digikam running after a full scan, I repeated the same exact scan @ 80% accuracy and it crashed minutes later.

I did not make any other changes to my Digikam album between these runs.  I did not suggest names, review Unconfirmed or Unknown faces, load other photos or otherwise reconfigure Digikam.

I have not yet tried any of the commits mentioned in this thread.

@Timakau, can you please with the current git/master version try if the problem is still to reproduce?

Maik

Still crashes. With yesterday's master (66580b3892f67c1233c14946637b00f3aa053b69):

> digikam.dimg: "/home/timo/Pictures/pictures/Telegram/Telegram Images/246921120_45306.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/Telegram/Telegram Images/246921164_244594.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/Telegram/Telegram Images/246921168_292326.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/Telegram/Telegram Images/246921436_238780.jpg"  : JPEG file identified
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale gave (QRect(200,114 50x50), QRect(336,105 104x104))
> digikam.facesengine: Verifying face QRect(200,114 50x50) using cascade 0
> digikam.facesengine: detectMultiScale: image size ( 130 , 130 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 30 , 30 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(200,114 50x50) using cascade 1
> digikam.facesengine: detectMultiScale: image size ( 130 , 130 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 30 , 30 )
> 
> Thread 3994 "Thread (pooled)" received signal SIGBUS, Bus error.
> [Switching to Thread 0x7ffeea7fc700 (LWP 10727)]
> QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff04611890)
>     at /nix/store/701978sbb55nhqhq1kbrrcyax3pz5brm-qtbase-5.11.1-dev/include/QtCore/qmap.h:228
> 228                 new (&n->key) Key(k);
> (gdb) 
> (gdb) bt
> #0  QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff04611890)
>     at /nix/store/701978sbb55nhqhq1kbrrcyax3pz5brm-qtbase-5.11.1-dev/include/QtCore/qmap.h:228
> #1  QMapNode<QString, QVariant>::copy (this=0x79636e65677255, d=d@entry=0x7fff04611890)
>     at /nix/store/701978sbb55nhqhq1kbrrcyax3pz5brm-qtbase-5.11.1-dev/include/QtCore/qmap.h:258
> #2  0x00007ffff6241af2 in QMap<QString, QVariant>::detach_helper (this=0x7fffac092ba8)
>     at /nix/store/701978sbb55nhqhq1kbrrcyax3pz5brm-qtbase-5.11.1-dev/include/QtCore/qmap.h:1006
> #3  QMap<QString, QVariant>::detach (this=0x7fffac092ba8) at /nix/store/701978sbb55nhqhq1kbrrcyax3pz5brm-qtbase-5.11.1-dev/include/QtCore/qmap.h:364
> #4  QMap<QString, QVariant>::insert (this=0x7fffac092ba8, akey=..., avalue=...) at /nix/store/701978sbb55nhqhq1kbrrcyax3pz5brm-qtbase-5.11.1-dev/include/QtCore/qmap.h:704
> #5  0x00007ffff6250621 in Digikam::DImgLoader::uniqueHashV2 (filePath=..., img=img@entry=0x344b888) at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:364
> #6  0x00007ffff6250907 in Digikam::DImgLoader::createHistoryImageId (filePath=..., image=..., metadata=...)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:276
> #7  0x00007ffff6250df3 in Digikam::DImgLoader::readMetadata (this=0x7ffeea7ec7d0, filePath=...) at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:224
> #8  0x00007ffff62588e1 in Digikam::JPEGLoader::load (this=this@entry=0x7ffeea7ec7d0, filePath=..., observer=observer@entry=0x344b6a0)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:92
> #9  0x00007ffff63aeb60 in Digikam::DImg::load (this=this@entry=0x344b888, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x344b6a0, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #10 0x00007ffff63af160 in Digikam::DImg::load (this=this@entry=0x344b888, filePath=..., observer=observer@entry=0x344b6a0, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #11 0x00007ffff6463deb in Digikam::PreviewLoadingTask::execute (this=0x344b690) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #12 0x00007ffff647b40e in Digikam::LoadSaveThread::run (this=0x4881aa0) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #13 0x00007ffff649cf0e in Digikam::DynamicThread::Private::run (this=0x493b700) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #14 0x00007fffe955b602 in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #15 0x00007fffe956412b in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #16 0x00007ffff51ee5a7 in start_thread () from /nix/store/xdsjx0gba4id3yyqxv66bxnm2sqixkjj-glibc-2.27/lib/libpthread.so.0
> #17 0x00007fffe5b8222f in clone () from /nix/store/xdsjx0gba4id3yyqxv66bxnm2sqixkjj-glibc-2.27/lib/libc.so.6

@Timakau, if I see it correctly in this thread, we have not yet tested the AppImage. Please test if you can reproduce it as well, just download the AppImage and set the executable flag and start it.

https://files.kde.org/digikam/

Maik

Git commit 968f695fe523aeec741c80c64f0c992eb51933e2 by Maik Qualmann.
Committed on 28/12/2018 at 21:24.
Pushed by mqualmann into branch 'master'.

try with non static methods in threads

M  +5    -5    core/libs/dimg/dimg.cpp
M  +124  -0    core/libs/dimg/loaders/dimgloader.cpp
M  +9    -3    core/libs/dimg/loaders/dimgloader.h
M  +24   -0    core/libs/threadimageio/fileio/loadsavethread.cpp
M  +5    -2    core/libs/threadimageio/fileio/loadsavethread.h
M  +2    -2    core/libs/threadimageio/preview/previewtask.cpp
M  +1    -1    core/utilities/queuemanager/tools/decorate/watermark.cpp

https://commits.kde.org/digikam/968f695fe523aeec741c80c64f0c992eb51933e2

Same issue with appimage:

> Digikam::OpenCVFaceDetector::verifyFace: Verifying face QRect(76,118 167x167) using cascade 8
> Digikam::OpenCVFaceDetector::verifyFace: feature QRectF(0.1,0.4 0.8x0.6) QRect(76,118 167x167) QRect(93,185 134x100)
> Digikam::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size ( 134 , 100 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 17 , 17 )
> Digikam::DImg::load: "/home/timo/Pictures/pictures/Telegram/Telegram Images/245013596_303678.jpg"  : JPEG file identified
> Digikam::OpenCVFaceDetector::cascadeResult: detectMultiScale gave (QRect(33,47 64x38))
> Digikam::DetectionWorker::process: Found 1 faces in "239311953_98930.jpg" QSize(1280, 739) QSize(1280, 739)
> Digikam::DatabaseWriter::process: Removing old entries ()
> Digikam::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size ( 613 , 800 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 26 , 26 )
> Digikam::DImg::load: "/home/timo/Pictures/pictures/Telegram/Telegram Images/245013652_48507.jpg"  : JPEG file identified
> 
> Thread 1715 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff229bd700 (LWP 31333)]
> 0x00007ffff63a6e47 in Digikam::LoadingTask::progressInfo (this=0x2c83710, img=0x2c83908, progress=0.345333338)
>     at /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65      /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavetask.cpp: No such file or directory.
> (gdb) bt
> #0  0x00007ffff63a6e47 in Digikam::LoadingTask::progressInfo (this=0x2c83710, img=0x2c83908, progress=0.345333338)
>     at /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007ffff61615c5 in Digikam::JPEGLoader::load (this=this@entry=0x7fff229bc9d0, filePath=..., observer=observer@entry=0x2c83720)
>     at /b/dktemp/digikam-master/core/libs/dimg/loaders/jpegloader.cpp:470
> #2  0x00007ffff62bd649 in Digikam::DImg::load (this=this@entry=0x2c83908, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x2c83720, 
>     rawDecodingSettings=...) at /b/dktemp/digikam-master/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff62bdbc0 in Digikam::DImg::load (this=this@entry=0x2c83908, filePath=..., observer=observer@entry=0x2c83720, rawDecodingSettings=...)
>     at /b/dktemp/digikam-master/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff6383912 in Digikam::PreviewLoadingTask::execute (this=0x2c83710) at /b/dktemp/digikam-master/core/libs/threadimageio/preview/previewtask.cpp:292
> #5  0x00007ffff639abf5 in Digikam::LoadSaveThread::run (this=0x269a9a0) at /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff63c0bee in Digikam::DynamicThread::Private::run (this=0x26a6d30) at /b/dktemp/digikam-master/core/libs/threads/dynamicthread.cpp:183
> #7  0x00000030a10aa60f in ?? () from /home/timo/.cache/appimage-run/4cbb3f534a41c1b5fa3cb473a278eef1129363ad8a0ac0376e4da1932b8a045d/squashfs-root/usr/lib/libQt5Core.so.5
> #8  0x00000030a10ae18a in ?? () from /home/timo/.cache/appimage-run/4cbb3f534a41c1b5fa3cb473a278eef1129363ad8a0ac0376e4da1932b8a045d/squashfs-root/usr/lib/libQt5Core.so.5
> #9  0x00007ffff56dc5a7 in start_thread () from /usr/lib/libpthread.so.0
> #10 0x00007fffefdd122f in clone () from /usr/lib/libc.so.6

Haven't tested the latest patch yet.

With the latest patch:

> digikam.facesengine: detectMultiScale: image size ( 400 , 225 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 0 , 0 )
> digikam.general: Removing old entries ()
> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/Kamera-Uploads/2013-12-11 14.38.38.jpg"  : JPEG file identified
> 
> Thread 460 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff1d7fa700 (LWP 10775)]
> 0x00007fffe95d9120 in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007fffe95d9120 in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #1  0x00007fffe95df25a in operator<(QString const&, QString const&) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #2  0x00007ffff6240ca1 in qMapLessThanKey<QString> (key2=..., key1=...) at /nix/store/701978sbb55nhqhq1kbrrcyax3pz5brm-qtbase-5.11.1-dev/include/QtCore/qmap.h:71
> #3  QMap<QString, QVariant>::insert (this=0x11faf38, akey=..., avalue=...) at /nix/store/701978sbb55nhqhq1kbrrcyax3pz5brm-qtbase-5.11.1-dev/include/QtCore/qmap.h:711
> #4  0x00007ffff6250001 in Digikam::DImgLoader::uniqueHashV2 (this=this@entry=0x7fff1d7ea7d0, filePath=..., img=img@entry=0x3bd3cb8)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:437
> #5  0x00007ffff625027f in Digikam::DImgLoader::createHistoryImageId (this=this@entry=0x7fff1d7ea7d0, filePath=..., image=..., metadata=...)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:302
> #6  0x00007ffff6250706 in Digikam::DImgLoader::readMetadata (this=0x7fff1d7ea7d0, filePath=...) at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:224
> #7  0x00007ffff62586a1 in Digikam::JPEGLoader::load (this=this@entry=0x7fff1d7ea7d0, filePath=..., observer=observer@entry=0x3bd3ad0)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:92
> #8  0x00007ffff63ae920 in Digikam::DImg::load (this=this@entry=0x3bd3cb8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x3bd3ad0, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #9  0x00007ffff63aef20 in Digikam::DImg::load (this=this@entry=0x3bd3cb8, filePath=..., observer=observer@entry=0x3bd3ad0, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #10 0x00007ffff6463baf in Digikam::PreviewLoadingTask::execute (this=0x3bd3ac0) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #11 0x00007ffff647b2be in Digikam::LoadSaveThread::run (this=0x2ea1350) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #12 0x00007ffff649cece in Digikam::DynamicThread::Private::run (this=0x3189540) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #13 0x00007fffe955a602 in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #14 0x00007fffe956312b in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #15 0x00007ffff51ed5a7 in start_thread () from /nix/store/xdsjx0gba4id3yyqxv66bxnm2sqixkjj-glibc-2.27/lib/libpthread.so.0
> #16 0x00007fffe5b8122f in clone () from /nix/store/xdsjx0gba4id3yyqxv66bxnm2sqixkjj-glibc-2.27/lib/libc.so.6


I also got another crash when I was a bit impatient and tried to start face detection while digkam was still looking for new images. Probably unrelated but maybe not:

> digikam.dimg: "/home/timo/Pictures/pictures/images/old-pics/Camera/IMG_20170809_180048.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/images/old-pics/Camera/IMG_20170809_180041.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/images/old-pics/Camera/IMG_20170809_180031.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/images/old-pics/Camera/IMG_20170809_180024.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/images/old-pics/Camera/IMG_20170809_180018.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/Pictures/pictures/images/old-pics/Camera/IMG_20170809_180013.jpg"  : JPEG file identified
> 
> Thread 1 "digikam" received signal SIGSEGV, Segmentation fault.
> 0x0000000002ae4c20 in ?? ()
> (gdb) bt
> #0  0x0000000002ae4c20 in ?? ()
> #1  0x00007fffe9743c49 in QMetaObject::cast(QObject const*) const () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #2  0x00007fffea4fe9d8 in QWidget::isActiveWindow() const () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #3  0x00007fffea547e55 in QStyleOption::init(QWidget const*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #4  0x00007fffea721ec2 in QAbstractItemView::viewOptions() const () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #5  0x00007fffea7236a5 in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #6  0x00007fffea78e6cd in QTreeView::indexRowSizeHint(QModelIndex const&) const () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #7  0x00007fffea78ef5e in QTreeViewPrivate::layout(int, bool, bool) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #8  0x00007fffea796e1e in QTreeView::doItemsLayout() () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #9  0x00007fffea72472d in QAbstractItemView::timerEvent(QTimerEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #10 0x00007fffea79736a in QTreeView::timerEvent(QTimerEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #11 0x00007fffe976a32b in QObject::event(QEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #12 0x00007fffea516223 in QWidget::event(QEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #13 0x00007fffea5b759e in QFrame::event(QEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #14 0x00007fffea5ba2d3 in QAbstractScrollArea::event(QEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #15 0x00007fffea72ddbb in QAbstractItemView::event(QEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #16 0x00007fffea4d702c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #17 0x00007fffea4de5df in QApplication::notify(QObject*, QEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Widgets.so.5
> #18 0x00007fffe973a608 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #19 0x00007fffe9793bce in QTimerInfoList::activateTimers() () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #20 0x00007fffe9794391 in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #21 0x00007fffe55a098f in g_main_context_dispatch () from /nix/store/k2cl7p0cnzvgz36fb7ibj4fpamdg3sax-glib-2.58.1/lib/libglib-2.0.so.0
> #22 0x00007fffe55a0c40 in g_main_context_iterate.isra () from /nix/store/k2cl7p0cnzvgz36fb7ibj4fpamdg3sax-glib-2.58.1/lib/libglib-2.0.so.0
> #23 0x00007fffe55a0e1c in g_main_context_iteration () from /nix/store/k2cl7p0cnzvgz36fb7ibj4fpamdg3sax-glib-2.58.1/lib/libglib-2.0.so.0
> #24 0x00007fffe979473f in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
>    from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #25 0x00007fffc1479461 in ?? ()
>    from /nix/store/316i5ypkifykyggnkqqp538qa1wz50ng-qtbase-5.11.1-bin/lib/qt-5.11/plugins/platforms/../../../../../95njj2bn313887dx4mbpfapqmznfqs7m-qtbase-5.11.1/lib/libQt5XcbQpa.so.5
> #26 0x00007fffe973893a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #27 0x00007fffe9741e00 in QCoreApplication::exec() () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #28 0x0000000000407638 in main (argc=<optimized out>, argv=<optimized out>) at /home/timo/tmp/source/core/app/main/main.cpp:290

For me, the last backtrace is a non sense.

I suspect that the backtrace is corrupted, as GDB is not able to handle the reality in memory to hack.

Following all the stages in this file where Maik try to fix the crash, i propose a different approach. My ideas:

- the crash is related to thumbnail creation, and something in thumbnail storage is broken with face workflow (why, i don't know).
Perhaps the problem is with libpgf codec...

So, to be sure, there is a simple way to switch thumb compression to JPEG for ex, and to see if something look better. This is just for testing of course.

- the crash is related to OpenCV and OpenCL low level uses. Look this comment for ex (also relevant of Face management):

https://bugs.kde.org/show_bug.cgi?id=401253#c4

In this case, at least the OpenCV must be compiled without OpenCL support. This can be certainly done with AppImage, but for distro packages, we need another way in digiKam implementation to make the code more robust when OpenCL is used by OpenCV.

Note: this is why i hate all low level performance improvements as OpenCL framework, which depend in deed of video card device and proprietary modules. You have no control in application level, and no way to stabilize the user code. We want a working code first, the improvements if possible without to degrade all workflow can come later...

And my 10cts €: proprietary video cards modules under Linux are now enough stable as under Windows. So for me this OpenCL framework is not portable and safe. And i do not talk about MacOS...

Gilles

Edit : proprietary video cards modules under Linux are _NOT_ enough
stable as under Windows.

I'll test with OpenCL disabled (currently building opencv with `-DWITH_OPENCL=OFF`).

> there is a simple way to switch thumb compression to JPEG for ex

If you can provide a patch or explain how to do that I'll test that too.

Even if digiKam use PGF to store thumbnails in database, we have plenty of codec usable as alternative. The code is here:

https://cgit.kde.org/digikam.git/tree/core/libs/threadimageio/thumb/thumbnailcreator.cpp#n758


The switch to the right codec depends of the format recorded to the thumbnail database schema at DB creation.

If you compile digiKam yourself, the magic line to touch is located here:

https://cgit.kde.org/digikam.git/tree/core/libs/threadimageio/thumb/thumbnailcreator.cpp#n769

The storage of thumbnails is always forced to PGF. Just use another one as PNG for ex. Of course the thumbnail database size will grow more quickly that PGF as there is no wavelets compression in PNG.

The possible formats as listed here :

https://cgit.kde.org/digikam.git/tree/core/libs/database/thumbsdb/thumbsdb.h#n48

Note that JPEG do not support alpha channel, and cannot be used everywhere, but for JPEG or RAW photo it's fine. I don't recommend to use JPEG2000...

Before to change this line of code, make a database backup before !

recompile code and before to run, remove the database sqlite file for thumbnail. Only this one must be affected by this change.

Start digiKam, the thumbnail database must be recreated from scratch. Start to navigate in album, thumbnail must be generated and appear in icon-view. The cache mechanism must be active when you switch to another album and go back to previous one, the thumbnails already calculated and stored in database must appear immediately.

Now run the face management again, all look if stability is improved.

When you complete the test, revert the source code line changes and recompile. restore the database backup done previously.

Voilà

Gilles

So basically I apply this patch: https://gist.github.com/timokau/4b4ec516d975c0ebdb3f4b65e8b5dc82 correct?

I always test with a copy of my pictures, wipe the databases (`rm ~/pictures/*.db`) and rebuild from scratch so the rest shouldn't be an issue.

yes patch is fine...
Gilles

No luck.


With OpenCL disabled:

> digikam.facesengine: Verifying face QRect(308,190 47x47) using cascade 7
> digikam.facesengine: feature QRectF(0.2,0.25 0.6x0.6) QRect(308,190 47x47) QRect(317,202 28x28)
> digikam.facesengine: detectMultiScale: image size ( 28 , 28 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 5 , 5 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(308,190 47x47) using cascade 8
> digikam.facesengine: feature QRectF(0.1,0.4 0.8x0.6) QRect(308,190 47x47) QRect(313,209 38x28)
> digikam.facesengine: detectMultiScale: image size ( 38 , 28 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 5 , 5 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "2014-02-25 19.05.25.jpg" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> [Thread 0x7ffef8ff9700 (LWP 12284) exited]
> 
> Thread 56 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff34bdd700 (LWP 5075)]
> 0x00007ffff6487767 in Digikam::LoadingTask::progressInfo (this=0x3d488a0, img=0x3d48a98, progress=0.649999976)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) 
> (gdb) bt
> #0  0x00007ffff6487767 in Digikam::LoadingTask::progressInfo (this=0x3d488a0, img=0x3d48a98, progress=0.649999976)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007ffff6259ca0 in Digikam::JPEGLoader::load (this=this@entry=0x7fff34bcd7d0, filePath=..., observer=observer@entry=0x3d488b0)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:470
> #2  0x00007ffff63ae920 in Digikam::DImg::load (this=this@entry=0x3d48a98, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x3d488b0, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff63aef20 in Digikam::DImg::load (this=this@entry=0x3d48a98, filePath=..., observer=observer@entry=0x3d488b0, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff6463baf in Digikam::PreviewLoadingTask::execute (this=0x3d488a0) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #5  0x00007ffff647b2be in Digikam::LoadSaveThread::run (this=0x31cd1e0) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff649cece in Digikam::DynamicThread::Private::run (this=0x2e9b100) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe955a602 in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe956312b in ?? () from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51ed5a7 in start_thread () from /nix/store/xdsjx0gba4id3yyqxv66bxnm2sqixkjj-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe5c9d22f in clone () from /nix/store/xdsjx0gba4id3yyqxv66bxnm2sqixkjj-glibc-2.27/lib/libc.so.6
> (gdb) 


With the patch applied (PNG instead of PGF):


> digikam.dimg: "/home/timo/Pictures/pictures/Pictures/[...]/IMG_7334.JPG"  : JPEG file identified
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "IMG_2038.jpg" QSize(1600, 1200) QSize(1600, 1200)
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size ( 800 , 600 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 26 , 26 )
> 
> Thread 27 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff479ff700 (LWP 15737)]
> 0x00007ffff6487767 in Digikam::LoadingTask::progressInfo (this=0x30299d0, 
>     img=0x3029bc8, progress=0.699999988)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) 
> (gdb) 
> (gdb) bt
> #0  0x00007ffff6487767 in Digikam::LoadingTask::progressInfo (this=0x30299d0, 
>     img=0x3029bc8, progress=0.699999988)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007ffff6259ca0 in Digikam::JPEGLoader::load (this=this@entry=0x7fff479ef7d0, 
>     filePath=..., observer=observer@entry=0x30299e0)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:470
> #2  0x00007ffff63ae920 in Digikam::DImg::load (this=this@entry=0x3029bc8, 
>     filePath=..., loadFlagsInt=loadFlagsInt@entry=63, 
>     observer=observer@entry=0x30299e0, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff63aef20 in Digikam::DImg::load (this=this@entry=0x3029bc8, 
>     filePath=..., observer=observer@entry=0x30299e0, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff6463baf in Digikam::PreviewLoadingTask::execute (this=0x30299d0)
>     at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #5  0x00007ffff647b2be in Digikam::LoadSaveThread::run (this=0x3630640)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff649cece in Digikam::DynamicThread::Private::run (this=0x3437be0)
>     at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffe955a602 in ?? ()
>    from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #8  0x00007fffe956312b in ?? ()
>    from /nix/store/m76jw52z58353hksxm6ws9qw21l7lmmq-qtbase-5.11.1/lib/libQt5Core.so.5
> #9  0x00007ffff51ed5a7 in start_thread ()
>    from /nix/store/xdsjx0gba4id3yyqxv66bxnm2sqixkjj-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffe5c9d22f in clone ()
>    from /nix/store/xdsjx0gba4id3yyqxv66bxnm2sqixkjj-glibc-2.27/lib/libc.so.6
> (gdb)

Here at the last backtrace we find ourselves in the wrong overloaded function, which should never be possible. I think it will be something very strange, causing these crashes. So far only on Intel processors. Although Intel is probably more common, I find it strange. I can not confirm a memory leak, nor any crash here with me.

Maik

Well the backtrace revelate 2 points :

1/ None of thumbnail storage format or OpenCL introduce the dysfunction.

2/ As Maik said, a wrong overloaded function is called from the thumbnail engine.

So i suspect a dysfunction with your compiler, probably a compilation option used when you run make.

Please give me the result of "g++ -v" from a console.

Here with the g++ 5.5.0 (the base from Mageia6), i never seen this kind of crash from the thumbnail engine.

Also, I'm curious to see which options are passed by KF5::ECM component to g++ while compilation. To check this point it's very simple. From a console, go to build directory of thumbnail engine and run :

[gilles@localhost threadimageio]$ export VERBOSE=1
[gilles@localhost threadimageio]$ pwd
/mnt/devel/GIT/6.x/build/core/libs/threadimageio
[gilles@localhost threadimageio]$ make clean
cd /mnt/devel/GIT/6.x/build && make -f CMakeFiles/Makefile2 core/libs/threadimageio/clean
make[1] : on entre dans le répertoire « /mnt/devel/GIT/6.x/build »
make -f core/libs/threadimageio/CMakeFiles/threadimageio_src.dir/build.make core/libs/threadimageio/CMakeFiles/threadimageio_src.dir/clean
make[2] : on entre dans le répertoire « /mnt/devel/GIT/6.x/build »
cd /mnt/devel/GIT/6.x/build/core/libs/threadimageio && /usr/bin/cmake -P CMakeFiles/threadimageio_src.dir/cmake_clean.cmake
make[2] : on quitte le répertoire « /mnt/devel/GIT/6.x/build »
make -f core/libs/threadimageio/CMakeFiles/threadimageio_src_automoc.dir/build.make core/libs/threadimageio/CMakeFiles/threadimageio_src_automoc.dir/clean
make[2] : on entre dans le répertoire « /mnt/devel/GIT/6.x/build »
cd /mnt/devel/GIT/6.x/build/core/libs/threadimageio && /usr/bin/cmake -P CMakeFiles/threadimageio_src_automoc.dir/cmake_clean.cmake
make[2] : on quitte le répertoire « /mnt/devel/GIT/6.x/build »
make[1] : on quitte le répertoire « /mnt/devel/GIT/6.x/build »
[gilles@localhost threadimageio]$ make
cd /mnt/devel/GIT/6.x/build && /usr/bin/cmake -H/mnt/devel/GIT/6.x -B/mnt/devel/GIT/6.x/build --check-build-system CMakeFiles/Makefile.cmake 0
cd /mnt/devel/GIT/6.x/build && /usr/bin/cmake -E cmake_progress_start /mnt/devel/GIT/6.x/build/CMakeFiles /mnt/devel/GIT/6.x/build/core/libs/threadimageio/CMakeFiles/progress.marks
cd /mnt/devel/GIT/6.x/build && make -f CMakeFiles/Makefile2 core/libs/threadimageio/all
make[1] : on entre dans le répertoire « /mnt/devel/GIT/6.x/build »
make -f core/libs/threadimageio/CMakeFiles/threadimageio_src_automoc.dir/build.make core/libs/threadimageio/CMakeFiles/threadimageio_src_automoc.dir/depend
make[2] : on entre dans le répertoire « /mnt/devel/GIT/6.x/build »
cd /mnt/devel/GIT/6.x/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /mnt/devel/GIT/6.x /mnt/devel/GIT/6.x/core/libs/threadimageio /mnt/devel/GIT/6.x/build /mnt/devel/GIT/6.x/build/core/libs/threadimageio /mnt/devel/GIT/6.x/build/core/libs/threadimageio/CMakeFiles/threadimageio_src_automoc.dir/DependInfo.cmake --color=
make[2] : on quitte le répertoire « /mnt/devel/GIT/6.x/build »
make -f core/libs/threadimageio/CMakeFiles/threadimageio_src_automoc.dir/build.make core/libs/threadimageio/CMakeFiles/threadimageio_src_automoc.dir/build
make[2] : on entre dans le répertoire « /mnt/devel/GIT/6.x/build »
[  0%] Automatic moc for target threadimageio_src
...

Capture all the compiler traces generated and post the result here...

Gilles

Gilles, to 2, the crash is not in the thumbnail engine. It is the PreviewTask to load images for perform the face recognition.

Maik

Not face recognition...face detection.

Maik

When I see the last backtrace, the progress is already 0.6999, so almost 70%. It's definitely the same thread, because the DImg address is the same and the jump LoadingTask::progressInfo() instead of SharedLoadingTask::progressInfo(), impossible...

Maik

> So i suspect a dysfunction with your compiler, probably a compilation option used when you run make.

Note that at Maik's request I also tried the appimage and got the same result (https://bugs.kde.org/show_bug.cgi?id=399923#c106).

> Please give me the result of "g++ -v" from a console.

Using built-in specs.
COLLECT_GCC=/nix/store/p7j7qg5cri229ihf8nllwjhzgbvgx5d0-gcc-7.3.0/bin/g++
COLLECT_LTO_WRAPPER=/nix/store/p7j7qg5cri229ihf8nllwjhzgbvgx5d0-gcc-7.3.0/libexec/gcc/x86_64-unknown-linux-gnu/7.3.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: 
Thread model: posix
gcc version 7.3.0 (GCC) 

> Also, I'm curious to see which options are passed by KF5::ECM component to g++ while compilation.

Pastebin: http://sprunge.us/yJdEwP

Can you reproduce the dysfunction using the last digiKam 6.0.0-beta3 just
released ?

https://www.digikam.org/news/2018-12-30-6.0.0-beta3_release_announcement/

There were no relevant commits since the last revision I tested (66580b3892f67c1233c14946637b00f3aa053b69). I assume you just automatically post this in every ticket when a new beta is released?

The only two commits touch the changelog and a couple of comments.

yes, this is an automatic message from a multiple selection of bugzilla files.

Gilles

Hi,

I think i may encounter the same issue. I wrote about my issues in #401253

This is what happens when i run Beta 3 and run scan faces on my collection.
nCVFaceDetector::cascadeResult: detectMultiScale: image size ( 800 , 450 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 26 , 26 )
Digikam::IccManager::IccManager: Cannot open workspace color profile ""
Digikam::DImg::load: "/home/mathias/photos/Osorterat/Mathias/osorterade bilder/Messenger/received_10208789201436022.jpeg"  : JPEG file identified
Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
Digikam::DImg::load: "/home/mathias/photos/Osorterat/Mathias/osorterade bilder/Messenger/received_10208790012496298.jpeg"  : JPEG file identified
/tmp/.mount_digika4WzBy9/AppRun: line 115:  1381 Segmentation fault      (core dumped) digikam $@


Cam I help you in anyway? 

I have just started to test out Digikam to see if I like the workflow.

However my Face detect still crashes in BETA 3 as it did in BETA 2.

I have tested it with both a local database, Centralized MySQL environment.
Tested it on 2 different Windows 10 and also installed a xUbuntu 18.04 and all of them crashes in BETA 3.

I then tested in 5.9 and it seems stable (not had that much time to test).

If you guys need help with debugging just give me a shout. I have both windows / Linux machine running.

We're back to this problem here, and we're pretty sure that the memory corruption is triggered by active OpenCL hardware acceleration. We were able to prevent a crash in the Windows bundle in the OpenCL module for AMD. It is not enough for OpenCV not only to set the option WITH_OPENCL=OFF, a lot of special hardware accelerations are still active (AMD / VA Intel, etc.). Can you please post the output from:

opencv_version --hw
opencv_version --opencl
opencv_version -v --opencl

Maik

Sure, happy this bug isn't being forgotten.

$ opencv_version --hw
3.4.5
OpenCV's HW features list:
    ID=  1 (MMX) -> ON
    ID=  2 (SSE) -> ON
    ID=  3 (SSE2) -> ON
    ID=  4 (SSE3) -> ON
    ID=  5 (SSSE3) -> ON
    ID=  6 (SSE4.1) -> ON
    ID=  7 (SSE4.2) -> ON
    ID=  8 (POPCNT) -> ON
    ID=  9 (FP16) -> ON
    ID= 10 (AVX) -> ON
    ID= 11 (AVX2) -> N/A
    ID= 12 (FMA3) -> N/A
    ID= 13 (AVX512F) -> N/A
    ID= 14 (AVX512BW) -> N/A
    ID= 15 (AVX512CD) -> N/A
    ID= 16 (AVX512DQ) -> N/A
    ID= 17 (AVX512ER) -> N/A
    ID= 18 (AVX512IFMA) -> N/A
    ID= 19 (AVX512PF) -> N/A
    ID= 20 (AVX512VBMI) -> N/A
    ID= 21 (AVX512VL) -> N/A
    ID=100 (NEON) -> N/A
    ID=200 (VSX) -> N/A
    ID=201 (VSX3) -> N/A
    ID=256 (AVX512-SKX) -> N/A
Total available: 10

$ opencv_version --opencl
3.4.5
OpenCL is disabled

$ opencv_version -v --opencl
(build info elided)
OpenCL is disabled

I get a lot more output here at "opencv_version -v --opencl". Please try "opencv_version -v".

Maik

That only gives the first part:

$ opencv_version -v
(build info elided)


(note that that is the literal output, I didn't elide anything)

Seems like my distro is doing that on purpose: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/opencv/3.x.nix#L172

I'll try changing that.

Okay, here it is:

$ opencv_version -v --opencl

General configuration for OpenCV 3.4.5 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            /build/opencv_contrib
    Version control (extra):     unknown

  Platform:
    Timestamp:                   1970-01-01T00:00:01Z
    Host:                        Linux 4.14.104 x86_64
    CMake:                       3.12.1
    CMake generator:             Unix Makefiles
    CMake build tool:            /nix/store/sf2wwp8m7vk61vk34rhn9pslgs0ci2v4-gnumake-4.2.1/bin/make
    Configuration:               Release

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (4 files):          + SSSE3 SSE4_1
      SSE4_2 (1 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (0 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (5 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (10 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (1 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_SKX

  C/C++:
    Built as dynamic libs?:      YES
    C++11:                       YES
    C++ Compiler:                /nix/store/klci955kxil7q32ggran6lnhpnkk8yjz-gcc-wrapper-7.3.0/bin/g++  (ver 7.3.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /nix/store/klci955kxil7q32ggran6lnhpnkk8yjz-gcc-wrapper-7.3.0/bin/gcc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fopenmp -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fopenmp -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      
    Linker flags (Debug):        
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:          dl m pthread rt
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 aruco bgsegm bioinspired calib3d ccalib core datasets dnn dnn_objdetect dpm face features2d flann fuzzy hdf hfs highgui img_hash imgcodecs imgproc java_bindings_generator line_descriptor ml objdetect optflow phase_unwrapping photo plot python_bindings_generator reg rgbd saliency sfm shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv freetype java js matlab ovis python2 python3 ts viz
    Applications:                apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI: 
    GTK+:                        NO
    VTK support:                 NO

  Media I/O: 
    ZLib:                        /nix/store/nsrwlly5ikwhppcslaazp547kh7q7j6s-zlib-1.2.11/lib/libz.so (ver 1.2.11)
    JPEG:                        /nix/store/yr8mbackq6bks0g3vhr7hkzh841r1sv2-libjpeg-turbo-2.0.1/lib/libjpeg.so (ver 62)
    WEBP:                        /nix/store/va9xi1k1w5bsd182k0lqmwki34m7skja-libwebp-1.0.1/lib/libwebp.so (ver encoder: 0x020e)
    PNG:                         /nix/store/px5w0kjsn31dcs1zscqss8zjxkb9j45m-libpng-apng-1.6.35/lib/libpng.so (ver 1.6.35)
    TIFF:                        /nix/store/plihhafcap43qq6qab4f8gzfcmfiy00b-libtiff-4.0.10/lib/libtiff.so (ver 42 / 4.0.10)
    JPEG 2000:                   /nix/store/3bpn824b669kh58ip7165ddmm8sqvjyn-jasper-2.0.14/lib/libjasper.so (ver 2.0.14)
    OpenEXR:                     /nix/store/l6z0wxmsb8psnzsj344ga0v06r685c0a-ilmbase-2.3.0/lib/libImath.so /nix/store/a97cislj7mcj449m9lmmzi2pfki4gwbh-openexr-2.3.0/lib/libIlmImf.so /nix/store/l6z0wxmsb8psnzsj344ga0v06r685c0a-ilmbase-2.3.0/lib/libIex.so /nix/store/l6z0wxmsb8psnzsj344ga0v06r685c0a-ilmbase-2.3.0/lib/libHalf.so /nix/store/l6z0wxmsb8psnzsj344ga0v06r685c0a-ilmbase-2.3.0/lib/libIlmThread.so (ver Unknown)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      NO
      avcodec:                   NO
      avformat:                  NO
      avutil:                    NO
      swscale:                   NO
      avresample:                NO
    GStreamer:                   NO
    libv4l/libv4l2:              NO
    v4l/v4l2:                    linux/videodev2.h

  Parallel framework:            OpenMP

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      YES (/nix/store/m7qq6kqm1zwkjc3li2ar9iwmlr8cfdww-openblas-0.3.4/lib/libopenblas.so)
    Eigen:                       YES (ver 3.3.5)
    Custom HAL:                  NO
    Protobuf:                    /nix/store/m5ssl9mglx7lycznxvik3n2kbrbwn159-protobuf-3.6.1/lib/libprotobuf.so (3.6.1)

  OpenCL:                        YES (no extra features)
    Include path:                /build/source/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python (for build):            NO

  Java:                          
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /nix/store/rw6zc3wpl7ansi127man9bx92f9jr95i-opencv-3.4.5
-----------------------------------------------------------------


OpenCL is disabled

from your last output :

OpenCL:                        YES (no extra features)
    Include path:                /build/source/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

==> Opencl is not disabled

Maik,

What's about to check for CUDA support in OpenCV ?

Gilles

CUDA also seems to be active with me, but I do not have an NVIDIA graphics card anymore. Timokau do you have an NVIDIA graphics card with proprietary drivers? Another thing is OpenMP, my OpenCV version of openSUSE is without OpenMP, timokau's OpenCV is with OpenMP.

Maik

Its possible that I didn't properly disable opencl last time. It looks like I only edited the opencv 2.x build recipe, but then used opencv 3.x for digikam. I'll re-build with `-DWITH_OPENCL=OFF` and `-DWITH_LAPACK=OFF` and try to reproduce.

My graphics card on my desktop is AMD radeon with open source drivers. On my laptop its intel hd graphics with i915. I've seen the segfault on both.

My favorite is OpenMP with you. Build an OpenCV without OpenMP and OpenCL. Look here for other OpenCL options: https://bugs.kde.org/show_bug.cgi?id=401253#c12

Maik

Git commit 6e1533e3e13018e94394409e33be087cad0c8061 by Gilles Caulier.
Committed on 05/03/2019 at 16:22.
Pushed by cgilles into branch 'master'.

disable openmp while compiling OpenCV

M  +1    -0    project/bundles/3rdparty/ext_opencv/CMakeLists.txt

https://commits.kde.org/digikam/6e1533e3e13018e94394409e33be087cad0c8061

Good news,

After 2 weeks of works, the Linux AppImage bundle 64 bits is now reconstructed from scratch with:

- All OpenCV options for CUDA, OPenMP, and OPenCL disabled
- A large upgrade of Qt5 from 5.9.7 to 5.11.3.
- An upgrade to KF5 5.55.
- An upgrade to Ffmpeg 3.3.9
- The fontconfig/freetype integration in the bundle to reduce system dependencies


Files can be downloaded here :

https://files.kde.org/digikam/

Please test and report.

Thanks in advance

Gilles Caulier

The appimage (digikam-6.1.0-git-20190307T101229-x86-64.appimage) also still segfaults.

No backtrace because I don't know of an easy way to do that with appimage. But I doubt that it would be particularly useful anyhow.

Very simple : "./digikam-6.1.0-git-20190307T101229-x86-64.appimage debug"

Gilles Caulier

Okay, did that. Interestingly, the crash happened at the exact moment when I connected a second monitor and set it up with `xrandr`.

> Digikam::DImg::load: "/home/timo/Pictures/pictures/Whatsapp/Media/WhatsApp Images/IMG-20170906-WA0005.jpg"  : JPEG file identified
> Digikam::DImg::load: "/home/timo/Pictures/pictures/Whatsapp/Media/WhatsApp Images/IMG-20170906-WA0006.jpeg"  : JPEG file identified
> Digikam::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size ( 450 , 800 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 26 , 26 )
> Digikam::ActionThreadBase::slotJobFinished: One job is done
> Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> Digikam::FacePipeline::Private::checkFinished: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> Digikam::DImg::load: "/home/timo/Pictures/pictures/Whatsapp/Media/WhatsApp Images/IMG-20170906-WA0008.jpeg"  : JPEG file identified
> Digikam::ActionThreadBase::cancel: Cancel Main Thread
> [Thread 0x7fff557fa700 (LWP 14683) exited]
> 
> Thread 435 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff577fe700 (LWP 13067)]
> 0x0000000000000000 in ?? ()
> (gdb) bt
> #0  0x0000000000000000 in ?? ()
> #1  0x00007ffff69f796d in Digikam::LoadingTask::progressInfo (this=0x4d8f290, img=<optimized out>, progress=0.150000006)
>     at /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #2  0x00007ffff67ac7fd in Digikam::JPEGLoader::load (this=this@entry=0x7fff577fd940, filePath=..., observer=observer@entry=0x4d8f2a0)
>     at /b/dktemp/digikam-master/core/libs/dimg/loaders/jpegloader.cpp:470
> #3  0x00007ffff690a609 in Digikam::DImg::load (this=this@entry=0x4d8f488, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x4d8f2a0, 
>     rawDecodingSettings=...) at /b/dktemp/digikam-master/core/libs/dimg/dimg.cpp:429
> #4  0x00007ffff690ab80 in Digikam::DImg::load (this=this@entry=0x4d8f488, filePath=..., observer=observer@entry=0x4d8f2a0, rawDecodingSettings=...)
>     at /b/dktemp/digikam-master/core/libs/dimg/dimg.cpp:373
> #5  0x00007ffff69d4405 in Digikam::PreviewLoadingTask::execute (this=0x4d8f290) at /b/dktemp/digikam-master/core/libs/threadimageio/preview/previewtask.cpp:292
> #6  0x00007ffff69eb6e5 in Digikam::LoadSaveThread::run (this=0x7ffff000e3a0) at /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #7  0x00007ffff6a1148e in Digikam::DynamicThread::Private::run (this=0x2f52930) at /b/dktemp/digikam-master/core/libs/threads/dynamicthread.cpp:183
> #8  0x0000003aa4eaf7ef in ?? () from /home/timo/.cache/appimage-run/f4633b6497585e35b63894a60bde62dc842e791757a89beb70b9479b22a47952/squashfs-root/usr/lib/libQt5Core.so.5
> #9  0x0000003aa4eb8f14 in ?? () from /home/timo/.cache/appimage-run/f4633b6497585e35b63894a60bde62dc842e791757a89beb70b9479b22a47952/squashfs-root/usr/lib/libQt5Core.so.5
> #10 0x00007ffff6456ef7 in start_thread () from /usr/lib/libpthread.so.0
> #11 0x00007ffff585122f in clone () from /usr/lib/libc.so.6
> (gdb)

The crash is completely different now.

I suppose that you use Color Management with your screens. Can you disable the option in Setup dialog and try again ?

Gilles Caulier

I'm not sure what you mean. What color management? Where do I disable it?

All settings are at their defaults.

Color Management is a workflow settings use to render colors on devices as natural color for human eyes. This include screens, printers, etc... 

More information : https://en.wikipedia.org/wiki/Color_management

In digiKam, the settings is located in Setup dialog to Color Management/Behavior tab. Just turn off the main option on top of this view.

Gilles Caulier

With color management disabled:

> Digikam::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
> Digikam::OpenCVFaceDetector::verifyFace: Verifying face QRect(192,392 46x46) using cascade 8
> Digikam::OpenCVFaceDetector::verifyFace: feature QRectF(0.1,0.4 0.8x0.6) QRect(192,392 46x46) QRect(197,410 37x28)
> Digikam::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size ( 37 , 28 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 5 , 5 )
> Digikam::OpenCVFaceDetector::cascadeResult: detectMultiScale gave ()
> Digikam::DetectionWorker::process: Found 0 faces in "IMG-20131105-WA0002.jpg" QSize(816, 612) QSize(816, 612)
> Digikam::DatabaseWriter::process: Removing old entries ()
> Digikam::OpenCVFaceDetector::cascadeResult: detectMultiScale: image size ( 600 , 800 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 24 , 24 )
> [New Thread 0x7fff557fa700 (LWP 14010)]
> 
> Thread 38 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff60c28700 (LWP 6473)]
> 0x00007ffff69f7967 in Digikam::LoadingTask::progressInfo (this=0x3158ad0, 
>     img=0x3158cc8, progress=0.850000024)
>     at /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65      /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavetask.cpp: No such file or directory.
> (gdb) bt
> #0  0x00007ffff69f7967 in Digikam::LoadingTask::progressInfo (this=0x3158ad0, img=0x3158cc8, progress=0.850000024)
>     at /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007ffff67ac7fd in Digikam::JPEGLoader::load (this=this@entry=0x7fff60c27940, filePath=..., observer=observer@entry=0x3158ae0)
>     at /b/dktemp/digikam-master/core/libs/dimg/loaders/jpegloader.cpp:470
> #2  0x00007ffff690a609 in Digikam::DImg::load (this=this@entry=0x3158cc8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x3158ae0, 
>     rawDecodingSettings=...) at /b/dktemp/digikam-master/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff690ab80 in Digikam::DImg::load (this=this@entry=0x3158cc8, filePath=..., observer=observer@entry=0x3158ae0, rawDecodingSettings=...)
>     at /b/dktemp/digikam-master/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff69d4405 in Digikam::PreviewLoadingTask::execute (this=0x3158ad0) at /b/dktemp/digikam-master/core/libs/threadimageio/preview/previewtask.cpp:292
> #5  0x00007ffff69eb6e5 in Digikam::LoadSaveThread::run (this=0x3eb29e0) at /b/dktemp/digikam-master/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff6a1148e in Digikam::DynamicThread::Private::run (this=0x59e7830) at /b/dktemp/digikam-master/core/libs/threads/dynamicthread.cpp:183
> #7  0x0000003aa4eaf7ef in ?? () from /home/timo/.cache/appimage-run/f4633b6497585e35b63894a60bde62dc842e791757a89beb70b9479b22a47952/squashfs-root/usr/lib/libQt5Core.so.5
> #8  0x0000003aa4eb8f14 in ?? () from /home/timo/.cache/appimage-run/f4633b6497585e35b63894a60bde62dc842e791757a89beb70b9479b22a47952/squashfs-root/usr/lib/libQt5Core.so.5
> #9  0x00007ffff6456ef7 in start_thread () from /usr/lib/libpthread.so.0
> #10 0x00007ffff585122f in clone () from /usr/lib/libc.so.6
> (gdb)

Git commit e31d8c418b9b334d9aa31d4835e34a0c781ae6f4 by Maik Qualmann.
Committed on 15/03/2019 at 22:13.
Pushed by mqualmann into branch 'master'.

fix big memory allocation failure if rotate DImg

M  +4    -4    core/libs/dimg/dimg.cpp

https://commits.kde.org/digikam/e31d8c418b9b334d9aa31d4835e34a0c781ae6f4

Git commit abdf66b5c6ba7fc37fec0853db0e08a45b6a3ad3 by Maik Qualmann.
Committed on 15/03/2019 at 22:29.
Pushed by mqualmann into branch 'master'.

Revert "fix big memory allocation failure if rotate DImg"
Sorry for noise

M  +4    -4    core/libs/dimg/dimg.cpp

https://commits.kde.org/digikam/abdf66b5c6ba7fc37fec0853db0e08a45b6a3ad3

Git commit b89f553cce1174f2a328fb7e271729897346dd44 by Maik Qualmann.
Committed on 16/03/2019 at 06:06.
Pushed by mqualmann into branch 'master'.

test with malloc to allocate DImg memory

M  +9    -9    core/libs/dimg/dimg.cpp
M  +1    -1    core/libs/dimg/dimg_p.h
M  +1    -1    core/libs/dimg/loaders/dimgloader.cpp
M  +3    -8    core/libs/dimg/loaders/dimgloader.h
M  +8    -7    core/libs/dimg/loaders/jp2kloader.cpp
M  +3    -3    core/libs/dimg/loaders/jpegloader.cpp
M  +2    -2    core/libs/dimg/loaders/pngloader.cpp
M  +2    -2    core/libs/dimg/loaders/rawloader.cpp
M  +49   -23   core/libs/dimg/loaders/tiffloader.cpp

https://commits.kde.org/digikam/b89f553cce1174f2a328fb7e271729897346dd44

With the latest commit (build from source, all-default settings):

> digikam.facesengine: detectMultiScale: image size ( 103 , 77 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 13 , 13 )
> digikam.facesengine: detectMultiScale gave (QRect(27,38 46x27))
> digikam.facesengine: Verifying face QRect(357,331 66x66) using cascade 0
> digikam.facesengine: detectMultiScale: image size ( 146 , 146 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 40 , 40 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(357,331 66x66) using cascade 1
> digikam.facesengine: detectMultiScale: image size ( 146 , 146 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 40 , 40 )
> 
> Thread 477 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff42ffc700 (LWP 15258)]
> --Type <RET> for more, q to quit, c to continue without paging--
> 0x00007ffff6d366c8 in Digikam::DImgLoader::granularity (this=0x7fff42fec7d0, 
>     observer=0x3364350, total=3240, progressSlice=<optimized out>)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:81
> 81              granularity = (int)((total / (20 * progressSlice)) / observer->granularity());
> (gdb) 
> (gdb) bt
> #0  0x00007ffff6d366c8 in Digikam::DImgLoader::granularity (this=0x7fff42fec7d0, observer=0x3364350, total=3240, progressSlice=<optimized out>)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:81
> #1  0x00007ffff6d411be in Digikam::JPEGLoader::load (this=this@entry=0x7fff42fec7d0, filePath=..., observer=observer@entry=0x3364350)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:460
> #2  0x00007ffff6e95c80 in Digikam::DImg::load (this=this@entry=0x3364538, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x3364350, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:429
> #3  0x00007ffff6e96280 in Digikam::DImg::load (this=this@entry=0x3364538, filePath=..., observer=observer@entry=0x3364350, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #4  0x00007ffff6f522bb in Digikam::PreviewLoadingTask::execute (this=0x3364340) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #5  0x00007ffff6f6994e in Digikam::LoadSaveThread::run (this=0x4db0000) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff6f8b3ce in Digikam::DynamicThread::Private::run (this=0x2b9d420) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007ffff0f2b022 in ?? () from /nix/store/9mdqkdb7iwdw4ripbqdpazbqnx7wc2qy-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007ffff0f268c2 in ?? () from /nix/store/9mdqkdb7iwdw4ripbqdpazbqnx7wc2qy-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff6500ef7 in start_thread () from /nix/store/681354n3k44r8z90m35hm8945vsp95h1-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffef49922f in clone () from /nix/store/681354n3k44r8z90m35hm8945vsp95h1-glibc-2.27/lib/libc.so.6

Git commit a09223b38b261427b08241b6275c35b5235661cc by Maik Qualmann.
Committed on 16/03/2019 at 11:36.
Pushed by mqualmann into branch 'master'.

For a test the Jpeg loader is deactivated, now the QImage loader is used

M  +2    -1    core/libs/dimg/dimg.cpp

https://commits.kde.org/digikam/a09223b38b261427b08241b6275c35b5235661cc

With a09223b38b261427b08241b6275c35b5235661cc:

> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Action Thread run  1  new jobs
> digikam.dimg: "/home/timo/Pictures/pictures/Whatsapp/Media/WhatsApp Images/IMG-20170904-WA0004.jpg"  : QIMAGE file identified
> [New Thread 0x7fff44de8700 (LWP 2077)]
> 
> Thread 4140 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff37fff700 (LWP 769)]
> 0xffffffffffffffff in ?? ()
> (gdb) bt
> #0  0xffffffffffffffff in ?? ()
> #1  0x00007ffff6f75e5d in Digikam::LoadingTask::progressInfo (this=0x36a62a0, img=<optimized out>, progress=0.899999976)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #2  0x00007ffff6d49329 in Digikam::QImageLoader::load (this=this@entry=0x7fff37fef7d0, filePath=..., observer=observer@entry=0x36a62b0)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/qimageloader.cpp:62
> #3  0x00007ffff6e95751 in Digikam::DImg::load (this=this@entry=0x36a6498, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x36a62b0, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:566
> #4  0x00007ffff6e96370 in Digikam::DImg::load (this=this@entry=0x36a6498, filePath=..., observer=observer@entry=0x36a62b0, rawDecodingSettings=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg.cpp:373
> #5  0x00007ffff6f5240b in Digikam::PreviewLoadingTask::execute (this=0x36a62a0) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #6  0x00007ffff6f69a9e in Digikam::LoadSaveThread::run (this=0x2f4d670) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #7  0x00007ffff6f8b51e in Digikam::DynamicThread::Private::run (this=0x5288720) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #8  0x00007ffff0f2b022 in ?? () from /nix/store/9mdqkdb7iwdw4ripbqdpazbqnx7wc2qy-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff0f268c2 in ?? () from /nix/store/9mdqkdb7iwdw4ripbqdpazbqnx7wc2qy-qtbase-5.12.0/lib/libQt5Core.so.5
> #10 0x00007ffff6500ef7 in start_thread () from /nix/store/681354n3k44r8z90m35hm8945vsp95h1-glibc-2.27/lib/libpthread.so.0
> #11 0x00007fffef49922f in clone () from /nix/store/681354n3k44r8z90m35hm8945vsp95h1-glibc-2.27/lib/libc.so.6
> (gdb)

Maik, i'm back to this bug as i found some help with the clang 8 used to compile whole digiKam.

Maegia 7 beta 3 come with this version of clang (in Mageia 6, only clang 3.9 was provided). The step is enormous and the results very interesting.

The compilation with clang 8 generate plenty of warnings about virtual functions.

Gilles

I will generate a full backtrace and post here. Please be patient, i must update the whole Mageia 7 before.

Note : Compilation with Clang is more slower than Gcc 8 (also available under Mageia 7, but there are plenty of test and cache generated in background. The clang-tools include also a collection of CLI programs that we can use to format or port digiKam code to a more recent C++11 syntax (as zero pointer init => numptr). Look this commit for ex :

https://cgit.kde.org/kipi-plugins.git/commit/?id=df28fc170c916cad267993a532b08b8fe4fd4c43

Gilles

Thank you both for not forgetting about this.

Created attachment 119667 [details]
clang 8 compilation trace

Maik,

Look well the warnings around core/libs/threadimageio/*

There are plenty of "-Winconsistent-missing-override" issues. I don't know why GCC 8 is not verbose like this...

Gilles

We suppress the messages with GCC. I will reactivate the warnings and fix everything step by step...there are many...
I suspect that Clang does not have a CLI tool for these warningst o automatically fix it in the source code..

Maik

I can still reproduce this (or at least some segfault during face dection) with current master (33d0457e20adda97c003f3dee652a1749406ff9f):

> digikam.facesengine: detectMultiScale: image size ( 480 , 800 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 24 , 24 )
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.dimg: "/home/timo/tmp/deletable-pics/Android/Whatsapp/IMG-20130807-WA0006.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/tmp/deletable-pics/Android/Whatsapp/IMG-20130809-WA0000.jpg"  : JPEG file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "IMG-20130705-WA0000.jpg" QSize(480, 800) QSize(480, 800)
> digikam.facesengine: detectMultiScale: image size ( 800 , 600 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 24 , 24 )
> digikam.dimg: "/home/timo/tmp/deletable-pics/P/G/IMG_1683.JPG"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Using  4  CPU core to run threads
> [New Thread 0x7ffeebfff700 (LWP 15844)]
> digikam.general: Action Thread run  1  new jobs
> [New Thread 0x7fff23ffe700 (LWP 15845)]
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images        INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );" 
>  (QVariant(int, 281), QVariant(int, 281), QVariant(QString, "autodetectedPerson"), QVariant(int, 281), QVariant(int, 281), QVariant(QString, "autodetectedFace"), QVariant(int, 281), QVariant(int, 281), QVariant(QString, "tagRegion"))
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> corrupted size vs. prev_size
> digikam.database: Search result: 3031
> digikam.general: Removing old entries ()
> 
> Thread 48 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3491f700 (LWP 18350)]
> 0x00007fffef39672a in operator==(QString const&, QString const&) ()
>    from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007fffef39672a in operator==(QString const&, QString const&) ()
>    from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #1  0x00007ffff6f5a575 in Digikam::LoadingDescription::equalsIgnoreReducedVersion (this=<optimized out>, other=...)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadingdescription.cpp:313
> #2  0x00007ffff6f65f43 in Digikam::SharedLoadingTask::notifyNewLoadingProcess (this=0x2edc370, process=<optimized out>, 
>     description=...) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavetask.cpp:429
> #3  0x00007ffff6f5bcb0 in Digikam::LoadingCache::notifyNewLoadingProcess (this=this@entry=0x5e59e0, 
>     process=process@entry=0x3a9ef60, description=...) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadingcache.cpp:240
> #4  0x00007ffff6f416c4 in Digikam::PreviewLoadingTask::execute (this=0x3a9ed90)
>     at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:149
> #5  0x00007ffff6f58cde in Digikam::LoadSaveThread::run (this=0x6e416d0)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff6f79ffe in Digikam::DynamicThread::Private::run (this=0x6d497c0)
>     at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffef3127d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007fffef30e062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff64e1ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffed6eb22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 913484add447c51d0dc6d93fc4324870368f8136 by Maik Qualmann.
Committed on 08/08/2019 at 21:15.
Pushed by mqualmann into branch 'master'.

clear LoadingProcessListener QList

M  +5    -0    core/libs/threadimageio/fileio/loadsavetask.h

https://invent.kde.org/kde/digikam/commit/913484add447c51d0dc6d93fc4324870368f8136

With commit 913484add447c51d0dc6d93fc4324870368f8136:

> digikam.facesengine: detectMultiScale: image size ( 122 , 136 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 34 , 34 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(718,194 56x56) using cascade 4
> digikam.facesengine: detectMultiScale: image size ( 122 , 136 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 34 , 34 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(718,194 56x56) using cascade 5
> digikam.facesengine: feature QRectF(0,0 0.6x0.6) QRect(718,194 56x56) QRect(718,194 34x34)
> digikam.facesengine: detectMultiScale: image size ( 34 , 34 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 6 , 6 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(718,194 56x56) using cascade 6
> digikam.facesengine: feature QRectF(0.4,0 0.6x0.6) QRect(718,194 56x56) QRect(740,194 34x34)
> digikam.facesengine: detectMultiScale: image size ( 34 , 34 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 6 , 6 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(718,194 56x56) using cascade 7
> digikam.facesengine: feature QRectF(0.2,0.25 0.6x0.6) QRect(718,194 56x56) QRect(729,208 34x34)
> digikam.facesengine: detectMultiScale: image size ( 34 , 34 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 6 , 6 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(718,194 56x56) using cascade 8
> digikam.facesengine: feature QRectF(0.1,0.4 0.8x0.6) QRect(718,194 56x56) QRect(724,216 45x34)
> digikam.facesengine: detectMultiScale: image size ( 45 , 34 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 6 , 6 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 1 faces in "IMG-20130919-WA0005.jpg" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> 
> Thread 42 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff23b6a700 (LWP 356)]
> QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff1461f710) at /nix/store/hy0z060pmjwrdrbb0q
> rii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:228
> 228                 new (&n->key) Key(k);
> (gdb) bt
> #0  QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff1461f710)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:228
> #1  QMapNode<QString, QVariant>::copy (this=0x61, d=d@entry=0x7fff1461f710)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:258
> #2  0x00007ffff78d6150 in QMapNode<QString, QVariant>::copy (this=0x7fff146217d0, d=d@entry=0x7fff1461f710)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:261
> #3  0x00007ffff78d6150 in QMapNode<QString, QVariant>::copy (this=0x7fff1460f9a0, d=d@entry=0x7fff1461f710)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:261
> #4  0x00007ffff7c28198 in QMap<QString, QVariant>::detach_helper (this=0x27ae498)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:1008
> #5  0x00007ffff6e8638d in QMap<QString, QVariant>::detach (this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:364
> #6  QMap<QString, QVariant>::insert (avalue=..., akey=..., this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:706
> #7  Digikam::DImg::setAttribute (this=<optimized out>, key=..., value=...)
>     at /home/timo/tmp/source/core/libs/dimg/dimg_props.cpp:207
> #8  0x00007ffff6f5a812 in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:336
> #9  0x00007ffff6f42c7f in Digikam::PreviewLoadingTask::execute (this=0x2f20f40)
>     at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:302
> #10 0x00007ffff6f5a55e in Digikam::LoadSaveThread::run (this=0x295c600)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #11 0x00007ffff6f7b84e in Digikam::DynamicThread::Private::run (this=0x2b6f480)
>     at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #12 0x00007fffef3137d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #13 0x00007fffef30f062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #14 0x00007ffff64e2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #15 0x00007fffed6ec22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Maik,

Probably not relevant of this crash, but at least this can be fixed easily :

https://www.digikam.org/reports/cppcheck/master/147.html#line-468

Gilles

timokau,

The face detection algorithm will be completely new and based on DNN. This algorithm come from OpenCV library.

It's the first step to rewrite whole Face detection and recognition engine, all based on DNN.

https://community.kde.org/GSoC/2019/StatusReports/ThanhTrungDinh#Work_report

I recognition is really improved. Here no crash with 20000 items scanned and 2500 faces detected. Only few false positive (less than 10), and algorithm can now detect dog, cat, cut faces, masked faced. The results are really impressive.

Gilles Caulier

(In reply to caulier.gilles from comment #159)
> Probably not relevant of this crash, but at least this can be fixed easily :
> 
> https://www.digikam.org/reports/cppcheck/master/147.html#line-468

Yes, I had not fixed that yet, because then the merge with the branch of Thanh is no longer possible and it is already fixed there. But I also think that it is not the cause. Strange is just that very few users can ever reproduce it. But I have another idea...

Maik

Git commit fb025c9ca6c4274d20a3af4571f58094113eb734 by Maik Qualmann.
Committed on 09/08/2019 at 10:31.
Pushed by mqualmann into branch 'master'.

just another attempt to fix the crash

M  +2    -2    core/libs/threadimageio/fileio/loadsavetask.cpp

https://invent.kde.org/kde/digikam/commit/fb025c9ca6c4274d20a3af4571f58094113eb734

The face tooling updates are exciting! I was aware of the ongoing recognition work, but didn't know it also covered detection. Lots of interesting areas of improvement here, for example using active learning and semi-supervised clustering to minimize human input. Exciting to see where this goes :)

Unfortunately no luck with fb025c9ca6c4274d20a3af4571f58094113eb734:

> digikam.dimg: "/home/timo/tmp/deletable-pics/Android/Whatsapp/IMG-20131016-WA0002.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/tmp/deletable-pics/Android/Whatsapp/IMG-20131016-WA0003.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/tmp/deletable-pics/Android/Whatsapp/IMG-20131016-WA0004.jpg"  : JPEG file identified
> digikam.dimg: "/home/timo/tmp/deletable-pics/Android/Whatsapp/IMG-20131018-WA0001.jpg"  : JPEG file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.dimg: "/home/timo/tmp/deletable-pics/Android/Whatsapp/IMG-20131020-WA0000.jpg"  : JPEG file identified
> digikam.general: Cancel Main Thread
> [Thread 0x7fff27fff700 (LWP 7030) exited]
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/home/timo/tmp/deletable-pics/Android/Whatsapp/IMG-20131020-WA0002.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "IMG-20131014-WA0000.jpg" QSize(800, 450) QSize(800, 450)
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size ( 640 , 480 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 22 , 22 )
> 
> Thread 30 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff2ffff700 (LWP 22236)]
> 0x0000006c00657261 in ?? ()
> (gdb) bt
> #0  0x0000006c00657261 in ?? ()
> #1  0x00007ffff6072945 in Exiv2::ExifKey::clone() const ()
>    from /nix/store/56932a3lcqp48131p0w0ggr1lhxxna3v-exiv2-0.27.1/lib/libexiv2.so.27
> #2  0x00007ffff6024bb2 in Exiv2::Exifdatum::Exifdatum(Exiv2::Exifdatum const&) ()
>    from /nix/store/56932a3lcqp48131p0w0ggr1lhxxna3v-exiv2-0.27.1/lib/libexiv2.so.27
> #3  0x00007ffff6eae6dd in __gnu_cxx::new_allocator<std::_List_node<Exiv2::Exifdatum> >::construct<Exiv2::Exifdatum, Exiv2::Exifdatum
>  const&> (this=<optimized out>, __p=<optimized out>)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/ext/new_allocator.h:136
> #4  std::allocator_traits<std::allocator<std::_List_node<Exiv2::Exifdatum> > >::construct<Exiv2::Exifdatum, Exiv2::Exifdatum const&>
>  (__a=..., __p=<optimized out>) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/alloc_traits.h:475
> #5  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::_M_create_node<Exiv2::Exifdatum const&> (
>     this=0x7fff2ffeecf0) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:575
> #6  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::_M_insert<Exiv2::Exifdatum const&> (__position=..., 
>     this=0x7fff2ffeecf0) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:1801
> #7  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::emplace_back<Exiv2::Exifdatum const&> (
>     this=0x7fff2ffeecf0) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:1133
> #8  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::_M_initialize_dispatch<std::_List_const_iterator<Exiv2:
> :Exifdatum> > (__last=..., __first=..., this=0x7fff2ffeecf0)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:1730
> #9  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::list (__x=..., this=0x7fff2ffeecf0)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:655
> #10 Exiv2::ExifData::ExifData (this=0x7fff2ffeecf0)
>     at /nix/store/291xr43ngjk60aw7ldshmajbavn5ajn8-exiv2-0.27.1-dev/include/exiv2/exif.hpp:434
> #11 Digikam::MetaEngine::getExifTagData (this=this@entry=0x7fff2ffeedf0, 
>     exifTagName=exifTagName@entry=0x7ffff73790cd "Exif.Image.InterColorProfile")
>     at /home/timo/tmp/source/core/libs/metadataengine/engine/metaengine_exif.cpp:682
> #12 0x00007ffff6eeb904 in Digikam::DMetadata::getIccProfile (this=this@entry=0x7fff2ffeedf0)
>     at /home/timo/tmp/source/core/libs/metadataengine/dmetadata/dmetadata_exif.cpp:85
> #13 0x00007ffff6d2282f in Digikam::DImgLoader::checkExifWorkingColorSpace (this=this@entry=0x7fff2ffef590)
>     at /home/timo/tmp/source/core/libs/dimg/loaders/dimgloader.cpp:301
> #14 0x00007ffff6d2bfeb in Digikam::JPEGLoader::load (this=this@entry=0x7fff2ffef590, filePath=..., 
>     observer=observer@entry=0x438adc0) at /home/timo/tmp/source/core/libs/dimg/loaders/jpegloader.cpp:637
> #15 0x00007ffff6e7eb37 in Digikam::DImg::load (this=this@entry=0x438afa8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, 
>     observer=observer@entry=0x438adc0, rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg_fileio.cpp:126
> #16 0x00007ffff6e7f370 in Digikam::DImg::load (this=this@entry=0x438afa8, filePath=..., observer=observer@entry=0x438adc0, 
>     rawDecodingSettings=...) at /home/timo/tmp/source/core/libs/dimg/dimg_fileio.cpp:63
> #17 0x00007ffff6f419ab in Digikam::PreviewLoadingTask::execute (this=0x438adb0)
>     at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:292
> #18 0x00007ffff6f590ae in Digikam::LoadSaveThread::run (this=0x7fffd0015b30)
>     at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #19 0x00007ffff6f7a35e in Digikam::DynamicThread::Private::run (this=0x2a940a0)
>     at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #20 0x00007fffef3127d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #21 0x00007fffef30e062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #22 0x00007ffff64e1ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #23 0x00007fffed6eb22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Timokau,

If you want to test the current development code for the new face detection/recognition based on DNN, the git branch is here :

https://cgit.kde.org/digikam.git/log/?h=gsoc19-face-recognition

This branch is synchronized with master and is up to date.

You need at least opencv 3.4.6 (i tested with this version) or opencv 4.x

To test, you need to setup a new repository with test images and new database.
after a fresh DK session, and a complete scan for new items to populate the database, go to Face tab from left side bar and run a face detection only process. Do not touch any other option than one to select DNN algorithm in advanced section.

Let's the process running and wait to see if faces are detected properly, without crash and good result.

The face recognition is not yet finalized. It do not give results here yet.

Gilles Caulier

Git commit 1c0cc93ec8d4707473eeaf0a15f12c1687bce653 by Maik Qualmann.
Committed on 09/08/2019 at 20:29.
Pushed by mqualmann into branch 'master'.

a new interesting attempt to fix the crash

M  +26   -14   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +18   -14   core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/1c0cc93ec8d4707473eeaf0a15f12c1687bce653

Git commit b436cb9c0b04da9efff0bda66cf10b71271a1857 by Maik Qualmann.
Committed on 09/08/2019 at 21:44.
Pushed by mqualmann into branch 'master'.

and this in addition

M  +1    -1    core/libs/threadimageio/fileio/loadsavetask.cpp

https://invent.kde.org/kde/digikam/commit/b436cb9c0b04da9efff0bda66cf10b71271a1857

Unfortunately no luck with either approach :/

With b436cb9c0b04da9efff0bda66cf10b71271a1857:

> digikam.facesengine: Verifying face QRect(552,144 49x49) using cascade 8
> digikam.facesengine: feature QRectF(0.1,0.4 0.8x0.6) QRect(552,144 49x49) QRect(557,164 39x29)
> digikam.facesengine: detectMultiScale: image size ( 39 , 29 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 5 , 5 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "IMG-20171231-WA0001.jpg" QSize(1280, 1105) QSize(1280, 1105)
> digikam.facesengine: detectMultiScale: image size ( 800 , 599 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 26 , 26 )
> 
> Thread 30 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff377fe700 (LWP 25666)]
> 0x00007ffff6e8536b in std::__atomic_base<int>::load (__m=std::memory_order_relaxed, this=<optimized out>)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/atomic_base.h:396
> 396             return __atomic_load_n(&_M_i, __m);
> (gdb) bt
> #0  0x00007ffff6e8536b in std::__atomic_base<int>::load (__m=std::memory_order_relaxed, this=<optimized out>) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/atomic_base.h:396
> #1  QAtomicOps<int>::load<int> (_q_value=...) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qatomic_cxx11.h:227
> #2  QBasicAtomicInteger<int>::load (this=<optimized out>) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qbasicatomic.h:103
> #3  QtPrivate::RefCount::isShared (this=<optimized out>) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qrefcount.h:101
> #4  QMap<QString, QVariant>::detach (this=<optimized out>) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:364
> #5  QMap<QString, QVariant>::insert (avalue=..., akey=..., this=<optimized out>) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:706
> #6  Digikam::DImg::setAttribute (this=0x2b41058, key=..., value=...) at /home/timo/tmp/source/core/libs/dimg/dimg_props.cpp:207
> #7  0x00007ffff6f592d2 in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:336
> #8  0x00007ffff6f41a61 in Digikam::PreviewLoadingTask::execute (this=0x2b40e60) at /home/timo/tmp/source/core/libs/threadimageio/preview/previewtask.cpp:298
> #9  0x00007ffff6f5901e in Digikam::LoadSaveThread::run (this=0x2abfc40) at /home/timo/tmp/source/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #10 0x00007ffff6f7a28e in Digikam::DynamicThread::Private::run (this=0x2ceb1d0) at /home/timo/tmp/source/core/libs/threads/dynamicthread.cpp:183
> #11 0x00007fffef3127d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #12 0x00007fffef30e062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #13 0x00007ffff64e1ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #14 0x00007fffed6eb22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6

And with the new detection/recognition code (at 0b749b2623ee3b0b9561bf9a47178e8fbdbe8c55):

> String, "autodetectedFace"), QVariant(int, 281), QVariant(int, 281), QVariant(QString, "tagRegion"))
> digikam.facedb: nnmodel:  "" , nndata  ""
> digikam.facesengine: cv::Exception: OpenCV(3.4.7) /build/source/modules/dnn/src/caffe/caffe_io.cpp:1121: error: (-2:Unspecified error
> ) FAILED: fs.is_open(). Can't open "" in function 'ReadProtoFromTextFile'
> 
> digikam.general: Found 0 faces in "IMG-20170907-WA0003.jpg" QSize(1280, 720) QSize(1280, 720)
> digikam.facedb: nnmodel:  "" , nndata  ""
> digikam.facesengine: cv::Exception: OpenCV(3.4.7) /build/source/modules/dnn/src/caffe/caffe_io.cpp:1121: error: (-2:Unspecified error
> ) FAILED: fs.is_open(). Can't open "" in function 'ReadProtoFromTextFile'
> 
> digikam.general: Found 0 faces in "IMG-20170907-WA0004.jpg" QSize(1032, 581) QSize(1032, 581)
> digikam.dimg: "/home/timo/tmp/deletable-pics/Syncthing-3TB/Oneplus One/Whatsapp/Media/WhatsApp Images/IMG-20170907-WA0006.jpg"  : JPE
> G file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.database: Search result: 36
> digikam.general: Removing old entries ()
> digikam.general: Cancel Main Thread
> [Thread 0x7fff37fff700 (LWP 17296) exited]
> digikam.general: One job is done
> digikam.general: Cancel Main Thread
> digikam.general: Cannot extract preview for "/home/timo/tmp/deletable-pics/Syncthing-3TB/Oneplus One/Whatsapp/Media/WhatsApp Images/I
> MG-20171021-WA0000.jpeg"
> digikam.dimg: "/home/timo/tmp/deletable-pics/Syncthing-3TB/Oneplus One/Whatsapp/Media/WhatsApp Images/IMG-20170907-WA0007.jpg"  : JPE
> G file identified
> 
> Thread 4830 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff36ffd700 (LWP 16919)]
> 0x00007fffef48e7e0 in operator<(QString const&, QString const&) ()
>    from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007fffef48e7e0 in operator<(QString const&, QString const&) ()
>    from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #1  0x00007ffff6f760f9 in qMapLessThanKey<QString> (key2=..., key1=...)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:71
> #2  QMap<QString, QVariant>::insert (avalue=..., akey=..., this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:713
> #3  Digikam::DImg::setAttribute (this=<optimized out>, key=..., value=...)
>     at /home/timo/tmp/digikam/core/libs/dimg/dimg_props.cpp:207
> #4  0x00007ffff6e1e211 in Digikam::DImgLoader::uniqueHashV2 (filePath=..., img=img@entry=0x25d4748)
>     at /home/timo/tmp/digikam/core/libs/dimg/loaders/dimgloader.cpp:374
> #5  0x00007ffff6e1e4f7 in Digikam::DImgLoader::createHistoryImageId (filePath=..., image=..., metadata=...)
>     at /home/timo/tmp/digikam/core/libs/dimg/loaders/dimgloader.cpp:286
> #6  0x00007ffff6e1e9e3 in Digikam::DImgLoader::readMetadata (this=0x7fff36fed5a0, filePath=...)
>     at /home/timo/tmp/digikam/core/libs/dimg/loaders/dimgloader.cpp:229
> #7  0x00007ffff6e264f1 in Digikam::JPEGLoader::load (this=this@entry=0x7fff36fed5a0, filePath=..., 
>     observer=observer@entry=0x25d4560) at /home/timo/tmp/digikam/core/libs/dimg/loaders/jpegloader.cpp:92
> #8  0x00007ffff6f6f867 in Digikam::DImg::load (this=this@entry=0x25d4748, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, 
>     observer=observer@entry=0x25d4560, rawDecodingSettings=...) at /home/timo/tmp/digikam/core/libs/dimg/dimg_fileio.cpp:126
> #9  0x00007ffff6f700a0 in Digikam::DImg::load (this=this@entry=0x25d4748, filePath=..., observer=observer@entry=0x25d4560, 
>     rawDecodingSettings=...) at /home/timo/tmp/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #10 0x00007ffff703265e in Digikam::PreviewLoadingTask::execute (this=0x25d4550)
>     at /home/timo/tmp/digikam/core/libs/threadimageio/preview/previewtask.cpp:288
> #11 0x00007ffff7049d5e in Digikam::LoadSaveThread::run (this=0x2b656c0)
>     at /home/timo/tmp/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #12 0x00007ffff706afce in Digikam::DynamicThread::Private::run (this=0x2b67ab0)
>     at /home/timo/tmp/digikam/core/libs/threads/dynamicthread.cpp:183
> #13 0x00007fffef40a7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #14 0x00007fffef406062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #15 0x00007ffff65d9ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #16 0x00007fffed53a22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Which Linux version do you use exactly? I could not determine it in the thread. Since even Gilles with his Intel machine can not reproduce the problem and we should have a lot more feedback on a problem in digiKam, I think the problem is somewhere else. For me, it looks like the loading task is being killed from "outside"...

Maik

> Which Linux version do you use exactly? I could not determine it in the thread.

It's in the first post: NixOS. Or do you mean the actual linux kernel version? That would be 4.19.65.

> For me, it looks like the loading task is being killed from "outside"...

How could that happen? Its a segfault after all, not a reaction to a SIGKILL or something. Also remember that the same issue occurs with the snap, which probably rules out some external factors. Though I agree that a hardware-related failure doesn't seem much more likely. We do have some people in the CC list, not sure if they're seeing the same error.

Please post the output of:

cat /proc/sys/vm/overcommit_memory

and:

cat /proc/sys/vm/overcommit_ratio

Maik

Overcommit is set to the default setting ("however much the kernel heuristic sees fit"):

$ cat /proc/sys/vm/overcommit_memory
0
$ cat /proc/sys/vm/overcommit_ratio
50

The machine has 8G of ram, though I have reproduced it on my 16G laptop as well.

I tried out a couple of old NixOS versions, and the oldest one with a working digikam with face detection had digikam 4.12.0. I could reproduce the crash with that version too.

Timokau,

I would to know if the Google Summer of Code 2019 branch about face recognition port to OpenCV DNN will solve this issue.

Typically, i currently perform a bug triage in Bugzilla, especially about duplicates files. I discovered that more than 150 files are open about Face management crashes. It's enormous.

Here i test in-deed the new face detection and recognition code, fully rewritten. 

Even if recognition need more adjustment to improve results, the face detection is fabulous. Compared to previous one, the results are more than expected. The neuronal network with a previous training for faces performed by OpenCV team is just wonderful. It work in all cases, including the critical shot conditions.

I tested with plenty of different collections, all  the detection is very stable. It never crash with or without multi-core support. 

To test you need :

- to checkout the code from development branch :

git clone https://invent.kde.org/kde/digikam.git digikam
cd digikam
git checkout -b gsoc19-face-recognition remotes/origin/gsoc19-face-recognition
./bootstrap.linux
cd build 
make -j4
su
make install/fast

Warning this will overwrite your current digiKam build from git/master branch. There is no risk to re-install master later to go back. only internal source code is touched in this developement branch. There is no difference with installed files compared with git/master.

- i recommend to run this digiKam version in a new account. This will create a fresh database. Import your collection which introduce crash during faces scan.

- Go to People tab from left sidebar. Select Scan for faces only. Scan whole collection (no need to scan tags). In advanced options, don't forget to select DNN module.

- Start face scan and wait to see if the operation can be completed without crash. Look also the face detection results to see if you have not too much false positives.

Best

Gilles Caulier

As I reported previously (https://bugs.kde.org/show_bug.cgi?id=399923#c167), this is still an issue with the new face detection code.

Oh, i don't read well your last comment. It's interresting, especially this trace :

> String, "autodetectedFace"), QVariant(int, 281), QVariant(int, 281), QVariant(QString, "tagRegion"))
> digikam.facedb: nnmodel:  "" , nndata  ""
> digikam.facesengine: cv::Exception: OpenCV(3.4.7) /build/source/modules/dnn/src/caffe/caffe_io.cpp:1121: error: (-2:Unspecified error
> ) FAILED: fs.is_open(). Can't open "" in function 'ReadProtoFromTextFile'
> 
> digikam.general: Found 0 faces in "IMG-20170907-WA0003.jpg" QSize(1280, 720) QSize(1280, 720)
> digikam.facedb: nnmodel:  "" , nndata  ""
> digikam.facesengine: cv::Exception: OpenCV(3.4.7) /build/source/modules/dnn/src/caffe/caffe_io.cpp:1121: error: (-2:Unspecified error
> ) FAILED: fs.is_open(). Can't open "" in function 'ReadProtoFromTextFile'
> 
> digikam.general: Found 0 faces in "IMG-20170907-WA0004.jpg" QSize(1032, 581) QSize(1032, 581)
> digikam.dimg: "/home/timo/tmp/deletable-pics/Syncthing-3TB/Oneplus One/Whatsapp/Media/WhatsApp Images/IMG-20170907-WA0006.jpg"  : JPE
> G file identified

It's clear : your OpenCV 3.4.7 is not well build. It miss symbols and generate a C++ exception.

I build myself OpenCV 3.4.7, using OpenCV git repository and tag "3.4.7". I use a minimal configuration with this shell script :

https://cgit.kde.org/digikam.git/tree/project/scripts/bootstrap.opencv-minimal-linux.sh

Checkout opencv 3.4.7 tag from git repository, copy this shell script on root OpenCV directory, run it and go to build subdir to run make -j4 (as with digiKam source).

Before to install this new minimal OpenCV, uninstall all previous development versions from OpenCV. Do not mix configurations. I already faced this kind of situation, and this make an unusable OpenCV with digiKam. You can let's OpenCv2 binaries as shared libraries, but all cmake config files, headers, and pkgconfig files must be removed and replaced by new compiled OpenCV version.

Take a care to not use in parallel OpenCv4 files. Some part are binary compatible with OpenCv3 and OpenCv2.

When this minimal OpenCv 3.4.7 is installed, from digiKam face recognition branch, remove build sub-dir, and reconfigure with the bootstrap script. Look while configuration stage is all OpenCv 3.4.7 modules are found properly. 

Compile and install this digiKam. Test again the scan for faces function.

Gilles Caulier

PS : here i test with Mageia6 which only includes OpenCv2. I uninstalled all OPenCv2 devel packages and checkout/compile/install OpenCv 3.4.7 as well.
Mageia6 is installed on a real computer, not a VM. I use a NVIDIA video Card, even if my minimal OpenCv 3.4.7 is compiled without CUDA support. In fact i disabled all video and other system accelerations: it's not optimum but safe to run.

How do I then tell digikam to use that opencv build instead of the system version?

You have no choice. Uninstall the development package from the system version. You can let's the binary packages. digiKam will try to find opencv devel files, so if only the compiled version is present, it will work.

I do it under Mageia6 which provide only OpenCV 2 instead OpenCV 3.

Gilles Caulier

So does it have to be compiled in a subfolder of the digikam checkout? Or how does digikam find the build result?

Checkout OpenCV3 in a dedicated directory, outside digiKam.
Compile and install OpenCV3 in your system.
Bootstrap digiKam, and check if right OpenCV3 is found from your system.

Gilles Caulier

It still crashes with opencv 3.4.7 built with your configure script :/

> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/tmp/tmp.Uw8rYQpimd/Pictures/p/iPod/IMG_0905.MOV"  : Try to load with QImage
> digikam.dimg: QImageLoader support only the image mime type
> digikam.general: Cannot extract preview for "/tmp/tmp.Uw8rYQpimd/Pictures/p/iPod/IMG_0905.MOV"
> digikam.dimg: "/tmp/tmp.Uw8rYQpimd/Pictures/p/iPod/IMG_0906.MOV"  : Try to load with QImage
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: QImageLoader support only the image mime type
> digikam.general: Cannot extract preview for "/tmp/tmp.Uw8rYQpimd/Pictures/p/iPod/IMG_0906.MOV"
> digikam.dimg: "/tmp/tmp.Uw8rYQpimd/Pictures/p/iPod/IMG_0907.MOV"  : Try to load with QImage
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: QImageLoader support only the image mime type
> digikam.dimg: "/tmp/tmp.Uw8rYQpimd/Pictures/p/iPod/IMG_0908.MOV"  : Try to load with QImage
> digikam.dimg: QImageLoader support only the image mime type
> digikam.general: Found 0 faces in "IMG_0861.JPG" QSize(960, 720) QSize(960, 720)
> digikam.general: Removing old entries ()
> 
> Thread 295 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff49006700 (LWP 27860)]
> 0x00007ffff7032313 in Digikam::PreviewLoadingTask::execute (this=0x4d309e0)
>     at /tmp/tmp.Uw8rYQpimd/digikam/core/libs/threadimageio/preview/previewtask.cpp:322
> 322                 LoadSaveNotifier* const notifier = l->loadSaveNotifier();
> (gdb) 
> (gdb) 
> (gdb) bt
> #0  0x00007ffff7032313 in Digikam::PreviewLoadingTask::execute (this=0x4d309e0)
>     at /tmp/tmp.Uw8rYQpimd/digikam/core/libs/threadimageio/preview/previewtask.cpp:322
> #1  0x00007ffff7049d0e in Digikam::LoadSaveThread::run (this=0x512b660)
>     at /tmp/tmp.Uw8rYQpimd/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #2  0x00007ffff706af5e in Digikam::DynamicThread::Private::run (this=0x61f9fa0)
>     at /tmp/tmp.Uw8rYQpimd/digikam/core/libs/threads/dynamicthread.cpp:183
> #3  0x00007fffef40a7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #4  0x00007fffef406062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #5  0x00007ffff65d9ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #6  0x00007fffed72e22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb) 

This was with digikam 820034951db297a2d31ce0b3141786b295a55ce0 (gsoc19-face-recognition).

Git commit 69e0c11b852299c8ed2a47f7e84b34aa2d49d67a by Maik Qualmann.
Committed on 22/08/2019 at 05:57.
Pushed by mqualmann into branch 'gsoc19-face-recognition'.

Merge branch 'master' into gsoc19-face-recognition


https://invent.kde.org/kde/digikam/commit/69e0c11b852299c8ed2a47f7e84b34aa2d49d67a

Your last crash is one that can never happen. We do not test the pointer, but it can never be zero. The QList from which the pointer comes is only read or written under QMutex condition. Since I suspect extreme compiler optimizations, I've changed some little thing in the head of the gsoc19-face-recognition branch. Please test again...

Maik

To quote douglas adams ;)
> The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair.

Here's the run with latest gsoc (8e9fdf3ab6c581152d8c2f9dd8d764fbb3dfff96):

> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.dimg: "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_121952.jpg"  : JPEG file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Found 0 faces in "20171222_114024.jpg" QSize(4032, 3024) QSize(4032, 3024)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_121956.mp4"  : Try to load with QImage
> digikam.dimg: QImageLoader support only the image mime type
> digikam.general: Cannot extract preview for "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_121956.mp4"
> digikam.dimg: "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_122009.mp4"  : Try to load with QImage
> digikam.dimg: QImageLoader support only the image mime type
> digikam.general: Cannot extract preview for "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_122009.mp4"
> digikam.dimg: "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_122616.mp4"  : Try to load with QImage
> digikam.dimg: QImageLoader support only the image mime type
> digikam.general: Cannot extract preview for "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_122616.mp4"
> digikam.dimg: "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_123142.mp4"  : Try to load with QImage
> digikam.dimg: QImageLoader support only the image mime type
> digikam.general: Cannot extract preview for "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_123142.mp4"
> digikam.dimg: "/tmp/tmp.VjfYQHZygB/Pictures/p/20171222_123152.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> Trying to construct an instance of an invalid type, type id: 49213440
> 
> Thread 38 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff48d63700 (LWP 16659)]
> 0x00007fffef48772a in operator==(QString const&, QString const&) ()
>    from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007fffef48772a in operator==(QString const&, QString const&) ()
>    from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #1  0x00007ffff7044eb5 in Digikam::LoadingDescription::equalsIgnoreReducedVersion (this=<optimized out>, other=...)
>     at /tmp/tmp.VjfYQHZygB/digikam/core/libs/threadimageio/fileio/loadingdescription.cpp:313
> #2  0x00007ffff704fcbb in Digikam::SharedLoadingTask::notifyNewLoadingProcess (this=0x318d560, process=<optimized out>, 
>     description=...) at /tmp/tmp.VjfYQHZygB/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:429
> #3  0x00007ffff70465f0 in Digikam::LoadingCache::notifyNewLoadingProcess (this=this@entry=0x7fffb0017f70, 
>     process=process@entry=0x320a7c0, description=...)
>     at /tmp/tmp.VjfYQHZygB/digikam/core/libs/threadimageio/fileio/loadingcache.cpp:240
> #4  0x00007ffff702bc77 in Digikam::PreviewLoadingTask::execute (this=0x320a5f0)
>     at /tmp/tmp.VjfYQHZygB/digikam/core/libs/threadimageio/preview/previewtask.cpp:145
> #5  0x00007ffff704361e in Digikam::LoadSaveThread::run (this=0x61bdb80)
>     at /tmp/tmp.VjfYQHZygB/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff706488e in Digikam::DynamicThread::Private::run (this=0x37a70a0)
>     at /tmp/tmp.VjfYQHZygB/digikam/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffef4037d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007fffef3ff062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff65d2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffed72722f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit de9adb9f0051bff5da4a0ffa2e85dbc198e3b33f by Maik Qualmann.
Committed on 23/08/2019 at 04:44.
Pushed by mqualmann into branch 'master'.

add dynamic cast

M  +21   -16   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +16   -11   core/libs/threadimageio/preview/previewtask.cpp
M  +8    -8    core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/kde/digikam/commit/de9adb9f0051bff5da4a0ffa2e85dbc198e3b33f

Git commit 7948db0711fa81018cf0e6c473d3ea1a59d97388 by Maik Qualmann.
Committed on 23/08/2019 at 04:46.
Pushed by mqualmann into branch 'gsoc19-face-recognition'.

Merge branch 'master' into gsoc19-face-recognition


https://invent.kde.org/kde/digikam/commit/7948db0711fa81018cf0e6c473d3ea1a59d97388

With 7948db0711fa81018cf0e6c473d3ea1a59d97388:

> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Found 0 faces in "20171222_123227.jpg" QSize(4032, 3024) QSize(4032, 3024)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/tmp/tmp.XBJ09aYEaP/Pictures/p/20171222_123618.jpg"  : JPEG file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Found 0 faces in "20171222_123231.jpg" QSize(4032, 3024) QSize(4032, 3024)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.dimg: "/tmp/tmp.XBJ09aYEaP/Pictures/p/20171222_123651.mp4"  : Try to load with QImage
> digikam.dimg: QImageLoader support only the image mime type
> digikam.general: Cannot extract preview for "/tmp/tmp.XBJ09aYEaP/Pictures/p/20171222_123651.mp4"
> digikam.dimg: "/tmp/tmp.XBJ09aYEaP/Pictures/p/20171222_123708.mp4"  : Try to load with QImage
> digikam.dimg: QImageLoader support only the image mime type
> digikam.general: Cannot extract preview for "/tmp/tmp.XBJ09aYEaP/Pictures/p/20171222_123708.mp4"
> digikam.dimg: "/tmp/tmp.XBJ09aYEaP/Pictures/p/20171222_123714.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> 
> Thread 328 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff177fe700 (LWP 4385)]
> 0x00007fffef48772a in operator==(QString const&, QString const&) ()
>    from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007fffef48772a in operator==(QString const&, QString const&) ()
>    from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #1  0x00007ffff7044a05 in Digikam::LoadingDescription::equalsIgnoreReducedVersion (this=<optimized out>, other=...)
>     at /tmp/tmp.XBJ09aYEaP/digikam/core/libs/threadimageio/fileio/loadingdescription.cpp:313
> #2  0x00007ffff704f80b in Digikam::SharedLoadingTask::notifyNewLoadingProcess (this=0x3e349c0, process=<optimized out>, 
>     description=...) at /tmp/tmp.XBJ09aYEaP/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:433
> #3  0x00007ffff7046140 in Digikam::LoadingCache::notifyNewLoadingProcess (this=this@entry=0xa80020, 
>     process=process@entry=0x3d13120, description=...)
>     at /tmp/tmp.XBJ09aYEaP/digikam/core/libs/threadimageio/fileio/loadingcache.cpp:240
> #4  0x00007ffff702bac7 in Digikam::PreviewLoadingTask::execute (this=0x3d12f50)
>     at /tmp/tmp.XBJ09aYEaP/digikam/core/libs/threadimageio/preview/previewtask.cpp:145
> #5  0x00007ffff704316e in Digikam::LoadSaveThread::run (this=0x5b23b00)
>     at /tmp/tmp.XBJ09aYEaP/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff706440e in Digikam::DynamicThread::Private::run (this=0x26991b0)
>     at /tmp/tmp.XBJ09aYEaP/digikam/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007fffef4037d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007fffef3ff062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff65d2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #10 0x00007fffed72722f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

This crash from the face-recognition branch is just not understandable , and not reproducible here. I play with this code since few weeks, and it do not crash, including the video file parsing (which are not handled in face workflow).

Gilles Caulier

Can you just parse only albums containing images please ?

Gilles Caulier

So you want me to run the face detection only on images, not videos? How would I do that?

Very simple. In advanced settings section of face scan dialog, only select right albums to scan, and not the whole albums collection.

Don't forget also to deselect tag albums, just to be sure.

It's not that simple, since videos and pictures are not separated. Since I'm working on a copy of my collection anyway, it would be fine to just delete all videos. Is that possible somehow?

Very simple : go to search tool, enter mp4 in text field on top.
Wait the response from the database, all .mp4 files will be listed on icon view. CTRL+A to select all .mp4 from icon-view and SHIFT+DEL to delete definitively all files.

Ah, I was hoping there was a way to filter for all files with a video mime type (since I have videos in different formats), but I got it done with `find` instead. It came to 95% this time, but still crashed in the end:

> [New Thread 0x7fff3effd700 (LWP 25249)]
> digikam.general: Action Thread run  1  new jobs
> digikam.dimg: "/tmp/tmp.p0vuUbXE4s/Pictures/p/images/new-pics/DCIM/Camera/IMG_20180110_101827.jpg"  : JPEG file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> [New Thread 0x7fff059b5700 (LWP 25250)]
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageT
> agProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AN
> D ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pi
> d=?)) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,   
>      ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformatio
> n.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Imag
> es        INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=Image
> Information.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagPropertie
> s.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagP
> roperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (I
> mageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  )
>  )  );" 
>  (QVariant(int, 281), QVariant(int, 281), QVariant(QString, "autodetectedPerson"), QVariant(int, 281), QVariant(int, 281), QVariant(
> QString, "autodetectedFace"), QVariant(int, 281), QVariant(int, 281), QVariant(QString, "tagRegion"))
> digikam.database: Search result: 3735
> 
> Thread 45 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff267fb700 (LWP 16009)]
> 0x00007fffed6d3626 in __memmove_sse2_unaligned_erms () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007fffed6d3626 in __memmove_sse2_unaligned_erms ()
>    from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> #1  0x00007ffff6f65e93 in memcpy (__len=<optimized out>, __src=<optimized out>, __dest=<optimized out>)
>     at /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/bits/string_fortified.h:34
> #2  Digikam::DImg::detach (this=0x2bab688) at /tmp/tmp.p0vuUbXE4s/digikam/core/libs/dimg/dimg_data.cpp:67
> #3  0x00007ffff702b542 in Digikam::PreviewLoadingTask::execute (this=0x2bab490)
>     at /tmp/tmp.p0vuUbXE4s/digikam/core/libs/threadimageio/preview/previewtask.cpp:381
> #4  0x00007ffff704316e in Digikam::LoadSaveThread::run (this=0x2d091a0)
>     at /tmp/tmp.p0vuUbXE4s/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #5  0x00007ffff706440e in Digikam::DynamicThread::Private::run (this=0x2d09b40)
>     at /tmp/tmp.p0vuUbXE4s/digikam/core/libs/threads/dynamicthread.cpp:183
> #6  0x00007fffef4037d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #7  0x00007fffef3ff062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007ffff65d2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #9  0x00007fffed72722f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 264482ec662a28db39d7a0237ce6d1d2474ba585 by Maik Qualmann.
Committed on 25/08/2019 at 07:59.
Pushed by mqualmann into branch 'master'.

disable the communication between threads for a test

M  +4    -4    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/264482ec662a28db39d7a0237ce6d1d2474ba585

Git commit 04179402304ee8d9d813d5761dabf3ea8c769048 by Maik Qualmann.
Committed on 25/08/2019 at 08:01.
Pushed by mqualmann into branch 'gsoc19-face-recognition'.

Merge branch 'master' into gsoc19-face-recognition


https://invent.kde.org/kde/digikam/commit/04179402304ee8d9d813d5761dabf3ea8c769048

*** Bug 379470 has been marked as a duplicate of this bug. ***

Sorry for the delay. The test is still running.

One thing I can say for sure is that with 04179402304ee8d9d813d5761dabf3ea8c769048 the performance has degraded a lot. It takes up to 10G of ram at times (4G on average) and after over 10h of running I'm now at 17%. Is that to be expected?

Git commit 27f663c1155498c03c174ce93d7c94e69d77489a by Maik Qualmann.
Committed on 19/10/2019 at 16:28.
Pushed by mqualmann into branch 'master'.

review loading tasks
Related: bug 406019

M  +42   -37   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +49   -46   core/libs/threadimageio/preview/previewtask.cpp
M  +1    -1    core/libs/threadimageio/thumb/thumbnailcreator_engine.cpp
M  +41   -40   core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/kde/digikam/commit/27f663c1155498c03c174ce93d7c94e69d77489a

With 27f663c1155498c03c174ce93d7c94e69d77489a the scan finishes in just a few seconds, seemingly doing nothing.

Presumably it's because you tested the developer branch with the new DNN face recognition. Check the facial recognition settings and use a new recognition.db.

Maik

I set $HOME to a tmpdir for each test to avoid carrying over configuration and I deleted all databases in the picture dir. Is there anything else that could cause this?

In principle, it should be a clean startup configuration. It is difficult to say why the images are not processed. Maybe it helps to post the console log.

Maik

I doesn't look very helpful to me, but here's the full log of the scan (completed in 7 seconds): http://sprunge.us/fVfY1j

OK, JPEG is an unknown format. Now the log would be good from the beginning. With digiKam-6.4.0, the image loaders will now also be plugins. These plugins are probably not found in your installation. The search is done in the Qt system plugin path. But there is the possibility to set an env variable to the plugin path.

export DK_PLUGIN_PATH=path_to_plugin_dir

Maik

Okay, setting that environment variable appropriately fixed that issue. So here's the latest crash (with 27f663c1155498c03c174ce93d7c94e69d77489a):


> digikam.facesengine: detectMultiScale: image size ( 52 , 52 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 9 , 9 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(262,470 86x86) using cascade 7
> digikam.facesengine: feature QRectF(0.2,0.25 0.6x0.6) QRect(262,470 86x86) QRect(279,492 52x52)
> digikam.facesengine: detectMultiScale: image size ( 52 , 52 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 9 , 9 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(262,470 86x86) using cascade 8
> digikam.facesengine: feature QRectF(0.1,0.4 0.8x0.6) QRect(262,470 86x86) QRect(271,504 69x52)
> digikam.facesengine: detectMultiScale: image size ( 69 , 52 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 9 , 9 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "some_file_.JPG" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.facesengine: detectMultiScale: image size ( 800 , 600 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 26 , 26 )
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "some_file_.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "some_file_.JPG" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "some_file_.JPG" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size ( 800 , 600 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 26 , 26 )
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "some_file_.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "some_file_.JPG" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> 
> Thread 3275 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff38ff9700 (LWP 17945)]
> QMapNode<QString, QVariant>::copy (this=0x7ffff3b807b8 <vtable for Exiv2::Exifdatum+16>, d=d@entry=0x7ffef04259d0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:258
> 258         QMapNode<Key, T> *n = d->createNode(key, value);
> (gdb) bt
> #0  QMapNode<QString, QVariant>::copy (this=0x7ffff3b807b8 <vtable for Exiv2::Exifdatum+16>, d=d@entry=0x7ffef04259d0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:258
> #1  0x00007ffff7887758 in QMap<QString, QVariant>::detach_helper (this=0x341f0b8)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:1008
> #2  0x00007ffff78a8ead in QMap<QString, QVariant>::detach (this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:364
> #3  QMap<QString, QVariant>::insert (avalue=..., akey=..., this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:706
> #4  Digikam::DImg::setAttribute (this=<optimized out>, key=..., value=...)
>     at some_file_:207
> #5  0x00007ffff797c1c2 in Digikam::LoadSaveThread::exifRotate (image=..., filePath=...)
>     at some_file_:336
> #6  0x00007ffff7965e4c in Digikam::PreviewLoadingTask::execute (this=0x2702bf0)
>     at some_file_:296
> #7  0x00007ffff797bbbe in Digikam::LoadSaveThread::run (this=0x7fffd80123e0)
>     at some_file_:133
> #8  0x00007ffff799c54e in Digikam::DynamicThread::Private::run (this=0x1aaef60)
>     at some_file_:183
> #9  0x00007ffff4a357d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #10 0x00007ffff4a31062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #11 0x00007ffff43fcef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #12 0x00007ffff433222f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 6a77ab5528a85bdf1bb0d5f30bb70f928d99c7b7 by Maik Qualmann.
Committed on 28/10/2019 at 17:32.
Pushed by mqualmann into branch 'master'.

move these two static functions for a test in PreviewLoadingTask

M  +26   -2    core/libs/threadimageio/preview/previewtask.cpp
M  +3    -0    core/libs/threadimageio/preview/previewtask.h

https://invent.kde.org/kde/digikam/commit/6a77ab5528a85bdf1bb0d5f30bb70f928d99c7b7

One full scan actually went through, but on the second try:

> digikam.general: Found 2 faces in "2012-08-31 18.24.29.jpg" QSize(640, 480) QSize(640, 480)
> digikam.general: Removing old entries (FaceTagsIface(1, image 5429, tag 281, regionQRect(509,142 48x48), FaceTagsIface(1, image 5429,
>  tag 281, regionQRect(300,22 60x60))
> digikam.facesengine: detectMultiScale: image size ( 480 , 640 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 22 , 22 )
> digikam.facesengine: detectMultiScale gave (QRect(138,268 178x178))
> digikam.facesengine: Verifying face QRect(138,268 178x178) using cascade 0
> digikam.facesengine: detectMultiScale: image size ( 258 , 258 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 107 , 107 )
> digikam.facesengine: detectMultiScale gave (QRect(26,33 196x196))
> digikam.facesengine: Verifying face QRect(138,268 178x178) using cascade 1
> digikam.facesengine: detectMultiScale: image size ( 258 , 258 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 107 , 107 )
> digikam.facesengine: detectMultiScale gave (QRect(26,37 196x196))
> digikam.facesengine: Verifying face QRect(138,268 178x178) using cascade 2
> digikam.facesengine: Verifying face QRect(138,268 178x178) using cascade 3
> digikam.facesengine: detectMultiScale: image size ( 258 , 258 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 107 , 107 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(138,268 178x178) using cascade 4
> digikam.facesengine: detectMultiScale: image size ( 258 , 258 ) searchIncrement 1.1 grouping 3 flags 0 min size ( 107 , 107 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.facesengine: Verifying face QRect(138,268 178x178) using cascade 5
> digikam.facesengine: feature QRectF(0,0 0.6x0.6) QRect(138,268 178x178) QRect(138,268 107x107)
> digikam.facesengine: detectMultiScale: image size ( 107 , 107 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 18 , 18 )
> digikam.facesengine: detectMultiScale gave (QRect(26,51 59x39), QRect(4,60 39x26))
> digikam.facesengine: Verifying face QRect(138,268 178x178) using cascade 6
> digikam.facesengine: feature QRectF(0.4,0 0.6x0.6) QRect(138,268 178x178) QRect(209,268 107x107)
> digikam.facesengine: detectMultiScale: image size ( 107 , 107 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 18 , 18 )
> digikam.facesengine: detectMultiScale gave (QRect(26,58 40x27))
> digikam.facesengine: Verifying face QRect(138,268 178x178) using cascade 7
> digikam.facesengine: feature QRectF(0.2,0.25 0.6x0.6) QRect(138,268 178x178) QRect(174,313 107x107)
> digikam.facesengine: detectMultiScale: image size ( 107 , 107 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 18 , 18 )
> digikam.facesengine: detectMultiScale gave (QRect(33,46 51x42))
> digikam.facesengine: Verifying face QRect(138,268 178x178) using cascade 8
> digikam.facesengine: feature QRectF(0.1,0.4 0.8x0.6) QRect(138,268 178x178) QRect(156,339 142x107)
> digikam.facesengine: detectMultiScale: image size ( 142 , 107 ) searchIncrement 1.1 grouping 2 flags 0 min size ( 18 , 18 )
> digikam.facesengine: detectMultiScale gave (QRect(30,19 33x20), QRect(50,36 51x30), QRect(31,62 67x40))
> digikam.general: Found 1 faces in "2012-09-01 19.43.55.jpg" QSize(480, 640) QSize(480, 640)
> digikam.facesengine: detectMultiScale: image size ( 480 , 640 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 22 , 22 )
> digikam.general: Removing old entries (FaceTagsIface(1, image 5430, tag 281, regionQRect(138,268 178x178))
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "2012-09-01 21.24.15.jpg" QSize(480, 640) QSize(480, 640)
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size ( 640 , 480 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 22 , 22 )
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "2012-09-02 19.48.21.jpg" QSize(640, 480) QSize(640, 480)
> digikam.general: Removing old entries ()
> digikam.facesengine: detectMultiScale: image size ( 480 , 640 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 22 , 22 )
> 
> Thread 1107 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff4cffa700 (LWP 14912)]
> 0xffffffffffffffff in ?? ()
> (gdb) bt
> #0  0xffffffffffffffff in ?? ()
> #1  0x00007ffff7987ffd in Digikam::LoadingTask::progressInfo (this=0x3397410, img=<optimized out>, progress=0.200000003)
>     at /tmp/tmp.tagPkiKadL/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #2  0x00007fffb2369a40 in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x7fff240a6f50, filePath=..., observer=0x3397420)
>     at /tmp/tmp.tagPkiKadL/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:434
> #3  0x00007ffff78a18bc in Digikam::DImg::load (this=this@entry=0x33975f8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63,
>     observer=observer@entry=0x3397420, rawDecodingSettings=...) at /tmp/tmp.tagPkiKadL/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #4  0x00007ffff78a2b50 in Digikam::DImg::load (this=this@entry=0x33975f8, filePath=..., observer=observer@entry=0x3397420,
>     rawDecodingSettings=...) at /tmp/tmp.tagPkiKadL/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #5  0x00007ffff796602a in Digikam::PreviewLoadingTask::execute (this=0x3397410)
>     at /tmp/tmp.tagPkiKadL/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #6  0x00007ffff797c2be in Digikam::LoadSaveThread::run (this=0x49d95d0)
>     at /tmp/tmp.tagPkiKadL/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #7  0x00007ffff799cc4e in Digikam::DynamicThread::Private::run (this=0x355b600)
>     at /tmp/tmp.tagPkiKadL/digikam/core/libs/threads/dynamicthread.cpp:183
> #8  0x00007ffff4a337d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff4a2f062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #10 0x00007ffff43faef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #11 0x00007ffff433022f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Ok, here is the crash that should never occur, we are in the wrong overriden function. The crash in the static function is already strange. Although static functions with threads are considered critical, no problems should arise with modern compilers. Here I will try one more experiment with DImg as a pointer. How do you compile digiKam? Do you use additional compiler options?

Maik

My reproduction script is as follows:

https://gist.github.com/timokau/a50d1033bce78f56804406bb8585b607

The first couple of lines are to set up an clean environment in a tmpdir for the build. The `nix-shell` line creates a shell with all the dependencies needed for the build. The actual compilation is very standard, I juts pass the prefix and the build type. `wrapQtApp` sets up the appropriate env vars so that digikam can find QT (unfortunately QT and nixos is a bit of an annoying combination).

Git commit a3614106b068d39ed820af7bc06f3036e8799d60 by Gilles Caulier.
Committed on 30/10/2019 at 09:20.
Pushed by cgilles into branch 'master'.

fix out of bound index reported by cppcheck

M  +5    -2    core/libs/facesengine/facedb/facedb.cpp

https://invent.kde.org/kde/digikam/commit/a3614106b068d39ed820af7bc06f3036e8799d60

Git commit dbcaeafec2ef2627dbf61095b94d7385ec9107f3 by Gilles Caulier.
Committed on 30/10/2019 at 09:47.
Pushed by cgilles into branch 'master'.

Cppcheck is right here : we have 2 decalration of i shadow variables. With recent compiler, this must be fine, but with older?

M  +11   -7    core/utilities/facemanagement/faceutils.cpp
M  +9    -7    core/utilities/facemanagement/faceutils.h

https://invent.kde.org/kde/digikam/commit/dbcaeafec2ef2627dbf61095b94d7385ec9107f3

Git commit 707891ca729fa69c1cbcad6c133f0c5585773747 by Gilles Caulier.
Committed on 30/10/2019 at 09:56.
Pushed by cgilles into branch 'master'.

Cppcheck is right here : we have 2 declarations of "send" shadow outter function. With recent compiler, this must be fine, but with older?

M  +14   -14   core/utilities/facemanagement/scanstatefilter.cpp
M  +2    -2    core/utilities/facemanagement/scanstatefilter.h

https://invent.kde.org/kde/digikam/commit/707891ca729fa69c1cbcad6c133f0c5585773747

With 707891ca729fa69c1cbcad6c133f0c5585773747:

> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "some_image.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "some_image.JPG" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "some_image.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "some_image.JPG" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "some_image.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "some_image.JPG" : "JPEG" file identified
> digikam.facesengine: detectMultiScale gave ()
> digikam.general: Found 0 faces in "some_image.JPG" QSize(1152, 1728) QSize(1152, 1728)
> digikam.general: Removing old entries ()
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Cancel Main Thread
> [Thread 0x7fff209b3700 (LWP 10573) exited]
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "some_image.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "some_image.JPG" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "some_image.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "some_image.JPG" : "JPEG" file identified
> digikam.facesengine: detectMultiScale: image size ( 533 , 800 ) searchIncrement 1.1 grouping 3 flags 1 min size ( 26 , 26 )
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> 
> Thread 67 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff237fe700 (LWP 19814)]
> 0x00007ffff78ce30e in QSharedDataPointer<Digikam::MetaEngineData::Private>::operator= (o=..., this=0x40e7190)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:99
> 99                  if (old && !old->ref.deref())
> (gdb) bt
> #0  0x00007ffff78ce30e in QSharedDataPointer<Digikam::MetaEngineData::Private>::operator= (o=..., this=0x40e7190) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:99
> #1  Digikam::MetaEngineData::operator= (this=0x40e7190, other=...) at /tmp/tmp.inM9rwW9gU/digikam/core/libs/metadataengine/engine/metaengine_data.cpp:52
> #2  0x00007ffff78a5a1c in Digikam::DImg::setMetadata (this=this@entry=0x304b8e8, data=...) at /tmp/tmp.inM9rwW9gU/digikam/core/libs/dimg/dimg_props.cpp:172
> #3  0x00007ffff775bf73 in Digikam::DImgLoader::readMetadata (this=0x4dee7c0, filePath=...) at /tmp/tmp.inM9rwW9gU/digikam/core/libs/dimg/loaders/dimgloader.cpp:224
> #4  0x00007fffb236843c in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x4dee7c0, filePath=..., observer=0x304b710) at /tmp/tmp.inM9rwW9gU/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:56
> #5  0x00007ffff78a080c in Digikam::DImg::load (this=this@entry=0x304b8e8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x304b710, rawDecodingSettings=...) at /tmp/tmp.inM9rwW9gU/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #6  0x00007ffff78a1aa0 in Digikam::DImg::load (this=this@entry=0x304b8e8, filePath=..., observer=observer@entry=0x304b710, rawDecodingSettings=...) at /tmp/tmp.inM9rwW9gU/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #7  0x00007ffff7964d7a in Digikam::PreviewLoadingTask::execute (this=0x304b700) at /tmp/tmp.inM9rwW9gU/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #8  0x00007ffff797b00e in Digikam::LoadSaveThread::run (this=0x7fffd8009b50) at /tmp/tmp.inM9rwW9gU/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #9  0x00007ffff799b99e in Digikam::DynamicThread::Private::run (this=0x31128a0) at /tmp/tmp.inM9rwW9gU/digikam/core/libs/threads/dynamicthread.cpp:183
> #10 0x00007ffff4a337d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #11 0x00007ffff4a2f062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #12 0x00007ffff43faef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #13 0x00007ffff433022f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit a902eddf0d84b1561e7c7ee75164e58fdeaf8672 by Maik Qualmann.
Committed on 31/10/2019 at 10:51.
Pushed by mqualmann into branch 'master'.

use DImg as pointer in static functions

M  +2    -2    core/dplugins/bqm/decorate/watermark/watermark.cpp
M  +1    -1    core/dplugins/bqm/transform/rotate/rotate.cpp
M  +3    -3    core/libs/dimg/dimg_metadata.cpp
M  +54   -54   core/libs/dimg/loaders/dimgloader.cpp
M  +2    -2    core/libs/dimg/loaders/dimgloader.h
M  +1    -1    core/libs/facesengine/detection/dnn/dnnfacedetectorbase.cpp
M  +35   -19   core/libs/threadimageio/fileio/loadsavethread.cpp
M  +4    -4    core/libs/threadimageio/fileio/loadsavethread.h
M  +3    -3    core/libs/threadimageio/preview/previewtask.cpp
M  +1    -1    core/utilities/imageeditor/core/editorcore.cpp

https://invent.kde.org/kde/digikam/commit/a902eddf0d84b1561e7c7ee75164e58fdeaf8672

On a902eddf0d84b1561e7c7ee75164e58fdeaf8672 face detection takes unreasonably long again, I assume because the new face recognition branch was merged in. I left it running for some 10 hours and got to 25%, and it didn't crash in that time at least. I can't really leave it running much longer than that for a test in a single run, since I want to be able to use the computer and digikam takes up a lot of resources.

Timokau,

The long time to scan for face detection is reproducible here and is not a dysfunction.

In fact it's just a setting to adjust (the level of detection). Currently, the value is too low and here after 4 h of one core processing, it detects a huge list of faces, with a minimal false positive. This DNN algorithm rock really.

So i propose to change the sensitivity values in Advanced Options view.

Gilles Caulier

Maik,

about crash, i recommend to check new cppcheck reports around threadimageio, dimg, and faceengine.

https://www.digikam.org/reports/cppcheck/master/

Look well the virtual calls in constructor warnings.
The copy constructor and equality operator warnings can be just disable definitions in classes to add, but it's not sure.

I'm sure that errors are located in these reports and must be fixed. the new cppcheck reports better warnings and the list grow a lots compared to previous version (2y old).

Gilles

Unfortunately turning down the accuracy doesn't impact the runtime very significantly. With 2% instead  of 100% it still didn't get very far (to 12% scanned) in ~6h. It didn't crash either though, so maybe the crash is fixed.

I agree that the results are impressive though. For a final scan the runtime may be tolerable, since it only needs to be done once.

Git commit 85af2a46ebcd7b19ec3b6809c85f587dbbd14f2e by Gilles Caulier.
Committed on 11/11/2019 at 14:11.
Pushed by cgilles into branch 'master'.

Face Detection Neural Network optimizations:
Switch from Yolo to SSD model for DNN face detection, as SSD model are lightweight compared to Yolo model and provide very good results.
This commit prevent to install 250Mb of binaray data on the system coming from Yolo inference engine.

M  +8    -7    core/data/facesengine/CMakeLists.txt
M  +1    -1    core/libs/facesengine/detection/facedetector.cpp

https://invent.kde.org/kde/digikam/commit/85af2a46ebcd7b19ec3b6809c85f587dbbd14f2e

That is definitely a lot faster! The scan finished once with a specificity of 1%. Then I set it back to 80% and it crashed as follows:

> digikam.general: Cancel Main Thread
> [Thread 0x7fff237fe700 (LWP 9711) exited]
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140814-WA0000.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140814-WA0000.jpg" : "JPEG" file identified
> digikam.general: Found 0 faces in "IMG-20140801-WA0006.jpg" QSize(960, 1280) QSize(960, 1280)
> digikam.general: Try to get preview from "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140815-WA0000.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140815-WA0000.jpg" : "JPEG" file identified
> digikam.general: Try to get preview from "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140815-WA0001.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140815-WA0001.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140819-WA0000.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140819-WA0000.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Found 0 faces in "IMG-20140801-WA0007.jpg" QSize(960, 1280) QSize(960, 1280)
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140819-WA0001.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140819-WA0001.jpg" : "JPEG" file identified
> digikam.general: Found 0 faces in "IMG-20140801-WA0008.jpg" QSize(960, 1280) QSize(960, 1280)
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140820-WA0000.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140820-WA0000.jpg" : "JPEG" file identified
> digikam.general: Found 0 faces in "IMG-20140801-WA0009.jpg" QSize(960, 1280) QSize(960, 1280)
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140820-WA0001.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.r5KKLL1qk4/Pictures/p/Android/Whatsapp/IMG-20140820-WA0001.jpg" : "JPEG" file identified
> 
> Thread 12847 "Thread (pooled)" received signal SIGBUS, Bus error.
> [Switching to Thread 0x7fff32ffd700 (LWP 7077)]
> QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff2400ecd0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:228
> 228                 new (&n->key) Key(k);
> (gdb) bt
> #0  QMapData<QString, QVariant>::createNode (left=false, parent=0x0, v=..., k=..., this=0x7fff2400ecd0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:228
> #1  QMapNode<QString, QVariant>::copy (this=0x696769724f656d69, d=d@entry=0x7fff2400ecd0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:258
> #2  0x00007ffff788a9d8 in QMap<QString, QVariant>::detach_helper (this=0x2e9df08)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:1008
> #3  0x00007ffff78ac04d in QMap<QString, QVariant>::detach (this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:364
> #4  QMap<QString, QVariant>::insert (avalue=..., akey=..., this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:706
> #5  Digikam::DImg::setAttribute (this=<optimized out>, key=..., value=...)
>     at /tmp/tmp.r5KKLL1qk4/digikam/core/libs/dimg/dimg_props.cpp:207
> #6  0x00007ffff797f30a in Digikam::LoadSaveThread::exifRotate (image=image@entry=0x473fc98, filePath=...)
>     at /tmp/tmp.r5KKLL1qk4/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:346
> #7  0x00007ffff7969014 in Digikam::PreviewLoadingTask::execute (this=0x473fab0)
>     at /tmp/tmp.r5KKLL1qk4/digikam/core/libs/threadimageio/preview/previewtask.cpp:296
> #8  0x00007ffff797ecee in Digikam::LoadSaveThread::run (this=0x7fff440dfec0)
>     at /tmp/tmp.r5KKLL1qk4/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #9  0x00007ffff799f69e in Digikam::DynamicThread::Private::run (this=0x3725bd0)
>     at /tmp/tmp.r5KKLL1qk4/digikam/core/libs/threads/dynamicthread.cpp:183
> #10 0x00007ffff4b227d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #11 0x00007ffff4b1e062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #12 0x00007ffff44e9ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #13 0x00007ffff441f22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Timokau,

I tested DNN face detection while this week end on my huge collection. It crash also few time, and after restart it complete the job.

I investigate to found the bug on my computer, but i'm not sure that it's the same, as i cannot get a backtrace under GDB. Your trace is instructive... Maik, did you already seen a similar trace before...

Gilles

This crash is one of many here in the thread. It crashes in a static function to the Exif rotation, writing the DImg attribute into the QMap. The DImg is alone at this moment and not shared with another thread. Problem could be the static function if another thread should go through the function at the same time. As I said modern compilers should prevent it.

Gilles, a bugtrace from your crash would be interesting now. you probably need many images that are not yet rotated, like the WhatsApp images by Timokau. But then there is still the strange crash in the overridden function.

Maik

Git commit c3476af439fa8351eee23f4477aaed8b2c00f56c by Maik Qualmann.
Committed on 12/11/2019 at 17:51.
Pushed by mqualmann into branch 'master'.

protect ExifRotate with a QMutex

M  +2    -0    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/c3476af439fa8351eee23f4477aaed8b2c00f56c

Inter thread static function uses are definitively the hell. Protect with a mutex, or make this function not static, included in DImg class for ex.

My crash is difficult to capture. Even is a core dumped file is generated, as said the system, i cannot found it. systemd report to core dumped file to list. This is nothing stored digiKam run path. So perhaps something is missing on system configuration...

Gilles

Okay, then the best way to move these functions to the DImg class.

Maik

We'll wait and see what the QMutex changes. We have access to the InfoProvider() in the static exifOrientation() function. This can not be easily moved to DImg.

Maik

Git commit 66346eddf9faafc0d8a16c603385e4b450a96968 by Maik Qualmann.
Committed on 12/11/2019 at 21:46.
Pushed by mqualmann into branch 'master'.

first step to move static function to DImg
next step move static uniqueHash() functions

M  +2    -3    core/dplugins/bqm/decorate/watermark/watermark.cpp
M  +1    -2    core/dplugins/bqm/transform/rotate/rotate.cpp
M  +22   -1    core/libs/dimg/dimg.h
M  +10   -0    core/libs/dimg/dimg_metadata.cpp
M  +1    -0    core/libs/dimg/dimg_p.h
M  +25   -0    core/libs/dimg/dimg_transform.cpp
M  +0    -56   core/libs/threadimageio/fileio/loadsavethread.cpp
M  +0    -16   core/libs/threadimageio/fileio/loadsavethread.h
M  +3    -5    core/libs/threadimageio/preview/previewtask.cpp
M  +2    -4    core/utilities/imageeditor/core/editorcore.cpp

https://invent.kde.org/kde/digikam/commit/66346eddf9faafc0d8a16c603385e4b450a96968

With the mutex, it crash after a long time to detect faces (completed at 80%):


digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/cicada.jpg" : "JPEG" file identified
digikam.general: Using  8  CPU core to run threads
digikam.general: Using  8  CPU core to run threads
digikam.general: Action Thread run  1  new jobs
digikam.general: Action Thread run  1  new jobs
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam.general: One job is done
digikam.general: Cancel Main Thread
digikam.general: One job is done
digikam.general: Cancel Main Thread
digikam.general: Try to get preview from "/mnt/data/photos/TESTS/JPEG/Horizontal/DSC00626.JPG"
digikam.general: Preview quality:  1
digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/DSC00626.JPG" : "JPEG" file identified
digikam.metaengine: Exiv2 ( 2 ) :  Directory SonyMinolta, entry 0x0088: Data area exceeds data buffer, ignoring it.

digikam.general: Found 0 faces in "CANON-IXUS850.JPG" QSize(3072, 2304) QSize(3072, 2304)
digikam.general: Removing old entries ()
digikam.general: Check for finish:  42 packages, 0 infos to filter, hasFinished() false
digikam.general: Found 0 faces in "CANON-POWERSHOTA70-01.JPG" QSize(2048, 1536) QSize(2048, 1536)
digikam.general: Removing old entries ()
digikam.general: Check for finish:  41 packages, 0 infos to filter, hasFinished() false
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam.general: Try to get preview from "/mnt/data/photos/TESTS/JPEG/Horizontal/DSCF2180.JPG"
digikam.general: Preview quality:  1
digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/DSCF2180.JPG" : "JPEG" file identified
digikam.general: Found 0 faces in "CANON-POWERSHOTA70-02.JPG" QSize(2048, 1536) QSize(2048, 1536)
digikam.general: Removing old entries ()
digikam.general: Check for finish:  40 packages, 0 infos to filter, hasFinished() false
digikam.facesengine: Good rect =  QRect(222,213 41x43) , conf =  0.684318
digikam.general: Found 1 faces in "car.jpg" QSize(1600, 1200) QSize(1600, 1200)
digikam.general: Removing old entries (FaceTagsIface(1, image 33634, tag 3, regionQRect(1184,939 219x229))
digikam.general: Check for finish:  39 packages, 0 infos to filter, hasFinished() false
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam.general: Try to get preview from "/mnt/data/photos/TESTS/JPEG/Horizontal/DSCF2260.JPG"
digikam.general: Preview quality:  1
digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/DSCF2260.JPG" : "JPEG" file identified
digikam.general: Found 0 faces in "cicada.jpg" QSize(1280, 960) QSize(1280, 960)
digikam.general: Removing old entries ()
digikam.general: Check for finish:  38 packages, 0 infos to filter, hasFinished() false
digikam.general: Using  8  CPU core to run threads
digikam.general: Action Thread run  1  new jobs
digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) ) "
digikam.database: Search query:
 "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images        INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );" 
 (QVariant(int, 3), QVariant(int, 3), QVariant(QString, "autodetectedPerson"), QVariant(int, 3), QVariant(int, 3), QVariant(QString, "autodetectedFace"), QVariant(int, 3), QVariant(int, 3), QVariant(QString, "tagRegion"))
digikam.general: Found 0 faces in "DSC00626.JPG" QSize(3872, 2592) QSize(3872, 2592)
digikam.database: Search result: 8466
digikam.general: Removing old entries ()
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam.general: Try to get preview from "/mnt/data/photos/TESTS/JPEG/Horizontal/flower.jpg"
digikam.general: Preview quality:  1
digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/flower.jpg" : "JPEG" file identified
digikam.general: Check for finish:  37 packages, 0 infos to filter, hasFinished() false
digikam.general: Try to get preview from "/mnt/data/photos/TESTS/JPEG/Horizontal/FromEFPost01.JPG"
digikam.general: Preview quality:  1
digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/FromEFPost01.JPG" : "JPEG" file identified
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam.general: Try to get preview from "/mnt/data/photos/TESTS/JPEG/Horizontal/FromEFPost02.JPG"
digikam.general: Preview quality:  1
digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/FromEFPost02.JPG" : "JPEG" file identified
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam.general: Try to get preview from "/mnt/data/photos/TESTS/JPEG/Horizontal/FromEFPost03.JPG"
digikam.general: Preview quality:  1
digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/FromEFPost03.JPG" : "JPEG" file identified
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam.general: Try to get preview from "/mnt/data/photos/TESTS/JPEG/Horizontal/FUJI-FINEPIXS7000.JPG"
digikam.general: Preview quality:  1
digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/FUJI-FINEPIXS7000.JPG" : "JPEG" file identified
digikam.general: Found 0 faces in "DSCF2180.JPG" QSize(3648, 2736) QSize(3648, 2736)
digikam.general: Removing old entries ()
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
digikam.general: Try to get preview from "/mnt/data/photos/TESTS/JPEG/Horizontal/FUJI-FINEPIXS9500.JPG"
digikam.general: Preview quality:  1
digikam.dimg: "/mnt/data/photos/TESTS/JPEG/Horizontal/FUJI-FINEPIXS9500.JPG" : "JPEG" file identified
digikam.general: Cancel Main Thread
digikam.general: One job is done
digikam.general: Check for finish:  36 packages, 0 infos to filter, hasFinished() false
digikam.general: Cancel Main Thread
digikam.general: Found 0 faces in "DSCF2260.JPG" QSize(3648, 2736) QSize(3648, 2736)
digikam.general: Removing old entries ()
digikam.general: Check for finish:  35 packages, 0 infos to filter, hasFinished() false
Segmentation fault (core dumped)
[gilles@localhost build]$ ll
total 65976

Note : i don't yet tested your last commit #66346eddf9faafc0d8a16c603385e4b450a96968

Test was processed with this one : #c3476af439fa8351eee23f4477aaed8b2c00f56c

Gilles

The scan came very close to finishing, then this (with 816d4e61b9cf9ddbaa41381c99e67e0c60ecae75, which came after your latest fix):

> digikam.general: Found 1 faces in "IMG_20171228_134932_1.jpg" QSize(1944, 2592) QSize(1944, 2592)
> digikam.general: Try to get preview from "/tmp/tmp.2P4nPAFBno/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171228_162912.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.2P4nPAFBno/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171228_162912.jpg" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.2P4nPAFBno/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171228_162921.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.2P4nPAFBno/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171228_162921.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.facesengine: Good rect =  QRect(164,93 63x88) , conf =  0.999894
> digikam.general: Found 1 faces in "IMG_20171228_134933.jpg" QSize(1944, 2592) QSize(1944, 2592)
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Using  4  CPU core to run threads
> [New Thread 0x7fff257fa700 (LWP 25987)]
> digikam.general: Action Thread run  1  new jobs
> [New Thread 0x7fff26ffd700 (LWP 25988)]
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=
> ?)) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images
>         INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagPrope
> rties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );"
>  (QVariant(int, 6), QVariant(int, 6), QVariant(QString, "autodetectedPerson"), QVariant(int, 6), QVariant(int, 6), QVariant(QString, "autodetectedFace"), QVariant(int, 6), QVariant(int, 6), QVariant(QString, "tagRegion"))
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.2P4nPAFBno/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171228_162932.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.2P4nPAFBno/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171228_162932.jpg" : "JPEG" file identified
> digikam.general: Try to get preview from "/tmp/tmp.2P4nPAFBno/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171228_163036.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.2P4nPAFBno/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171228_163036.jpg" : "JPEG" file identified
> digikam.database: Search result: 3677
> digikam.general: Cancel Main Thread
> digikam.general: One job is done
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Found 0 faces in "IMG_20171228_135010.jpg" QSize(3120, 4208) QSize(3120, 4208)
> digikam.general: Cancel Main Thread
> digikam.general: Found 0 faces in "IMG_20171228_135012.jpg" QSize(3120, 4208) QSize(3120, 4208)
> [Thread 0x7fff257fa700 (LWP 25987) exited]
> [Thread 0x7fff26ffd700 (LWP 25988) exited]
> 
> Thread 31 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff5effb700 (LWP 17967)]
> 0x00007ffff78ac43b in std::__atomic_base<int>::load (__m=std::memory_order_relaxed, this=<optimized out>)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/atomic_base.h:396
> 396             return __atomic_load_n(&_M_i, __m);
> (gdb) bt
> #0  0x00007ffff78ac43b in std::__atomic_base<int>::load (__m=std::memory_order_relaxed, this=<optimized out>)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/atomic_base.h:396
> #1  QAtomicOps<int>::load<int> (_q_value=...)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qatomic_cxx11.h:227
> #2  QBasicAtomicInteger<int>::load (this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qbasicatomic.h:103
> #3  QtPrivate::RefCount::isShared (this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qrefcount.h:101
> #4  QMap<QString, QVariant>::detach (this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:364
> #5  QMap<QString, QVariant>::insert (avalue=..., akey=..., this=<optimized out>)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:706
> #6  Digikam::DImg::setAttribute (this=0x3055ee8, key=..., value=...)
>     at /tmp/tmp.2P4nPAFBno/digikam/core/libs/dimg/dimg_props.cpp:207
> #7  0x00007ffff78a4696 in Digikam::DImg::load (this=this@entry=0x3055ee8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63,
>     observer=observer@entry=0x3055d10, rawDecodingSettings=...) at /tmp/tmp.2P4nPAFBno/digikam/core/libs/dimg/dimg_fileio.cpp:135
> #8  0x00007ffff78a5970 in Digikam::DImg::load (this=this@entry=0x3055ee8, filePath=..., observer=observer@entry=0x3055d10,
>     rawDecodingSettings=...) at /tmp/tmp.2P4nPAFBno/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #9  0x00007ffff7969180 in Digikam::PreviewLoadingTask::execute (this=0x3055d00)
>     at /tmp/tmp.2P4nPAFBno/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #10 0x00007ffff797f3ae in Digikam::LoadSaveThread::run (this=0x2eb68d0)
>     at /tmp/tmp.2P4nPAFBno/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #11 0x00007ffff799fa8e in Digikam::DynamicThread::Private::run (this=0x199ebf0)
>     at /tmp/tmp.2P4nPAFBno/digikam/core/libs/threads/dynamicthread.cpp:183
> #12 0x00007ffff4b227d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #13 0x00007ffff4b1e062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #14 0x00007ffff44e9ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #15 0x00007ffff441f22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit de875df40545d59cf038a0134c92377aed8d6187 by Maik Qualmann.
Committed on 13/11/2019 at 19:37.
Pushed by mqualmann into branch 'master'.

move static functions to DImg for more possible thread safe

M  +11   -5    core/libs/dimg/dimg.h
M  +24   -0    core/libs/dimg/dimg_fileio.cpp
M  +110  -6    core/libs/dimg/dimg_metadata.cpp
M  +1    -1    core/libs/dimg/dimg_p.h
M  +1    -157  core/libs/dimg/loaders/dimgloader.cpp
M  +0    -4    core/libs/dimg/loaders/dimgloader.h

https://invent.kde.org/kde/digikam/commit/de875df40545d59cf038a0134c92377aed8d6187

This time (de875df40545d59cf038a0134c92377aed8d6187) two full scans succeeded. The third one crashed:

> digikam.general: Try to get preview from "/tmp/tmp.pp4d0tIJXq/Pictures/p/Android/Camera/IMG_20140924_125327.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.pp4d0tIJXq/Pictures/p/Android/Camera/IMG_20140924_125327.jpg" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.pp4d0tIJXq/Pictures/p/Android/Camera/IMG_20141006_172222.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.pp4d0tIJXq/Pictures/p/Android/Camera/IMG_20141006_172222.jpg" : "JPEG" file identified
> digikam.general: Using  4  CPU core to run threads
> [New Thread 0x7fff3ffff700 (LWP 19696)]
> digikam.general: Cancel Main Thread
> digikam.general: Action Thread run  1  new jobs
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> [New Thread 0x7fff13fff700 (LWP 19697)]
> [Thread 0x7fff2b7fe700 (LWP 19669) exited]
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)
> ) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images
>       INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagPropertie
> s.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );"
>  (QVariant(int, 6), QVariant(int, 6), QVariant(QString, "autodetectedPerson"), QVariant(int, 6), QVariant(int, 6), QVariant(QString, "autodetectedFace"), QVariant(int, 6), QVariant(int, 6), QVariant(QString, "tagRegion"))
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.database: Search result: 3849
> digikam.general: Found 0 faces in "IMG_20140910_121849.jpg" QSize(2448, 3264) QSize(2448, 3264)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.pp4d0tIJXq/Pictures/p/Android/Camera/IMG_20141011_001014.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.pp4d0tIJXq/Pictures/p/Android/Camera/IMG_20141011_001014.jpg" : "JPEG" file identified
> digikam.general: Cancel Main Thread
> [Thread 0x7fff3ffff700 (LWP 19696) exited]
> digikam.general: One job is done
> digikam.general: Using  4  CPU core to run threads
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> [New Thread 0x7fff3ffff700 (LWP 19704)]
> 
> Thread 14741 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3f7fe700 (LWP 12948)]
> 0x00007ffff79693e2 in Digikam::PreviewLoadingTask::execute (this=0x3044550)
>     at /tmp/tmp.pp4d0tIJXq/digikam/core/libs/threadimageio/preview/previewtask.cpp:404
> 404                 postProcess();
> (gdb) bt
> #0  0x00007ffff79693e2 in Digikam::PreviewLoadingTask::execute (this=0x3044550)
>     at /tmp/tmp.pp4d0tIJXq/digikam/core/libs/threadimageio/preview/previewtask.cpp:404
> #1  0x00007ffff797f3be in Digikam::LoadSaveThread::run (this=0x2fd17d0)
>     at /tmp/tmp.pp4d0tIJXq/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #2  0x00007ffff799fa9e in Digikam::DynamicThread::Private::run (this=0x4ddada0)
>     at /tmp/tmp.pp4d0tIJXq/digikam/core/libs/threads/dynamicthread.cpp:183
> #3  0x00007ffff4b227d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #4  0x00007ffff4b1e062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #5  0x00007ffff44e9ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #6  0x00007ffff441f22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Ok, I think we are on the right way, because several attempts are needed to reproduce a crash. Also this crash we have rarely seen here in this long thread. I already have an idea...

Maik

I commit this fix too :

https://invent.kde.org/kde/digikam/commit/ce0cac88500e3bcfc7219baa805dabb6d91688b1

WorkerObject class in used especially in face management pipeline for face detection and recognition process...

Gilles

With 35339b4dbf2253dd1f60d3a374203d8717b05d86 (including Gilles' fix):

> digikam.dimg: "/tmp/tmp.0Tdwmd06eD/Pictures/p/Kamera-Uploads/2014-03-19 13.07.31.jpg" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.0Tdwmd06eD/Pictures/p/Kamera-Uploads/2014-03-19 13.07.34.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.0Tdwmd06eD/Pictures/p/Kamera-Uploads/2014-03-19 13.07.34.jpg" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.0Tdwmd06eD/Pictures/p/Kamera-Uploads/2014-03-21 12.16.50.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.0Tdwmd06eD/Pictures/p/Kamera-Uploads/2014-03-21 12.16.50.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Found 0 faces in "2014-03-04 06.04.54.png" QSize(720, 1280) QSize(720, 1280)
> digikam.general: Removing old entries ()
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.0Tdwmd06eD/Pictures/p/Kamera-Uploads/2014-03-22 13.33.55.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.0Tdwmd06eD/Pictures/p/Kamera-Uploads/2014-03-22 13.33.55.jpg" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Found 0 faces in "2014-03-04 06.05.02.png" QSize(720, 1280) QSize(720, 1280)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Found 0 faces in "2014-03-04 08.34.23.png" QSize(720, 1280) QSize(720, 1280)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.facesengine: Good rect =  QRect(96,102 96x121) , conf =  0.999628
> digikam.general: Found 1 faces in "2014-03-04 09.35.03.jpg" QSize(1392, 1392) QSize(1392, 1392)
> digikam.general: Removing old entries ()
> 
> Thread 24 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff5fffd700 (LWP 6656)]
> 0x00007ffff798ad87 in Digikam::LoadingTask::progressInfo (this=0x364dc60, img=0x364de48, progress=1)
>     at /tmp/tmp.0Tdwmd06eD/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) bt
> #0  0x00007ffff798ad87 in Digikam::LoadingTask::progressInfo (this=0x364dc60, img=0x364de48, progress=1) at /tmp/tmp.0Tdwmd06eD/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007fffb209e417 in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x7fff40383700, filePath=..., observer=0x364dc70) at /tmp/tmp.0Tdwmd06eD/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:621
> #2  0x00007ffff78a3a8c in Digikam::DImg::load (this=this@entry=0x364de48, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x364dc70, rawDecodingSettings=...)
>     at /tmp/tmp.0Tdwmd06eD/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #3  0x00007ffff78a4d20 in Digikam::DImg::load (this=this@entry=0x364de48, filePath=..., observer=observer@entry=0x364dc70, rawDecodingSettings=...)
>     at /tmp/tmp.0Tdwmd06eD/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #4  0x00007ffff79690f0 in Digikam::PreviewLoadingTask::execute (this=0x364dc60) at /tmp/tmp.0Tdwmd06eD/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #5  0x00007ffff797f31e in Digikam::LoadSaveThread::run (this=0x287c7e0) at /tmp/tmp.0Tdwmd06eD/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff799f9fe in Digikam::DynamicThread::Private::run (this=0x2c9ff30) at /tmp/tmp.0Tdwmd06eD/digikam/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007ffff4b227d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007ffff4b1e062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff44e9ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #10 0x00007ffff441f22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

This time it crashed during the first attempt, though it got quite far.

Git commit cab5995cacb2bfd97b304c2338b00e2ae176fdff by Maik Qualmann.
Committed on 14/11/2019 at 20:48.
Pushed by mqualmann into branch 'master'.

do not add this task to the listeners in the postprocess

M  +2    -2    core/libs/threadimageio/fileio/loadsavetask.cpp

https://invent.kde.org/kde/digikam/commit/cab5995cacb2bfd97b304c2338b00e2ae176fdff

With 47e8d093858421c1b95ea17acb396067ffccc8eb:

> digikam.general: Found 0 faces in "IMG_20150502_140220.jpg" QSize(3120, 4208) QSize(3120, 4208)
> digikam.general: Removing old entries ()
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Cancel Main Thread
> digikam.general: One job is done
> QPainter::begin: Paint device returned engine == 0, type: 2
> QPainter::setCompositionMode: Painter not active
> QPainter::end: Painter not active, aborted
> QPainter::begin: Paint device returned engine == 0, type: 2
> QPainter::setCompositionMode: Painter not active
> QPainter::end: Painter not active, aborted
> digikam.general: Cancel Main Thread
> QPainter::begin: Paint device returned engine == 0, type: 2
> QPainter::setCompositionMode: Painter not active
> QPainter::end: Painter not active, aborted
> QPainter::begin: Paint device returned engine == 0, type: 2
> QPainter::setCompositionMode: Painter not active
> QPainter::end: Painter not active, aborted
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> [Thread 0x7fff20ff9700 (LWP 26871) exited]
> [Thread 0x7fff237fe700 (LWP 26872) exited]
> 
> Thread 32 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3f7fe700 (LWP 25750)]
> Digikam::MetaEngineData::MetaEngineData (this=0x7fff3f7fcda8, other=...)
>     at /tmp/tmp.dCPXcFOXaA/digikam/core/libs/metadataengine/engine/metaengine_data.cpp:43
> 43          d = other.d;
> (gdb) bt
> #0  Digikam::MetaEngineData::MetaEngineData (this=0x7fff3f7fcda8, other=...)
>     at /tmp/tmp.dCPXcFOXaA/digikam/core/libs/metadataengine/engine/metaengine_data.cpp:43
> #1  0x00007ffff78a99e0 in Digikam::DImg::getMetadata (this=<optimized out>)
>     at /tmp/tmp.dCPXcFOXaA/digikam/core/libs/dimg/dimg_props.cpp:167
> #2  0x00007ffff7769d81 in Digikam::DImgLoader::checkExifWorkingColorSpace (this=this@entry=0x7fff2c099670)
>     at /tmp/tmp.dCPXcFOXaA/digikam/core/libs/dimg/loaders/dimgloader.cpp:242
> #3  0x00007fffb209e486 in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x7fff2c099670, filePath=..., observer=0x3eed580)
>     at /tmp/tmp.dCPXcFOXaA/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:601
> #4  0x00007ffff78a3a8c in Digikam::DImg::load (this=this@entry=0x3eed758, filePath=..., loadFlagsInt=loadFlagsInt@entry=63,
>     observer=observer@entry=0x3eed580, rawDecodingSettings=...) at /tmp/tmp.dCPXcFOXaA/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #5  0x00007ffff78a4d20 in Digikam::DImg::load (this=this@entry=0x3eed758, filePath=..., observer=observer@entry=0x3eed580,
>     rawDecodingSettings=...) at /tmp/tmp.dCPXcFOXaA/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #6  0x00007ffff79690f0 in Digikam::PreviewLoadingTask::execute (this=0x3eed570)
>     at /tmp/tmp.dCPXcFOXaA/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #7  0x00007ffff797f31e in Digikam::LoadSaveThread::run (this=0x37280f0)
>     at /tmp/tmp.dCPXcFOXaA/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #8  0x00007ffff799f97e in Digikam::DynamicThread::Private::run (this=0x263d1a0)
>     at /tmp/tmp.dCPXcFOXaA/digikam/core/libs/threads/dynamicthread.cpp:183
> #9  0x00007ffff4b217d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #10 0x00007ffff4b1d062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #11 0x00007ffff44e8ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #12 0x00007ffff441e22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 6aaa6622d706972bf8a532c0a92ea721b11c7fa0 by Maik Qualmann.
Committed on 15/11/2019 at 21:41.
Pushed by mqualmann into branch 'master'.

do not add this task to the listeners here

M  +3    -3    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/6aaa6622d706972bf8a532c0a92ea721b11c7fa0

With 2e3d03a2cea443f32d8a23c85393c345e8308e5b:

> digikam.dimg: "/tmp/tmp.7dg66TNVh9/Pictures/p/images/Telegram Images/223024373_562302.jpg" : "JPEG" file identified
> digikam.general: Found 0 faces in "223023849_735468.jpg" QSize(1280, 1258) QSize(1280, 1258)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.7dg66TNVh9/Pictures/p/images/Telegram Images/223024391_291148.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.7dg66TNVh9/Pictures/p/images/Telegram Images/223024391_291148.jpg" : "JPEG" file identified
> digikam.general: Try to get preview from "/tmp/tmp.7dg66TNVh9/Pictures/p/images/Telegram Images/223024438_254640.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.7dg66TNVh9/Pictures/p/images/Telegram Images/223024438_254640.jpg" : "JPEG" file identified
> digikam.general: Found 0 faces in "223023852_356023.jpg" QSize(1129, 1280) QSize(1129, 1280)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.7dg66TNVh9/Pictures/p/images/Telegram Images/223024462_270324.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.7dg66TNVh9/Pictures/p/images/Telegram Images/223024462_270324.jpg" : "JPEG" file identified
> 
> Thread 1342 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff41ffb700 (LWP 20326)]
> 0x00007ffff7968321 in Digikam::PreviewLoadingTask::execute (this=0x8e4f920)
>     at /tmp/tmp.7dg66TNVh9/digikam/core/libs/threadimageio/preview/previewtask.cpp:404
> 404                 postProcess();
> (gdb) bt
> #0  0x00007ffff7968321 in Digikam::PreviewLoadingTask::execute (this=0x8e4f920) at /tmp/tmp.7dg66TNVh9/digikam/core/libs/threadimageio/preview/previewtask.cpp:404
> #1  0x00007ffff797e2ee in Digikam::LoadSaveThread::run (this=0x2577e60) at /tmp/tmp.7dg66TNVh9/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #2  0x00007ffff799e94e in Digikam::DynamicThread::Private::run (this=0x2f6f240) at /tmp/tmp.7dg66TNVh9/digikam/core/libs/threads/dynamicthread.cpp:183
> #3  0x00007ffff4b1f7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #4  0x00007ffff4b1b062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #5  0x00007ffff44e6ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #6  0x00007ffff441c22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit eb5a4a61fddaded66f07e57e44cd2d98862a1fa9 by Maik Qualmann.
Committed on 16/11/2019 at 19:12.
Pushed by mqualmann into branch 'master'.

fix ICC Manager, must use a DImg reference

M  +9    -8    core/libs/dimg/filters/icc/iccmanager.cpp
M  +3    -5    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +5    -2    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/eb5a4a61fddaded66f07e57e44cd2d98862a1fa9

Git commit da67056beb32c461251661e90d8f4c73f984ce26 by Maik Qualmann.
Committed on 16/11/2019 at 19:39.
Pushed by mqualmann into branch 'master'.

revert last commit, private DImg data is shared

M  +6    -6    core/libs/dimg/filters/icc/iccmanager.cpp
M  +1    -1    core/libs/dimg/filters/icc/iccmanager.h

https://invent.kde.org/kde/digikam/commit/da67056beb32c461251661e90d8f4c73f984ce26

Maik, 

The problem from this file can be relevant of the long time problem located in DSharedData container and never fixed.

Look this clang report:

https://www.digikam.org/reports/clang/master/

==> Memory error; Use-after-free; mnt/devel/GIT/6.x/core/libs/dimg/dshareddata.h; operator Digikam::DImg::Private; ligne 120

And look well the analysis. Clang crying about this operator :

inline operator T* ()
{
    return d; <== Use of memory after it is freed
}

...because the memory is released here :

inline DSharedDataPointer& operator=(T* const o)
{
   delete assign(o);    <=== Memory is released	
   return *this;
}

... when this detach function is called :

void DImg::detach()
{
   if (!m_priv->hasMoreReferences())
   {
      return;
   }

   DSharedDataPointer<Private> old = m_priv; <=== Calling 'DSharedDataPointer::operator='; Returning; memory was released	
   m_priv = new Private;

   copyImageData(old); <=== Calling 'DSharedDataPointer::operator Digikam::DImg::Private *' !!!!!!!! BOUM !!!!!!!!
   copyMetaData(old);

   if (old->data)
   {
       size_t size = allocateData();
       memcpy(m_priv->data, old->data, size);
   }
}

Gilles

Yes, I have already thought about that and switched in DImg to QSharedDataPointer. I can apply the patch as a trial.

Maik

Git commit f037b05d531c90f95d8f41b7f81d837ace6aeb5b by Maik Qualmann.
Committed on 17/11/2019 at 07:25.
Pushed by mqualmann into branch 'master'.

use for a test QExplicitlySharedDataPointer for DImg

M  +2    -2    core/libs/dimg/dimg.cpp
M  +2    -2    core/libs/dimg/dimg.h
M  +2    -2    core/libs/dimg/dimg_copy.cpp
M  +4    -4    core/libs/dimg/dimg_data.cpp
M  +2    -2    core/libs/dimg/dimg_p.h

https://invent.kde.org/kde/digikam/commit/f037b05d531c90f95d8f41b7f81d837ace6aeb5b

With b306f5aa14358dd04167b1d197de1fdc86e37bbc (on the 4th run):

> digikam.general: Try to get preview from "/tmp/tmp.sIBLW3QdU7/Pictures/p/Kamera-Uploads/2014-03-13 20.01.02.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.sIBLW3QdU7/Pictures/p/Kamera-Uploads/2014-03-13 20.01.02.jpg" : "JPEG" file identified
> digikam.general: Using  4  CPU core to run threads
> [New Thread 0x7fff3ffff700 (LWP 16592)]
> digikam.general: Action Thread run  1  new jobs
> [New Thread 0x7fff31ffb700 (LWP 16593)]
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)
> ) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images
>       INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagPropertie
> s.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );"
>  (QVariant(int, 6), QVariant(int, 6), QVariant(QString, "autodetectedPerson"), QVariant(int, 6), QVariant(int, 6), QVariant(QString, "autodetectedFace"), QVariant(int, 6), QVariant(int, 6), QVariant(QString, "tagRegion"))
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.database: Search result: 3849
> digikam.general: Found 0 faces in "2014-01-07 14.27.05.jpg" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "2014-02-05 14.26.20.jpg" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "2014-02-07 15.43.22.jpg" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> 
> Thread 13957 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fffa51ee700 (LWP 20947)]
> 0x0000000000000000 in ?? ()
> (gdb) bt
> #0  0x0000000000000000 in ?? ()
> #1  0x00007ffff7966b0f in Digikam::PreviewLoadingTask::execute (this=0x292ea70) at /tmp/tmp.sIBLW3QdU7/digikam/core/libs/threadimageio/preview/previewtask.cpp:405
> #2  0x00007ffff797cafe in Digikam::LoadSaveThread::run (this=0x3ac4dd0) at /tmp/tmp.sIBLW3QdU7/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #3  0x00007ffff799d1de in Digikam::DynamicThread::Private::run (this=0x26c1940) at /tmp/tmp.sIBLW3QdU7/digikam/core/libs/threads/dynamicthread.cpp:183
> #4  0x00007ffff4b1b7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #5  0x00007ffff4b17062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #6  0x00007ffff44e2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #7  0x00007ffff441822f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit d7b1ea9ccdd65d9a7d85be0ed035b33f0f7d5d31 by Maik Qualmann.
Committed on 18/11/2019 at 19:49.
Pushed by mqualmann into branch 'master'.

prevent to access a QList simultaneously from multiple threads

M  +10   -3    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +11   -3    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/d7b1ea9ccdd65d9a7d85be0ed035b33f0f7d5d31

Git commit ba2893d27dd4542034f90b56bc4d75f1cb727d42 by Maik Qualmann.
Committed on 19/11/2019 at 07:10.
Pushed by mqualmann into branch 'master'.

fix wrong DImgLoaderObserver in postProcess()

M  +3    -3    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +1    -1    core/libs/threadimageio/fileio/loadsavetask.h
M  +1    -1    core/libs/threadimageio/preview/previewtask.cpp
M  +2    -2    core/libs/threadimageio/thumb/thumbnailtask.cpp
M  +2    -2    core/libs/threadimageio/thumb/thumbnailtask.h

https://invent.kde.org/kde/digikam/commit/ba2893d27dd4542034f90b56bc4d75f1cb727d42

I cannot start face detection with ba2893d27dd4542034f90b56bc4d75f1cb727d42. It gives me the error:

Face recognition is aborted, because there are no identities to recognize. Please add new identities.

Even though I've selected face detection, not recognition.

Yes, my mistake is with an empty face database. You would have to add a face by name manually. I fix it...

Maik

Git commit fe0020243fa27001b5daf575fb894663fdbe534b by Maik Qualmann.
Committed on 19/11/2019 at 17:35.
Pushed by mqualmann into branch 'master'.

check only for empty identities for face recognition

M  +5    -2    core/utilities/facemanagement/widgets/facescandialog.cpp

https://invent.kde.org/kde/digikam/commit/fe0020243fa27001b5daf575fb894663fdbe534b

With fe0020243fa27001b5daf575fb894663fdbe534b:

> digikam.general: Found 0 faces in "IMG_20171125_194502.jpg" QSize(3120, 4208) QSize(3120, 4208)
> digikam.general: Removing old entries ()
> digikam.general: Try to get preview from "/tmp/tmp.LFZ59UShRA/Pictures/p/android_pics/DCIM/Camera/IMG_20171126_004126.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.LFZ59UShRA/Pictures/p/android_pics/DCIM/Camera/IMG_20171126_004126.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.LFZ59UShRA/Pictures/p/android_pics/DCIM/Camera/IMG_20171126_112717.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.LFZ59UShRA/Pictures/p/android_pics/DCIM/Camera/IMG_20171126_112717.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Using  4  CPU core to run threads
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Found 0 faces in "IMG_20171125_194503.jpg" QSize(3120, 4208) QSize(3120, 4208)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "IMG_20171125_211335.jpg" QSize(3120, 4208) QSize(3120, 4208)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "IMG_20171125_211336.jpg" QSize(3120, 4208) QSize(3120, 4208)
> digikam.general: Removing old entries ()
> [New Thread 0x7fff267fc700 (LWP 27668)]
> 
> Thread 786 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff38f54700 (LWP 14401)]
> 0x00007ffff7988637 in Digikam::LoadingTask::progressInfo (this=0x363d540, img=0x363d728, progress=0.75)
>     at /tmp/tmp.LFZ59UShRA/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) bt
> #0  0x00007ffff7988637 in Digikam::LoadingTask::progressInfo (this=0x363d540, img=0x363d728, progress=0.75)
>     at /tmp/tmp.LFZ59UShRA/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007fffc28d7a40 in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x7fff303bb0b0, filePath=..., observer=0x363d550)
>     at /tmp/tmp.LFZ59UShRA/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:434
> #2  0x00007ffff78a0adc in Digikam::DImg::load (this=this@entry=0x363d728, filePath=..., loadFlagsInt=loadFlagsInt@entry=63,
>     observer=observer@entry=0x363d550, rawDecodingSettings=...) at /tmp/tmp.LFZ59UShRA/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #3  0x00007ffff78a1d70 in Digikam::DImg::load (this=this@entry=0x363d728, filePath=..., observer=observer@entry=0x363d550,
>     rawDecodingSettings=...) at /tmp/tmp.LFZ59UShRA/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #4  0x00007ffff796695c in Digikam::PreviewLoadingTask::execute (this=0x363d540)
>     at /tmp/tmp.LFZ59UShRA/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #5  0x00007ffff797cbce in Digikam::LoadSaveThread::run (this=0x2df45f0)
>     at /tmp/tmp.LFZ59UShRA/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff799d30e in Digikam::DynamicThread::Private::run (this=0x3247510)
>     at /tmp/tmp.LFZ59UShRA/digikam/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007ffff4b1a7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007ffff4b16062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff44e1ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #10 0x00007ffff441722f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit d235ea6eff1cff7a8240fd9ac1a67ce441aae940 by Maik Qualmann.
Committed on 20/11/2019 at 06:44.
Pushed by mqualmann into branch 'master'.

do not receive progress info in postProcess()

M  +3    -13   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +1    -1    core/libs/threadimageio/fileio/loadsavetask.h
M  +1    -12   core/libs/threadimageio/preview/previewtask.cpp
M  +2    -2    core/libs/threadimageio/thumb/thumbnailtask.cpp
M  +2    -2    core/libs/threadimageio/thumb/thumbnailtask.h

https://invent.kde.org/kde/digikam/commit/d235ea6eff1cff7a8240fd9ac1a67ce441aae940

With 6845671c7e6b60e337fae81b097d14bb32e0b6e8:

> digikam.general: Try to get preview from "/tmp/tmp.oF2HC2AYd3/Pictures/p/android_pics/DCIM/Camera/IMG_20171221_153250.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.oF2HC2AYd3/Pictures/p/android_pics/DCIM/Camera/IMG_20171221_153250.jpg" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.oF2HC2AYd3/Pictures/p/android_pics/DCIM/Camera/IMG_20171221_153251.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.oF2HC2AYd3/Pictures/p/android_pics/DCIM/Camera/IMG_20171221_153251.jpg" : "JPEG" file identified
> digikam.facesengine: Good rect =  QRect(179,146 35x47) , conf =  0.622759
> digikam.general: Found 1 faces in "IMG_20171221_125943.jpg" QSize(3120, 4208) QSize(3120, 4208)
> digikam.general: Removing old entries ()
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.oF2HC2AYd3/Pictures/p/android_pics/DCIM/Camera/IMG_20171221_153449.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.oF2HC2AYd3/Pictures/p/android_pics/DCIM/Camera/IMG_20171221_153449.jpg" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.oF2HC2AYd3/Pictures/p/android_pics/DCIM/Camera/IMG_20171221_153456.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.oF2HC2AYd3/Pictures/p/android_pics/DCIM/Camera/IMG_20171221_153456.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> 
> Thread 46 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff427fc700 (LWP 739)]
> 0x00007ffff43c3626 in __memmove_sse2_unaligned_erms () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007ffff43c3626 in __memmove_sse2_unaligned_erms () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> #1  0x00007ffff789f323 in memcpy (__len=<optimized out>, __src=<optimized out>, __dest=<optimized out>) at /nix/store/bniand9afisrgrsfi7kr093334iv3ibv-glibc-2.27-dev/include/bits/string_fortified.h:34
> #2  Digikam::DImg::detach (this=0x799fd8) at /tmp/tmp.oF2HC2AYd3/digikam/core/libs/dimg/dimg_data.cpp:67
> #3  0x00007ffff7966787 in Digikam::PreviewLoadingTask::execute (this=0x799df0) at /tmp/tmp.oF2HC2AYd3/digikam/core/libs/threadimageio/preview/previewtask.cpp:380
> #4  0x00007ffff797cb1e in Digikam::LoadSaveThread::run (this=0x3100e60) at /tmp/tmp.oF2HC2AYd3/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #5  0x00007ffff799d17e in Digikam::DynamicThread::Private::run (this=0x3466ba0) at /tmp/tmp.oF2HC2AYd3/digikam/core/libs/threads/dynamicthread.cpp:183
> #6  0x00007ffff4b1a7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #7  0x00007ffff4b16062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007ffff44e1ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #9  0x00007ffff441722f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Maik,

When i compile with Clang static ananlyzer 8.0 under Mageia 7.1 i can see that:


In file included from /home/gilles/Documents/7.x/core/libs/dimg/dimg_data.cpp:26:
In file included from /home/gilles/Documents/7.x/core/libs/dimg/dimg_p.h:52:
In file included from /usr/lib64/qt5/include/QtCore/QSharedData:1:
/usr/lib64/qt5/include/QtCore/qshareddata.h:161:41: warning: Use of memory after it is freed
    inline const T *constData() const { return d; }
                                        ^~~~~~~~
1 warning generated.

These king of report is dropped from complete report published in digikam.org/reports/ as it's outside of digiKam space, but this can be a way to explore...

Gilles

I think that in this case we still have 2 references to QSharedData. It crashes in memcpy(). We do not check the memory allocation, whether we have received it. Timokau, what is the memory usage during face detection?

Maik

Memory usage is pretty high. I have 8G of ram and swap space is still needed for the scan to complete. It seems to me that the crash occurs more often when I use the system otherwise at the same time, so it may very well be that it has something to do with load.

But for a memcpy to fail, both physical ram and swap would have to be fully exhausted right?

Another data point is that IIRC I reproduced the crash on my laptop too at some point. The laptop has 16G of RAM.

Git commit ea529419b90d54ce5bc6052a14ec2c23c5f79b1f by Maik Qualmann.
Committed on 21/11/2019 at 16:58.
Pushed by mqualmann into branch 'master'.

various optimization for image loading

M  +4    -5    core/libs/dimg/dimg_data.cpp
M  +1    -1    core/libs/threadimageio/fileio/loadingcache.cpp
M  +1    -7    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/ea529419b90d54ce5bc6052a14ec2c23c5f79b1f

I kept a loose eye on my the ram usage during this run, seemed to hover around ~5-6G actual memory and 1-2G swap. The system definitely didn't OOM. I was using it at the same time, and I should've noticed that.  With 7f0595ab85f3bf305876771e96671da3dce7122e:

> Gigikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.tlWfpH3zHy/Pictures/p/path/file.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.tlWfpH3zHy/Pictures/p/path/file.JPG" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.tlWfpH3zHy/Pictures/p/path/file.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.tlWfpH3zHy/Pictures/p/path/file.JPG" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.tlWfpH3zHy/Pictures/p/path/file.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.tlWfpH3zHy/Pictures/p/path/file.JPG" : "JPEG" file identified
> [New Thread 0x7fff187d2700 (LWP 23549)]
> 
> Thread 31 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fffa51ee700 (LWP 28349)]
> 0x00007ffff7988567 in Digikam::LoadingTask::progressInfo (this=0x4993e60, img=0x4994048, progress=0.5) at /tmp/tmp.tlWfpH3zHy/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) bt
> #0  0x00007ffff7988567 in Digikam::LoadingTask::progressInfo (this=0x4993e60, img=0x4994048, progress=0.5)
>     at /tmp/tmp.tlWfpH3zHy/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007fffb2098a40 in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x7fff48325680, filePath=..., observer=0x4993e70)
>     at /tmp/tmp.tlWfpH3zHy/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:434
> #2  0x00007ffff78a0aec in Digikam::DImg::load (this=this@entry=0x4994048, filePath=..., loadFlagsInt=loadFlagsInt@entry=63,
>     observer=observer@entry=0x4993e70, rawDecodingSettings=...) at /tmp/tmp.tlWfpH3zHy/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #3  0x00007ffff78a1d80 in Digikam::DImg::load (this=this@entry=0x4994048, filePath=..., observer=observer@entry=0x4993e70,
>     rawDecodingSettings=...) at /tmp/tmp.tlWfpH3zHy/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #4  0x00007ffff7966900 in Digikam::PreviewLoadingTask::execute (this=0x4993e60)
>     at /tmp/tmp.tlWfpH3zHy/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #5  0x00007ffff797cafe in Digikam::LoadSaveThread::run (this=0xf4a270)
>     at /tmp/tmp.tlWfpH3zHy/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff799d15e in Digikam::DynamicThread::Private::run (this=0x2b876d0)
>     at /tmp/tmp.tlWfpH3zHy/digikam/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007ffff4b1a7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007ffff4b16062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff44e1ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #10 0x00007ffff441722f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit c31a8b4e3650fe05392e01c2388f2f2847a191b4 by Maik Qualmann.
Committed on 21/11/2019 at 22:09.
Pushed by mqualmann into branch 'master'.

for a test we disable the QCache

M  +2    -2    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/c31a8b4e3650fe05392e01c2388f2f2847a191b4

It is only interested in what has changed in the memory usage.

Maik

Post of commits with CCBUGS seems broken. Please test this experiment:

https://invent.kde.org/kde/digikam/commit/7c50ae256ee7eb59e4c00dc101d7d7339a420ab9

Maik

I tracked the memory usage with psrecord (attached). You can easily see the point where it crashed, when the CPU usage dropped to 0 and the memory usage flattened out.
With b50ee49d2455c936f02178d59389669e0796e506:

> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images
>       INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagPropertie
> s.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );"
>  (QVariant(int, 4), QVariant(int, 4), QVariant(QString, "autodetectedPerson"), QVariant(int, 4), QVariant(int, 4), QVariant(QString, "autodetectedFace"), QVariant(int, 4), QVariant(int, 4), QVariant(QString, "tagRegion"))
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.m1S6vaoNjT/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171226_173931.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.m1S6vaoNjT/Pictures/p/images/new-pics/DCIM/Camera/IMG_20171226_173931.jpg" : "JPEG" file identified
> digikam.database: Search result: 5669
> digikam.general: Cancel Main Thread
> digikam.facesengine: Good rect =  QRect(10,116 63x74) , conf =  0.999788
> digikam.general: Found 1 faces in "IMG_20171224_155639.jpg" QSize(2592, 1944) QSize(2592, 1944)
> digikam.general: Removing old entries ()
> digikam.general: One job is done
> digikam.general: Cancel Main Thread
> digikam.general: Found 0 faces in "IMG_20171224_155649.jpg" QSize(3120, 4208) QSize(3120, 4208)
> digikam.general: Removing old entries ()
> [Thread 0x7fff29a91700 (LWP 11815) exited]
> [Thread 0x7fff28a8f700 (LWP 11814) exited]
> 
> Thread 171 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff2a292700 (LWP 29547)]
> 0x00007ffff43c33cf in __memmove_sse2_unaligned_erms () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007ffff43c33cf in __memmove_sse2_unaligned_erms () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> #1  0x00007ffff78c7d67 in std::char_traits<char>::copy (__n=32, __s2=0x4 <error: Cannot access memory at address 0x4>, __s1=<optimized out>)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/char_traits.h:350
> #2  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy (__n=32,
>     __s=0x4 <error: Cannot access memory at address 0x4>, __d=<optimized out>)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/basic_string.h:340
> #3  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_copy_chars (__k2=<optimized out>,
>     __k1=0x4 <error: Cannot access memory at address 0x4>, __p=<optimized out>)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/basic_string.h:382
> #4  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*> (this=this@entry=0x7fff08086c88,
>     __beg=0x4 <error: Cannot access memory at address 0x4>, __end=<optimized out>)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/basic_string.tcc:225
> #5  0x00007ffff78c83dd in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct_aux<char*> (
>     __end=<optimized out>, __beg=<optimized out>, this=0x7fff08086c88)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/basic_string.h:236
> #6  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*> (__end=<optimized out>,
>     __beg=<optimized out>, this=0x7fff08086c88) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/basic_string.h:255
> #7  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string (__str=..., this=0x7fff08086c88)
>     at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/basic_string.h:440
> #8  Digikam::MetaEngineData::Private::Private (this=0x7fff08086c80)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/metadataengine/engine/metaengine_data_p.h:34
> #9  QSharedDataPointer<Digikam::MetaEngineData::Private>::clone (this=0x7fff0829adb0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:253
> #10 QSharedDataPointer<Digikam::MetaEngineData::Private>::detach_helper (this=this@entry=0x7fff0829adb0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:259
> #11 0x00007ffff78cd22d in QSharedDataPointer<Digikam::MetaEngineData::Private>::detach (this=0x7fff0829adb0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:74
> #12 QSharedDataPointer<Digikam::MetaEngineData::Private>::data (this=0x7fff0829adb0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:81
> #13 Digikam::MetaEngine::Private::exifMetadata (this=0x7fff0829ad80)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/metadataengine/engine/metaengine_p.cpp:104
> #14 0x00007ffff78d39d8 in Digikam::MetaEngine::getExifTagData (this=this@entry=0x7fff2a290db0,
>     exifTagName=exifTagName@entry=0x7ffff7d94f5d "Exif.Image.InterColorProfile")
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/metadataengine/engine/metaengine_exif.cpp:686
> #15 0x00007ffff7910dd4 in Digikam::DMetadata::getIccProfile (this=this@entry=0x7fff2a290db0)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/metadataengine/dmetadata/dmetadata_exif.cpp:85
> #16 0x00007ffff7766def in Digikam::DImgLoader::checkExifWorkingColorSpace (this=this@entry=0x7fff0806b5a0)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/dimg/loaders/dimgloader.cpp:243
> #17 0x00007fffc28d8486 in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x7fff0806b5a0, filePath=..., observer=0x41ec540)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:601
> #18 0x00007ffff78a0b6c in Digikam::DImg::load (this=this@entry=0x41ec718, filePath=..., loadFlagsInt=loadFlagsInt@entry=63,
>     observer=observer@entry=0x41ec540, rawDecodingSettings=...) at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #19 0x00007ffff78a1e00 in Digikam::DImg::load (this=this@entry=0x41ec718, filePath=..., observer=observer@entry=0x41ec540, rawDecodingSettings=...)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #20 0x00007ffff7966a50 in Digikam::PreviewLoadingTask::execute (this=0x41ec530)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #21 0x00007ffff797cc4e in Digikam::LoadSaveThread::run (this=0x7fffd80092e0)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #22 0x00007ffff799d2ae in Digikam::DynamicThread::Private::run (this=0x2e7c7e0)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/threads/dynamicthread.cpp:183
> #23 0x00007ffff4b1a7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #24 0x00007ffff4b16062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #25 0x00007ffff44e1ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #26 0x00007ffff441722f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Created attachment 124119 [details]
digikam resource usage with b50ee49d2455c936f02178d59389669e0796e506

The backtrace from comment #262 is very interresting :

> #8  Digikam::MetaEngineData::Private::Private (this=0x7fff08086c80)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/metadataengine/engine/metaengine_data_p.h:34
> #9  QSharedDataPointer<Digikam::MetaEngineData::Private>::clone (this=0x7fff0829adb0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:253
> #10 QSharedDataPointer<Digikam::MetaEngineData::Private>::detach_helper (this=this@entry=0x7fff0829adb0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:259
> #11 0x00007ffff78cd22d in QSharedDataPointer<Digikam::MetaEngineData::Private>::detach (this=0x7fff0829adb0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:74
> #12 QSharedDataPointer<Digikam::MetaEngineData::Private>::data (this=0x7fff0829adb0)
>     at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:81
> #13 Digikam::MetaEngine::Private::exifMetadata (this=0x7fff0829ad80)
>     at /tmp/tmp.m1S6vaoNjT/digikam/core/libs/metadataengine/engine/metaengine_p.cpp:104
> #14 0x00007ffff78d39d8 in Digikam::MetaEngine::getExifTagData (this=this@entry=0x7fff2a290db0,
>     exifTagName=exifTagName@entry=0x7ffff7d94f5d "Exif.Image.InterColorProfile")

It's relevant of QSharedData C++ clone operation from the private container of MetaEngineDatata private container. This class do not have an overloaded  clone method, default one is used instead.

Coverity Report static analyzer start to tip to implement this kind of C++ operator with data containers (not this one). It's not a simple task and an obscur implementation to do. I tried all the way explained on the web to fix the reports from Coverity without success.

Gilles Caulier

Yes, I think the crash is clear. We use a QSharedDataPointer, read access is shared, write access calls detach(). I do not think he is able to correctly clone the Exiv2::ExifData container. We have the getExifdata() function 2 times, once as const, read only and once to read and write. I think here is the QExplicitlySharedDataPointer the right choice, which does not execute an automatic detach(). The write operations should be intercepted by our mutex.

Maik

Git commit 33735b564c7cb1276d1a0588a518190300f83d31 by Maik Qualmann.
Committed on 26/11/2019 at 17:47.
Pushed by mqualmann into branch 'master'.

use QExplicitlySharedDataPointer for MetaEngineData

M  +2    -2    core/libs/metadataengine/engine/metaengine_data.h
M  +1    -1    core/libs/metadataengine/engine/metaengine_p.h

https://invent.kde.org/kde/digikam/commit/33735b564c7cb1276d1a0588a518190300f83d31

With 105ccfd47cc8ac73037eac0d6873581a9e49c1aa:

> digikam.general: Try to get preview from "/tmp/tmp.CebnuTHvLs/Pictures/p/sth/P1020970.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.CebnuTHvLs/Pictures/p/sth/P1020970.JPG" : "JPEG" file identified
> digikam.general: Cancel Main Thread
> [Thread 0x7fff48d1a700 (LWP 9071) exited]
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Found 0 faces in "P1020949.JPG" QSize(3072, 2304) QSize(3072, 2304)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.CebnuTHvLs/Pictures/p/sth/P1020971.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.CebnuTHvLs/Pictures/p/sth/P1020971.JPG" : "JPEG" file identified
> 
> Thread 57 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff337fe700 (LWP 27911)]
> 0x0000000000000018 in ?? ()
> (gdb) bt
> #0  0x0000000000000018 in ?? ()
> #1  0x00007ffff3aa6945 in Exiv2::ExifKey::clone() const () from /nix/store/56932a3lcqp48131p0w0ggr1lhxxna3v-exiv2-0.27.1/lib/libexiv2.so.27
> #2  0x00007ffff3a58bb2 in Exiv2::Exifdatum::Exifdatum(Exiv2::Exifdatum const&) () from /nix/store/56932a3lcqp48131p0w0ggr1lhxxna3v-exiv2-0.27.1/lib/libexiv2.so.27
> #3  0x00007ffff78d45cd in __gnu_cxx::new_allocator<std::_List_node<Exiv2::Exifdatum> >::construct<Exiv2::Exifdatum, Exiv2::Exifdatum const&> (this=<optimized out>, __p=<optimized out>) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/ext/new_allocator.h:136
> #4  std::allocator_traits<std::allocator<std::_List_node<Exiv2::Exifdatum> > >::construct<Exiv2::Exifdatum, Exiv2::Exifdatum const&> (__a=..., __p=<optimized out>) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/alloc_traits.h:475
> #5  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::_M_create_node<Exiv2::Exifdatum const&> (this=0x7fff337fccb0) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:575
> #6  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::_M_insert<Exiv2::Exifdatum const&> (__position=..., this=0x7fff337fccb0) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:1801
> #7  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::emplace_back<Exiv2::Exifdatum const&> (this=0x7fff337fccb0) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:1133
> #8  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::_M_initialize_dispatch<std::_List_const_iterator<Exiv2::Exifdatum> > (__last=..., __first=..., this=0x7fff337fccb0) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:1730
> #9  std::__cxx11::list<Exiv2::Exifdatum, std::allocator<Exiv2::Exifdatum> >::list (__x=..., this=0x7fff337fccb0) at /nix/store/wr5dlcw1asw8dwgm4bwjmga8f52m5lfx-gcc-7.4.0/include/c++/7.4.0/bits/stl_list.h:655
> #10 Exiv2::ExifData::ExifData (this=0x7fff337fccb0) at /nix/store/291xr43ngjk60aw7ldshmajbavn5ajn8-exiv2-0.27.1-dev/include/exiv2/exif.hpp:434
> #11 Digikam::MetaEngine::getExifTagData (this=this@entry=0x7fff337fcdb0, exifTagName=exifTagName@entry=0x7ffff7d95f5d "Exif.Image.InterColorProfile") at /tmp/tmp.CebnuTHvLs/digikam/core/libs/metadataengine/engine/metaengine_exif.cpp:686
> #12 0x00007ffff7911984 in Digikam::DMetadata::getIccProfile (this=this@entry=0x7fff337fcdb0) at /tmp/tmp.CebnuTHvLs/digikam/core/libs/metadataengine/dmetadata/dmetadata_exif.cpp:85
> #13 0x00007ffff7767ddf in Digikam::DImgLoader::checkExifWorkingColorSpace (this=this@entry=0x7fff1c19bc00) at /tmp/tmp.CebnuTHvLs/digikam/core/libs/dimg/loaders/dimgloader.cpp:243
> #14 0x00007fffc28d8486 in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x7fff1c19bc00, filePath=..., observer=0x2e82e20) at /tmp/tmp.CebnuTHvLs/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:601
> #15 0x00007ffff78a1dfc in Digikam::DImg::load (this=this@entry=0x2e82ff8, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x2e82e20, rawDecodingSettings=...) at /tmp/tmp.CebnuTHvLs/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #16 0x00007ffff78a3090 in Digikam::DImg::load (this=this@entry=0x2e82ff8, filePath=..., observer=observer@entry=0x2e82e20, rawDecodingSettings=...) at /tmp/tmp.CebnuTHvLs/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #17 0x00007ffff7967600 in Digikam::PreviewLoadingTask::execute (this=0x2e82e10) at /tmp/tmp.CebnuTHvLs/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #18 0x00007ffff797d7fe in Digikam::LoadSaveThread::run (this=0x2bd8070) at /tmp/tmp.CebnuTHvLs/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #19 0x00007ffff799de5e in Digikam::DynamicThread::Private::run (this=0x2b9e240) at /tmp/tmp.CebnuTHvLs/digikam/core/libs/threads/dynamicthread.cpp:183
> #20 0x00007ffff4b1b7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #21 0x00007ffff4b17062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #22 0x00007ffff44e2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #23 0x00007ffff441822f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Created attachment 124143 [details]
digikam resource usage with 105ccfd47cc8ac73037eac0d6873581a9e49c1aa

It's clear. your last backtrace as nothing to do with face management.

The crash come from Exiv2 shared library when an ICC color profile is taken from a JPEG image to render an image.

1/ did you have the Color Management enabled on your digiKam settings ? If yes, please turn off and try again.

2/ Which Exiv2 version do you use exactly ? I hope that it's the last stable 0.27.2. If now, please update this component and recompile digiKam. It's important, as Exiv2 is a critical component for digiKam, it's used everywhere.

3/ Can you process the facemanagement step by step to parse small parts of the collection. The idea is to try to localize the JPEG file which crash Exiv2.

Best

Gilles Caulier

I was using exiv2 0.27.1. I've updated to 0.27.2, disabled color management (which was enabled in the other runs, since every run starts from a clean digikam with default configuration). With 105ccfd47cc8ac73037eac0d6873581a9e49c1aa:

> digikam.general: Found 0 faces in "IMG_20180620_172836326_HDR.jpg" QSize(4160, 3120) QSize(4160, 3120)
> digikam.general: Removing old entries ()
> digikam.general: Cancel Main Thread
> [Thread 0x7fff1affd700 (LWP 23850) exited]
> digikam.general: One job is done
> digikam.general: Cancel Main Thread
> [Thread 0x7fff1a7fc700 (LWP 23851) exited]
> digikam.general: Check for finish:  10 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.PyUwvaBSNi/Pictures/p/something/IMG_20180621_105141093.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.PyUwvaBSNi/Pictures/p/something/IMG_20180621_105141093.jpg" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.facesengine: Good rect =  QRect(248,151 32x42) , conf =  0.81234
> digikam.general: Found 1 faces in "IMG_20180620_215959940_HDR.jpg" QSize(4160, 3120) QSize(4160, 3120)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "IMG_20180621_100509050.jpg" QSize(3120, 4160) QSize(3120, 4160)
> digikam.general: Removing old entries ()
> 
> Thread 1137 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff5cf59700 (LWP 4434)]
> Digikam::DImg::rotate (this=0x256ef48, angle=angle@entry=Digikam::DImg::ROT90)
>     at /tmp/tmp.PyUwvaBSNi/digikam/core/libs/dimg/dimg_transform.cpp:119
> 119                             *to = *from++;
> (gdb) bt
> #0  Digikam::DImg::rotate (this=0x256ef48, angle=angle@entry=Digikam::DImg::ROT90) at /tmp/tmp.PyUwvaBSNi/digikam/core/libs/dimg/dimg_transform.cpp:119
> #1  0x00007ffff78b4997 in Digikam::DImg::rotateAndFlip (this=this@entry=0x256ef48, orientation=<optimized out>) at /tmp/tmp.PyUwvaBSNi/digikam/core/libs/dimg/dimg_transform.cpp:450
> #2  0x00007ffff78b4b8e in Digikam::DImg::exifRotate (this=this@entry=0x256ef48, filePath=...) at /tmp/tmp.PyUwvaBSNi/digikam/core/libs/dimg/dimg_transform.cpp:562
> #3  0x00007ffff7967c74 in Digikam::PreviewLoadingTask::execute (this=0x256ed60) at /tmp/tmp.PyUwvaBSNi/digikam/core/libs/threadimageio/preview/previewtask.cpp:296
> #4  0x00007ffff797d84e in Digikam::LoadSaveThread::run (this=0x2525630) at /tmp/tmp.PyUwvaBSNi/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #5  0x00007ffff799deae in Digikam::DynamicThread::Private::run (this=0x2c4b4f0) at /tmp/tmp.PyUwvaBSNi/digikam/core/libs/threads/dynamicthread.cpp:183
> #6  0x00007ffff4b1b7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #7  0x00007ffff4b17062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007ffff44e2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #9  0x00007ffff441822f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

I don't think isolating a single image is going to work, since there is no failure that is deterministically triggered by a single image. Sometimes the scan succeeds without crash.

The cause here might be that we did not get the requested memory as a block. Hmm, yes we have to check if we got the memory, but what if not? Just do not rotate? On the other hand, we have to look at where we lose memory somewhere, the memory consumption seems to me a bit too high.

Maik

Created attachment 124172 [details]
here's one additional trace taken during the last run

However, I see no error message that the memory could not be requested. Another possibility is that the shared image data is changed by another thread at the same time.

Maik

Git commit 62ab5314c3b32a83c0b765b67c251ee2bd810b37 by Maik Qualmann.
Committed on 28/11/2019 at 21:53.
Pushed by mqualmann into branch 'master'.

add debug, we need the loading task status

M  +2    -0    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/62ab5314c3b32a83c0b765b67c251ee2bd810b37

Not sure how useful that debug information is, loading task status seems to always be 0. With e69b8317e24ee6487b6a79a75052aab6e7024f43:


> digikam.general: Loading task status: 0
> digikam.general: Try to get preview from "/tmp/tmp.jRPmEBcJjg/Pictures/p/something/IMG_7056.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.jRPmEBcJjg/Pictures/p/something/IMG_7056.JPG" : "JPEG" file identified
> digikam.general: Using  4  CPU core to run threads
> [New Thread 0x7fff5d3ff700 (LWP 28951)]
> digikam.general: Action Thread run  1  new jobs
> [New Thread 0x7fff23fff700 (LWP 28952)]
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images        INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (  (
> ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );"
>  (QVariant(int, 4), QVariant(int, 4), QVariant(QString, "autodetectedPerson"), QVariant(int, 4), QVariant(int, 4), QVariant(QString, "autodetectedFace"), QVariant(int, 4), QVariant(int, 4), QVariant(QString, "tagRegion"))
> digikam.database: Search result: 2618
> digikam.general: Trying to get thumbnail with Exiv2 for "/tmp/tmp.jRPmEBcJjg/Pictures/p/Android/Camera/IMG_20140731_200523.jpg"
> digikam.general: Trying to get thumbnail with DImg preview for "/tmp/tmp.jRPmEBcJjg/Pictures/p/Android/Camera/IMG_20140731_200523.jpg"
> digikam.dimg: "/tmp/tmp.jRPmEBcJjg/Pictures/p/Android/Camera/IMG_20140731_200523.jpg" : "JPEG" file identified
> digikam.general: Cancel Main Thread
> [Thread 0x7fff5d3ff700 (LWP 28951) exited]
> digikam.general: One job is done
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Cancel Main Thread
> [Thread 0x7fff23fff700 (LWP 28952) exited]
> 
> Thread 79 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3effd700 (LWP 20118)]
> 0x00007ffff4b9e72a in operator==(QString const&, QString const&) ()
>    from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007ffff4b9e72a in operator==(QString const&, QString const&) () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #1  0x00007ffff797e275 in Digikam::LoadingDescription::equalsIgnoreReducedVersion (this=<optimized out>, other=...) at /tmp/tmp.jRPmEBcJjg/digikam/core/libs/threadimageio/fileio/loadingdescription.cpp:313
> #2  0x00007ffff798868b in Digikam::SharedLoadingTask::notifyNewLoadingProcess (this=0x4d316a0, process=<optimized out>, description=...)
>     at /tmp/tmp.jRPmEBcJjg/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:434
> #3  0x00007ffff797f6e0 in Digikam::LoadingCache::notifyNewLoadingProcess (this=this@entry=0x5ae960, process=process@entry=0x323d7d0, description=...)
>     at /tmp/tmp.jRPmEBcJjg/digikam/core/libs/threadimageio/fileio/loadingcache.cpp:240
> #4  0x00007ffff79668b5 in Digikam::PreviewLoadingTask::execute (this=0x323d610) at /tmp/tmp.jRPmEBcJjg/digikam/core/libs/threadimageio/preview/previewtask.cpp:146
> #5  0x00007ffff797c93e in Digikam::LoadSaveThread::run (this=0x2cd24c0) at /tmp/tmp.jRPmEBcJjg/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff799cf9e in Digikam::DynamicThread::Private::run (this=0x3373e90) at /tmp/tmp.jRPmEBcJjg/digikam/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007ffff4b1a7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007ffff4b16062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff44e1ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #10 0x00007ffff441722f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 330d3cea6749f1a50783bd0acd6ac57b95cf0519 by Maik Qualmann.
Committed on 30/11/2019 at 21:52.
Pushed by mqualmann into branch 'master'.

polish set loading status

M  +7    -3    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +0    -26   core/libs/threadimageio/preview/previewtask.cpp
M  +1    -2    core/libs/threadimageio/preview/previewtask.h

https://invent.kde.org/kde/digikam/commit/330d3cea6749f1a50783bd0acd6ac57b95cf0519

With 330d3cea6749f1a50783bd0acd6ac57b95cf0519:

> digikam.dimg: "/tmp/tmp.CLpH7QxTNQ/Pictures/p/Android/Whatsapp/IMG-20140822-WA0033.jpg" : "JPEG" file identified
> digikam.general: Found 0 faces in "IMG-20140809-WA0001.jpg" QSize(1280, 721) QSize(1280, 721)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.CLpH7QxTNQ/Pictures/p/Android/Whatsapp/IMG-20140822-WA0034.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.CLpH7QxTNQ/Pictures/p/Android/Whatsapp/IMG-20140822-WA0034.jpg" : "JPEG" file identified
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Removing old entries ()
> digikam.general: Try to get preview from "/tmp/tmp.CLpH7QxTNQ/Pictures/p/Android/Whatsapp/IMG-20140822-WA0035.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.CLpH7QxTNQ/Pictures/p/Android/Whatsapp/IMG-20140822-WA0035.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.CLpH7QxTNQ/Pictures/p/Android/Whatsapp/IMG-20140822-WA0036.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.CLpH7QxTNQ/Pictures/p/Android/Whatsapp/IMG-20140822-WA0036.jpg" : "JPEG" file identified
> digikam.general: Found 0 faces in "IMG-20140809-WA0002.jpg" QSize(1280, 720) QSize(1280, 720)
> digikam.general: Removing old entries ()
> digikam.facesengine: Good rect =  QRect(95,71 92x119) , conf =  0.998028
> digikam.general: Found 1 faces in "IMG-20140809-WA0003.jpg" QSize(500, 609) QSize(500, 609)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "IMG-20140809-WA0004.jpg" QSize(1280, 721) QSize(1280, 721)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "IMG-20140809-WA0005.jpg" QSize(550, 555) QSize(550, 555)
> digikam.general: Removing old entries ()
> 
> Thread 33 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff373ff700 (LWP 7826)]
> 0x00007ffff43a252c in malloc_consolidate () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb) bt
> #0  0x00007ffff43a252c in malloc_consolidate () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> #1  0x00007ffff43a3448 in _int_free () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> #2  0x00007ffff789c0a6 in Digikam::DImg::Private::~Private (this=0x7fff3c064580, __in_chrg=<optimized out>) at /tmp/tmp.CLpH7QxTNQ/digikam/core/libs/dimg/dimg_p.h:140
> #3  QExplicitlySharedDataPointer<Digikam::DImg::Private>::~QExplicitlySharedDataPointer (this=<optimized out>, __in_chrg=<optimized out>) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qshareddata.h:182
> #4  Digikam::DImg::~DImg (this=<optimized out>, __in_chrg=<optimized out>) at /tmp/tmp.CLpH7QxTNQ/digikam/core/libs/dimg/dimg.cpp:188
> #5  0x00007ffff7982d87 in QCache<QString, Digikam::DImg>::unlink (n=..., this=0x569f20) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qcache.h:69
> #6  QCache<QString, Digikam::DImg>::trim (m=204869632, this=0x569f20) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qcache.h:193
> #7  QCache<QString, Digikam::DImg>::insert (acost=4845568, aobject=0x7fff2809c950, akey=..., this=0x569f20) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qcache.h:173
> #8  Digikam::LoadingCache::putImage (this=this@entry=0x5cced0, cacheKey=..., img=..., filePath=...) at /tmp/tmp.CLpH7QxTNQ/digikam/core/libs/threadimageio/fileio/loadingcache.cpp:193
> #9  0x00007ffff79675df in Digikam::PreviewLoadingTask::execute (this=0x25fe5a0) at /tmp/tmp.CLpH7QxTNQ/digikam/core/libs/threadimageio/preview/previewtask.cpp:305
> #10 0x00007ffff797d77e in Digikam::LoadSaveThread::run (this=0x7fff3c1c9160) at /tmp/tmp.CLpH7QxTNQ/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #11 0x00007ffff799ddce in Digikam::DynamicThread::Private::run (this=0x3370680) at /tmp/tmp.CLpH7QxTNQ/digikam/core/libs/threads/dynamicthread.cpp:183
> #12 0x00007ffff4b1b7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #13 0x00007ffff4b17062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #14 0x00007ffff44e2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #15 0x00007ffff441822f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit aacced0438943b163ffa95a3e1d82d74e4d065ae by Maik Qualmann.
Committed on 01/12/2019 at 13:00.
Pushed by mqualmann into branch 'master'.

return new DImg from QCache

M  +4    -2    core/libs/threadimageio/fileio/loadingcache.cpp

https://invent.kde.org/kde/digikam/commit/aacced0438943b163ffa95a3e1d82d74e4d065ae

With aacced0438943b163ffa95a3e1d82d74e4d065ae:

> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.HClqSMevcG/Pictures/p/Syncthing-3TB/Telegram-Android/Telegram Images/238228297_1801.jpg" : "JPEG" file identified
> digikam.general: Found 0 faces in "238224963_303731.jpg" QSize(720, 1280) QSize(720, 1280)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.HClqSMevcG/Pictures/p/Syncthing-3TB/Telegram-Android/Telegram Images/238228345_51256.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.HClqSMevcG/Pictures/p/Syncthing-3TB/Telegram-Android/Telegram Images/238228345_51256.jpg" : "JPEG" file identified
> digikam.general: Found 0 faces in "238224970_75850.jpg" QSize(949, 1280) QSize(949, 1280)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "238224982_164876.jpg" QSize(1280, 949) QSize(1280, 949)
> digikam.general: Found 0 faces in "238225065_102724.jpg" QSize(589, 429) QSize(589, 429)
> digikam.general: Found 0 faces in "238225204_14626.jpg" QSize(720, 1280) QSize(720, 1280)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.HClqSMevcG/Pictures/p/Syncthing-3TB/Telegram-Android/Telegram Images/238228411_98872.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.HClqSMevcG/Pictures/p/Syncthing-3TB/Telegram-Android/Telegram Images/238228411_98872.jpg" : "JPEG" file identified
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.HClqSMevcG/Pictures/p/Syncthing-3TB/Telegram-Android/Telegram Images/238228432_91266.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.HClqSMevcG/Pictures/p/Syncthing-3TB/Telegram-Android/Telegram Images/238228432_91266.jpg" : "JPEG" file identified
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.HClqSMevcG/Pictures/p/Syncthing-3TB/Telegram-Android/Telegram Images/238228438_206539.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.HClqSMevcG/Pictures/p/Syncthing-3TB/Telegram-Android/Telegram Images/238228438_206539.jpg" : "JPEG" file identified
> digikam.facesengine: Good rect =  QRect(76,85 71x104) , conf =  0.999438
> digikam.general: Found 1 faces in "238225234_157978.jpg" QSize(1280, 959) QSize(1280, 959)
> digikam.general: Removing old entries ()
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Found 0 faces in "238225255_170020.jpg" QSize(1280, 949) QSize(1280, 949)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "238225346_22680.jpg" QSize(949, 1280) QSize(949, 1280)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "238225406_274036.jpg" QSize(1280, 720) QSize(1280, 720)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "238225432_9767.jpg" QSize(960, 894) QSize(960, 894)
> digikam.general: Removing old entries ()
> 
> Thread 52 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff2af58700 (LWP 21794)]
> Digikam::PreviewLoadingTask::execute (this=0x31e1640)
>     at /tmp/tmp.HClqSMevcG/digikam/core/libs/threadimageio/preview/previewtask.cpp:325
> 325                         if (l->accessMode() == LoadSaveThread::AccessModeReadWrite)
> (gdb) bt
> #0  Digikam::PreviewLoadingTask::execute (this=0x31e1640) at /tmp/tmp.HClqSMevcG/digikam/core/libs/threadimageio/preview/previewtask.cpp:325
> #1  0x00007ffff797d77e in Digikam::LoadSaveThread::run (this=0x5525c00) at /tmp/tmp.HClqSMevcG/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #2  0x00007ffff799de0e in Digikam::DynamicThread::Private::run (this=0x3acb730) at /tmp/tmp.HClqSMevcG/digikam/core/libs/threads/dynamicthread.cpp:183
> #3  0x00007ffff4b1b7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #4  0x00007ffff4b17062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #5  0x00007ffff44e2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #6  0x00007ffff441822f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit a0774df21a8f039e3355c8ad3ffa0ca20924dbbf by Maik Qualmann.
Committed on 01/12/2019 at 17:11.
Pushed by mqualmann into branch 'master'.

latest DImg change has a memory leak

M  +1    -3    core/libs/threadimageio/fileio/loadingcache.cpp
M  +1    -1    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +1    -1    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/kde/digikam/commit/a0774df21a8f039e3355c8ad3ffa0ca20924dbbf

With 43ef007a130c058982aaf61a900a0c95d99fd748, crashed on the 4th run after 3 successful ones:

> digikam.general: Cancel Main Thread
> [Thread 0x7fff22ffd700 (LWP 17652) exited]
> digikam.general: One job is done
> digikam.general: Cancel Main Thread
> [Thread 0x7fff23fff700 (LWP 17653) exited]
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.G5Y5JzCHGg/Pictures/p/Private Bilder/some-folder/sth-086.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.G5Y5JzCHGg/Pictures/p/Private Bilder/some-folder/sth-086.JPG" : "JPEG" file identified
> digikam.general: Found 0 faces in "sth-042.JPG" QSize(2448, 3264) QSize(2448, 3264)
> digikam.general: Removing old entries ()
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.G5Y5JzCHGg/Pictures/p/Private Bilder/some-folder/sth-087.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.G5Y5JzCHGg/Pictures/p/Private Bilder/some-folder/sth-087.JPG" : "JPEG" file identified
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Try to get preview from "/tmp/tmp.G5Y5JzCHGg/Pictures/p/Private Bilder/some-folder/sth-088.JPG"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.G5Y5JzCHGg/Pictures/p/Private Bilder/some-folder/sth-088.JPG" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Found 0 faces in "sth-043.JPG" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Found 0 faces in "sth-044.JPG" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "sth-045.JPG" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> digikam.general: Found 0 faces in "sth-046.JPG" QSize(3264, 2448) QSize(3264, 2448)
> digikam.general: Removing old entries ()
> 
> Thread 47 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3cf54700 (LWP 4085)]
> 0x0000000000000000 in ?? ()
> (gdb) bt
> #0  0x0000000000000000 in ?? ()
> #1  0x00007ffff7967a6d in Digikam::PreviewLoadingTask::execute (this=0x41f7db0) at /tmp/tmp.G5Y5JzCHGg/digikam/core/libs/threadimageio/preview/previewtask.cpp:338
> #2  0x00007ffff797d79e in Digikam::LoadSaveThread::run (this=0x7fff40072190) at /tmp/tmp.G5Y5JzCHGg/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #3  0x00007ffff799de0e in Digikam::DynamicThread::Private::run (this=0x3847210) at /tmp/tmp.G5Y5JzCHGg/digikam/core/libs/threads/dynamicthread.cpp:183
> #4  0x00007ffff4b1b7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #5  0x00007ffff4b17062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #6  0x00007ffff44e2ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #7  0x00007ffff441822f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 58e483a0a3bddc9cb339ab4b5548623a2d18bc3d by Maik Qualmann.
Committed on 05/12/2019 at 22:29.
Pushed by mqualmann into branch 'master'.

small changes on the preview task

M  +44   -52   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +1    -2    core/libs/threadimageio/fileio/loadsavetask.h
M  +34   -42   core/libs/threadimageio/preview/previewtask.cpp
M  +32   -37   core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/kde/digikam/commit/58e483a0a3bddc9cb339ab4b5548623a2d18bc3d

With 8dd3f5099466dede99fd300e55db84fbff6dbad0 (from 2019-12-07):

> digikam.general: Try to get preview from "/tmp/tmp.848xADW9lH/Pictures/p/some-path/DSCN1115.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.848xADW9lH/Pictures/p/some-path/DSCN1115.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.facesengine: Good rect =  QRect(108,73 65x90) , conf =  0.999973
> digikam.facesengine: Good rect =  QRect(172,106 56x77) , conf =  0.997096
> digikam.general: Found 2 faces in "DSCN0912.jpg" QSize(4320, 3240) QSize(4320, 3240)
> digikam.general: Removing old entries ()
> digikam.general: Entry QRect(1490,556 1215x1215)227overlapsQRect(1555,518 936x1296)6, skipping
> digikam.general: Entry QRect(2241,1031 1004x1005)99overlapsQRect(2477,994 806x1108)6, skipping
> digikam.dimg:  : return null image! ( false ,  0 ,  0 )
> digikam.dimg:  : return null image! ( false ,  0 ,  0 )
> digikam.general: Try to get preview from "/tmp/tmp.848xADW9lH/Pictures/p/some-path/DSCN1116.jpg"
> digikam.general: Preview quality:  1
> digikam.dimg: "/tmp/tmp.848xADW9lH/Pictures/p/some-path/DSCN1116.jpg" : "JPEG" file identified
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.facesengine: Good rect =  QRect(94,100 56x76) , conf =  0.995798
> digikam.facesengine: Good rect =  QRect(148,90 62x74) , conf =  0.950529
> digikam.general: Found 2 faces in "DSCN0913.jpg" QSize(4320, 3240) QSize(4320, 3240)
> digikam.general: Removing old entries ()
> digikam.general: Using  4  CPU core to run threads
> digikam.facesengine: Good rect =  QRect(98,85 62x87) , conf =  0.999666
> digikam.facesengine: Good rect =  QRect(23,90 73x88) , conf =  0.998955
> digikam.facesengine: Good rect =  QRect(179,85 73x66) , conf =  0.866344
> digikam.general: Found 3 faces in "DSCN0923.jpg" QSize(4320, 3240) QSize(4320, 3240)
> digikam.general: Removing old entries ()
> digikam.facesengine: Good rect =  QRect(176,137 39x50) , conf =  0.978829
> digikam.facesengine: Good rect =  QRect(125,72 31x49) , conf =  0.721479
> digikam.general: Found 2 faces in "DSCN0945.jpg" QSize(4320, 3240) QSize(4320, 3240)
> digikam.general: Removing old entries ()
> [New Thread 0x7fff237fe700 (LWP 21359)]
> 
> Thread 42 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff5cbfe700 (LWP 17198)]
> 0x00007ffff7989737 in Digikam::LoadingTask::progressInfo (this=0x3968070, img=0x3968258, progress=0.798271596)
>     at /tmp/tmp.848xADW9lH/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> 65              if (m_thread && m_thread->querySendNotifyEvent())
> (gdb) bt
> #0  0x00007ffff7989737 in Digikam::LoadingTask::progressInfo (this=0x3968070, img=0x3968258, progress=0.798271596) at /tmp/tmp.848xADW9lH/digikam/core/libs/threadimageio/fileio/loadsavetask.cpp:65
> #1  0x00007fffb2099a40 in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x7fff24260050, filePath=..., observer=0x3968080) at /tmp/tmp.848xADW9lH/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:434
> #2  0x00007ffff78a2dcc in Digikam::DImg::load (this=this@entry=0x3968258, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x3968080, rawDecodingSettings=...) at /tmp/tmp.848xADW9lH/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #3  0x00007ffff78a4060 in Digikam::DImg::load (this=this@entry=0x3968258, filePath=..., observer=observer@entry=0x3968080, rawDecodingSettings=...) at /tmp/tmp.848xADW9lH/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #4  0x00007ffff7967fcf in Digikam::PreviewLoadingTask::execute (this=0x3968070) at /tmp/tmp.848xADW9lH/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #5  0x00007ffff797dcce in Digikam::LoadSaveThread::run (this=0x25b18f0) at /tmp/tmp.848xADW9lH/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #6  0x00007ffff799e37e in Digikam::DynamicThread::Private::run (this=0x2974e10) at /tmp/tmp.848xADW9lH/digikam/core/libs/threads/dynamicthread.cpp:183
> #7  0x00007ffff4b1d7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #8  0x00007ffff4b19062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #9  0x00007ffff44e4ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #10 0x00007ffff441a22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 8229ef53b49b0c62244a96cb8f7d06a7452b4ce3 by Maik Qualmann.
Committed on 09/12/2019 at 21:59.
Pushed by mqualmann into branch 'master'.

various changes to the loading task classes

M  +7    -7    core/dplugins/dimg/heif/dimgheifloader_load.cpp
M  +2    -2    core/dplugins/dimg/heif/dimgheifloader_save.cpp
M  +7    -7    core/dplugins/dimg/imagemagick/dimgimagemagickloader.cpp
M  +9    -9    core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp
M  +6    -6    core/dplugins/dimg/jpeg/dimgjpegloader_save.cpp
M  +4    -4    core/dplugins/dimg/jpeg2000/dimgjpeg2000loader_load.cpp
M  +4    -4    core/dplugins/dimg/jpeg2000/dimgjpeg2000loader_save.cpp
M  +2    -2    core/dplugins/dimg/pgf/dimgpgfloader.cpp
M  +1    -1    core/dplugins/dimg/pgf/dimgpgfloader_load.cpp
M  +1    -1    core/dplugins/dimg/pgf/dimgpgfloader_save.cpp
M  +6    -6    core/dplugins/dimg/png/dimgpngloader_load.cpp
M  +3    -3    core/dplugins/dimg/png/dimgpngloader_save.cpp
M  +4    -4    core/dplugins/dimg/qimage/dimgqimageloader.cpp
M  +6    -6    core/dplugins/dimg/raw/dimgrawloader.cpp
M  +9    -9    core/dplugins/dimg/tiff/dimgtiffloader_load.cpp
M  +4    -4    core/dplugins/dimg/tiff/dimgtiffloader_save.cpp
M  +1    -2    core/dplugins/generic/tools/expoblending/manager/expoblendingthread.cpp
M  +2    -3    core/dplugins/generic/tools/panorama/tasks/preprocesstask.cpp
M  +5    -5    core/libs/dimg/dimg_fileio.cpp
M  +3    -3    core/libs/dimg/filters/icc/icctransform.cpp
M  +1    -2    core/libs/dimg/filters/icc/icctransformfilter.cpp
M  +2    -2    core/libs/dimg/filters/icc/icctransformfilter.h
M  +2    -2    core/libs/dimg/filters/rawprocessingfilter.cpp
M  +2    -4    core/libs/dimg/loaders/dimgloaderobserver.h
M  +1    -1    core/libs/threadimageio/fileio/loadingcache.h
M  +11   -22   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +28   -28   core/libs/threadimageio/fileio/loadsavetask.h
M  +8    -8    core/libs/threadimageio/preview/previewtask.cpp
M  +3    -3    core/libs/threadimageio/thumb/thumbnailtask.cpp
M  +1    -2    core/utilities/queuemanager/manager/batchtool.cpp

https://invent.kde.org/kde/digikam/commit/8229ef53b49b0c62244a96cb8f7d06a7452b4ce3

Again after 3 successful runs, with 8229ef53b49b0c62244a96cb8f7d06a7452b4ce3:

> digikam.dimg: "/tmp/tmp.x7EaxS04Zv/Pictures/p/Kamera-Uploads/2014-03-14 12.09.33-2.jpg" : "JPEG" file identified
> digikam.general: Using  4  CPU core to run threads
> [New Thread 0x7fff237fe700 (LWP 13164)]
> digikam.general: Action Thread run  1  new jobs
> [New Thread 0x7fff22ffd700 (LWP 13165)]
> digikam.general: Check for finish:  51 packages, 0 infos to filter, hasFinished() false
> digikam.general: Try to get preview from "/tmp/tmp.x7EaxS04Zv/Pictures/p/Kamera-Uploads/2014-03-14 12.21.56.jpg"
> digikam.general: Preview quality:  1
> digikam.database: " ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) ) "
> digikam.database: Search query:
>  "SELECT DISTINCT Images.id, Images.name, Images.album,        Albums.albumRoot,        ImageInformation.rating, Images.category,        ImageInformation.format, ImageInformation.creationDate,        Images.modificationDate, Images.fileSize,        ImageInformation.width,  ImageInformation.height,        ImageTagProperties.value, ImageTagProperties.property, ImageTagProperties.tagid  FROM Images        INNER JOIN ImageTagProperties ON ImageTagProperties.imageid=Images.id        LEFT JOIN ImageInformation ON Images.id=ImageInformation.imageid        INNER JOIN Albums           ON Albums.id=Images.album WHERE Images.status=1 AND (
> ( ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) OR ( (ImageTagProperties.tagid=? OR ImageTagProperties.tagid IN (SELECT id FROM TagsTree WHERE pid=?)) AND ImageTagProperties.property=?  ) )  );"
>  (QVariant(int, 6), QVariant(int, 6), QVariant(QString, "autodetectedPerson"), QVariant(int, 6), QVariant(int, 6), QVariant(QString, "autodetectedFace"), QVariant(int, 6), QVariant(int, 6), QVariant(QString, "tagRegion"))
> digikam.dimg: "/tmp/tmp.x7EaxS04Zv/Pictures/p/Kamera-Uploads/2014-03-14 12.21.56.jpg" : "JPEG" file identified
> digikam.database: Search result: 3849
> Premature end of JPEG file
> digikam.general: Cancel Main Thread
> digikam.general: One job is done
> digikam.general: Cancel Main Thread
> digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.
> digikam.facesengine: Good rect =  QRect(139,98 41x54) , conf =  0.942572
> digikam.general: Found 1 faces in "2013-12-21 21.29.17-1.jpg" QSize(2448, 3264) QSize(2448, 3264)
> digikam.general: Removing old entries (FaceTagsIface(1, image 4610, tag 6, regionQRect(1110,1066 446x588))
> [Thread 0x7fff237fe700 (LWP 13164) exited]
> [Thread 0x7fff22ffd700 (LWP 13165) exited]
> 
> Thread 20650 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fff3cff9700 (LWP 773)]
> 0x00007ffff4ba17e0 in operator<(QString const&, QString const&) () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> (gdb) bt
> #0  0x00007ffff4ba17e0 in operator<(QString const&, QString const&) () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #1  0x00007ffff78abac9 in qMapLessThanKey<QString> (key2=..., key1=...) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:71
> #2  QMap<QString, QVariant>::insert (avalue=..., akey=..., this=<optimized out>) at /nix/store/hy0z060pmjwrdrbb0qrii7hnwif8cz1y-qtbase-5.12.0-dev/include/QtCore/qmap.h:713
> #3  Digikam::DImg::setAttribute (this=<optimized out>, key=..., value=...) at /tmp/tmp.x7EaxS04Zv/digikam/core/libs/dimg/dimg_props.cpp:207
> #4  0x00007ffff78a6cd6 in Digikam::DImg::getUniqueHashV2 (this=0x2ad5a18) at /tmp/tmp.x7EaxS04Zv/digikam/core/libs/dimg/dimg_metadata.cpp:93
> #5  0x00007ffff78a849d in Digikam::DImg::createHistoryImageId (this=<optimized out>, filePath=..., type=type@entry=Digikam::HistoryImageId::Current) at /tmp/tmp.x7EaxS04Zv/digikam/core/libs/dimg/dimg_metadata.cpp:312
> #6  0x00007ffff7768995 in Digikam::DImgLoader::readMetadata (this=0x7fff0c057a80, filePath=...) at /tmp/tmp.x7EaxS04Zv/digikam/core/libs/dimg/loaders/dimgloader.cpp:223
> #7  0x00007fffb209843c in DigikamJPEGDImgPlugin::DImgJPEGLoader::load (this=0x7fff0c057a80, filePath=..., observer=0x2ad5840) at /tmp/tmp.x7EaxS04Zv/digikam/core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp:56
> #8  0x00007ffff78a2d6f in Digikam::DImg::load (this=this@entry=0x2ad5a18, filePath=..., loadFlagsInt=loadFlagsInt@entry=63, observer=observer@entry=0x2ad5840, rawDecodingSettings=...) at /tmp/tmp.x7EaxS04Zv/digikam/core/libs/dimg/dimg_fileio.cpp:138
> #9  0x00007ffff78a4010 in Digikam::DImg::load (this=this@entry=0x2ad5a18, filePath=..., observer=observer@entry=0x2ad5840, rawDecodingSettings=...) at /tmp/tmp.x7EaxS04Zv/digikam/core/libs/dimg/dimg_fileio.cpp:63
> #10 0x00007ffff7967f5b in Digikam::PreviewLoadingTask::execute (this=0x2ad5830) at /tmp/tmp.x7EaxS04Zv/digikam/core/libs/threadimageio/preview/previewtask.cpp:286
> #11 0x00007ffff797dc3e in Digikam::LoadSaveThread::run (this=0x38f4ea0) at /tmp/tmp.x7EaxS04Zv/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:133
> #12 0x00007ffff799e2ee in Digikam::DynamicThread::Private::run (this=0x34a3680) at /tmp/tmp.x7EaxS04Zv/digikam/core/libs/threads/dynamicthread.cpp:183
> #13 0x00007ffff4b1d7d2 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #14 0x00007ffff4b19062 in ?? () from /nix/store/683732s7dq6bxrl267lcadl5lvn6i4nh-qtbase-5.12.0/lib/libQt5Core.so.5
> #15 0x00007ffff44e4ef7 in start_thread () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libpthread.so.0
> #16 0x00007ffff441a22f in clone () from /nix/store/iykxb0bmfjmi7s53kfg6pjbfpd8jmza6-glibc-2.27/lib/libc.so.6
> (gdb)

Git commit 95594f2438a4f6f80340e70b20964e3685b4e235 by Maik Qualmann.
Committed on 11/12/2019 at 11:41.
Pushed by mqualmann into branch 'master'.

try with a QMutex for DImg data

M  +4    -0    core/libs/dimg/dimg_data.cpp
M  +10   -10   core/libs/dimg/dimg_metadata.cpp
M  +2    -0    core/libs/dimg/dimg_p.h
M  +31   -11   core/libs/dimg/dimg_props.cpp
M  +5    -3    core/libs/dimg/dimg_transform.cpp
M  +2    -0    core/libs/dimg/loaders/dimgloader.cpp

https://invent.kde.org/kde/digikam/commit/95594f2438a4f6f80340e70b20964e3685b4e235

*** Bug 415046 has been marked as a duplicate of this bug. ***

I think that just might have fixed it. I just completed the 10th (!) successful scan with 95594f2438a4f6f80340e70b20964e3685b4e235. I don't think I ever had more than 3 scans without a crash before.

Since testing is quite time consuming (~2h per run) and the crash isn't entirely reliably reproducible, its difficult to be entirely certain. But I think it is fixed.

Timokau, thank you very much for testing these many patches. We have made digiKam a lot more stable and can now close some bug reports.

Maik

My pleasure! Thank you for staying on this for so long Maik. I'm looking forward to digikam 7.0 :)

Congratulations Maik for your effort. 

So we can plan a first official 7.0.0-beta1 release before Christmas. 

Let's me hear the best date to plan...

Gilles

Hi Gilles,

I think if the Beta1 appears shortly before or on the 24th, that would be a good date ((:-))

Maik

Ok, i will plan the release next week to be ready for next week end.

Gilles

timokau,

First i wish you an happy new year 2020.

I would to ask you if it's possible to check the current implementation of digiKam with the Similarities feature (Find duplicates).
I perform currently a huge bug triage, and i see a lots of reports about memory leak which crash digiKam while populating the database with Haar fingerprints with huge collection.

I tried to reproduce the problem here without success (ouff: thanks to huge Maik hack...). But as we know that your system is very sensible to this kind of dysfunction, this can be a good double check.  

Thanks in advance

Gilles Caulier

Sorry for the slow reply. A happy new year to you too :)

Unfortunately I don't currently have access to the computer I usually used to reproduce this issue. I'm not sure when exactly I'll have access again. I'll reproduce it then if its still relevant, but it might take some time.