Bug 427333 - Face detection crashes due to pure virtual method call in LoadingCache
Summary: Face detection crashes due to pure virtual method call in LoadingCache
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Faces-Detection (show other bugs)
Version: 7.2.0
Platform: Manjaro Linux
: NOR crash
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-10-04 18:21 UTC by Simon Westersund
Modified: 2021-01-19 17:24 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 7.2.0
Sentry Crash Report:


Attachments
GDB backtrace with latest master 1b1b3f044c (30.84 KB, text/plain)
2020-10-04 18:21 UTC, Simon Westersund
Details
bootstrap.local output of CMake configuration (15.96 KB, text/plain)
2020-10-04 18:22 UTC, Simon Westersund
Details
GDB backtrace with latest master b7d3d74918 (41.74 KB, text/plain)
2020-10-06 19:59 UTC, Simon Westersund
Details
GDB backtrace with latest master b12004d560 (30.68 KB, text/plain)
2020-10-07 20:51 UTC, Simon Westersund
Details
GDB backtrace with latest master 793a18d716 (32.08 KB, text/plain)
2020-10-08 16:58 UTC, Simon Westersund
Details
GDB backtrace with latest master 793a18d716 after deleting DImg_ImageMagick_Plugin.so (29.91 KB, text/plain)
2020-10-08 17:01 UTC, Simon Westersund
Details
GDB backtrace with latest master 182bef3be8 (27.96 KB, text/plain)
2020-10-09 08:55 UTC, Simon Westersund
Details
GDB backtrace with latest master 66f96e26b5 (34.93 KB, text/plain)
2020-10-09 20:20 UTC, Simon Westersund
Details
GDB backtrace with latest master 925e074a71 (38.69 KB, text/plain)
2020-10-10 10:07 UTC, Simon Westersund
Details
GDB backtrace with latest master 3d387983fe (36.03 KB, text/plain)
2020-11-11 21:25 UTC, Simon Westersund
Details
Debug logs + GDB backtrace from 3d387983fe (36.58 KB, text/plain)
2020-11-11 21:58 UTC, Simon Westersund
Details
Debug logs from stuck face detection in 8f1b5e900f (23.94 KB, text/plain)
2020-11-15 20:29 UTC, Simon Westersund
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Westersund 2020-10-04 18:21:03 UTC
Created attachment 132116 [details]
GDB backtrace with latest master 1b1b3f044c

SUMMARY
Face detection crashes due to pure virtual method call in LoadingCache.

STEPS TO REPRODUCE
1. Build digikam from master commit 1b1b3f044c (snapshot of "7.2.0 beta").
2. Open digikam with a large collection of images (over 27000 JPEGs).
3. In the face detection settings, choose the "Work on all processor cores" option.
4. For workflow, use "Detect faces" and "Scan again and merge results".
5. Start the face detection by clicking the "Scan collection for faces" button.


OBSERVED RESULT
Digikam crashes.

EXPECTED RESULT
Digikam should not crash. It should detect faces in my collection.

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Linux 5.8.11-1-MANJARO
KDE Plasma Version: 5.19.5
KDE Frameworks Version: 5.74.0
Qt Version: 5.15.1

ADDITIONAL INFORMATION
Similar crashes have been reported by myself in this bug: https://bugs.kde.org/show_bug.cgi?id=425723
It was closed as resolved at the same time as this bug: https://bugs.kde.org/show_bug.cgi?id=426175
See the GDB backtrace attached. I attempted to run without GDB at first, but crashed after around 20 minutes with around 30 % completion. When I ran again with GDB, digikam crashed around 79 % completion.
Comment 1 Simon Westersund 2020-10-04 18:22:18 UTC
Created attachment 132117 [details]
bootstrap.local output of CMake configuration

Attached also my build configuration, if it might help differentiate from the other related bugs, which were resolved.
Comment 2 Maik Qualmann 2020-10-04 19:10:46 UTC
Git commit 6812249e06eb14efd0ad9fd24604f4c59c3e211f by Maik Qualmann.
Committed on 04/10/2020 at 19:09.
Pushed by mqualmann into branch 'master'.

add dynamic cast to SharedLoadingTask

M  +11   -1    core/libs/threadimageio/fileio/loadingcache.cpp

https://invent.kde.org/graphics/digikam/commit/6812249e06eb14efd0ad9fd24604f4c59c3e211f
Comment 3 Simon Westersund 2020-10-04 20:28:58 UTC
I tested again with GDB enabled and now the face scan crashed with a segmentation fault, after it printed the warning that was added. The backtrace was as follows:

[New Thread 0x7fff390e8640 (LWP 92762)]
digikam.general: SharedLoadingTask not found!!!
--Type <RET> for more, q to quit, c to continue without paging--

Thread 70 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffd2dee640 (LWP 92023)]
0x00007ffff42e2493 in ?? () from /usr/lib/libQt5Core.so.5
(gdb) bt
#0  0x00007ffff42e2493 in  () at /usr/lib/libQt5Core.so.5
#1  0x00007ffff6224c3a in QMutexLocker::QMutexLocker(QBasicMutex*) (this=0x7fffd2ded3e0, m=0x55556109c020) at /usr/include/qt/QtCore/qmutex.h:233
#2  0x00007ffff63b0df9 in Digikam::DImg::setAttribute(QString const&, QVariant const&) (this=0x555561860510, key=..., value=...) at /home/simon/Development/kde/digikam/core/libs/dimg/dimg_props.cpp:233
#3  0x00007ffff63a9984 in Digikam::DImg::load(QString const&, int, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&)
    (this=0x555561860510, filePath=..., loadFlagsInt=63, observer=0x555561860340, rawDecodingSettings=...) at /home/simon/Development/kde/digikam/core/libs/dimg/dimg_fileio.cpp:113
#4  0x00007ffff63a96b1 in Digikam::DImg::load(QString const&, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (this=0x555561860510, filePath=..., observer=0x555561860340, rawDecodingSettings=...)
    at /home/simon/Development/kde/digikam/core/libs/dimg/dimg_fileio.cpp:62
#5  0x00007ffff6479082 in Digikam::PreviewLoadingTask::execute() (this=0x555561860330) at /home/simon/Development/kde/digikam/core/libs/threadimageio/preview/previewtask.cpp:306
#6  0x00007ffff648beac in Digikam::LoadSaveThread::run() (this=0x5555612fb160) at /home/simon/Development/kde/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:154
#7  0x00007ffff64bb2ce in Digikam::DynamicThread::Private::run() (this=0x5555612fb1f0) at /home/simon/Development/kde/digikam/core/libs/threads/dynamicthread.cpp:191
#8  0x00007ffff42e5dc2 in  () at /usr/lib/libQt5Core.so.5
#9  0x00007ffff42e1e8f in  () at /usr/lib/libQt5Core.so.5
#10 0x00007ffff3e7d3e9 in start_thread () at /usr/lib/libpthread.so.0
#11 0x00007ffff3da9293 in clone () at /usr/lib/libc.so.6
Comment 4 Maik Qualmann 2020-10-04 20:44:38 UTC
Git commit 6a3e2da4943e992bd81ab8820095f1df5587badd by Maik Qualmann.
Committed on 04/10/2020 at 20:43.
Pushed by mqualmann into branch 'master'.

revert dynamic cast and lock cache again

M  +1    -11   core/libs/threadimageio/fileio/loadingcache.cpp
M  +37   -31   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +4    -0    core/libs/threadimageio/preview/previewtask.cpp
M  +30   -24   core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/graphics/digikam/commit/6a3e2da4943e992bd81ab8820095f1df5587badd
Comment 5 Maik Qualmann 2020-10-05 18:18:38 UTC
Git commit 80be654bda3352554eb8f5e8905734f30c7d7a29 by Maik Qualmann.
Committed on 05/10/2020 at 18:17.
Pushed by mqualmann into branch 'master'.

i think this new loader code fix all race conditions

M  +1    -15   core/libs/threadimageio/fileio/loadingcache.cpp
M  +0    -2    core/libs/threadimageio/fileio/loadingcache.h
M  +49   -51   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +49   -51   core/libs/threadimageio/preview/previewtask.cpp
M  +42   -39   core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/graphics/digikam/commit/80be654bda3352554eb8f5e8905734f30c7d7a29
Comment 6 Simon Westersund 2020-10-06 19:59:10 UTC
Created attachment 132178 [details]
GDB backtrace with latest master b7d3d74918

Hi Maik,
I tested again with the latest master, but the same function still somehow does a pure virtual method call:

Digikam::LoadingCache::notifyNewLoadingProcess(Digikam::LoadingProcess*, Digikam::LoadingDescription const&)

To me, this backtrace looks exactly like the original backtrace.
Comment 7 Maik Qualmann 2020-10-06 20:40:14 UTC
Git commit 9519ba801a52577565ac53143ca01a0320e687af by Maik Qualmann.
Committed on 06/10/2020 at 20:39.
Pushed by mqualmann into branch 'master'.

use a loading flag

M  +5    -17   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +0    -1    core/libs/threadimageio/fileio/loadsavetask.h
M  +5    -9    core/libs/threadimageio/preview/previewtask.cpp
M  +5    -9    core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/graphics/digikam/commit/9519ba801a52577565ac53143ca01a0320e687af
Comment 8 Maik Qualmann 2020-10-07 16:28:14 UTC
Git commit 6478053c9490a73bb8b8c499750b2961627e18d2 by Maik Qualmann.
Committed on 07/10/2020 at 16:27.
Pushed by mqualmann into branch 'master'.

further changes to the preview loader

M  +33   -40   core/libs/threadimageio/fileio/loadsavetask.cpp
M  +38   -45   core/libs/threadimageio/preview/previewtask.cpp
M  +27   -34   core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/graphics/digikam/commit/6478053c9490a73bb8b8c499750b2961627e18d2
Comment 9 Maik Qualmann 2020-10-07 19:01:18 UTC
Git commit b12004d56014da117887938741897b171e9f9167 by Maik Qualmann.
Committed on 07/10/2020 at 19:00.
Pushed by mqualmann into branch 'master'.

use a QHash with pointer as key

M  +11   -11   core/libs/threadimageio/fileio/loadingcache.cpp

https://invent.kde.org/graphics/digikam/commit/b12004d56014da117887938741897b171e9f9167
Comment 10 caulier.gilles 2020-10-07 19:02:11 UTC
Maik, 

Since few days, I managed to parse whole digiKam code with Clazy static analyzer, more centered on Qt used.

And some results are interesting. I recommend to take a look :

https://www.digikam.org/reports/clazy/master/

If you want to make a Clazy try, look here :

https://github.com/KDE/clazy

It's just another Clang plugin in fact. Just install clang-devel package, configure, compile, and install Clazy code.

After that, just use my script here :

https://invent.kde.org/graphics/digikam/-/blob/master/project/reports/clazy.sh

The compilation traces with the Clazy warnings are exported to a log file and later exported to an HTML page generated with a Python 3 script. There are few dependencies but nothing exotic. At end results are exported to the Web site.

Perhaps you will found something about Preview Thread...

Gilles

Gilles
Comment 11 Simon Westersund 2020-10-07 19:40:47 UTC
(In reply to Maik Qualmann from comment #8)
> Git commit 6478053c9490a73bb8b8c499750b2961627e18d2 by Maik Qualmann.
> Committed on 07/10/2020 at 16:27.
> Pushed by mqualmann into branch 'master'.
> 
> further changes to the preview loader
> 
> M  +33   -40   core/libs/threadimageio/fileio/loadsavetask.cpp
> M  +38   -45   core/libs/threadimageio/preview/previewtask.cpp
> M  +27   -34   core/libs/threadimageio/thumb/thumbnailtask.cpp
> 
> https://invent.kde.org/graphics/digikam/commit/
> 6478053c9490a73bb8b8c499750b2961627e18d2

Hello Maik,

I tested with this commit now and the whole collection was scanned through without crashing :)
I will still test with your latest QHash patch. Hopefully my previous test run was not just a coincidence!
Comment 12 Simon Westersund 2020-10-07 20:51:17 UTC
Created attachment 132199 [details]
GDB backtrace with latest master b12004d560

Tested again and this time digikam crashed again, unfortunately. The backtrace remains the same as earlier.
Comment 13 Maik Qualmann 2020-10-07 21:02:55 UTC
Hmm, strange that GetMagickInfo is always called in a different thread in all backtraces. Can you delete the "DImg_ImageMagick_Plugin.so" plugin for testing?

Maik
Comment 14 Maik Qualmann 2020-10-08 05:50:41 UTC
Git commit 18b0993b36d01e436e637dae9bad22d65a8092be by Maik Qualmann.
Committed on 08/10/2020 at 05:49.
Pushed by mqualmann into branch 'master'.

disable MagickInfo for a test and use small static list

M  +10   -2    core/dplugins/dimg/imagemagick/dimgimagemagickplugin.cpp

https://invent.kde.org/graphics/digikam/commit/18b0993b36d01e436e637dae9bad22d65a8092be
Comment 15 Maik Qualmann 2020-10-08 06:10:34 UTC
Git commit 793a18d716ee5bbe8835733118cccd25cbeea04d by Maik Qualmann.
Committed on 08/10/2020 at 06:09.
Pushed by mqualmann into branch 'master'.

read Magick formats only at plugin start

M  +47   -47   core/dplugins/dimg/imagemagick/dimgimagemagickplugin.cpp
M  +7    -0    core/dplugins/dimg/imagemagick/dimgimagemagickplugin.h

https://invent.kde.org/graphics/digikam/commit/793a18d716ee5bbe8835733118cccd25cbeea04d
Comment 16 Simon Westersund 2020-10-08 16:58:39 UTC
Created attachment 132217 [details]
GDB backtrace with latest master 793a18d716

I simply built the latest master, without removing DImg_ImageMagick_Plugin.so first. That caused in a very quick crash (within seconds from starting the face recognition).
Comment 17 Simon Westersund 2020-10-08 17:01:02 UTC
Created attachment 132218 [details]
GDB backtrace with latest master 793a18d716 after deleting DImg_ImageMagick_Plugin.so

Then I tested deleting DImg_ImageMagick_Plugin.so from here:
rm /home/simon/local/opt/digikam/plugins/digikam/dimg/DImg_ImageMagick_Plugin.so

Then the face recognition ran for a few minutes before crashing.
Comment 18 Maik Qualmann 2020-10-08 18:22:34 UTC
Git commit 59ee1d16a091cf5d85150ccaaf5601292f8e1928 by Maik Qualmann.
Committed on 08/10/2020 at 18:22.
Pushed by mqualmann into branch 'master'.

do not notify the current loading task

M  +5    -5    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +5    -5    core/libs/threadimageio/preview/previewtask.cpp
M  +5    -5    core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/graphics/digikam/commit/59ee1d16a091cf5d85150ccaaf5601292f8e1928
Comment 19 Maik Qualmann 2020-10-08 18:58:46 UTC
Git commit 43171bcf551344f450cb815bb4b3de309e78f747 by Maik Qualmann.
Committed on 08/10/2020 at 18:58.
Pushed by mqualmann into branch 'master'.

new variant to send notifications from loader task

M  +2    -6    core/libs/threadimageio/fileio/loadingcache.cpp
M  +2    -2    core/libs/threadimageio/fileio/loadingcache.h
M  +15   -5    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +4    -0    core/libs/threadimageio/fileio/loadsavetask.h
M  +5    -5    core/libs/threadimageio/preview/previewtask.cpp
M  +5    -5    core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/graphics/digikam/commit/43171bcf551344f450cb815bb4b3de309e78f747
Comment 20 Simon Westersund 2020-10-09 08:55:52 UTC
Created attachment 132231 [details]
GDB backtrace with latest master 182bef3be8

Here is a backtrace from the latest testing. The face detection crashed after around 8% completion.
Comment 21 Maik Qualmann 2020-10-09 17:51:42 UTC
Git commit d02237e021ae5dfe104052cb855154e05949a959 by Maik Qualmann.
Committed on 09/10/2020 at 17:51.
Pushed by mqualmann into branch 'master'.

try with url encoded file path as cache key

M  +38   -27   core/libs/threadimageio/fileio/loadingdescription.cpp

https://invent.kde.org/graphics/digikam/commit/d02237e021ae5dfe104052cb855154e05949a959
Comment 22 Simon Westersund 2020-10-09 20:20:06 UTC
Created attachment 132247 [details]
GDB backtrace with latest master 66f96e26b5

With the latest master, there is a segmentation fault crash, instead of pure virtual fuction call. I attached a backtrace.
Comment 23 Maik Qualmann 2020-10-09 21:24:42 UTC
There is this message in the last backtrace and it surprises me after research:

__memmove_avx_unaligned_erms () at /usr/lib/libc.so.6

Please post the output of "uname -a"

Maik
Comment 24 Simon Westersund 2020-10-10 09:51:30 UTC
(In reply to Maik Qualmann from comment #23)
> There is this message in the last backtrace and it surprises me after
> research:
> 
> __memmove_avx_unaligned_erms () at /usr/lib/libc.so.6
> 
> Please post the output of "uname -a"
> 
> Maik

Here is the "uname -a" output. I redacted the hostname.
Linux $(hostname) 5.8.11-1-MANJARO #1 SMP PREEMPT Wed Sep 23 14:35:40 UTC 2020 x86_64 GNU/Linux
Comment 25 Simon Westersund 2020-10-10 10:07:49 UTC
Created attachment 132258 [details]
GDB backtrace with latest master 925e074a71

Here is the latest backtrace. We are now back to the original pure virtual function call problem.
Comment 26 Maik Qualmann 2020-10-10 11:20:45 UTC
You have installed a preemptive kernel. Does your distribution also have a "normal" kernel? Can you install this to test?

Maik
Comment 27 caulier.gilles 2020-10-10 11:49:21 UTC
Maik,

I use pre-emptive Linux kernel (recompiled by hand) in my office to reduce time-latency between process. It's not a real time kernel, but it reduce time glitch in kernel scheduler loop. Remember that in my office we implement real time data acquisition and measurement systems.

I'm not sure if this kind of Kernel patches can have a side effect in user space.

Gilles
Comment 28 Maik Qualmann 2020-10-10 12:27:13 UTC
Well, I know it's not an "RT". But now I rule out that the Pronlem is within our reach. OpenSUSE Tumbleweed doesn't use a pre-emptive kernel. I think if it was really safe they would. Another thing is this message on memcpy:

__memmove_avx_unaligned_erms () at /usr/lib/libc.so.6

This indicates a bug in libc6 if it is compiled with X32 support. There are corresponding entries at Red Hat, they assert that they are not affected because they are not using X32 support. We have vtable problems, the loading notification and the jumg into the virtual function postProcess(). In the meantime I can rule out that we leave "dead" loading tasks in the cache.

Maik
Comment 29 Simon Westersund 2020-10-10 13:04:46 UTC
Maik and Gilles,

I'm using Manjaro's default kernels. They have their own Kernel settings through which users can install some kernels, and that is what I have used.

I can try to look for a way to install a generic kernel and test that. Would you prefer a 5.8.x kernel, or the latest 5.4 LTS?

--
Simon
Comment 30 caulier.gilles 2020-10-11 08:37:37 UTC
Maik,

I just enable this Clazy check for you :

https://github.com/KDE/clazy/blob/master/docs/checks/README-virtual-call-ctor.md

Report will be updated in one hour

Gilles
Comment 31 caulier.gilles 2020-11-11 09:13:33 UTC
Maik,

I think i discovered the origin of the wrong virtual calls which crash here.

Clazy static analyzer report plenty of warnings relevant of this check:

https://github.com/KDE/clazy/blob/master/docs/checks/README-copyable-polymorphic.md

Solution: disabled copy constructor in relevant classes if possible (i currently do it) or implemented clone() operator.   

Some pointers :

https://stackoverflow.com/questions/4122789/copy-object-keep-polymorphism

https://www.fluentcpp.com/2017/09/08/make-polymorphic-copy-modern-cpp/

Did you already take a look of this approach ?

Gilles
Comment 32 Maik Qualmann 2020-11-11 19:49:36 UTC
I will deal with this approach.
Despite all that, I would like to have a new backtrace.
Because additional debug messages are included. So I would have a little more of the terminal output before the crash.

Maik
Comment 33 Simon Westersund 2020-11-11 21:25:20 UTC
Created attachment 133246 [details]
GDB backtrace with latest master 3d387983fe

Hello Gilles and Maik,

I rebuilt the master branch from today and reran the same old face detection test. The crash reproduced around 5% into the process. The backtrace is attached.

Maik, you mentioned that we should expect some added debug messages right before the crash. However, I didn't see anything other than the "New Thread" and "Thread exited" messages from GDB right before the crash. Some hundred lines before the crash, there were a couple of errors about unknown image types, due to *.xcf files in my library. For example:

> digikam.metaengine: Cannot load metadata from file /path/to/image.xcf  (Error # 11 :  /path/to/image.xcf: The file contains data of an unknown image type
> digikam.metaengine: Error: source file is not HEIF image.
Comment 34 Maik Qualmann 2020-11-11 21:34:43 UTC
Did you activate the debug output with:

export QT_LOGGING_RULES="digikam.*=true"

I expect debug output like this:

digikam.general: LoadingTask status is set to stopping now... 0x47aefb0
digikam.general: PreviewLoadingTask is at the end............ 0x47aefb0
digikam.general: PreviewLoadingTask destructor was called.... 0x47aefb0

Maik
Comment 35 Simon Westersund 2020-11-11 21:39:54 UTC
(In reply to Maik Qualmann from comment #34)
> Did you activate the debug output with:
> 
> export QT_LOGGING_RULES="digikam.*=true"
> 
> I expect debug output like this:
> 
> digikam.general: LoadingTask status is set to stopping now... 0x47aefb0
> digikam.general: PreviewLoadingTask is at the end............ 0x47aefb0
> digikam.general: PreviewLoadingTask destructor was called.... 0x47aefb0
> 
> Maik

Thanks for pointing this out, I didn't do that!
Comment 36 Simon Westersund 2020-11-11 21:58:26 UTC
Created attachment 133247 [details]
Debug logs + GDB backtrace from 3d387983fe

Here is a bit of the debug log and the backtraces again. I took the liberty of redacting the file paths partially :)
Comment 37 Maik Qualmann 2020-11-12 06:37:44 UTC
Ok, very interesting backtrace. A destructor from the preview task (0x555555fd48e0) was called although it is not finished yet. That confirms my theory, but at the moment I have no idea. In principle it is not possible in the program sequence. So we need a backtrace that may contain this. i will add a breakpoint.

Maik
Comment 38 Maik Qualmann 2020-11-12 06:49:17 UTC
Git commit cb4d73e665223ca3aad005e566a732e891151fbc by Maik Qualmann.
Committed on 12/11/2020 at 06:48.
Pushed by mqualmann into branch 'master'.

add breakpoint to the destructor from preview task

M  +9    -3    core/libs/threadimageio/preview/previewtask.cpp
M  +1    -0    core/libs/threadimageio/preview/previewtask.h

https://invent.kde.org/graphics/digikam/commit/cb4d73e665223ca3aad005e566a732e891151fbc
Comment 39 Maik Qualmann 2020-11-12 11:37:43 UTC
Git commit f14ffc9e1a3f636fbb8a0ea798e18570843d5e48 by Maik Qualmann.
Committed on 12/11/2020 at 11:36.
Pushed by mqualmann into branch 'master'.

fix breakpoint patch when stopping loading

M  +5    -4    core/libs/threadimageio/preview/previewtask.cpp
M  +1    -1    core/libs/threadimageio/preview/previewtask.h

https://invent.kde.org/graphics/digikam/commit/f14ffc9e1a3f636fbb8a0ea798e18570843d5e48
Comment 40 Maik Qualmann 2020-11-12 11:40:25 UTC
Git commit 5db0ea0bacb8bb9f06636aac2f57418f782b0055 by Maik Qualmann.
Committed on 12/11/2020 at 11:39.
Pushed by mqualmann into branch 'master'.

fix breakpoint patch

M  +1    -0    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/graphics/digikam/commit/5db0ea0bacb8bb9f06636aac2f57418f782b0055
Comment 41 Maik Qualmann 2020-11-13 05:06:33 UTC
Git commit 1bc64bfd86efcf228075f3d1b0979716bde4bdb8 by Maik Qualmann.
Committed on 13/11/2020 at 05:05.
Pushed by mqualmann into branch 'master'.

i think we should use dynamic_cast here

M  +30   -16   core/libs/threadimageio/engine/managedloadsavethread.cpp

https://invent.kde.org/graphics/digikam/commit/1bc64bfd86efcf228075f3d1b0979716bde4bdb8
Comment 42 caulier.gilles 2020-11-14 22:46:35 UTC
Git commit 06304b820fcee383b5a747af806c82cc44655b3b by Gilles Caulier.
Committed on 14/11/2020 at 22:43.
Pushed by cgilles into branch 'master'.

Fix Clazy report about copy constructor and copy operator used with polymorphic classes from FaceEngine.
Store data provider classes instance while training faces stage as map of pointers. data are allocated with new operator.

M  +12   -8    core/libs/facesengine/recognition/dataproviders.cpp
M  +11   -1    core/libs/facesengine/recognition/dataproviders.h
M  +2    -1    core/libs/facesengine/recognition/facialrecognition_wrapper_recognize.cpp
M  +2    -2    core/libs/facesengine/recognition/recognitiontrainingprovider.cpp
M  +5    -5    core/utilities/facemanagement/workers/databasewriter.cpp
M  +2    -2    core/utilities/facemanagement/workers/databasewriter.h
M  +2    -2    core/utilities/facemanagement/workers/detectionworker.cpp
M  +2    -2    core/utilities/facemanagement/workers/detectionworker.h
M  +4    -4    core/utilities/facemanagement/workers/recognitionworker.cpp
M  +2    -2    core/utilities/facemanagement/workers/recognitionworker.h
M  +11   -10   core/utilities/facemanagement/workers/trainerworker.cpp
M  +1    -1    core/utilities/facemanagement/workers/trainerworker.h

https://invent.kde.org/graphics/digikam/commit/06304b820fcee383b5a747af806c82cc44655b3b
Comment 43 caulier.gilles 2020-11-14 22:50:15 UTC
Maik,

Please, double check well my important last commit with data providers classes which are polymorphic and provide a copy constructor/operator. My fix must be safe (at least after plenty of tests, all is fine here).

Gilles
Comment 44 Maik Qualmann 2020-11-15 11:51:00 UTC
Hi Gilles,

digiKam now crashes when I start it. A fresh build was created.

(gdb) bt
#0  0x00007ffff7c98937 in Digikam::FacePipeline::Private::createThumbnailLoadThread() ()
    at /usr/lib64/libdigikamgui.so.7.2.0
#1  0x00007ffff7ca984e in Digikam::DatabaseWriter::DatabaseWriter(Digikam::FacePipeline::WriteMode, Digikam::FacePipeline::Private*) () at /usr/lib64/libdigikamgui.so.7.2.0
#2  0x00007ffff7c95bf2 in Digikam::FacePipeline::plugDatabaseEditor() () at /usr/lib64/libdigikamgui.so.7.2.0
#3  0x00007ffff7928599 in Digikam::DigikamItemView::DigikamItemView(QWidget*) ()
    at /usr/lib64/libdigikamgui.so.7.2.0
#4  0x00007ffff79922c8 in Digikam::StackedView::StackedView(QWidget*) () at /usr/lib64/libdigikamgui.so.7.2.0
#5  0x00007ffff7982009 in Digikam::ItemIconView::ItemIconView(QWidget*, Digikam::DModelFactory*) ()
    at /usr/lib64/libdigikamgui.so.7.2.0
#6  0x00007ffff78e8c05 in Digikam::DigikamApp::setupView() () at /usr/lib64/libdigikamgui.so.7.2.0
#7  0x00007ffff78db212 in Digikam::DigikamApp::DigikamApp() () at /usr/lib64/libdigikamgui.so.7.2.0
#8  0x0000000000405de3 in main ()

Maik
Comment 45 Maik Qualmann 2020-11-15 12:00:15 UTC
Git commit d3cdb8c2f92612c8660ba46984da7c36f5691ba6 by Maik Qualmann.
Committed on 15/11/2020 at 11:58.
Pushed by mqualmann into branch 'master'.

fix crash in the DatabaseWriter

M  +1    -1    core/utilities/facemanagement/workers/databasewriter.cpp

https://invent.kde.org/graphics/digikam/commit/d3cdb8c2f92612c8660ba46984da7c36f5691ba6
Comment 46 Simon Westersund 2020-11-15 13:23:11 UTC
Hello Maik,

I reproduced the problem and ran into the ASSERT breakpoint. Is there anything special you would want me to provide from gdb, or are backtraces and debug logs still enough?

The short backtrace currently looks like this:

> (gdb) bt
> #0  0x00007ffff3cbe615 in raise () at /usr/lib/libc.so.6
> #1  0x00007ffff3ca7862 in abort () at /usr/lib/libc.so.6
> #2  0x00007ffff427c9ac in  () at /usr/lib/libQt5Core.so.5
> #3  0x00007ffff427bd59 in qt_assert_x(char const*, char const*, char const*, int) () at /usr/lib/libQt5Core.so.5
> #4  0x00007ffff645cbf4 in Digikam::PreviewLoadingTask::~PreviewLoadingTask() (this=0x555560f13860, __in_chrg=<optimized out>) at /home/simon/Development/kde/digikam/core/libs/threadimageio/preview/previewtask.cpp:58
> #5  0x00007ffff645cc62 in Digikam::PreviewLoadingTask::~PreviewLoadingTask() (this=0x555560f13860, __in_chrg=<optimized out>) at /home/simon/Development/kde/digikam/core/libs/threadimageio/preview/previewtask.cpp:59
> #6  0x00007ffff6470750 in Digikam::LoadSaveThread::run() (this=0x555560e1a980) at /home/simon/Development/kde/digikam/core/libs/threadimageio/fileio/loadsavethread.cpp:130
> #7  0x00007ffff649fb1e in Digikam::DynamicThread::Private::run() (this=0x555560df6020) at /home/simon/Development/kde/digikam/core/libs/threads/dynamicthread.cpp:191
> #8  0x00007ffff42bddc2 in  () at /usr/lib/libQt5Core.so.5
> #9  0x00007ffff42b9e8f in  () at /usr/lib/libQt5Core.so.5
> #10 0x00007ffff3e553e9 in start_thread () at /usr/lib/libpthread.so.0
> #11 0x00007ffff3d81293 in clone () at /usr/lib/libc.so.6

Best regards,
Simon
Comment 47 Maik Qualmann 2020-11-15 14:53:19 UTC
Git commit 006263c2beefdd97a793af8d38a38dba068566ac by Maik Qualmann.
Committed on 15/11/2020 at 14:52.
Pushed by mqualmann into branch 'master'.

revert old change to the DynamicThread

M  +2    -2    core/libs/threads/dynamicthread.cpp

https://invent.kde.org/graphics/digikam/commit/006263c2beefdd97a793af8d38a38dba068566ac
Comment 48 Simon Westersund 2020-11-15 20:29:37 UTC
Created attachment 133371 [details]
Debug logs from stuck face detection in 8f1b5e900f

I retested with the 8f1b5e900f commit, and now it seems like the face detection gets stuck and doesn't progress. My GDB console with debug prints enabled printed nothing for several minutes, which is definitely not normal. Digikam did not crash, it just could not progress somehow... I was able to cancel the detection by pressing the button next to the progress indicator in the bottom right corner and then exit Digikam. I have attached the last bit of debug logs, including when I interrupted the face detection after it had stalled.
Comment 49 Maik Qualmann 2020-11-16 12:33:07 UTC
Please try it again. Very rarely have I had it before that the face detection "stops" - it should be another problem.

Maik
Comment 50 Maik Qualmann 2020-11-22 21:20:37 UTC
Git commit 1fcedb1fa11d6f48c1265dc309ea56138a5c45bc by Maik Qualmann.
Committed on 22/11/2020 at 21:19.
Pushed by mqualmann into branch 'master'.

try to fix the face detection stopping

M  +4    -2    core/utilities/maintenance/facesdetector.cpp

https://invent.kde.org/graphics/digikam/commit/1fcedb1fa11d6f48c1265dc309ea56138a5c45bc
Comment 51 Maik Qualmann 2021-01-05 11:43:43 UTC
Git commit df6e4fcf0157246f6d1d646f6d768d2b27e48f89 by Maik Qualmann.
Committed on 05/01/2021 at 11:42.
Pushed by mqualmann into branch 'master'.

this could be a cause of the crash in the preview loader
Related: bug 429307, bug 427380, bug 421043

M  +4    -0    core/libs/threadimageio/fileio/loadsavetask.cpp
M  +5    -0    core/libs/threadimageio/preview/previewtask.cpp
M  +2    -0    core/libs/threadimageio/thumb/thumbnailtask.cpp

https://invent.kde.org/graphics/digikam/commit/df6e4fcf0157246f6d1d646f6d768d2b27e48f89
Comment 52 Maik Qualmann 2021-01-14 06:35:56 UTC
Git commit d62937a9e0c3a496f8ccbdbe72835d08899d6718 by Maik Qualmann.
Committed on 14/01/2021 at 06:33.
Pushed by mqualmann into branch 'master'.

the DynamicThread should not be a QRunnable
Otherwise it is a bit strange that a QRunnable
started a QRunnable, only the QThreadPool should do this.
Related: bug 429307, bug 427380, bug 421043

M  +2    -3    core/libs/threads/dynamicthread.cpp
M  +7    -3    core/libs/threads/dynamicthread.h

https://invent.kde.org/graphics/digikam/commit/d62937a9e0c3a496f8ccbdbe72835d08899d6718
Comment 53 Maik Qualmann 2021-01-14 06:50:41 UTC
Git commit 46d4abd0cfcbdbdbab1f1189186e1737caec2839 by Maik Qualmann.
Committed on 14/01/2021 at 06:49.
Pushed by mqualmann into branch 'master'.

call wait() after stop() before call start() again
Otherwise a running task has not yet ended and and
is deleted when the QThreadPool is started again.
Related: bug 429307, bug 427380, bug 421043

M  +1    -0    core/utilities/facemanagement/threads/facepreviewloader.cpp

https://invent.kde.org/graphics/digikam/commit/46d4abd0cfcbdbdbab1f1189186e1737caec2839