Bug 385592

Summary: digikam segfaults occasionally, when renaming/moving multiple images [patch]
Product: [Applications] digikam Reporter: Johannes Hirte <johannes.hirte>
Component: Database-ThumbsAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: crash CC: BrianEdmond, caulier.gilles, metzpinguin
Priority: NOR    
Version: 5.8.0   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In: 5.8.0
Sentry Crash Report:
Attachments: ImageInfo.patch

Description Johannes Hirte 2017-10-11 10:58:41 UTC
When renaming and/or moving images around, from time to time digikam dies with a segfault.

backtrace with gdb:

Thread 41 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f8b047f8700 (LWP 15472)]
__memcpy_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:109
109     ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S: No such file or directory.
(gdb) bt
#0  __memcpy_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:109
#1  0x00007f8c0a563573 in memcpy (__len=<optimized out>, __src=<optimized out>, __dest=<optimized out>) at /usr/include/bits/string_fortified.h:34
#2  QConcatenable<QString>::appendTo (out=<synthetic pointer>: <optimized out>, a=...) at /usr/include/qt5/QtCore/qstringbuilder.h:261
#3  QConcatenable<QStringBuilder<QStringBuilder<QStringBuilder<QString, QString>, QLatin1Char>, QString> >::appendTo<QChar> (out=<synthetic pointer>: <optimized out>, p=...) at /usr/include/qt5/QtCore/qstringbuilder.h:358
#4  QStringBuilder<QStringBuilder<QStringBuilder<QString, QString>, QLatin1Char>, QString>::convertTo<QString> (this=<optimized out>) at /usr/include/qt5/QtCore/qstringbuilder.h:118
#5  QStringBuilder<QStringBuilder<QStringBuilder<QString, QString>, QLatin1Char>, QString>::operator QString (this=<optimized out>) at /usr/include/qt5/QtCore/qstringbuilder.h:131
#6  Digikam::ImageInfo::filePath (this=this@entry=0x7f8b047f7868) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/database/item/imageinfo.cpp:807
#7  0x00007f8c0a5686aa in Digikam::ImageInfo::thumbnailInfo (this=this@entry=0x7f8b047f7868) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/database/item/imageinfo.cpp:1803
#8  0x00007f8c0a53026b in Digikam::ThumbsDbInfoProvider::thumbnailInfo (this=<optimized out>, identifier=...) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/database/coredb/coredbthumbinfoprovider.cpp:62
#9  0x00007f8c0b4a9d33 in Digikam::ThumbnailCreator::makeThumbnailInfo (this=this@entry=0x198b9b0, identifier=..., rect=...) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/thumbnailcreator.cpp:374
#10 0x00007f8c0b4ad7c6 in Digikam::ThumbnailCreator::load (this=this@entry=0x198b9b0, identifier=..., rect=..., pregenerate=pregenerate@entry=false) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/thumbnailcreator.cpp:226
#11 0x00007f8c0b4ae0e8 in Digikam::ThumbnailCreator::load (this=this@entry=0x198b9b0, identifier=...) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/thumbnailcreator.cpp:181
#12 0x00007f8c0b4bc9dd in Digikam::ThumbnailLoadingTask::execute (this=0x5bb3920) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/thumbnailtask.cpp:166
#13 0x00007f8c0b4946fe in Digikam::LoadSaveThread::run (this=0x7f8c0c0ee1c0 <Digikam::(anonymous namespace)::Q_QGS_defaultObject::innerFunction()::holder>)
    at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/loadsavethread.cpp:133
#14 0x00007f8c0b4d1ade in Digikam::DynamicThread::DynamicThreadPriv::run (this=0x198b6c0) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threads/dynamicthread.cpp:183
#15 0x00007f8c09550385 in QThreadPoolThread::run (this=0x2cbd660) at thread/qthreadpool.cpp:99
#16 0x00007f8c095533d0 in QThreadPrivate::start (arg=0x2cbd660) at thread/qthread_unix.cpp:368
#17 0x00007f8c060f68aa in start_thread (arg=0x7f8b047f8700) at pthread_create.c:465
#18 0x00007f8c08bc70bf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

looks like something multithreaded is not synchronized correctly
Comment 1 caulier.gilles 2017-10-11 11:35:23 UTC
Which digiKam version do you use exactly ?

Gilles Caulier
Comment 2 Johannes Hirte 2017-10-11 12:20:09 UTC
(In reply to caulier.gilles from comment #1)
> Which digiKam version do you use exactly ?
> 
> Gilles Caulier

From git at commit 841cd2947cccfff1b541e97e5a626ccdc2cef979
Comment 3 Johannes Hirte 2017-10-11 12:26:41 UTC
Btw. I've seen those segfaults and hangs on renaming pictures several times before. It's only the first time I digged into this.
Comment 4 Maik Qualmann 2017-10-11 16:53:39 UTC
I do not see an probleme in the ImageInfo::filePath() function. And it crashes in system or Qt. Which distribution do you use?

Maik
Comment 5 Johannes Hirte 2017-10-11 18:55:20 UTC
(In reply to Maik Qualmann from comment #4)
> I do not see an probleme in the ImageInfo::filePath() function. And it
> crashes in system or Qt. Which distribution do you use?
> 
> Maik

I'm using Gentoo with live-ebuild for digikam.

I don't think ImageInfo::filePath() is the problem in the first place. It seems to me, m_data is invalidated by some other thread, and this way a nullptr is pushed down.
Comment 6 Johannes Hirte 2017-10-11 18:56:28 UTC
(In reply to Johannes Hirte from comment #5)
> (In reply to Maik Qualmann from comment #4)
> > I do not see an probleme in the ImageInfo::filePath() function. And it
> > crashes in system or Qt. Which distribution do you use?
> > 
> > Maik
> 
> I'm using Gentoo with live-ebuild for digikam.
> 
> I don't think ImageInfo::filePath() is the problem in the first place. It
> seems to me, m_data is invalidated by some other thread, and this way a
> nullptr is pushed down.

Sorry, not a nullptr, an invalid pointer.
Comment 7 Maik Qualmann 2017-10-12 04:48:53 UTC
Created attachment 108290 [details]
ImageInfo.patch

Johannes,

can you apply this patch and test it? This crash I have never seen or reproduced here.

Maik
Comment 8 Johannes Hirte 2017-10-12 08:08:53 UTC
(In reply to Maik Qualmann from comment #7)
> Created attachment 108290 [details]
> ImageInfo.patch
> 
> Johannes,
> 
> can you apply this patch and test it? This crash I have never seen or
> reproduced here.
> 
> Maik

got this

Thread 71 "Thread (pooled)" received signal SIGBUS, Bus error.
[Switching to Thread 0x7f5584d0e700 (LWP 24038)]
malloc_consolidate (av=0x7f551c000020) at malloc.c:4489
4489    malloc.c: No such file or directory.
Comment 9 Johannes Hirte 2017-10-12 08:44:26 UTC
I can confirm, that the error changed from SIGSEGV to SIGBUS. I can't reproduce the error reliable, but when it happens, it is SIGBUS now.
Comment 10 Johannes Hirte 2017-10-12 18:04:18 UTC
(In reply to Johannes Hirte from comment #9)
> I can confirm, that the error changed from SIGSEGV to SIGBUS. I can't
> reproduce the error reliable, but when it happens, it is SIGBUS now.

Okay, I have to revert this. Now it's SIGABRT:

Thread 23 "Thread (pooled)" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fb0317fa700 (LWP 6324)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007fb0f2936506 in __GI_abort () at abort.c:90
#2  0x00007fb0f297a7b7 in __libc_message (action=action@entry=(do_abort | do_backtrace), fmt=fmt@entry=0x7fb0f2a8a0a0 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:181
#3  0x00007fb0f29822e9 in malloc_printerr (action=<optimized out>, str=0x7fb0f2a8a7c0 "malloc(): smallbin double linked list corrupted", ptr=<optimized out>, ar_ptr=<optimized out>) at malloc.c:5423
#4  0x00007fb0f2985af9 in _int_malloc (av=0x7fb014000020, bytes=<optimized out>) at malloc.c:3593
#5  0x00007fb0f2987525 in __GI___libc_malloc (bytes=64) at malloc.c:3083
#6  0x00007fb0f2f93398 in operator new(unsigned long) () from /usr/lib/gcc/x86_64-pc-linux-gnu/7.2.0/libstdc++.so.6
#7  0x00007fb0f6b12b19 in QSqlField::QSqlField (this=0x7fb0317f9580, fieldName=..., type=QVariant::ByteArray) at kernel/qsqlfield.cpp:163
#8  0x00007fb0dc966290 in qToField (field=0x7fb01413f190, tc=<optimized out>) at qsql_mysql.cpp:334
#9  0x00007fb0dc9668a3 in QMYSQLResult::record (this=<optimized out>) at qsql_mysql.cpp:836
#10 0x00007fb0f6b088b8 in QSqlQuery::record (this=this@entry=0x7fb0317f9708) at kernel/qsqlquery.cpp:915
#11 0x00007fb0f50b8d4f in Digikam::BdEngineBackend::readToList (this=this@entry=0x1b65d20, query=...) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/database/engine/dbenginebackend.cpp:838
#12 0x00007fb0f50b8edb in Digikam::BdEngineBackend::handleQueryResult (this=this@entry=0x1b65d20, query=..., values=values@entry=0x7fb0317f9750, lastInsertId=lastInsertId@entry=0x0)
    at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/database/engine/dbenginebackend.cpp:875
#13 0x00007fb0f50ba59b in Digikam::BdEngineBackend::execSql (this=this@entry=0x1b65d20, sql=..., boundValue1=..., boundValue2=..., values=values@entry=0x7fb0317f9750, lastInsertId=lastInsertId@entry=0x0)
    at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/database/engine/dbenginebackend.cpp:901
#14 0x00007fb0f50c42ca in Digikam::ThumbsDb::findByHash (this=<optimized out>, uniqueHash=..., fileSize=<optimized out>) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/database/thumbsdb/thumbsdb.cpp:131
#15 0x00007fb0f52e17bc in Digikam::ThumbnailCreator::loadThumbsDbInfo (this=this@entry=0x1ff6740, info=...) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/thumbnailcreator.cpp:882
#16 0x00007fb0f52e3fb7 in Digikam::ThumbnailCreator::loadFromDatabase (this=this@entry=0x1ff6740, info=...) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/thumbnailcreator.cpp:917
#17 0x00007fb0f52e5e8b in Digikam::ThumbnailCreator::load (this=this@entry=0x1ff6740, identifier=..., rect=..., pregenerate=pregenerate@entry=false) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/thumbnailcreator.cpp:246
#18 0x00007fb0f52e60e8 in Digikam::ThumbnailCreator::load (this=this@entry=0x1ff6740, identifier=...) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/thumbnailcreator.cpp:181
#19 0x00007fb0f52f49dd in Digikam::ThumbnailLoadingTask::execute (this=0x66baa60) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/thumbnailtask.cpp:166
#20 0x00007fb0f52cc6fe in Digikam::LoadSaveThread::run (this=0x7fb0f5f261c0 <Digikam::(anonymous namespace)::Q_QGS_defaultObject::innerFunction()::holder>)
    at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threadimageio/loadsavethread.cpp:133
#21 0x00007fb0f5309ade in Digikam::DynamicThread::DynamicThreadPriv::run (this=0x1ff6450) at /var/tmp/portage/media-gfx/digikam-9999/work/digikam-9999/libs/threads/dynamicthread.cpp:183
#22 0x00007fb0f3388385 in QThreadPoolThread::run (this=0x30465a0) at thread/qthreadpool.cpp:99
#23 0x00007fb0f338b3d0 in QThreadPrivate::start (arg=0x30465a0) at thread/qthread_unix.cpp:368
#24 0x00007fb0eff2e8aa in start_thread (arg=0x7fb0317fa700) at pthread_create.c:465
#25 0x00007fb0f29ff0bf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

I've put 30 pictures into an album and started renaming with the integrated renaming tool. This could take some time, but it will trigger the bug sooner or later.
Comment 11 Johannes Hirte 2017-10-12 18:46:08 UTC
And now SIGSEGV is back. The backtrace is similar to the one above. I can provide it, if you need it. Here is the workflow, how I reproduced it:

1. put 30 images into an album
2. rename the images  crashtest-001 - crashtest-030
3. rename the images  crashtest-101 - crashtest-130
4. repeat at 2.
Comment 12 Maik Qualmann 2017-10-12 20:16:54 UTC
Git commit f8d3249f580b78c0afad9dd83a929a7e12652759 by Maik Qualmann.
Committed on 12/10/2017 at 20:14.
Pushed by mqualmann into branch 'master'.

polish and add missing reference to ImageInfo lists

M  +1    -1    libs/database/item/imageinfolist.cpp
M  +1    -1    libs/database/item/imageinfolist.h
M  +6    -6    libs/database/utils/dio.cpp
M  +2    -2    libs/database/utils/dio.h
M  +3    -3    libs/database/utils/dio_p.h
M  +1    -1    libs/fileactionmanager/fileworkeriface.h
M  +1    -1    libs/fileactionmanager/metadatahub.cpp
M  +1    -1    libs/fileactionmanager/metadatahub.h
M  +1    -1    libs/fileactionmanager/metadatahubmngr.cpp
M  +1    -1    libs/fileactionmanager/metadatahubmngr.h
M  +1    -1    libs/imageproperties/disjointmetadata.cpp

https://commits.kde.org/digikam/f8d3249f580b78c0afad9dd83a929a7e12652759
Comment 13 Maik Qualmann 2017-10-12 20:18:27 UTC
Can you please test the latest git/master version?

Maik
Comment 14 Maik Qualmann 2017-10-12 20:46:57 UTC
I can now reproduce the crash...

Maik
Comment 15 Maik Qualmann 2017-10-12 21:15:37 UTC
Git commit 38ebdc7496037ad38bc356baa0242c5587ebd9c4 by Maik Qualmann.
Committed on 12/10/2017 at 21:14.
Pushed by mqualmann into branch 'master'.

try to find the renaming crash

M  +2    -2    libs/database/utils/dio.cpp

https://commits.kde.org/digikam/38ebdc7496037ad38bc356baa0242c5587ebd9c4
Comment 16 Maik Qualmann 2017-10-17 18:35:51 UTC
Git commit 89e61652264777a38578c5bf9b944a4c82173294 by Maik Qualmann.
Committed on 17/10/2017 at 18:32.
Pushed by mqualmann into branch 'master'.

add backward QHash to the ImageInfo cache to resolve changed names
Related: bug 382312, bug 384577

M  +6    -0    libs/database/item/imageinfocache.cpp
M  +1    -0    libs/database/item/imageinfocache.h
M  +2    -2    libs/database/utils/dio.cpp

https://commits.kde.org/digikam/89e61652264777a38578c5bf9b944a4c82173294
Comment 17 Maik Qualmann 2017-10-17 18:37:44 UTC
Johannes,

can you please test the latest git/master code?

Maik
Comment 18 Johannes Hirte 2017-10-20 12:22:47 UTC
Looks good, I wasn't able to reproduce a crash.
Comment 19 caulier.gilles 2017-10-21 15:30:44 UTC
Thanks Maik to patch the code.

Thanks Johannes for the feedback.

I close this file now.
Comment 20 Johannes Hirte 2017-10-22 15:28:58 UTC
Had another segfault, this time outside of a gdb session, so I don't have a backtrace. It happened when moving pictures with dolphin into a collection during a running digikam. So I'm not sure this is the same bug. I'm trying to reproduce inside gdb and reopen this bug or open a new one, as soon as I have some more facts.
Comment 21 Maik Qualmann 2018-02-11 22:17:13 UTC
*** Bug 390287 has been marked as a duplicate of this bug. ***