Bug 393635

Summary: Segfault when saving TIFF after modifying RAW
Product: [Applications] digikam Reporter: Victor Engmark <kde.org>
Component: Metadata-EngineAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: crash CC: caulier.gilles, gdiaz, metzpinguin
Priority: NOR    
Version: 5.9.0   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In: 6.3.0
Sentry Crash Report:

Description Victor Engmark 2018-04-29 09:59:40 UTC
`coredumpctl gdb 2643 <<< bt` output:

#0  0x00007fec1165e186 in Exiv2::IptcData::printStructure(std::ostream&, unsigned char const*, unsigned long, unsigned int) () from /usr/lib/libexiv2.so.26
[Current thread is 1 (Thread 0x7feae8ff9700 (LWP 2696))]
(gdb) #0  0x00007fec1165e186 in Exiv2::IptcData::printStructure(std::ostream&, unsigned char const*, unsigned long, unsigned int) () at /usr/lib/libexiv2.so.26
#1  0x00007fec11658a20 in Exiv2::Image::printIFDStructure(Exiv2::BasicIo&, std::ostream&, Exiv2::PrintStructureOption, unsigned int, bool, char, int) () at /usr/lib/libexiv2.so.26
#2  0x00007fec1165a558 in Exiv2::Image::printTiffStructure(Exiv2::BasicIo&, std::ostream&, Exiv2::PrintStructureOption, int, unsigned long) () at /usr/lib/libexiv2.so.26
#3  0x00007fec116d7115 in Exiv2::TiffImage::printStructure(std::ostream&, Exiv2::PrintStructureOption, int) ()
    at /usr/lib/libexiv2.so.26
#4  0x00007fec116d8bdb in Exiv2::TiffImage::readMetadata() () at /usr/lib/libexiv2.so.26
#5  0x00007fec1b23e771 in Digikam::MetaEngine::load(QString const&) const () at /usr/lib/libdigikamcore.so.5.9.0
#6  0x00007fec1b287f58 in Digikam::DMetadata::load(QString const&) const () at /usr/lib/libdigikamcore.so.5.9.0
#7  0x00007fec1b287fe5 in Digikam::DMetadata::DMetadata(QString const&) () at /usr/lib/libdigikamcore.so.5.9.0
#8  0x00007fec1b2dd428 in Digikam::ThumbnailCreator::createThumbnail(Digikam::ThumbnailInfo const&, QRect const&) const () at /usr/lib/libdigikamcore.so.5.9.0
#9  0x00007fec1b2e0223 in Digikam::ThumbnailCreator::load(Digikam::ThumbnailIdentifier const&, QRect const&, bool) const () at /usr/lib/libdigikamcore.so.5.9.0
#10 0x00007fec1b2e0b65 in Digikam::ThumbnailCreator::load(Digikam::ThumbnailIdentifier const&) const ()
    at /usr/lib/libdigikamcore.so.5.9.0
#11 0x00007fec1b2ef487 in  () at /usr/lib/libdigikamcore.so.5.9.0
#12 0x00007fec1b2c4869 in Digikam::LoadSaveThread::run() () at /usr/lib/libdigikamcore.so.5.9.0
#13 0x00007fec1b3075e0 in Digikam::DynamicThread::DynamicThreadPriv::run() () at /usr/lib/libdigikamcore.so.5.9.0
#14 0x00007fec18bfda92 in  () at /usr/lib/libQt5Core.so.5
#15 0x00007fec18c00abd in  () at /usr/lib/libQt5Core.so.5
#16 0x00007fec121840bc in start_thread () at /usr/lib/libpthread.so.0
#17 0x00007fec17f062ff in clone () at /usr/lib/libc.so.6

This happened by importing the RAW file, cropping it, and then saving. I was able to reproduce it twice afterwards with the same image, once (successfully) saving right after importing, then cropping, and then crashing again on save.

The cropped TIFF seems to be saved successfully, but when trying to preview it DigiKam segfaults again:

#0  0x00007f1cfa852186 in Exiv2::IptcData::printStructure(std::ostream&, unsigned char const*, unsigned long, unsigned int) () from /usr/lib/libexiv2.so.26
[Current thread is 1 (Thread 0x7f1bf17fa700 (LWP 4016))]
(gdb) #0  0x00007f1cfa852186 in Exiv2::IptcData::printStructure(std::ostream&, unsigned char const*, unsigned long, unsigned int) () at /usr/lib/libexiv2.so.26
#1  0x00007f1cfa84ca20 in Exiv2::Image::printIFDStructure(Exiv2::BasicIo&, std::ostream&, Exiv2::PrintStructureOption, unsigned int, bool, char, int) () at /usr/lib/libexiv2.so.26
#2  0x00007f1cfa84e558 in Exiv2::Image::printTiffStructure(Exiv2::BasicIo&, std::ostream&, Exiv2::PrintStructureOption, int, unsigned long) () at /usr/lib/libexiv2.so.26
#3  0x00007f1cfa8cb115 in Exiv2::TiffImage::printStructure(std::ostream&, Exiv2::PrintStructureOption, int) ()
    at /usr/lib/libexiv2.so.26
#4  0x00007f1cfa8ccbdb in Exiv2::TiffImage::readMetadata() () at /usr/lib/libexiv2.so.26
#5  0x00007f1d04432771 in Digikam::MetaEngine::load(QString const&) const () at /usr/lib/libdigikamcore.so.5.9.0
#6  0x00007f1d0447bf58 in Digikam::DMetadata::load(QString const&) const () at /usr/lib/libdigikamcore.so.5.9.0
#7  0x00007f1d0447bfe5 in Digikam::DMetadata::DMetadata(QString const&) () at /usr/lib/libdigikamcore.so.5.9.0
#8  0x00007f1d044cb5e9 in  () at /usr/lib/libdigikamcore.so.5.9.0
#9  0x00007f1d044cc399 in  () at /usr/lib/libdigikamcore.so.5.9.0
#10 0x00007f1d044b8869 in Digikam::LoadSaveThread::run() () at /usr/lib/libdigikamcore.so.5.9.0
#11 0x00007f1d044fb5e0 in Digikam::DynamicThread::DynamicThreadPriv::run() () at /usr/lib/libdigikamcore.so.5.9.0
#12 0x00007f1d01df1a92 in  () at /usr/lib/libQt5Core.so.5
#13 0x00007f1d01df4abd in  () at /usr/lib/libQt5Core.so.5
#14 0x00007f1cfb3780bc in start_thread () at /usr/lib/libpthread.so.0
#15 0x00007f1d010fa2ff in clone () at /usr/lib/libc.so.6
Comment 1 caulier.gilles 2018-04-29 10:02:07 UTC
What do you use as libexiv2 version with digiKam. Look in Help Components Info for details.

In all cases :

1/ report this problem to Exiv2 bugzilla as UPSTREAM bug. There is no DK code in fault here.

2/ try to use digiKam AppImage 5.9.0 offcial bundle to see uf problem is reproducible.

Gilles Caulier
Comment 2 Victor Engmark 2018-04-29 10:08:51 UTC
I use Exiv2 0.26.

PS: This happens with about one in ten images that I edit. It has happened at least once in every release for the last few months. Usually DigiKam crashes upon loading the catalogue after crashing during save, but now it crashes when clicking Preview.
Comment 3 Maik Qualmann 2018-04-29 10:11:45 UTC
This is the same Bug 387515

Maik
Comment 4 Victor Engmark 2018-04-29 10:13:46 UTC
Reported as https://dev.exiv2.org/issues/1347
Comment 5 Maik Qualmann 2018-04-29 10:16:57 UTC
I could not reproduce the crash in Bug 387515 with the test image. Can you also provide this image? It would be nice if they tried to reproduce the crash with the AppImage from www.digikam.org.

Maik
Comment 6 Victor Engmark 2018-04-29 10:21:37 UTC
(In reply to Maik Qualmann from comment #5)
> I could not reproduce the crash in Bug 387515 with the test image. Can you
> also provide this image? It would be nice if they tried to reproduce the
> crash with the AppImage from www.digikam.org.

Please see https://transfernow.net/612xx2l6shgd
Comment 7 caulier.gilles 2018-04-29 10:41:43 UTC
*** Bug 387515 has been marked as a duplicate of this bug. ***
Comment 8 Maik Qualmann 2018-04-29 10:42:34 UTC
Even if the crash does not always occur when calling the preview, it crashes clearly in Exiv2. The message also appears in the console:

digikam.metaengine: Exiv2 ( 2 ) :  Directory Image3, entry 0x0111: Strip 0 is outside of the data area; ignored.

Maik
Comment 9 caulier.gilles 2018-04-29 10:58:40 UTC
and typically, the Exiv2 interface written in digiKam core catch all Exiv2 C++ exceptions. So it's clear that in this case, Exiv2 do not generate a dedicated exception and crash.

To resume, Exiv2 implementation must be fixed.

Gilles Caulier
Comment 10 Maik Qualmann 2018-04-29 11:03:00 UTC
Maybe an interesting observation here, if I open the image with the CLI exiv2 tool before, digiKam will not crash anymore.

Maik
Comment 11 caulier.gilles 2018-04-29 12:07:25 UTC
Well, if it's always reproducible, i don't understand why...
Comment 12 Maik Qualmann 2018-04-29 16:10:47 UTC
I think it's related to the memory contents, whether it crashes.

Maik
Comment 13 caulier.gilles 2018-04-29 18:14:44 UTC
But a stand alone exiv2 session will run in a process which will not share the memory to another one as digiKam.

Or Exiv2 use a system mecanism which is badly managed, as mmap and munmap to speed up access to file metadata.

Look for ex to Exiv2::FileIo class for ex. (basicio.cpp)

https://github.com/Exiv2/exiv2/blob/master/src/basicio.cpp#L410

Gilles
Comment 14 Maik Qualmann 2018-08-03 17:08:04 UTC
*** Bug 397129 has been marked as a duplicate of this bug. ***
Comment 15 Victor Engmark 2018-09-22 20:32:45 UTC
I have reported this to the Exiv2 project at https://dev.exiv2.org/issues/1347 and they say it's not an Exiv2 issue.

When I run `exiv2 [offending file]` the first line printed is

> Warning: Directory Image3, entry 0x0111: Strip 0 is outside of the data area; ignored.

Could it be that Digikam does not handle this warning?
Comment 16 caulier.gilles 2018-09-22 20:47:29 UTC
Exiv2 team is serious ? The crash backtrace is clear :

#0  0x00007fec1165e186 in Exiv2::IptcData::printStructure(std::ostream&, unsigned char const*, unsigned long, unsigned int) () from /usr/lib/libexiv2.so.26
[Current thread is 1 (Thread 0x7feae8ff9700 (LWP 2696))]
(gdb) #0  0x00007fec1165e186 in Exiv2::IptcData::printStructure(std::ostream&, unsigned char const*, unsigned long, unsigned int) () at /usr/lib/libexiv2.so.26
#1  0x00007fec11658a20 in Exiv2::Image::printIFDStructure(Exiv2::BasicIo&, std::ostream&, Exiv2::PrintStructureOption, unsigned int, bool, char, int) () at /usr/lib/libexiv2.so.26
#2  0x00007fec1165a558 in Exiv2::Image::printTiffStructure(Exiv2::BasicIo&, std::ostream&, Exiv2::PrintStructureOption, int, unsigned long) () at /usr/lib/libexiv2.so.26
#3  0x00007fec116d7115 in Exiv2::TiffImage::printStructure(std::ostream&, Exiv2::PrintStructureOption, int) ()
    at /usr/lib/libexiv2.so.26
#4  0x00007fec116d8bdb in Exiv2::TiffImage::readMetadata() () at /usr/lib/libexiv2.so.26
#5  0x00007fec1b23e771 in Digikam::MetaEngine::load(QString const&) const () at 

It's in Exiv2 code, not digiKam. The tiff parser and after the IPTC metadata parser do something wrong with the image.

digiKam ask to Exiv2 to extract the image from standard IPTC preview tag (typically a JPEG image). That all. The action is delegate to Exiv2, and digiKam wait the data. The crash appear in the internal process from Exiv2. It's not so far a digiKam crash.

Note: the warning come from Exiv2 and is just informative that some tags structure cannot be handled properly. Exiv2 format the metadata container accordingly and pass as well the data to the client application. That all... In other word, this warning is not important for digiKam.

Gilles Caulier
Comment 17 Maik Qualmann 2018-09-22 21:24:32 UTC
It crashes at exactly this point with the test image "_MG_5144_v1.TIFF":

https://cgit.kde.org/digikam.git/tree/core/libs/dmetadata/metaengine.cpp#n292

Since it is not always reproducible, but under stress conditions when more metadata is loaded, it is probably due to the fact that Exiv2 is not 100% thread save.

We have to hope for Exiv2-0.27...

Maik
Comment 18 Victor Engmark 2018-09-22 22:43:19 UTC
https://dev.exiv2.org/issues/1347#note-11

> Please ask the DigiKam Engineers to reproduce your issue with the eviv2 command-line program and your issue will be investigated.
Comment 19 caulier.gilles 2018-09-23 09:38:56 UTC
Maik,

I clarify the situation with my last responses in Exiv2 bug report :


http://dev.exiv2.org/issues/1347

Gilles
Comment 20 caulier.gilles 2018-09-23 09:41:02 UTC
Maik,

Forget to said that we will need to lock all calls to DK Metadata Engine with Mutex, as the next 0.27 will take few months to be released and we will need to support Exiv2 0.26 again for a while. 

What do you think about ?

gilles Caulier
Comment 21 Maik Qualmann 2018-09-23 11:32:25 UTC
I think that QMutex will not help us here. It crashes in PreviewLoadingTask::loadImagePreview(), which I can reproduce well if I click on the image right after the start. But moving the function into the LoadingCache and calling it with a CacheLock does not improve the crash. And see also this Bugreport: https://github.com/Exiv2/exiv2/issues/427

Maik
Comment 22 Maik Qualmann 2018-09-23 11:40:56 UTC
I have let the address of all DMetadata objects debugged and even if they are destroyed. In principle, when the crash occurs, there is current no more DMetadata object in memory.

Maik
Comment 23 caulier.gilles 2019-07-24 05:43:15 UTC
Victor,

This crash still reproducible using last Linux AppImage bundle 6.2.0 pre release available here : https://files.kde.org/digikam/

Thanks in advance

Gilles Caulier
Comment 24 Victor Engmark 2019-07-25 19:59:16 UTC
I don't have the time to get a pre-release working, but I'll assume it's working and get back to you if it's not. Thank you very much for the help!
Comment 25 caulier.gilles 2019-07-25 21:45:02 UTC
We need a real feedback with a test. We provide a pre-release AppImage bundle for that.

Please take 5 mns to download, run, and test.

Thanks in advance

Gilles Caulier
Comment 26 Victor Engmark 2019-08-04 19:48:21 UTC
I downloaded and verified the image:

$ wget --continue https://download.kde.org/unstable/digikam/digikam-6.0.0-beta3-20181228T114626-x86-64.appimage https://download.kde.org/unstable/digikam/digikam-6.0.0-beta3-20181228T114626-x86-64.appimage.sig
…
$ gpg --keyserver keys.gnupg.net --receive-keys 4A77747BC2386E50
$ gpg --verify ./*.sig # pgp signature 
gpg: assuming signed data in './digikam-6.0.0-beta3-20181228T114626-x86-64.appimage'
gpg: Signature made Sat 29 Dec 2018 03:55:02 NZDT
gpg:                using RSA key 4A77747BC2386E50
gpg: Good signature from "digiKam.org (digiKam project) <digikamdeveloper@gmail.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: D1CF 2444 A785 8C5F 2FB0  95B7 4A77 747B C238 6E50
$ chmod u+x digikam-6.0.0-beta3-20181228T114626-x86-64.appimage

but it doesn't work:

$ ./digikam-6.0.0-beta3-20181228T114626-x86-64.appimage
bash: ./digikam-6.0.0-beta3-20181228T114626-x86-64.appimage: No such file or directory
Comment 27 caulier.gilles 2019-08-04 20:38:35 UTC
why do you want to use an instable version (beta3).

6.2.0 official release is published :

https://download.kde.org/stable/digikam/6.2.0/

Gilles Caulier
Comment 28 Victor Engmark 2019-08-05 05:57:02 UTC
So you *don't* "provide a pre-release AppImage bundle for that." Thanks. I'll just mark this as fixed then, and not bother wasting everybody's time reporting issues.