Bug 367640

Summary: Crash when decoding Canon 550D RAW file using VNG interpolation with libraw 0.18
Product: [Applications] digikam Reporter: Andrew Coles <andrew.i.coles>
Component: Plugin-DImg-RAWAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: crash CC: bafco001, caulier.gilles, lexa, metzpinguin
Priority: NOR    
Version: 5.1.0   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
URL: https://www.dropbox.com/s/b51ff1u2o8bthln/IMG_5249.CR2?dl=0
Latest Commit: Version Fixed In: 5.3.0
Sentry Crash Report:

Description Andrew Coles 2016-08-21 12:33:51 UTC
Opening a raw from my camera (e.g. https://www.dropbox.com/s/b51ff1u2o8bthln/IMG_5249.CR2?dl=0 ) causes a segfault in RAW decoding in digikam 5.1 and current HEAD from git; but not in digikam 5.0.

This is due to commit d0a60f5f02890de9e5a5a3e312511930f3cf72a5, which upgraded libraw to 0.18.0 beta.  Reverting the commit and resolving merge conflicts fixes the issue.

Reproducible: Always

Steps to Reproduce:
1. Open the Image Editor for a CR2 RAW file (e.g. https://www.dropbox.com/s/b51ff1u2o8bthln/IMG_5249.CR2?dl=0)

Actual Results:  
Segfault

Expected Results:  
Should open RAW image for editing

Relevant valgrind output:

==12797== Thread 17 Thread (pooled):
==12797== Invalid write of size 4
==12797==    at 0x6364DA3: LibRaw::vng_interpolate() (dcraw_common.cpp:4683)
==12797==    by 0x633BFF7: LibRaw::dcraw_process() (libraw_cxx.cpp:4143)
==12797==    by 0x5EB4826: RawEngine::DRawDecoder::Private::loadFromLibraw(QString const&, QByteArray&, int&, int&, int&) (drawdecoder_p.cpp:516)
==12797==    by 0x5EB23A2: RawEngine::DRawDecoder::decodeRAWImage(QString const&, RawEngine::DRawDecoderSettings const&, QByteArray&, int&, int&, int&) (drawdecoder.cpp:473)
==12797==    by 0x5C188FC: Digikam::RAWLoader::load(QString const&, Digikam::DImgLoaderObserver*) (rawloader.cpp:113)
==12797==    by 0x5D9EDAA: Digikam::DImg::load(QString const&, int, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (dimg.cpp:497)
==12797==    by 0x5D9E00E: Digikam::DImg::load(QString const&, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (dimg.cpp:369)
==12797==    by 0x5D9D226: Digikam::DImg::DImg(QString const&, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (dimg.cpp:107)
==12797==    by 0x5E5D540: Digikam::LoadingTask::execute() (loadsavetask.cpp:49)
==12797==    by 0x5E4D3FE: Digikam::LoadSaveThread::run() (loadsavethread.cpp:133)
==12797==    by 0x5E94AD7: Digikam::DynamicThread::DynamicThreadPriv::run() (dynamicthread.cpp:183)
==12797==    by 0x800C342: ??? (in /usr/lib/x86_64-linux-gnu/libQt5Core.so.5.5.1)
==12797==  Address 0x88cdd7d0 is 0 bytes after a block of size 20,480 alloc'd
==12797==    at 0x4C2FB55: calloc (vg_replace_malloc.c:711)
==12797==    by 0x6333B3E: calloc (libraw_alloc.h:48)
==12797==    by 0x6333B3E: LibRaw::calloc(unsigned long, unsigned long) (libraw_cxx.cpp:472)
==12797==    by 0x6364C06: LibRaw::vng_interpolate() (dcraw_common.cpp:4665)
==12797==    by 0x633BFF7: LibRaw::dcraw_process() (libraw_cxx.cpp:4143)
==12797==    by 0x5EB4826: RawEngine::DRawDecoder::Private::loadFromLibraw(QString const&, QByteArray&, int&, int&, int&) (drawdecoder_p.cpp:516)
==12797==    by 0x5EB23A2: RawEngine::DRawDecoder::decodeRAWImage(QString const&, RawEngine::DRawDecoderSettings const&, QByteArray&, int&, int&, int&) (drawdecoder.cpp:473)
==12797==    by 0x5C188FC: Digikam::RAWLoader::load(QString const&, Digikam::DImgLoaderObserver*) (rawloader.cpp:113)
==12797==    by 0x5D9EDAA: Digikam::DImg::load(QString const&, int, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (dimg.cpp:497)
==12797==    by 0x5D9E00E: Digikam::DImg::load(QString const&, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (dimg.cpp:369)
==12797==    by 0x5D9D226: Digikam::DImg::DImg(QString const&, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (dimg.cpp:107)
==12797==    by 0x5E5D540: Digikam::LoadingTask::execute() (loadsavetask.cpp:49)
==12797==    by 0x5E4D3FE: Digikam::LoadSaveThread::run() (loadsavethread.cpp:133)
Comment 1 caulier.gilles 2016-08-21 12:54:40 UTC
Right problem is located in libraw source code included in digiKam core.

Please report this problem to Libraw team. It must be fixed before 0.18 final release. 

Note I CC Alex Tutubalin which maintain Libraw source code.

Gilles Caulier
Comment 2 caulier.gilles 2016-08-21 12:56:11 UTC
Which Canon camera model did you use to take this CR2 file ?

Gilles Caulier
Comment 3 Andrew Coles 2016-08-21 17:10:45 UTC
550D (in the USA: T2i)

Thanks,

Andrew
Comment 4 caulier.gilles 2016-08-21 17:39:19 UTC
Question : Do you use OpenMP to compile digiKam. Look in Help/Components Info dialog for details. The info is "parallelized demosaicing = <yes | no>

Gilles Caulier
Comment 5 Maik Qualmann 2016-08-23 17:57:05 UTC
I can not reproduce the crash. Parallelized demosaicing is here "yes".

Maik
Comment 6 caulier.gilles 2016-08-23 20:49:46 UTC
Crash reproducible here :

digikam.rawengine: "/mnt/data/IMG_5249.CR2"
digikam.rawengine: 
-- RAW DECODING SETTINGS --------------------------------
-- autoBrightness:          true
-- sixteenBitsImage:        true
-- brightness:              1
-- RAWQuality:              1
-- inputColorSpace:         0
-- outputColorSpace:        1
-- RGBInterpolate4Colors:   false
-- DontStretchPixels:       false
-- unclipColors:            0
-- whiteBalance:            3
-- customWhiteBalance:      6500
-- customWhiteBalanceGreen: 1
-- halfSizeColorImage:      false
-- enableBlackPoint:        false
-- blackPoint:              0
-- enableWhitePoint:        false
-- whitePoint:              0
-- NoiseReductionType:      0
-- NoiseReductionThreshold: 0
-- enableCACorrection:      false
-- caMultiplier:            0, 0
-- medianFilterPasses:      0
-- inputProfile:            ""
-- outputProfile:           ""
-- deadPixelMap:            ""
-- whiteBalanceArea:        QRect(0,0 0x0)
-- dcbIterations:           -1
-- dcbEnhanceFl:            false
-- eeciRefine:              false
-- esMedPasses:             0
-- NRChrominanceThreshold:  0
-- expoCorrection:          false
-- expoCorrectionShift:     0.994
-- expoCorrectionHighlight: 0
---------------------------------------------------------

digikam.rawengine: LibRaw progress:  Reading metadata  pass  1  of  2
digikam.rawengine: LibRaw progress:  Reading RAW data  pass  0  of  2
digikam.rawengine: LibRaw progress:  Reading RAW data  pass  1  of  2
digikam.rawengine: Disabling LibRaw highlights adjustments
digikam.rawengine: LibRaw progress:  Scaling colors  pass  0  of  2
Scaling with darkness 0, saturation 13783, and
multipliers 2.383604 1.000000 1.391007 1.000000
digikam.rawengine: LibRaw progress:  Scaling colors  pass  1  of  2
digikam.rawengine: LibRaw progress:  Pre-interpolating  pass  0  of  2
digikam.rawengine: LibRaw progress:  Pre-interpolating  pass  1  of  2
Bilinear interpolation...
digikam.rawengine: LibRaw progress:  Interpolating  pass  0  of  3
digikam.rawengine: LibRaw progress:  Interpolating  pass  1  of  3
digikam.rawengine: LibRaw progress:  Interpolating  pass  2  of  3
VNG interpolation...
*** Error in `digikam': free(): corrupted unsorted chunks: 0x00007fe0f8048a00 ***
KCrash: Application 'digikam' crashing...
KCrash: Attempting to start /usr/libexec/drkonqi from kdeinit
sock_file=/run/user/1000/kdeinit5__0
Unable to start Dr. Konqi
[gilles@localhost data]$
Comment 7 caulier.gilles 2016-08-23 20:51:14 UTC
digikam.rawengine: LibRaw progress:  Interpolating  pass  0  of  3
digikam.rawengine: LibRaw progress:  Interpolating  pass  1  of  3
digikam.rawengine: LibRaw progress:  Interpolating  pass  2  of  3
VNG interpolation...
*** Error in `/usr/bin/digikam': free(): corrupted unsorted chunks: 0x00007fff04049b20 ***

Thread 47 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffeebfff700 (LWP 16782)]
LibRaw::vng_interpolate (this=this@entry=0x7ffeebfa54d0) at /home/gilles/Devel/5.x/core/libs/rawengine/libraw/internal/dcraw_common.cpp:4681
4681            *ip++ = weight;
(gdb) bt
#0  0x00007ffff6c23ef5 in LibRaw::vng_interpolate() (this=this@entry=0x7ffeebfa54d0) at /home/gilles/Devel/5.x/core/libs/rawengine/libraw/internal/dcraw_common.cpp:4681
#1  0x00007ffff6bfc268 in LibRaw::dcraw_process() (this=0x7ffeebfa54d0) at /home/gilles/Devel/5.x/core/libs/rawengine/libraw/src/libraw_cxx.cpp:4143
#2  0x00007ffff6793ad1 in RawEngine::DRawDecoder::Private::loadFromLibraw(QString const&, QByteArray&, int&, int&, int&) (this=0x7fff04014c60, filePath=..., imageData=..., width=@0x7ffeebffe15c: 0, height=@0x7ffeebffe158: 0, rgbmax=@0x7ffeebffe154: 0) at /home/gilles/Devel/5.x/core/libs/rawengine/drawdecoder_p.cpp:516
#3  0x00007ffff67916ff in RawEngine::DRawDecoder::decodeRAWImage(QString const&, RawEngine::DRawDecoderSettings const&, QByteArray&, int&, int&, int&) (this=0x7ffeebffe490, filePath=..., DRawDecoderSettings=..., imageData=..., width=@0x7ffeebffe15c: 0, height=@0x7ffeebffe158: 0, rgbmax=@0x7ffeebffe154: 0) at /home/gilles/Devel/5.x/core/libs/rawengine/drawdecoder.cpp:473
#4  0x00007ffff6515af8 in Digikam::RAWLoader::load(QString const&, Digikam::DImgLoaderObserver*) (this=0x7ffeebffe490, filePath=..., observer=0x34530e0) at /home/gilles/Devel/5.x/core/libs/dimg/loaders/rawloader.cpp:113
#5  0x00007ffff6687285 in Digikam::DImg::load(QString const&, int, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (this=0x7ffeebffead0, filePath=..., loadFlagsInt=63, observer=0x34530e0, rawDecodingSettings=...)
    at /home/gilles/Devel/5.x/core/libs/dimg/dimg.cpp:497
#6  0x00007ffff668656d in Digikam::DImg::load(QString const&, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (this=0x7ffeebffead0, filePath=..., observer=0x34530e0, rawDecodingSettings=...)
    at /home/gilles/Devel/5.x/core/libs/dimg/dimg.cpp:369
#7  0x00007ffff6685843 in Digikam::DImg::DImg(QString const&, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (this=0x7ffeebffead0, filePath=..., observer=0x34530e0, rawDecodingSettings=...)
    at /home/gilles/Devel/5.x/core/libs/dimg/dimg.cpp:107
#8  0x00007ffff6740144 in Digikam::LoadingTask::execute() (this=0x34530d0) at /home/gilles/Devel/5.x/core/libs/threadimageio/loadsavetask.cpp:49
#9  0x00007ffff6730f80 in Digikam::LoadSaveThread::run() (this=0x26c55e0) at /home/gilles/Devel/5.x/core/libs/threadimageio/loadsavethread.cpp:133
#10 0x00007ffff6774e22 in Digikam::DynamicThread::DynamicThreadPriv::run() (this=0x2839f30) at /home/gilles/Devel/5.x/core/libs/threads/dynamicthread.cpp:183
#11 0x00007fffea55ebe3 in QThreadPoolThread::run() () at /lib64/libQt5Core.so.5
#12 0x00007fffea562649 in QThreadPrivate::start(void*) () at /lib64/libQt5Core.so.5
#13 0x00007ffff530566d in start_thread () at /lib64/libpthread.so.0
#14 0x00007fffe53c8ded in clone () at /lib64/libc.so.6
(gdb)
Comment 8 Maik Qualmann 2016-08-23 21:09:58 UTC
OK, yes is also to reproduce here. One should already used VNG interpolation...

Maik
Comment 9 Colin Walls 2016-09-19 13:09:00 UTC
Exactly the same error importing a Nikon NEF from a D7200.
Comment 10 caulier.gilles 2016-11-01 17:11:53 UTC
Git commit 2c67e6ffd23d8e7fda2704e875378b7f2da20905 by Gilles Caulier.
Committed on 01/11/2016 at 17:00.
Pushed by cgilles into branch 'master'.

Internal Libraw updated to 0.18-beta1 with 78 cameras added,
floating point DNG support, decode exotic DNG formats e.g. 8-bit DNG,
and more metadata parsed while decoding as white balance presets,
DNG colordata, vendor specific metadata.

See Libraw announcement for details : http://www.libraw.org/news/libraw-0-18-beta1
Related: bug 368862, bug 328321, bug 257737, bug 347010

FIXED-IN: 5.3.0

M  +5    -0    NEWS
M  +1    -5    libs/rawengine/libraw/COPYRIGHT
M  +230  -235  libs/rawengine/libraw/Changelog.txt
M  +0    -0    libs/rawengine/libraw/LICENSE.CDDL
M  +0    -0    libs/rawengine/libraw/LICENSE.LGPL
M  +4    -5    libs/rawengine/libraw/README
A  +34   -0    libs/rawengine/libraw/README.DNGSDK.txt
M  +1    -4    libs/rawengine/libraw/internal/aahd_demosaic.cpp
M  +1682 -1038 libs/rawengine/libraw/internal/dcraw_common.cpp
M  +1    -4    libs/rawengine/libraw/internal/dcraw_fileio.cpp
M  +3    -6    libs/rawengine/libraw/internal/defines.h
M  +1    -10   libs/rawengine/libraw/internal/demosaic_packs.cpp
M  +1    -4    libs/rawengine/libraw/internal/dht_demosaic.cpp
M  +16   -6    libs/rawengine/libraw/internal/libraw_internal_funcs.h
M  +1080 -551  libs/rawengine/libraw/internal/libraw_x3f.cpp
M  +1    -3    libs/rawengine/libraw/internal/var_defines.h
M  +1    -10   libs/rawengine/libraw/internal/wf_filtering.cpp
M  +29   -21   libs/rawengine/libraw/libraw/libraw.h
M  +1    -4    libs/rawengine/libraw/libraw/libraw_alloc.h
M  +10   -32   libs/rawengine/libraw/libraw/libraw_const.h
M  +5    -8    libs/rawengine/libraw/libraw/libraw_datastream.h
M  +2    -6    libs/rawengine/libraw/libraw/libraw_internal.h
M  +67   -28   libs/rawengine/libraw/libraw/libraw_types.h
M  +3    -6    libs/rawengine/libraw/libraw/libraw_version.h
M  +1    -5    libs/rawengine/libraw/samples/4channels.cpp
M  +51   -10   libs/rawengine/libraw/samples/dcraw_emu.cpp
M  +1    -5    libs/rawengine/libraw/samples/dcraw_half.c
M  +1    -4    libs/rawengine/libraw/samples/half_mt.c
M  +1    -4    libs/rawengine/libraw/samples/half_mt_win32.c
M  +1    -4    libs/rawengine/libraw/samples/mem_image.cpp
M  +1    -5    libs/rawengine/libraw/samples/multirender_test.cpp
M  +1    -6    libs/rawengine/libraw/samples/postprocessing_benchmark.cpp
M  +124  -20   libs/rawengine/libraw/samples/raw-identify.cpp
M  +16   -5    libs/rawengine/libraw/samples/simple_dcraw.cpp
M  +2    -5    libs/rawengine/libraw/samples/unprocessed_raw.cpp
M  +38   -3    libs/rawengine/libraw/src/libraw_c_api.cpp
M  +458  -100  libs/rawengine/libraw/src/libraw_cxx.cpp
M  +1    -4    libs/rawengine/libraw/src/libraw_datastream.cpp
A  +811  -0    libs/rawengine/libraw/src/libraw_xtrans_compressed.cpp     [License: UNKNOWN]  *

The files marked with a * at the end have a non valid license. Please read: http://techbase.kde.org/Policies/Licensing_Policy and use the headers which are listed at that page.


http://commits.kde.org/digikam/2c67e6ffd23d8e7fda2704e875378b7f2da20905