Bug 426597

Summary: stack underflow crash in Digikam::DImg::load()
Product: [Applications] digikam Reporter: K D Murray <foss>
Component: Plugin-DImg-RAWAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: crash CC: caulier.gilles, metzpinguin
Priority: NOR    
Version: 7.2.0   
Target Milestone: ---   
Platform: Debian unstable   
OS: Linux   
Latest Commit: Version Fixed In: 7.2.0
Sentry Crash Report:

Description K D Murray 2020-09-16 11:55:03 UTC
SUMMARY

A stack underflow crash occurs with both 7.1.0 and the current master branch from git when compiled on debain sid. Traceback below, with asan. Appears to be in QString::QString() as part of the DRawInfo() ctor, while doing Digikam::DImg::load()


STEPS TO REPRODUCE
1. Run digikam, and trigger loading any RAW file (in my case CR2)

NB: i've re-compiled this with -fsanitize=address to hopefully help debugging, but the error initially occured without this flag. 


OBSERVED RESULT

```
digikam.general: Using  16  CPU core to run threads                                                                                                                                   
digikam.general: Action Thread run  1  new jobs                                                                                                                                       
digikam.general: Cancel Main Thread                                                                                                                                                                                                                                                                                                                                         
digikam.general: One job is done                                                                                                                                                                                                                                                                                                                                            
digikam.general: Cancel Main Thread                                                                                                                                                                                                                                                                                                                                         
digikam.database: Starting scan!                                                                                                                                                                                                                                                                                                                                            
digikam.general: Stacked View Mode :  1                                                                                                                                                                                                                                                                                                                                     
digikam.metaengine: index     :  0                                                                                                                                                    
digikam.metaengine: properties:  3                                                                                                                                                                                                                             
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.                                                                                                                                                                              
digikam.metaengine: index     :  0                                                                                                                                                                                                                                                                                                                                          
digikam.metaengine: properties:  3                                                                                                                                                                                                                             
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.                                              
digikam.general: Stacked View Mode :  1                                                                            
digikam.metaengine: index     :  0                                                                                 
digikam.metaengine: properties:  3                             
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.                                                                                                                                                      
digikam.metaengine: index     :  0                                                                                             
digikam.metaengine: properties:  3                                                                                 
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.                                                                                                                                                                              
digikam.general: Stacked View Mode :  1                                                                            
digikam.metaengine: index     :  0                                                                                                                                                                                                                             
digikam.metaengine: properties:  3                                                                                                                                                                                                                             
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.                                                                                                                                                                              
digikam.metaengine: index     :  0                                                                                 
digikam.metaengine: properties:  3                                                                                 
digikam.metaengine: Exif color-space tag is sRGB. Using default sRGB ICC profile.                                  
digikam.general: Shortcut value:  1                                                                                                                                                                                                                            
digikam.general: Detected change, triggering rescan of "/home/kevin/photos/library/2020/2020-09-99_around-home/todo/"                                                                                                                                          
digikam.general: Writing tags                                                                                                                                                                                                                                  
digikam.metaengine: MetaEngine::metadataWritingMode 3                                                                          
digikam.metaengine: Will write Metadata to file "/home/kevin/photos/library/2020/2020-09-99_around-home/todo/029A0172.CR2"                                                                                                                                     
digikam.metaengine: "029A0172.CR2" is a TIFF based RAW file,  writing to such a file is disabled by current settings.                                                                                                                                          
digikam.metaengine: Will write XMP sidecar for file "029A0172.CR2"                                                                                                                                                                                                                         
digikam.general: Detected change, triggering rescan of "/home/kevin/photos/library/2020/2020-09-99_around-home/todo/"                                                                                                                                          
digikam.metaengine: wroteComment:  false                                                                                       
digikam.metaengine: wroteEXIF:  true                                                                                           
digikam.metaengine: wroteIPTC:  true                                                                                           
digikam.metaengine: wroteXMP:  true                                                                                                                                                                                                                                                                                           
digikam.metaengine: Metadata for file "029A0172.CR2" written to XMP sidecar.                                                   
digikam.dimg: "/home/kevin/photos/library/2020/2020-09-99_around-home/todo/029A0172.CR2" : "RAW" file identified               
=================================================================                                                              
==1495583==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7f4eeb83eca0 at pc 0x7f4f3f5c5f72 bp 0x7f4eeb83e9b0 sp 0x7f4eeb83e9a8                                                                                                                  
WRITE of size 8 at 0x7f4eeb83eca0 thread T72 (Thread (pooled))                                                                               
    #0 0x7f4f3f5c5f71 in QString::QString() (/home/kevin/.homedir/opt/dk-compiled/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0xd95f71)                                                                                                                       
    #1 0x7f4f3fe3d2b9 in Digikam::DRawInfo::DRawInfo() (/home/kevin/.homedir/opt/dk-compiled/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x160d2b9)                                                                                                           
    #2 0x7f4efc56bcd0 in DigikamRAWDImgPlugin::DImgRAWLoader::load(QString const&, Digikam::DImgLoaderObserver*) (/usr/lib/x86_64-linux-gnu/qt5/plugins/digikam/dimg/DImg_RAW_Plugin.so+0x7cd0)                                                                
    #3 0x7f4f3d7b689f  (/lib/x86_64-linux-gnu/libQt5Core.so.5+0x38289f)                                                                                        
                                                                                                                                                               
Address 0x7f4eeb83eca0 is located in stack of thread T72 (Thread (pooled)) at offset 0 in frame                                                                                                                                                                                                                                                                             
    #0 0x7f4f3f9d3c2d in Digikam::DImg::load(QString const&, int, Digikam::DImgLoaderObserver*, Digikam::DRawDecoding const&) (/home/kevin/.homedir/opt/dk-compiled/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x11a3c2d)                                    
                                                                                                                   
  This frame has 37 object(s):                                                                                                                                                        
    [32, 36) 'loadFlags' (line 112)                                                                                                                                                   
    [48, 52) '<unknown>'                                                                                                                                                              
    [64, 68) '<unknown>'                                                                                           
    [80, 84) '<unknown>'                                                                                                                                                                                                                                                                                                                                                    
    [96, 100) '<unknown>'                                                                                                      
    [112, 116) '<unknown>'                                                                                                                                                            
    [128, 136) 'fileInfo' (line 103)                                                                                                                                                  
    [160, 168) '<unknown>'                                                                                                                                                            
    [192, 200) '<unknown>'                                                                                                     
    [224, 232) 'lock' (line 115)                                                                                                                                                      
    [256, 264) '<unknown>'                                                                                                                                                            
    [288, 296) '<unknown>'                                                                                                     
    [320, 328) '<unknown>'                                                                                                                                                            
    [352, 360) '<unknown>'                                                                                                                                                            
    [384, 392) '<unknown>'                                                                                                                                                            
    [416, 424) '<unknown>'                                                                                                     
    [448, 456) '<unknown>'                                                                                                                                                            
    [480, 488) '<unknown>'                                                                                                     
    [512, 520) '<unknown>'                                                                                                     
    [544, 552) '<unknown>'                                                                                                                   
    [576, 584) '<unknown>'                                                                                                                                                                                                                                     
    [608, 616) '<unknown>'                                                     
    [640, 656) '<unknown>'                                                                                                     
    [672, 688) '<unknown>'                                                                                                     
    [704, 720) '<unknown>'                                                                                                                                     
    [736, 752) '<unknown>'                                                                                                     
    [768, 784) '<unknown>'                                                                                                     
    [800, 816) '<unknown>'                                                                                                     
    [832, 848) '<unknown>'                                                                                                     
    [864, 880) '<unknown>'                                                                                                     
    [896, 912) '<unknown>'                                                                                                     
    [928, 944) '<unknown>'                                                                                                                                                            
    [960, 992) '<unknown>'                                                                                                     
    [1024, 1056) '<unknown>'                                                                                                   
    [1088, 1120) '<unknown>'                                                               
    [1152, 1184) '<unknown>'                                                                                                                                                          
    [1216, 1248) '<unknown>'                                                               
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork               
      (longjmp and C++ exceptions *are* supported)                                                                                                                                    
Thread T72 (Thread (pooled)) created by T0 here:                                           
    #0 0x7f4f450c52a2 in __interceptor_pthread_create (/lib/x86_64-linux-gnu/libasan.so.6+0x552a2)                             
    #1 0x7f4f3d4fc4da in QThread::start(QThread::Priority) (/lib/x86_64-linux-gnu/libQt5Core.so.5+0xc84da)                     
                                                                                                                                                                                      
SUMMARY: AddressSanitizer: stack-buffer-underflow (/home/kevin/.homedir/opt/dk-compiled/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0xd95f71) in QString::QString()                                                                                           
Shadow bytes around the buggy address:                                                                                         
  0x0fea5d6ffd40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                              
  0x0fea5d6ffd50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                              
  0x0fea5d6ffd60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                              
  0x0fea5d6ffd70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                              
  0x0fea5d6ffd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                              
=>0x0fea5d6ffd90: 00 00 00 00[f1]f1 f1 f1 04 f2 f8 f2 f8 f2 04 f2                                                              
  0x0fea5d6ffda0: 04 f2 04 f2 00 f2 f2 f2 00 f2 f2 f2 f8 f2 f2 f2                                                              
  0x0fea5d6ffdb0: 00 f2 f2 f2 f8 f2 f2 f2 f8 f2 f2 f2 f8 f2 f2 f2                                                              
  0x0fea5d6ffdc0: f8 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2                                                              
  0x0fea5d6ffdd0: 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2                                                              
  0x0fea5d6ffde0: 00 f2 f2 f2 f8 f8 f2 f2 f8 f8 f2 f2 f8 f8 f2 f2                                                              
Shadow byte legend (one shadow byte represents 8 application bytes):                                                           
  Addressable:           00                                                                                                    
  Partially addressable: 01 02 03 04 05 06 07                                                                                  
  Heap left redzone:       fa                                                                                                  
  Freed heap region:       fd                                                                                                  
  Stack left redzone:      f1                                                                                                  
  Stack mid redzone:       f2                                                                                                  
  Stack right redzone:     f3                                                                                                  
  Stack after return:      f5                                                                                                  
  Stack use after scope:   f8                                                                                                  
  Global redzone:          f9                                                                                                  
  Global init order:       f6                                                                                                  
  Poisoned by user:        f7                                                                                                  
  Container overflow:      fc                                                                                                  
  Array cookie:            ac                                                                                                  
  Intra object redzone:    bb                                                                                                  
  ASan internal:           fe                                                                                                  
  Left alloca redzone:     ca                                                                                                  
  Right alloca redzone:    cb                                                                                                  
  Shadow gap:              cc                                                                                                  
==1495583==ABORTING
```


EXPECTED RESULT

No crash.


SOFTWARE/OS VERSIONS
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 5.17.5
KDE Frameworks Version: 20.04.1 / 5.70.0
Qt Version: 5.14

ADDITIONAL INFORMATION
Comment 1 Maik Qualmann 2020-09-16 14:09:46 UTC
Well, some QStrings are put on the stack by DRawInfo. Some strings may be significant in size. What does the output of "ulimit -s" show in your system?

Maik
Comment 2 K D Murray 2020-09-16 22:59:21 UTC
Hi Maik

$ ulimit -s
8192

It would seem that

$ ulimit -s 65535
$ digikam

makes it not crash. I'll update/reopen this bug report if it starts crashing again. It would be great if digikam could increase the stack size, e.g. using setrlimit on linux if needing a larger stack size is a known issue. I'm happy to provide a patch if this is something you'd like included. 


Cheers,
Kevin
Comment 3 Maik Qualmann 2020-09-17 05:41:32 UTC
I open the bug again, a stack size of 8MB is a normal value for Linux and other operating systems (MacOS) are much less available. We should either create DRawInfo on the heap or find out which string that is passed by libraw is extremely long here. Can you provide the image (029A0172.CR2) that leads to the crash?

Maik
Comment 4 caulier.gilles 2020-09-19 10:29:11 UTC
Git commit 2601c0a8adbd51a3828353e24f5bc169260eb850 by Gilles Caulier.
Committed on 19/09/2020 at 10:27.
Pushed by cgilles into branch 'master'.

Use heap to create DRawInfo instance to prevent stack overflow
Related: bug 426175

M  +3    -3    core/libs/threadimageio/preview/previewtask.cpp

https://invent.kde.org/graphics/digikam/commit/2601c0a8adbd51a3828353e24f5bc169260eb850
Comment 5 caulier.gilles 2020-09-19 10:50:13 UTC
Git commit 2ed30b67282e725671e65f5499c6293dc0de7070 by Gilles Caulier.
Committed on 19/09/2020 at 10:50.
Pushed by cgilles into branch 'master'.

Use heap to create DRawInfo instance to prevent stack overflow
Related: bug 426175

M  +20   -18   core/dplugins/generic/tools/htmlgallery/generator/galleryelementfunctor.cpp

https://invent.kde.org/graphics/digikam/commit/2ed30b67282e725671e65f5499c6293dc0de7070
Comment 6 caulier.gilles 2020-09-19 10:52:49 UTC
Git commit edf82fbaf85e4131b3e39e4f787e9820a1e273ee by Gilles Caulier.
Committed on 19/09/2020 at 10:52.
Pushed by cgilles into branch 'master'.

Use heap to create DRawInfo instance to prevent stack overflow
Related: bug 426175

M  +5    -5    core/dplugins/dimg/raw/dimgrawloader.cpp

https://invent.kde.org/graphics/digikam/commit/edf82fbaf85e4131b3e39e4f787e9820a1e273ee
Comment 7 caulier.gilles 2020-09-19 10:55:00 UTC
Git commit f7521ff595caaee5a85fdd75354ac78a753e39dd by Gilles Caulier.
Committed on 19/09/2020 at 10:54.
Pushed by cgilles into branch 'master'.

Use heap to create DRawInfo instance to prevent stack overflow
Related: bug 426175

M  +80   -80   core/libs/metadataengine/dmetadata/dmetadata_libraw.cpp

https://invent.kde.org/graphics/digikam/commit/f7521ff595caaee5a85fdd75354ac78a753e39dd
Comment 8 caulier.gilles 2020-09-19 10:58:53 UTC
Git commit e662dbc0caea17d673cd84966655fe29fcc50828 by Gilles Caulier.
Committed on 19/09/2020 at 10:58.
Pushed by cgilles into branch 'master'.

Use heap to create DRawInfo instance to prevent stack overflow
Related: bug 426175

M  +1    -1    core/libs/rawengine/drawdecoder.cpp
M  +12   -8    core/libs/rawengine/drawdecoder_p.cpp

https://invent.kde.org/graphics/digikam/commit/e662dbc0caea17d673cd84966655fe29fcc50828
Comment 9 caulier.gilles 2020-09-19 11:18:56 UTC
Git commit 605e6875ca4386291a0b681ec26db9db0b305757 by Gilles Caulier.
Committed on 19/09/2020 at 11:18.
Pushed by cgilles into branch 'master'.

Use heap to create DRawInfo instance to prevent stack overflow
Related: bug 426175

M  +99   -99   core/libs/dngwriter/dngwriter_convert.cpp
M  +2    -0    core/libs/dngwriter/dngwriter_p.h

https://invent.kde.org/graphics/digikam/commit/605e6875ca4386291a0b681ec26db9db0b305757
Comment 10 caulier.gilles 2020-09-19 11:25:55 UTC
Git commit 96bc53a0682968ad3df03c698bf3e9bacb3a665d by Gilles Caulier.
Committed on 19/09/2020 at 11:25.
Pushed by cgilles into branch 'master'.

Use heap to create DRawInfo instance to prevent stack overflow
Related: bug 426175

M  +2    -1    core/dplugins/dimg/raw/dimgrawloader.cpp
M  +4    -2    core/dplugins/generic/tools/htmlgallery/generator/galleryelementfunctor.cpp
M  +2    -1    core/libs/metadataengine/dmetadata/dmetadata_libraw.cpp
M  +2    -1    core/libs/rawengine/drawdecoder_p.cpp
M  +2    -1    core/libs/threadimageio/preview/previewtask.cpp
M  +16   -13   core/tests/rawengine/raw2png.cpp

https://invent.kde.org/graphics/digikam/commit/96bc53a0682968ad3df03c698bf3e9bacb3a665d
Comment 11 caulier.gilles 2020-09-19 11:36:52 UTC
Git commit a1deb248f5e457b5abef17d207cb16daef1237b9 by Gilles Caulier.
Committed on 19/09/2020 at 11:36.
Pushed by cgilles into branch 'master'.

Use heap to create DRawDecoder instance to prevent stack overflow
Related: bug 426175

M  +1    -1    core/libs/progressmanager/workingwidget.cpp
M  +3    -2    core/libs/rawengine/drawdecoder.cpp
M  +1    -0    core/libs/threadimageio/preview/previewtask.cpp
M  +19   -6    core/tests/multithreading/myactionthread.cpp

https://invent.kde.org/graphics/digikam/commit/a1deb248f5e457b5abef17d207cb16daef1237b9
Comment 12 caulier.gilles 2020-09-19 11:43:12 UTC
Git commit 0507f9712e4cd76bcbe3aa4cec690ecb25e3fe9e by Gilles Caulier.
Committed on 19/09/2020 at 11:42.
Pushed by cgilles into branch 'master'.

Use heap to create DMetadata instance to prevent stack overflow
Related: bug 426175

M  +2    -2    core/app/main/digikamapp.cpp
M  +1    -0    core/app/main/digikamapp_p.h
M  +11   -10   core/app/views/stack/mapwidgetview.cpp

https://invent.kde.org/graphics/digikam/commit/0507f9712e4cd76bcbe3aa4cec690ecb25e3fe9e
Comment 13 caulier.gilles 2020-09-19 11:52:26 UTC
Kevin,

Please check if with my last commits to instantiate DRawInfo and DRawDecoder on heap, your memory allocation problem is fixed.

Thanks in advance

Gilles Caulier
Comment 14 K D Murray 2020-09-19 13:00:55 UTC
Gilles,

Many thanks for the patches. It does seem to have fixed the immediate cause of my crash. 

However, now with ASAN on, I'm getting a buffer overflow in LibRaw. Crash below, not sure why the line numbers aren't showing, i'm using CMAKE_BUILD_TYPE=Debug.

Cheers,
Kevin


==948374==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fffa8693892 at pc 0x7ffff2353fbf bp 0x7fffa86937d0 sp 0x7fffa86937c8                                                                                                                    
READ of size 1 at 0x7fffa8693892 thread T55 (Thread (pooled))                                                                                                                                                                                                  
    #0 0x7ffff2353fbe in LibRaw::tiff_set(tiff_hdr*, unsigned short*, unsigned short, unsigned short, int, int) [clone .constprop.0] (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x15bffbe)         
    #1 0x7ffff2355857 in LibRaw::tiff_head(tiff_hdr*, int) (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x15c1857)                                                                                                           
    #2 0x7ffff231acc3 in LibRaw::dcraw_make_mem_thumb(int*) (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x1586cc3)                                                                                                          
    #3 0x7ffff237f9dd in Digikam::DRawDecoder::Private::loadEmbeddedPreview(QByteArray&, LibRaw*) (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x15eb9dd)                                                                                                
    #4 0x7ffff236feb4 in Digikam::DRawDecoder::loadEmbeddedPreview(QByteArray&, QString const&) (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x15dbeb4)                                              
    #5 0x7ffff236f3f0 in Digikam::DRawDecoder::loadEmbeddedPreview(QImage&, QString const&) (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x15db3f0)                                                                                                      
    #6 0x7ffff212325f in Digikam::ThumbnailCreator::createThumbnail(Digikam::ThumbnailInfo const&, QRect const&) const (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x138f25f)                       
    #7 0x7ffff2117760 in Digikam::ThumbnailCreator::load(Digikam::ThumbnailIdentifier const&, QRect const&, bool) const (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x1383760)                                              
    #8 0x7ffff211646c in Digikam::ThumbnailCreator::load(Digikam::ThumbnailIdentifier const&) const (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x138246c)                                                                                                                                 
    #9 0x7ffff213906f in Digikam::ThumbnailLoadingTask::execute() (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x13a506f)                                                                                                    
    #10 0x7ffff213bee2 in Digikam::LoadSaveThread::run() (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x13a7ee2)                                                                                                                                                                            
    #11 0x7ffff219e15a in Digikam::DynamicThread::Private::run() (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x140a15a)                                                                                                     
    #12 0x7fffefa64691  (/lib/x86_64-linux-gnu/libQt5Core.so.5+0xcc691)                                            
    #13 0x7fffefa60a00  (/lib/x86_64-linux-gnu/libQt5Core.so.5+0xc8a00)                                                                                                                                                                                        
    #14 0x7fffef5caea6 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8ea6)                                                                                                                                                                          
    #15 0x7fffef6e7eae in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfdeae)                                        
                                                                                                                   
Address 0x7fffa8693892 is located in stack of thread T55 (Thread (pooled)) at offset 50 in frame                   
    #0 0x7ffff235487f in LibRaw::tiff_head(tiff_hdr*, int) (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x15c087f)                                                                                                           
                                                                                                                   
  This frame has 2 object(s):                                                                                                                                                                                                                                  
    [48, 50) 'latref' (line 123) <== Memory access at offset 50 overflows this variable                                        
    [64, 66) 'lonref' (line 124)                                                                                                                                                                                                                                                           
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork                                                                                                                                                                                                              
      (longjmp and C++ exceptions *are* supported)                                                                                                                                                                                                                                         
Thread T55 (Thread (pooled)) created by T0 here:                                                                               
    #0 0x7ffff76202a2 in __interceptor_pthread_create (/lib/x86_64-linux-gnu/libasan.so.6+0x552a2)                             
    #1 0x7fffefa604da in QThread::start(QThread::Priority) (/lib/x86_64-linux-gnu/libQt5Core.so.5+0xc84da)                                                                                                                                                                                                                    
                                                                                                                               
SUMMARY: AddressSanitizer: stack-buffer-overflow (/home/kevin/.homedir/opt/digikam/lib/x86_64-linux-gnu/libdigikamcore.so.7.2.0+0x15bffbe) in LibRaw::tiff_set(tiff_hdr*, unsigned short*, unsigned short, unsigned short, int, int) [clone .constprop.0]                                                                     
Shadow bytes around the buggy address:                                                                                         
  0x1000750ca6c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                  
  0x1000750ca6d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                  
  0x1000750ca6e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                                                                                                                                      
  0x1000750ca6f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                  
  0x1000750ca700: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1                                                                                                                                                                      
=>0x1000750ca710: f1 f1[02]f2 02 f3 f3 f3 00 00 00 00 00 00 00 00                                                                                                                                                                      
  0x1000750ca720: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 02 f2 f2                                                                                                                                                                      
  0x1000750ca730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                                                                                                                                      
  0x1000750ca740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                                                                                                                                      
  0x1000750ca750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                                                                                                                                      
  0x1000750ca760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00                                                                                                                                                                      
Shadow byte legend (one shadow byte represents 8 application bytes):                                                                                                                                                                   
  Addressable:           00                                                                                                                                                                                                            
  Partially addressable: 01 02 03 04 05 06 07                                                                                                                                                                                          
  Heap left redzone:       fa                                                                                                                                                                                                          
  Freed heap region:       fd                                  
  Stack left redzone:      f1                                  
  Stack mid redzone:       f2                                  
  Stack right redzone:     f3                                  
  Stack after return:      f5                                  
  Stack use after scope:   f8                                  
  Global redzone:          f9                                  
  Global init order:       f6                                  
  Poisoned by user:        f7                                  
  Container overflow:      fc                                  
  Array cookie:            ac                                  
  Intra object redzone:    bb                                  
  ASan internal:           fe                                  
  Left alloca redzone:     ca                                         
  Right alloca redzone:    cb                                         
  Shadow gap:              cc                                         
==948374==ABORTING
Comment 15 caulier.gilles 2020-09-19 13:22:05 UTC
Ha, this part of code is just imported from libraw project as well in digiKam core. The trace is clear : this memory corruption is done when you extract a preview image from Raw, not while Raw demosaicing.

For this kind of ASAN report, please :

1/ try to identify which Raw image introduce this memory leak,

2/ Use libraw CLi tool to extract preview image. No need to checkout libraw code outside digiKam, we compile all libraw CLI tool in digiKam tests suite. You will found all CLI tools in your build directory. Right tool is this one with "-e" option :

[gilles@pc-gilles rawengine]$ pwd
/mnt/devel/GIT/7.x/build/core/tests/rawengine
[gilles@pc-gilles rawengine]$ ./simple_dcraw 
simple_dcraw - LibRaw 0.20.0-Release sample. Emulates dcraw [-D] [-T] [-v] [-e] [-E]
 1134 cameras supported
Usage: ./simple_dcraw [-D] [-T] [-v] [-e] raw-files....
        -4 - 16-bit mode
        -L - list supported cameras and exit
        -v - verbose output
        -T - output TIFF files instead of .pgm/ppm
        -e - extract thumbnails (same as dcraw -e in separate run)
[gilles@pc-gilles rawengine]$ 

3/ Look if ASAN dysfunction still exists

4/ Report this upstream problem to libraw team : https://github.com/LibRaw/LibRaw/issues

Thanks in advance

Gilles Caulier
Comment 16 K D Murray 2020-09-19 13:37:57 UTC
OK, thanks Gilles, will do.

Any hints you have to easily try step 1?

> 1/ try to identify which Raw image introduce this memory leak,


Cheers,
Kevin
Comment 17 caulier.gilles 2020-09-19 14:04:32 UTC
Git commit 0815e2d88a9d3c69bca6f6fb8e0500a04c01d863 by Gilles Caulier.
Committed on 19/09/2020 at 14:03.
Pushed by cgilles into branch 'master'.

More debug trace to identfy raw file processed with preview extraction

M  +3    -1    core/libs/rawengine/drawdecoder.cpp

https://invent.kde.org/graphics/digikam/commit/0815e2d88a9d3c69bca6f6fb8e0500a04c01d863
Comment 18 caulier.gilles 2020-09-19 14:07:55 UTC
First it's clear, raw preview is only called with RAW files.

If you use one kind of camera to generate RAW, well it's simple. try with some RAW sample from your collection.

NEF for ex are based on TIFF/EP with Nikon customization. DNG is a pure TIFF/EP format too.

Else, the RAW preview is called to render thumbnails (your case here) or to show JPEG embedded preview when you press F3 in icon view. And yes, RAW container has smaller JPEG version inside. This permit to show quickly the RAW content with to process a complex demosaicing.

To identify the file, the Preview engine, which call libraw API must print the file in process on the console. You need to enable all debug traces on the console with this env. variable before to run digiKam :

export QT_LOGGING_RULES="digikam*=true"

With my last commit you must see something like this:

digikam.rawengine: LibRaw: loadEmbeddedPreview from "/mnt/data/photos/GILLES/NEW/HDR/img_1720.cr2"

Gilles Caulier
Comment 19 K D Murray 2020-09-19 23:19:00 UTC
OK, so now I get a possibly related issue:

Clicking on any CR2 leads to "Failed to load image" message in GUI, and this in console with debug logging on:


```
digikam.general: Try to get preview from "/home/kevin/photos/library/2020/2020-09-15_tidbinbilla/2020-09-15_12-44-37_029A0472.CR2"
digikam.general: Preview quality:  2
digikam.dimg: "/home/kevin/photos/library/2020/2020-09-15_tidbinbilla/2020-09-15_12-44-37_029A0472.CR2" : Unknown image format !!!
digikam.general: Cannot extract preview for "/home/kevin/photos/library/2020/2020-09-15_tidbinbilla/2020-09-15_12-44-37_029A0472.CR2"
digikam.general: Stacked View Mode :  1
```


Also seems as though it's failing to open or even create thumbnails of any JPEG.

Cheers,
K
Comment 20 caulier.gilles 2020-09-20 10:50:35 UTC
Git commit c96ede534820d3037aac1cdb1c65ad3b49db4b03 by Gilles Caulier.
Committed on 20/09/2020 at 10:48.
Pushed by cgilles into branch 'master'.

backport libraw source code from git/master rev. 3f701019d5abb44565229d5036ba0bf41a2d57a3

M  +1    -1    core/libs/rawengine/libraw/samples/dcraw_emu.cpp
M  +24   -29   core/libs/rawengine/libraw/samples/raw-identify.cpp
M  +1    -1    core/libs/rawengine/libraw/src/decoders/load_mfbacks.cpp
M  +1    -1    core/libs/rawengine/libraw/src/metadata/canon.cpp
M  +2    -2    core/libs/rawengine/libraw/src/metadata/ciff.cpp
M  +11   -1    core/libs/rawengine/libraw/src/metadata/cr3_parser.cpp
M  +19   -0    core/libs/rawengine/libraw/src/metadata/exif_gps.cpp
M  +26   -13   core/libs/rawengine/libraw/src/metadata/hasselblad_model.cpp
M  +11   -8    core/libs/rawengine/libraw/src/metadata/identify.cpp
M  +6    -4    core/libs/rawengine/libraw/src/metadata/misc_parsers.cpp
M  +1    -5    core/libs/rawengine/libraw/src/metadata/normalize_model.cpp
M  +1    -1    core/libs/rawengine/libraw/src/metadata/olympus.cpp
M  +5    -4    core/libs/rawengine/libraw/src/preprocessing/raw2image.cpp
M  +3    -3    core/libs/rawengine/libraw/src/tables/cameralist.cpp
M  +2    -2    core/libs/rawengine/libraw/src/write/file_write.cpp

https://invent.kde.org/graphics/digikam/commit/c96ede534820d3037aac1cdb1c65ad3b49db4b03
Comment 21 caulier.gilles 2020-09-20 11:42:17 UTC
Kevin,

About comment #19, i cannot reproduce this problem with CR2 files:

https://i.imgur.com/iFvqybj.jpg

...and JPEG:

https://i.imgur.com/X2FB6sj.png

Thumbnails, preview, ad loading in editor work as expected...

Gilles Caulier
Comment 22 caulier.gilles 2020-09-20 21:34:39 UTC
Git commit ef51f605528649e9f509bd37d2710409f3bf83f0 by Gilles Caulier.
Committed on 20/09/2020 at 21:32.
Pushed by cgilles into branch 'master'.

Add new compilation option to enable compiler sanitizers ASAN and UBSAN
Related: bug 426175

M  +1    -0    Mainpage.dox
M  +6    -1    core/CMakeLists.txt
M  +3    -1    core/cmake/modules/MacroCompiler.cmake

https://invent.kde.org/graphics/digikam/commit/ef51f605528649e9f509bd37d2710409f3bf83f0
Comment 23 K D Murray 2020-09-21 11:22:23 UTC
Hi Gilles,

Do you mind sharing the OS/library versions/etc you use in your screenshots? Maybe my issue is triggered by some broken dependency from debian I can work around. Or are there nightly appimage bundles that would include these fixes?

And regarding the asan issue: yes it appears fixed after the patch from upstream. Thanks to you/them for that :)

Cheers,
Kevin
Comment 24 caulier.gilles 2020-09-21 15:33:40 UTC
Hi Kevin,

I use Linux Mageia7. My dependencies are visible on dialog from the right screen side :

https://i.imgur.com/iFvqybj.jpg

Else the Nightly AppImage bundle builds are available here :

https://files.kde.org/digikam/

I close this file now. Note that i officialy add the ASAN support in digiKam build rules. A new cmake option "ENABLE_SANITIZERS" can be used for that:

https://invent.kde.org/graphics/digikam/-/blob/master/Mainpage.dox#L461

Best regards

Gilles Caulier

Gilles