Bug 308489

Summary: raw image development produces artifacts in digikam 3.0.0-beta 2
Product: [Applications] digikam Reporter: Robert Zeller <robert.zeller>
Component: Plugin-DImg-RAWAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: major CC: caulier.gilles, kenzo.zombie, lexa
Priority: NOR    
Version: 3.0.0   
Target Milestone: ---   
Platform: openSUSE   
OS: Linux   
URL: http://http://www.robert-zeller.org/transfer/DSC0607_v1-digikam_3.0_beta.JPG
Latest Commit: Version Fixed In: 3.3.0
Attachments: Raw_File_Decoding.jpg
Brightness_Contrast_Gamma.jpg

Description Robert Zeller 2012-10-16 13:24:54 UTC
I encountered a strange problem  developing a Nikon raw file with
digikam 3.0.0-beta2 which did not occur under digikam 2.9:

Under digikam 3.0.0-beta2 (latest version from git) a pattern of a faint
rectangular lattice of fine lines occurs over the entire .JPG that was
produced from the .NEF file. This does not happen when using digikam
2.9, nor do I see this pattern when developing the raw file by
rawtherapee or darktable.

You may view the results by downloading the original NEF and the JPGs
produced by digikam 2.9 and digikam 3.0.0-beta2 from the following website:

raw-file: www.robert-zeller.org/transfer/DSC0607.NEF
JPG digikam 2.9 : www.robert-zeller.org/transfer/DSC0607_v2-digikam_2.9.JPG
JPG digikam 3.0.0-beta2:
www.robert-zeller.org/transfer/DSC0607_v1-digikam_3.0_beta.JPG


Since the web-page is not open for directory search you have to specify
the full urls when using wget for download.
Caution: the files are huge since they were taken with a NIKON D800
which has a 36 Mpix sensor.

I used the same settings in digikam 2.9 and 3.0.0 when developing the raw.
I am wondering what the reason may be for this unacceptable result in
3.0.0-beta2?

Reproducible: Always

Steps to Reproduce:
1.download test images from web site given in "Details"
2.use digikam image editor to develop Nikon raw image: DSC0607.NEF
3.
Actual Results:  
digikam-3.0.0-beta2 produces a faint rectangular lattice across the entire jpg that is produced by the image editor

Expected Results:  
no artifacts !
Comment 1 Alex Tutubalin 2012-10-16 14:12:51 UTC
What interpolation settings do you use?
Comment 2 Alex Tutubalin 2012-10-16 14:16:16 UTC
Also: cannot reproduce for LibRaw 0.15-Beta1/ dcraw_emu with standard quality settings (-q 0 ... -q 3)
Comment 3 caulier.gilles 2012-10-21 07:04:36 UTC
*** Bug 308648 has been marked as a duplicate of this bug. ***
Comment 4 caulier.gilles 2012-10-22 09:45:03 UTC
Created attachment 74723 [details]
Raw_File_Decoding.jpg

Robert,

Please provide a screenshot of all settings used to process your RAW file.

Gilles Caulier
Comment 5 Robert Zeller 2012-10-22 13:09:33 UTC
Gilles,

please find attached two screenshots of Raw_File_decoding (I used 
default in 16 bit) and of the Brightness/Contrast/Gamma settings. I 
didn't change anything else from the defaults as far as I can remember. 
If you need more information, please let me know. The bug seems to be 
persistent.

Robert


On 10/22/2012 11:45 AM, Gilles Caulier wrote:
> https://bugs.kde.org/show_bug.cgi?id=308489
>
> --- Comment #4 from Gilles Caulier <caulier.gilles@gmail.com> ---
> Robert,
>
> Please provide a screenshot of all settings used to process your RAW file.
>
> Gilles Caulier
>
Comment 6 Robert Zeller 2012-10-22 13:09:34 UTC
Created attachment 74724 [details]
Brightness_Contrast_Gamma.jpg
Comment 7 caulier.gilles 2012-10-22 13:31:03 UTC
"Setup/Raw File Decoding" is ONLY used with automatization of RAW loading data in Image Editor.

Go to "Behavior" tab and set on "Always open in Raw Import tool"

Like this, for each Raw file open in editor, you will be able to customize RAW decoding settings. Try to change Quality settings, and look preview when artifacts. Give us the list of quality settings when dysfunction appears...

Gilles Caulier
Comment 8 Robert Zeller 2012-10-22 14:39:01 UTC
Gilles,

it looks like I found out, which settings cause the artifacts: it seems 
to be the option "Interpolate RGB as four colors". When this option is 
enabled I can see the pattern of the faint rectangular grid over the 
picture. If I disable this option the artifacts disappear.
Here is the list of options I used:

Demosaicing:
16 bits color depth
Interpolate RGB as four colors.
Quality: AHD v2

White Balance:
Method: Camera
Highlights: Rebuild
Correct false colors in highlights

Corrections:
Noise reduction: Wavelets
Enable Chromatic Aberration correction

Color management:
CameraProfile: embedded
Workspace: Adobe RGB

Robert



On 10/22/2012 03:31 PM, Gilles Caulier wrote:
> https://bugs.kde.org/show_bug.cgi?id=308489
>
> --- Comment #7 from Gilles Caulier <caulier.gilles@gmail.com> ---
> "Setup/Raw File Decoding" is ONLY used with automatization of RAW loading data
> in Image Editor.
>
> Go to "Behavior" tab and set on "Always open in Raw Import tool"
>
> Like this, for each Raw file open in editor, you will be able to customize RAW
> decoding settings. Try to change Quality settings, and look preview when
> artifacts. Give us the list of quality settings when dysfunction appears...
>
> Gilles Caulier
>
Comment 9 caulier.gilles 2012-10-26 13:10:54 UTC
This file is probably in relation with bug #305382

Gilles Caulier
Comment 10 caulier.gilles 2012-10-29 11:03:23 UTC
Git commit ca11950fa5926ccfb4438d8b9cbd072370be3f0b by Gilles Caulier.
Committed on 29/10/2012 at 11:55.
Pushed by cgilles into branch 'master'.

- Use -O4 compilation option to optimize speed with dosaicing process.

- Fix artifacts produced by libraw when OpenMP is used. This problem still present in libraw and must be fixed as DOWNSTREAM
  Please confirm if probelm still reproducible on your computer with this fix.
Related: bug 305382

M  +2    -1    libraw/CMakeLists.txt

http://commits.kde.org/libkdcraw/ca11950fa5926ccfb4438d8b9cbd072370be3f0b
Comment 11 Robert Zeller 2012-10-29 14:27:37 UTC
Gilles,

I am not sure if I did it right - I am not at all an expert in cmake. I 
copied your new version of libraw/CMakeLists.txt from the git repository 
into my digikam-3.0.0-beta2/extra/libkdcraw/libraw source and recompiled 
the entire stuff. In this case the artefacts are persistent with the 
setting "interpolate RGB as four colors" during the raw decoding 
process.  I suspect, that something else has to be done. So I copied the 
entire libkdcraw directory via git clone git://anongit.kde.org/libkdcraw 
into my source. But in this case I get compilation errors:

/usr/src/packages/packages.local/digikam/digikam-3.0.0-beta2/extra/kipi-plugins/common/libkipiplugins/tools/kprawthumbthread.cpp: 
In member function ‘virtual void KIPIPlugins::KPRawThumbThread::run()’:
/usr/src/packages/packages.local/digikam/digikam-3.0.0-beta2/extra/kipi-plugins/common/libkipiplugins/tools/kprawthumbthread.cpp:112:24: 
error: ‘loadDcrawPreview’ is not a member of ‘KDcrawIface::KDcraw’
make[2]: *** 
[extra/kipi-plugins/common/libkipiplugins/CMakeFiles/kipiplugins.dir/tools/kprawthumbthread.cpp.o] 
Error 1
make[1]: *** 
[extra/kipi-plugins/common/libkipiplugins/CMakeFiles/kipiplugins.dir/all] Error 
2
make: *** [all] Error 2

I think the best would be if you could point me to a complete source of 
digikam that contains the changes already and which can be compiled by 
the standard cmake / make procedures.

Robert


On 10/29/2012 12:03 PM, Gilles Caulier wrote:
> https://bugs.kde.org/show_bug.cgi?id=308489
>
> --- Comment #10 from Gilles Caulier <caulier.gilles@gmail.com> ---
> Git commit ca11950fa5926ccfb4438d8b9cbd072370be3f0b by Gilles Caulier.
> Committed on 29/10/2012 at 11:55.
> Pushed by cgilles into branch 'master'.
>
> - Use -O4 compilation option to optimize speed with dosaicing process.
>
> - Fix artifacts produced by libraw when OpenMP is used. This problem still
> present in libraw and must be fixed as DOWNSTREAM
>    Please confirm if probelm still reproducible on your computer with this fix.
> Related: bug 305382
>
> M  +2    -1    libraw/CMakeLists.txt
>
> http://commits.kde.org/libkdcraw/ca11950fa5926ccfb4438d8b9cbd072370be3f0b
>
Comment 12 caulier.gilles 2012-10-29 17:04:52 UTC
Robert,

Do not try to compile current implementation of libkdcraw with beta2. I changed API, for other reason, and digiKAm/Kipi-plugins from beta2 will not compile with it. 

Look my fix it's very simple. I just add -O4 option to compile libraw source code. Take libkdcraw source code from beta2 and just add -O4 option at the right place. In CMakeLists.txt you must find something like that :

FOREACH(_curentfile ${libraw_LIB_SRCS})
    IF(WIN32 AND MSVC)
        SET_SOURCE_FILES_PROPERTIES(${_curentfile} PROPERTIES COMPILE_FLAGS "-w")
    ELSE(WIN32 AND MSVC)
        SET_SOURCE_FILES_PROPERTIES(${_curentfile} PROPERTIES COMPILE_FLAGS "-w -fPIC")
    ENDIF(WIN32 AND MSVC)
ENDFOREACH(_curentfile ${libraw_LIB_SRCS})

In line including -fPIC option, just add -O4 at end. Recompile all, install, and test

Gilles Caulier
Comment 13 Robert Zeller 2012-10-29 18:12:09 UTC
Gilles,

I re-downloaded digikam-3.0.0-beta2; and changed CMakelists.txt in 
extra/libkdcraw : I added -O4 in the options right after -fPIC and 
recompiled the source.
With the demosaic option "Interpolate rgb as four colors" turned on, I 
don't see the artefacts any more in the sample picture that I provided 
to you. BUT: on another raw-picture the faint rectangular pattern is 
still there, which is very strange. So my guess is: -O4 does not remove 
completely the problem. It is clearly dependent on turning on the option 
"Interpolate rgb as four colors". Do you want me to provide the other 
raw-picture that still evaluates the dysfunction?

Robert


On 10/29/2012 06:04 PM, Gilles Caulier wrote:
> https://bugs.kde.org/show_bug.cgi?id=308489
>
> --- Comment #12 from Gilles Caulier <caulier.gilles@gmail.com> ---
> Robert,
>
> Do not try to compile current implementation of libkdcraw with beta2. I changed
> API, for other reason, and digiKAm/Kipi-plugins from beta2 will not compile
> with it.
>
> Look my fix it's very simple. I just add -O4 option to compile libraw source
> code. Take libkdcraw source code from beta2 and just add -O4 option at the
> right place. In CMakeLists.txt you must find something like that :
>
> FOREACH(_curentfile ${libraw_LIB_SRCS})
>      IF(WIN32 AND MSVC)
>          SET_SOURCE_FILES_PROPERTIES(${_curentfile} PROPERTIES COMPILE_FLAGS
> "-w")
>      ELSE(WIN32 AND MSVC)
>          SET_SOURCE_FILES_PROPERTIES(${_curentfile} PROPERTIES COMPILE_FLAGS "-w
> -fPIC")
>      ENDIF(WIN32 AND MSVC)
> ENDFOREACH(_curentfile ${libraw_LIB_SRCS})
>
> In line including -fPIC option, just add -O4 at end. Recompile all, install,
> and test
>
> Gilles Caulier
>
Comment 14 caulier.gilles 2012-10-29 18:13:48 UTC
yes, another picture can help to hack for this new problem...

Gilles Caulier
Comment 15 Alex Tutubalin 2012-10-29 18:54:49 UTC
I cannot reproduce the problem using LibRaw sample application.
I've tested on sample image shot with Canon 1Dmk2. 

Could you please provide your RAW for testing?
Comment 16 Robert Zeller 2012-10-29 22:16:52 UTC
Gilles, Alex,

please download: http://www.robert-zeller.org/transfer/DSC0630.NEF

This is a Nikon D800 raw file of about 46 MB, that also shows a faint 
pattern of blue squares (mainly in the upper sky part of the picture) 
after demosaic with the following settings:

Demosaicing:
16 bits color depth (libraw 0.15.0-Alpha4)
Interpolate RGB as four colors
Quality: Bilinear

White Balance:
Method : Camera
Highlights: Rebuild

Exposure Correction:
shift (linear) 1.31
Correct false colors in highlights

The artefacts vanish if I disable "Interpolate RGB as four colors"

Robert





On 10/29/2012 07:54 PM, Alex Tutubalin wrote:
> https://bugs.kde.org/show_bug.cgi?id=308489
>
> --- Comment #15 from Alex Tutubalin <lexa@lexa.ru> ---
> I cannot reproduce the problem using LibRaw sample application.
> I've tested on sample image shot with Canon 1Dmk2.
>
> Could you please provide your RAW for testing?
>
Comment 17 caulier.gilles 2013-05-25 07:51:28 UTC
Robert,

I recently updated libraw to 0.15.1 release in libkdcraw. I tried to repdoce the dysfunction in digiKam with you NEF RAW file, and i cannot...

==> with  "Interpolate RGB as four colors" option enabled :

http://www.flickr.com/photos/digikam/8812173451/sizes/o/

=> with  "Interpolate RGB as four colors" option disabled :

http://www.flickr.com/photos/digikam/8812197703/sizes/o/in/photostream/

Gilles Caulier