Bug 188573

Matthias,

Which libkdcraw and libraw you use exactly.

In digiKam, go to Help/Components info for details...

Gilles Caulier

I need sample RW2 image(s) to test.

Download link for test image:

http://rapidshare.com/files/216032879/P1030499.RW2.html
MD5: 257FAF67054192A93832974FA51E7530

Version info:

digiKam version 0.11.0-svn (rev.: 946212M)
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 support XMP metadata: Yes
LibCImg: 130
LibExiv2: 0.18
LibJPEG: 62
LibJasper: 1.900.1
LibKDE: 4.2.1 (KDE 4.2.1)
LibKExiv2: 0.5.0
LibKdcraw: 0.4.2
LibLCMS: 116
LibPNG: 1.2.27
LibQt: 4.4.3
LibRaw: 0.6.15-Release
LibTIFF: LIBTIFF, Version 3.8.2 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble widget: 0.7.1
LibGphoto2: 2.4.2
LibKipi: 0.3.0

Matthias,

in 'automatic' mode LibRaw (part of libKdcraw) tries to automatically adjust brightness of your image using 99% of pixels (1% of specular highlights is ignored). In your shot  more than 99% is black sky and less than 1% is eclipse picture.

So, you need to turn off image auto-brightness. 
Unfortunately, I'm not a digiKam user, so cannot help you with exact checkbox (or other control), hope Gilles will help.

To Gilles: for this shot '-W' (emulated) dcraw switch is needed.

No checkbox exists for that in digikam. I think a setting like that should not be enabled by default, not sure if it should be a setting at all. It only leads to blown out highlights that can later not be recovered in post processing. Not only on astrophotographic pictures like this one, there's a distinct danger for nightly city-light pictures as well.

I would keep lossy data manipulations like this under the control of the user.

Yes, this setting do not exist but it's easy to implement.

Matthias,

Can you use libkdcraw code from svn trunk instead KDE 4.2.1 ?

There is no risk to use it. i recommend to checkout kdegraphics/libs as explained here :

http://www.digikam.org/sharedlibs

And recompile/install libkipi, libkexiv2, and libkdcraw...
and after recompile digiKAm/kipi-plugins using these one...

Fine for you ?

Gilles Caulier

(In reply to comment #6)
> Yes, this setting do not exist but it's easy to implement.
> 
> Matthias,
> 
> Can you use libkdcraw code from svn trunk instead KDE 4.2.1 ?

Actually, I'm doing that all the time - or at least I thought so. How should the version info look like if I have the versions from svn?

Like this :

digiKam version 0.11.0-svn (rev.: 946707)
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 support XMP metadata: Yes
LibCImg: 130
LibExiv2: 0.18
LibJPEG: 62
LibJasper: 1.900.1
LibKDE: 4.1.3 (KDE 4.1.3)
LibKExiv2: 0.6.0
LibKdcraw: 0.5.0
LibLCMS: 117
LibPNG: 1.2.31
LibQt: 4.4.3
LibRaw: 0.7.1-Release
LibTIFF: LIBTIFF, Version 3.8.2 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble widget: 0.8SVN
LibGphoto2: 2.4.2
LibKipi: 0.4.0

Done :

http://farm4.static.flickr.com/3640/3404494476_b16fec185d_o.png

Gilles Caulier

SVN commit 947851 by cgilles:

use new auto brightness adjustements option from libkdcraw.
BUG: 188573


 M  +2 -0      digikam/project/project.kdevelop  
 M  +3 -1      digikam/utilities/imageeditor/editor/editorwindow.cpp  
 M  +10 -2     digikam/utilities/imageeditor/rawimport/rawsettingsbox.cpp  
 M  +5 -0      digikam/utilities/setup/setupdcraw.cpp  
 M  +9 -1      kipi-plugins/rawconverter/batchdialog.cpp  
 M  +12 -1     kipi-plugins/rawconverter/singledialog.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=947851

(In reply to comment #9)
> Done :
> 
> http://farm4.static.flickr.com/3640/3404494476_b16fec185d_o.png
> 
> Gilles Caulier

Yes, that looks very similar to the out-of-camera JPEG and to what comes out of ufraw. I'll check with a few more images when I'm back home.

Thanks Gilles and Alex.

I feel a bit dumb right now. In "Settings -> "Raw Decoding" I see no checkbox to disable the auto-brightness. I'm looking at your commit and I have the changes compiled, but where is that checkbox?

But I also saw that you have auto-brightness defaulting to On. That's not a good idea. This feature adds nothing not already available through the image editor and is potentially dangerous. It should not be enabled by default and have a big fat warning sign.

Version info:

digiKam version 0.11.0-svn (rev.: 947943M)
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 support XMP metadata: Yes
LibCImg: 130
LibExiv2: 0.18
LibJPEG: 62
LibJasper: 1.900.1
LibKDE: 4.2.1 (KDE 4.2.1)
LibKExiv2: 0.6.0
LibKdcraw: 0.5.0
LibLCMS: 116
LibPNG: 1.2.27
LibQt: 4.4.3
LibRaw: 0.7.1-Release
LibTIFF: LIBTIFF, Version 3.8.2 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble widget: 0.7.1
LibGphoto2: 2.4.2
LibKipi: 0.4.0

Look on the bottom of WB settings. checkbox is here.

This option is enabled by default because peoples whant generaly a correct exposure at first loading time. We have see a lots of bug report about this subject

Your case is special and need dedicated settings. uncheck box and digiKam will remember it between sessions

Gilles Caulier

Oh, ok. I found it. Thanks for the hint.

I wonder though. How is Ufraw able to produce perfect results even with my "special" pictures without the need for user intervention? I tried some normal portraits with disabled auto brightness and they come out dull and lack contrast in digikam, but are OK in Ufraw. I understand why that happens and I can fix that in digikam with the "Normalize" tool, but that step is not necessary in ufraw.

By the way, many of the "Color" tools are broken currently. Only Curves works, the target image in Levels is always blank, In Auto-Correction "OK" button behaves like "Cancel". Only on my build?

Matthias,

for virtually all pictures you need some kind of image data auto-scaling for first preview. 99%-rule will fail on black image with only few bright pixels (star sky, eclipse), but works well for most images.

(In reply to comment #15)
> Matthias,
> 
> for virtually all pictures you need some kind of image data auto-scaling for
> first preview. 99%-rule will fail on black image with only few bright pixels
> (star sky, eclipse), but works well for most images.

Alex, of course I acknowledge that you need some adjustment to auto-scale for a first preview. It is just that the 99%-rule is not appropriate for a RAW converter. You can do that to find a basic conversion curve in the Curves tool, but it must always be reversible. So yes, I agree, something has to be done so that the images don't look dull and underexposed, but the RAW converter is not the right place for that. It's a solution to the problem, but it's not the correct one.

There is a reason why photographers go through the (relative) pain of developing RAW files for themselves. It is to create better pictures than from using the in-camera JPEG engine. If the first conversion step is already lossy, what's the  point then?

First. You *need* to autoscale because RAW data is limited to some non-round value (i.e. 14735 for most ISO on Canon 5D mkII) and full-scale is 16 bit (0-65535).

Second. Autoscaling in linear space (before gamma conversion) works very well for *all* images (due to linear nature of RAW image). Autoscaling is virtually lossless (we need to round to nearest integer, but in 16 bit rounding effect is very tiny).

The only question is autoscaling size detector. If you will use real maximum value from RAW, autoscaling will be affected by hot pixels. If you ignore too many bright pixels, you'll have too scaled shadows (as in your eclipse shot).

May be, that 1% for highlights (clipped pixels) is too optimistic. It works well enough for most pictures, but photographers use different styles of exposure (exposure by meter, ETTR and so), so different setting for automatic scaling may be useful.

I'll think about adding some autoscaling control in LibRaw postprocessing routines.

(In reply to comment #17)
> First. You *need* to autoscale because RAW data is limited to some non-round
> value (i.e. 14735 for most ISO on Canon 5D mkII) and full-scale is 16 bit
> (0-65535).

Yes :-) That is due to the sensors quantization limit.

> Second. Autoscaling in linear space (before gamma conversion) works very well
> for *all* images (due to linear nature of RAW image). Autoscaling is virtually
> lossless (we need to round to nearest integer, but in 16 bit rounding effect is
> very tiny).

> The only question is autoscaling size detector. If you will use real maximum
> value from RAW, autoscaling will be affected by hot pixels. If you ignore too
> many bright pixels, you'll have too scaled shadows (as in your eclipse shot).

Exactly, that's it. But the hot pixel count is much lower than 1%, normally you only have a few and they're not really 'stuck' pixels like on a LCD, they show up under long time exposures only.

> May be, that 1% for highlights (clipped pixels) is too optimistic. It works
> well enough for most pictures, but photographers use different styles of
> exposure (exposure by meter, ETTR and so), so different setting for automatic
> scaling may be useful.

> I'll think about adding some autoscaling control in LibRaw postprocessing
> routines.

Cameras nowadays have 12bit to 13bit of sensor range (including noise). And there is always noise, so maybe that's a valid criterion? You compute a histogram, check what comes out as zero on the high end, then just scale to fill that gap to 16bit?