Bug 144155

Created attachment 20254 [details]
histogram for a picture with blown-out highlights

take a look at the histogram: due to the blown-out parts, the y-scale is waaaay
to big for the rest of the histogram, effectively making it useless.

Created attachment 20255 [details]
LIGHTROOM: histogram for a picture with blown-out highlights 

this is lightroom showing the same picture as digikam in the first attachment.
you can still see that the highlights are being clipped, but the rest of the
histogram still has a recognizable form.

in digikam you would have to use logarithmic scale to see anything --but that's
not intuitive for photography work.

>in digikam you would have to use logarithmic scale to see anything --but that's
>not intuitive for photography work. 

Why it's not intuitive ? It work perfectly...

Gilles caulier

> >in digikam you would have to use logarithmic scale to see anything --but that's
> >not intuitive for photography work.
>
> Why it's not intuitive ? It work perfectly...


Well, while it works perfectly, there are many people
who are not familiar with logarithmic representations ;-)

So the histogram y-axis is surely not wrong
for over/underexposure, cutting off the upper part
of the histogramm looks like a good solution.

But how to determine the cut-off value?
Maybe
  cut_off = min( maximal_value_of_histogram, 2*vertical_variance)
or one determines the cut-off as
the largest maximum on the interval [0.1, 0.9] ?

yes, the logarithmic histogram does work perfectly: the calculation and the curve are "correct"...

however, I think that logarithmic scale is great for some physics or math graphs.. but in photography, a logarithmic histogram does not tell (at least not to me, specially not at the first blick) the proportion of the color distribution in the image

Since the problem is normally on the extrem left or right of the histogram, you could simply cut off the 1% of either side (not really cut, but don't take into account for the Y-axis)

Other approach would be to use a given percentil to set the limit. 
I'm attaching an example-patch to the current SVN version (anonsvs) which ensures that 90% of the pixels take at least some 33% of the Y-scale.

The 90% and 33% are arbitrary values and you could play a bit with them, but they seem to give a nice result.
I'm also attaching a screenshot of digikam with(svn) and without the patch(0.9)

regards. Luis

Created attachment 20257 [details]
PATCH: limit the Y-axis of image histogram to 3*(90-percentil)

like I said, the limits are really arbitrary, but work for me.

I changed the "getMaximum" function in imagehistogram.cpp to cache the
results.. but that is really not relevant. only the "getPercentil" and the
changes in histogramwidget.cpp are interesting.

Created attachment 20258 [details]
screenshot of the patch for limiting the y-axis based on percentiles

Digikam standard and patched versions showing the same image...

I think you can read much more information from the histogram in the
patched-version.

> I think you can read much more information from the histogram in the
> patched-version.


Full agreement - very nice!
(Haven't tried the patch yet, but the result looks really good to me...)

Am Friday 13 April 2007 schrieb lparrab@gmx.net:
[bugs.kde.org quoted mail]

I think the contrary is true, in photography you __always__ use logarithmic 
representation because it corresponds more to the subjective impression 
called "Lightness". The actual subjective law is a 1/3 power law 
(http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html#lightness). 

So if we want to improve the histogram I think the best would be (because it 
corresponds to our visual perception) to represent the histogram as a power 
law, and neither log nor linear, cupped or otherwise.

My 2

human perception/sensitivity to light is roughly logarithmic, yes. but I still find it easier(faster) to read a linear histogram that a logarithmic one.

If you really want to use the perception, then maybe(??) a histogram-graph is not the right tool, and some other kind of representation would be better...

Luis

Comment on attachment 20257 [details]
PATCH: limit the Y-axis of image histogram to 3*(90-percentil)

PATCH OBSOLETE

Created attachment 20267 [details]
PATCH: limit the Y-axis of image histogram to 3*(90-percentil)

The first patch had a bug: it broke logarithmic scale (not on purpose, really
=)

This patch cleans up the code a bit and implements my proposed
percentile-solution.

Created attachment 20315 [details]
histogram example

When trying to test this patch I realized, that it has already been applied
to svn, right?

Attached is an example of an image
with a large blown-out region, for which
I am not sure, whether the algorithm works so well,
i.e., wouldn't a much smaller vertical range be better?

On the other hand, for cases where it works
well, it might be good to leave a little bit
of vertical space so that the largest peaks
don't touch the upper edge of the histogramm.

Apart from my last comment - shouldn't this one be closed?

Arnd,

Nothing have been applied on svn...

Gilles

Of course, Gilles, nothing has been applied to svn (somehow
I must have screwed that up last time I tried ...)

Created attachment 20849 [details]
example images and their histograms

I did a more thorough check of the patch, and find quite
a few cases, where it does not work optimally:
(A) too much is cut-off
(B) not enough is cut-off to the right
(C) too much is cut-off
(D) too much is cut-off

Maybe one should only use the largest 10% and lowest 10%
to detect blown out high-lights and shadows and
regard everything else as normal contents?

Unless there are any values which are being clipped, I would
suggest to always leave 2-3% of the height just blank,
so that it is clear, when something is clipped.

If you revise the patch, I am perfectly happy to test it again
(maybe we can still sneak it into the release candidate for 0.9.2 ;-)

Hi Luis,

did you have time to see whether it is possible to improve
the histograms for the examples I attached?

I think your patch is an important improvement, which maybe
can be improved even further! I am very eager if this could be 
be finalized in time for integration into 0.9.3.

Many thanks,

Arnd

Hi Luis,

any update on your patch (I'd really like if this could integrated in 0.9.3)?

Best, Arnd

Hello arnd

well.. the patch was  more or less working already.
the histogram is not 100% perfect for all images, but the results are IMVHO 
much better than the current implementation.

I havn't looked at it since then, and to be honest, I don't think I will find 
the time to do it anytime soon. so unless someone picks it up and finishes it 
(or merges it in its current state) it won't be in 0.9.3

I will probably give it another try if there are no volunteers, but like I 
said, it won't be anytime soon, sorry.

cheers. 
luis




On Monday 05 November 2007 09:57:52 Arnd Baecker wrote:
[bugs.kde.org quoted mail]

Hi Luis,

thanks a lot for the feedback!
The problem I see at the moment with the patch is that there
are cases where it does not work well enough in my opinion
(see the examples under #17).
It is clearly an improvement, but to me it is
not obvious wether the patch can be easily improved to 
always get it right, or not.
In my opinion it does not make sense to check in
some code for which it is not clear whether this
is the final solution, or if everything has to be ripped out
again later to make space for the real solution (tm) ;-).

Currently I am pretty busy with a long list of other issues,
so if you manage to have a look for an easy solution (e.g is some parameter tuning?), that's fine, otherwise this one will have to wait for a while.

Don't get me wrong: this is  a perfect example of an important contribution
to digikam to solve an issue, nicely illustrated with example images,
and a patch which improves things. So sooner (hopefully;-) or later 
this will get into svn.
(And: please keep your patches coming - contributions are very welcome!)

Thanks a lot, 

Arnd

I second Arnd here. 

This file is a good example of contribution to an open source project...

Gilles

Activity here ceased long ago but it's still open and valid, as I found working with some images showing the problem.
It seems the approach taken by the patch cuts off too much if there is a high, but real peak (because it cuts off 10% in any case) and cuts off not enough if the image is severly overexposed (more than 10% overexposed).

If we can't find the perfect technique we should not apply it to the (strictly) linear histogram mode but maybe introduce a third drawing mode?

Some thought on this:
- this is mostly about under- or overexposure creating artificial single-line peaks because the natural distribution is cut. Is there a natural image situation where a peak in the middle of the scale need to be cut?
- restricting to the upper and lower limit, it would be possible to first find the minimum and maximum (usually at 0 and 255), see if one of these takes more than e.g. 5% (*) of the pixels. We could additionally check if the mean is greater (overexposed) or less than (underexposed) the median (*). The amount of cutting depends on the maximum value of the remaining segments. I am currently not sure how far the following segments (255: 254, 253,...) need to be examined as well.

(*) needs empirical investigation

Marcel,

I'm agree to introduce a 3rd option to cut-off to 5% up/down histogram in high/low light.

Gilles Caulier

Created attachment 38729 [details]
PATCH max calculation with cutoff over/underexposure

in case of overexposure or underexposure we do want cut these values but not the peaks at the midvalues (if there are any). So my patch does a max calculation between 0.1 and 0.9 and a max calculation for 0.0 to 1.0. The returned max is the minimum between both. The cutoff max can also only get a visual high of 0.8 here. 

So if we have no over/underexposure (no peaks at both ends of the diagram) the overall max is returned, if there are peaks at the end the max between 0.1 and 0.9 is retuned lowered by 0.8 to show the cutoffs.

The patch also takes care that the max value do not hit the upper diagram frame with a cutoff of 0.05.

The patch and especially the mystic values 0.1, 0.9 and 0.8 are up for discussion. Please give it a try.

For me you rpatch work perfectly. With my all RAW files, when histogram is in linear mode, all is fine now. before, using log scale been necessary to show a suitable histogram. 

Arnd, lparrat, can you take a look ?

Gilles Caulier

SVN commit 1056933 by cgilles:

Apply patch from Jens Mueller about to adapt histogram scale when image is over/under exposed.
Sound like a good way to display a suitable histogram in this situation
BUGS: 144155


 M  +8 -7      histogram/imagehistogram.cpp  
 M  +1 -1      histogram/imagehistogram.h  
 M  +52 -22    widgets/common/histogrampainter.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1056933