Bug 144593 - New High Dynamic Range (HDR) tool
Summary: New High Dynamic Range (HDR) tool
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-Generic-ExpoBlending (show other bugs)
Version: 1.0.0
Platform: Debian testing Linux
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-04-24 09:28 UTC by Gerhard Kulzer
Modified: 2017-07-09 17:28 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In: 1.1.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gerhard Kulzer 2007-04-24 09:28:54 UTC
Version:            (using KDE KDE 3.5.6)
Installed from:    Debian testing/unstable Packages
OS:                Linux

This is to start a discussion on a new HDR plugin. A HDR tool will mix two or more (nearly) identical images having different exposure into a new image representing a wider dynamic range, which is closer to human perception of a photographic scene. So far so simple.

The two main tasks to be accomplished are:
1. the mixture algorithm
2. realignment of composing layers

Gimp-like applications allow for manual combination of layers through gradients and mask painting. This is not trivial, and with mask painting one can easily spend a couple of concentrated hours in front of the screen, the mouse arm ending seized in its socket. 

digiKam must accomplish some intelligent __automatic__ layer combination, as it doesn't want to compete in the Gimp arena and remain easy to use. Task 2 arises from the fact that bracketing often produces slightly shifted and rotated image series, which will blur the end result if combined without registering between the layers. But it is questionable in its priority compared to task 1, the HDR tool itself.

I've dug out an article on the relative merits of various known combination algorithms (see this link: http://www.gerhard.fr/files/HDR-tone-mapping.pdf, look at pages 641 and 645 in particular). The "Icam model" and "Photographic reproduction" algorithm seem to win the price. For the Icam model I found another article with all mathematical details required to define an algorithm (see http://www.gerhard.fr/files/MFairchildArticle01-2004.pdf). 

Question: does anybody know a coded implementation of this model?

Task 2 I haven't analyzed yet, but some simplified "minimum entropy " algorithm should be able do find the best match between layers. For example, one could sample 3 small areas widely separated and run entropy minimization, say LL, LR and center (because the upper half tends to be occupied by the sky which shows no exploitable details), or the 3 areas could be manually chosen.

What do you think?
Comment 1 Julien Narboux 2007-04-24 09:39:36 UTC
Concerning task 2: I think it is quite complicated, but Digikam could use Hugin (http://hugin.sourceforge.net/).

Here is a tutorial:

http://www.panotools.info/mediawiki/index.php?title=HDR_workflow_with_hugin

Comment 2 Arnd Baecker 2007-04-24 10:18:20 UTC
There is an interesting article on HDR with linux
  http://lwn.net/Articles/225652/
Another good overview  of the techniques
http://www.mpi-inf.mpg.de/resources/pfstools/papers/mantiuk07hdr_pipeline.pdf


Concerning the software side:
- http://www.mpi-inf.mpg.de/resources/pfstools/
- A step by guide using pfstools:
  http://www.flickr.com/groups/technique/discuss/72157594155731025/
- examples of tone-mapping algorithms (follow the links):
  http://www.mpi-inf.mpg.de/resources/tmo/
- Comparison of different tone-mapping algorithms:
  http://dativ.at/logmap/index.html
- on qpfstmo, tone-mapping
  http://theplaceofdeadroads.blogspot.com/2006/07/qpfstmo-hdr-tone-mapping-gui-for-linux_04.html
- Maybe this is presently the best all-in-one solution:
  http://qtpfsgui.sourceforge.net/

For images taken without a tripod one can use hugin
to get them aligned:
http://chris.quietlife.net/2006/09/06/hdr-and-tonemapping-in-linux/

Concerning integration in digikam:
I am not sure whether this is really a good idea;
the main strength, in my opinion, of digikam is
the administration of images + some simple manipulations.
For other tasks specialized tools should be used, i.e.:
- for image manipulation: gimp, krita, ...
- for building panoramae: hugin
- for HDR images: the above tools

For example for constructing panorama pictures from
single shots, I wrote a small script (showing up as a menue
on right-click in the album view), which copies
all selected images into a separate folder in /tmp
where I then start hugin, create the pano,
and then finally copy (using an automatically generated script)
them back into the corresponding digikam folder.

What about making the integration of external tools/scripts (even into
digikams menus + association of short-cuts) possible?

Of course, adding support for viewing HDR images
(e.g. OpenEXR format etc.) might be nice.
Comment 3 Julien Narboux 2007-04-24 10:33:10 UTC
I agree that stitching and tonemapping should be done by external apps. 
Digikam could just ease the communication with these apps and manage the generated pictures.

On my old Canon S40 the files taken in stitching mode are all called st_*. I don't if other camera do the same. When picture are taken using bracketing, I think the info is also in the exif.

This could be an hint for Digikam to group the picture (later when the 'group of pictures' is implemented) and add actions such "Stitch using Hugin." "Stitch using "qtpfsgui".  
Comment 4 Arnd Baecker 2007-04-24 11:11:51 UTC
One more link concerning the comparison of tone-mappers:
http://osp.wikidot.com/parameters-for-photographers

> ------- Additional Comments From Julien.Narboux inria fr  2007-04-24 10:33 -------

[...]
> On my old Canon S40 the files taken in stitching mode are all called st_*.
> I don't if other camera do the same.


For most DSLRs there won't be such stitching mode
because there is no live-preview (apart from those who
already have  a 1D Mark III ;-)

Usually takes such pictures in manual mode
(after selecting exposure for one direction) and with a fixed
white-balance. So this, together with specifying a time-interval
(i.e. shots taking in a range < 1 min.), should allow
automatic grouping.

> When picture are taken using bracketing,
> I think the info is also in the exif.


Yes, for the to-be HDRs (which I still have to process ;-),
my Canon 350D records:
  Exposure Mode: Autobracket
  Exposure Bias: 0, -2, 2       (in that order)

So it is straight-forward to sort them into a group.

> This could be an hint for Digikam to group the picture (later when the 'group of pictures' is implemented) and add actions such "Stitch using Hugin." "Stitch using "qtpfsgui".


Yes, grouping of pictures would definitively help a lot for such
of tasks (including the possibility to select more than one
representative image of such a group to be visible in the album view etc.)

But for this, a change in the database would be necessary +
adding quite a bit of code ...

Hmm, all this is more related to
http://bugs.kde.org/show_bug.cgi?id=121310
Comment 5 Marcel Wiesweg 2007-04-24 21:02:41 UTC
For integration of pfstools, QtPFSGui is the place to look, it seems they have integrated the PFS tools in their source code, may be as we include dcraw. (not looked at the source).

A blog entry from a Krita developer about the aligning problem:
http://cyrilleberger.blogspot.com/2007/04/aligning-images.html

As to creating a HDR, it can be a job for a KIPI plugin, if the process is mostly automatic. As Gerhard said, we dont want to mimic full-blown image editor capabilities.

Viewing HDR images can be integrated in the image viewer. Question is: Is loading+tonemapping a fast process? Or like loading a raw image? Or does it really take time? Furthermore, before opening such an image, a dialog might be needed asking for the preferred algorithm or settings, I dont know.

@Arnd: Yes there will be changes in the database, and there will be grouping of pictures, but dont expect it in the next months.
Comment 6 Marcel Wiesweg 2007-04-24 21:21:54 UTC
Thinking about this, it might _not_ be a task for a kipi plugin. If we need access to the pixel data of the full images, for aligning or for combining, an implementation using the image editor core will be better suited.
Comment 7 Gerhard Kulzer 2007-04-24 22:38:36 UTC
As I am a lazy guy by principle (lion) I don't like to switch applications during my workflow. As much as I want digiKam to remain easy to use, as much I thrive on sophisticated features like restauration or refocus. For example this morning i did 50 shots of commercial products to be integrated into a web site. I use digiKam's cropping, automatic correction, whitebalance, resizing and renaming tools. For the non-rectangulars I have to open another application to select and delete the background. So for me digiKam should have some selection tool, a stamp-copy and an eraser to be complete, all tools that don't employ layers. That's why I think that if we can integrate HDR into digiKam I would always prefer that. Looking at the dynamics of development, I also feel that digiKam is fast and might have to wait for another application to improve on a feature. I also feel that HDR is becoming popular.

Remark: the "Photographic reproduction" algorithm that i mentioned in #1 seems to be the Richard algorithm of the mpg site. So that one we can get a hold of.

I don't think matching layers needs a difficult algorithm as panorama tools do. HDR deals with minute in-plane shifts and even smaller rotations in a group of images, distortion don't need to be corrected.
Comment 8 Arnd Baecker 2007-04-25 09:31:42 UTC
Ad #5

> As to creating a HDR, it can be a job for a KIPI plugin, if the
> process is mostly automatic. As Gerhard said, we dont want to mimic
> full-blown image editor capabilities.


It seems it can be done automatic, but
in qtpfsgui there are also several default settings + the custom
mode in which one specify things like `confidence functions`,
`response curves` (or even use calibration to determine the
response curve of the camera) and different HDR creation models.

> Viewing HDR images can be integrated in the image viewer. Question
> is: Is loading+tonemapping a fast process?


This depends on the actual tonemapping procedure.
Some of them work pretty fast (on reduced size images),
but others can take much longer.
Tonemapping seems to be really the "art" part of the whole
procedure. For each algorithm there are several
parameters to tune ...

According to http://osp.wikidot.com/qtpfsgui-manual
for the quick display of the generated HDR a
``luminosity compression'' algorithm is used.
This might be something which could be incorporated
to display an HDR image as a thumbnail in the album-view.
Clicking on such a hdr could either bring up
an external tool (such as Qtpfsgui) or some additional
HDR editor (like the image editor itself).

The workflow could be
a) select a sequence of bracketed images (within digikam)
b) use Qtpfsgui to generate a HDR
  (optionally: save the hdr)
c) use Qtpfsgui to do the tone-mapping
d) save the resulting image as .png/.jpg etc.

One approach would be to make this sequence of steps
as simple as possible from within digikam:
a) - selecting images: no problem.
   - associating a program with right-mouse click: also no problem.
   - better option: have a menu entry (with definable shortcut)
   - problem: Qtpfsgui presently does not take parameters (=images)
     from the command line to directly generate a HDR.
b) minor detail for Qtpfsgui:
   for saving the hdr a reasonable name, composed of the input images could
   be suggested
   (no idea about exif info for HDR file formats)
d) For saving it might be nice if some of the exif information
   of the original images would be kept?

So for this to work it would be mostly a few small changes
on the Qtpfsgui side...

Of course, I can understand the wish to have everything
integrated within one application.
However, personally I am not in favour of re-doing a
HDR tool such as Qtpfsgui within digikam (or kipi-plugins, etc.).
It seems like a waste of resources, better needed at
other places of digikam and being against the unix philosophy
to "write programs that do one thing and do it well.
Write programs to work together."
but hey - I am not doing the coding ;-)
Comment 9 DrSlony 2008-04-02 19:13:14 UTC
HDR was and is hot, but tonemapping HDR images often does not render realistic results, even when care is taken to preserv the real-look. 

A new tool has recently been released called tufuse.
Tufuse is capable of exposure blending and FOCUS BLENDING.
Tufuse generates REALISTIC scenes without needing to go into HDR mode!

Check it out here:
http://www.tawbaware.com/tufuse.htm

I spoke with the author and unfortunately a cross-platform version isn't planned, so that would mean that in order to implement something as great as this in digikam, you (the programmers) would need to write a tool the works the same way as tufuse from scratch.

I, honstely, would prefer to see something realistic like tufuse in digikam, than hdr which renders less realistic tonemapped results.

A friend and I wrote a script to use tufuse.exe through wine and batch process whole directories of bracketed shots. 
See it here:
http://www.autopano.net/wiki/action/view/Bashfuse
Comment 10 DrSlony 2008-11-25 23:56:26 UTC
I wrote a new script to use Enfuse in digiKam, you can read about it and get it here:
http://www.panopixel.org/bashenfuse.php

I prefer it over tufuse because it does the same job but is open source, much faster, and allows me to process bigger files.
Comment 11 Oleg Atamanenko 2009-10-06 10:41:11 UTC
I voted for this feature.

I found qt-based application (http://qtpfsgui.sourceforge.net/) which performs HDR-imaging.

Maybe it is possible to refactor their code to some kind of new kipi plugin?
Comment 12 Julien Narboux 2009-11-28 09:30:23 UTC
Everyone can test the new Kipi plugin in preparation by Gilles Caulier based on qtpfsgui :
http://websvn.kde.org/branches/work/~cgilles/hdrcreator

Nice work Gilles ! 

To simplify interface:
Maybe one could delete some of the algorithms. I managed to get decent result with only three algorithms : Reinhard02, Reinhard05 and Fattal the others gives result which are not realistic.

Julien
Comment 13 Marcel Wiesweg 2009-11-28 13:11:00 UTC
Julien, for me playing with HDR is not only about getting realistic results, but about surprising and unrealistic but stunning results.
When changing the parameters for the individual tone-mapping modes, you can get a wide range of realistic to completely unrealistic results.

http://osp.wikidot.com/parameters-for-photographers
Comment 14 caulier.gilles 2009-12-21 16:10:07 UTC
SVN commit 1064720 by cgilles:

New tool to create pseudo HDR image using align_images_stack (Hugin project) and Enfuse (Enblend project).
It use a set of bracketed image, aligned automatically and fused
BUGS: 144593


 M  +2 -1      CMakeLists.txt  
 A             expoblending (directory)  
 AM            expoblending/CMakeLists.txt  
 AM            expoblending/Messages.sh  
 A             expoblending/TODO  
 AM            expoblending/aboutdata.h   [License: GPL (v2+)]
 A             expoblending/blendingdlg (directory)  
 AM            expoblending/blendingdlg/enfusesettings.cpp   [License: GPL (v2+)]
 AM            expoblending/blendingdlg/enfusesettings.h   [License: GPL (v2+)]
 AM            expoblending/blendingdlg/expoblendingdlg.cpp   [License: GPL (v2+)]
 AM            expoblending/blendingdlg/expoblendingdlg.h   [License: GPL (v2+)]
 AM            expoblending/expoblending.cpp   [License: GPL (v2+)]
 A             expoblending/expoblending.desktop  
 A             expoblending/icons (directory)  
 AM            expoblending/icons/CMakeLists.txt  
 AM            expoblending/icons/hi16-action-expoblending.png  
 AM            expoblending/icons/hi22-action-expoblending.png  
 AM            expoblending/icons/hi32-action-expoblending.png  
 AM            expoblending/icons/hi48-action-expoblending.png  
 A             expoblending/importwizard (directory)  
 AM            expoblending/importwizard/alignpage.cpp   [License: GPL (v2+)]
 AM            expoblending/importwizard/alignpage.h   [License: GPL (v2+)]
 AM            expoblending/importwizard/importwizarddlg.cpp   [License: GPL (v2+)]
 AM            expoblending/importwizard/importwizarddlg.h   [License: GPL (v2+)]
 AM            expoblending/importwizard/intropage.cpp   [License: GPL (v2+)]
 AM            expoblending/importwizard/intropage.h   [License: GPL (v2+)]
 AM            expoblending/importwizard/itemspage.cpp   [License: GPL (v2+)]
 AM            expoblending/importwizard/itemspage.h   [License: GPL (v2+)]
 AM            expoblending/importwizard/lastpage.cpp   [License: GPL (v2+)]
 AM            expoblending/importwizard/lastpage.h   [License: GPL (v2+)]
 A             expoblending/kipiplugin_expoblending.desktop  
 A             expoblending/manager (directory)  
 AM            expoblending/manager/actions.h   [License: GPL (v2+)]
 AM            expoblending/manager/actionthread.cpp   [License: GPL (v2+)]
 AM            expoblending/manager/actionthread.h   [License: GPL (v2+)]
 AM            expoblending/manager/manager.cpp   [License: GPL (v2+)]
 AM            expoblending/manager/manager.h   [License: GPL (v2+)]
 A             expoblending/pics (directory)  
 AM            expoblending/pics/assistant-align.png  
 AM            expoblending/pics/assistant-enfuse.png  
 AM            expoblending/pics/assistant-stack.png  
 AM            expoblending/plugin_expoblending.cpp   [License: GPL (v2+)]
 AM            expoblending/plugin_expoblending.h   [License: GPL (v2+)]


WebSVN link: http://websvn.kde.org/?view=rev&revision=1064720