Bug 338081 - Support HDR DNG images from HDRMerge
Summary: Support HDR DNG images from HDRMerge
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Thumbs-RAW (show other bugs)
Version: 4.2.0
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-08-06 23:31 UTC by DrSlony
Modified: 2017-07-27 10:57 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 4.3.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description DrSlony 2014-08-06 23:31:58 UTC
HDRMerge is an awesome FOSS program from Javier Celaya for generating a high dynamic range raw image in the DNG format from a bracketed raw set.
https://github.com/jcelaya/hdrmerge

Please add support for correctly displaying the thumbnails from these raw images. They follow standards and should therefore be supported.
You can download a sample image from:
http://rawtherapee.com/shared/test_images/hdrmerge_045.dng

Currently digiKam shows neither the thumbnail nor the embedded JPEG preview.
Comment 1 DrSlony 2014-08-11 14:46:33 UTC
With "[x] Embedded preview loads full-sized images" checked, thumbnails are not shown but clicking on a thumb does show the large preview.
http://i.imgur.com/abRoxR9.jpg
Comment 2 caulier.gilles 2014-08-13 22:21:38 UTC
RAW preview including DNG file are processed by libraw in background. Please report this problem to Libraw team.

Gilles Caulier
Comment 3 DrSlony 2014-08-14 09:29:37 UTC
Hi Gilles, thank your for responding. I can't find libraw's bugzilla, can you point me? I sent them a message but these are usually ignored if they're not filed in the correct place.
Comment 4 caulier.gilles 2014-08-14 09:52:25 UTC
Sound like libraw use GitHub issue manager. I can seen 8 entries closed :

https://github.com/LibRaw/LibRaw/issues?q=is%3Aissue+is%3Aclosed

Gilles Caulier
Comment 5 DrSlony 2014-08-14 10:15:16 UTC
Do you also use libraw to extract the embedded JPEG thumbnail and preview image?
Because digiKam DOES show the embedded preview, but it does NOT show the thumbnail. My request is for digiKam to show the embedded preview and thumbnail so that these DNG images can be managed.

ps. I wrote in comment 0 that it does not show the embedded preview - this is wrong, 4.2.0 does show the preview. Maybe I had an older version of some dependency back then. Now I'm using 4.2.0 and previews are shown after clicking on the DNG, but still no thumbs.

digiKam version 4.2.0
CPU cores: 8
Demosaic GPL2 pack support: No
Demosaic GPL3 pack support: No
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Pgf: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 supports XMP metadata: Yes
LibCImg: 130
LibEigen: 3.2.0
LibExiv2: 0.24
LibJPEG: 62
LibJasper: 1.900.1
LibKDE: 4.13.3
LibKExiv2: 2.3.1
LibKGeoMap: 2.0.0
LibKdcraw: 2.4.2
LibLCMS: 2060
LibLensFun: 0.2.8-0
LibPGF: 6.12.27 - external shared library
LibPNG: 1.6.12
LibQt: 4.8.5
LibRaw: 0.16.0-Beta1
LibTIFF: LIBTIFF, Version 4.0.3 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble Widget: 0.18.3 (stable release)
Parallelized PGF codec: No
Parallelized demosaicing: Yes
RawSpeed codec support: No
Database backend: QSQLITE
Kipi-Plugins: 4.2.0
LibKface: 3.0.0
LibKipi: 2.1.0
LibOpenCV: 2.4.5
Comment 6 caulier.gilles 2014-08-14 10:20:20 UTC
Yes, libraw is used in Both case.

Note Exiv2 has also a method to extract preview from DNG. Please report this problem too in Exiv2 bugzilla.

Gilles Caulier
Comment 8 DrSlony 2014-08-14 16:32:42 UTC
LibRaw refuse to support even simple extraction of the embedded JPEG thumbnail.
I can already extract BOTH preview images from that DNG using exiv2-0.24. Preview-2 is large, preview-1 is small, thumbnail-sized in fact (256x172). It could be used for the thumbnail in digiKam.

exiv2 -ep1 hdrmerge_045.dng

To reiterate:
1- digiKam already shows the large preview when you click on the thumbnail, but it shows no thumbnail.
2- exiv2 can extract both previews,
3- so you could use it to get the small preview for the thumbnail.

I removed the "resolved - upstream" status as one upstream refused to do anything (libraw), and the other already supports it (exiv2). So now it's up to digiKam to make use of exiv2's ability to get the thumb.
Comment 9 caulier.gilles 2014-08-14 16:37:00 UTC
Can you share some DNG files to test here ?

Gilles Caulier
Comment 10 DrSlony 2014-08-14 16:46:06 UTC
Sure, there is a link to one in my original comment here: 
http://rawtherapee.com/shared/test_images/hdrmerge_045.dng

Let me know if you need more.
Comment 11 DrSlony 2014-08-15 11:20:39 UTC
I discussed this with the Exiv2 devs (See comment 7 for the link) and it appears that Exiv2 is working correctly. No Exif thumbnail exists inside this image, so digiKam should use the embedded preview-1 image as the thumbnail.
Comment 12 caulier.gilles 2014-08-31 11:44:03 UTC
Yes, reproducible here :

digikam(72287)/digikam (core) Digikam::DImg::load: "/Users/gilles/Pictures/photos.test/hdrmerge_045.dng"  : RAW file identified
digikam(72287)/digikam (core) Digikam::JPEGUtils::isJpegImage: mimetype =  "TIFF"  ext =  "DNG"
digikam(72287)/digikam (core) Digikam::ThumbnailCreator::createThumbnail: Cannot create thumbnail for  "/Users/gilles/Pictures/photos.test/hdrmerge_045.dng" 
digikam(72287)/digikam (core) Digikam::ThumbnailCreator::load: Thumbnail is null for  "/Users/gilles/Pictures/photos.test/hdrmerge_045.dng" 

Gilles Caulier
Comment 13 caulier.gilles 2014-09-01 05:47:42 UTC
First problem confirmed Libraw 0.16 cannot extract embedded preview and half preview with this DNG file :

digikam(75049)/digikam (core) Digikam::ThumbnailCreator::load: Thumbnail is null for  "/Users/gilles/Pictures/photos.test/hdrmerge_045.dng" 
digikam(75049)/digikam (core) Digikam::ThumbnailCreator::createThumbnail: Trying to load Embedded preview with libraw
digikam(75049)/digikam (core) Digikam::ThumbnailCreator::createThumbnail: Trying to load half preview with libraw

This debug statements just proof that libraw is not able to work with this file without to use a full demosaicing method.

Typically this DNG is badly encoded by HDRMerge.

Gilles Caulier
Comment 14 caulier.gilles 2014-09-01 05:52:57 UTC
Other point : as libraw cannot play with this DNG, Image cannot be loaded in image editor...

Gilles Caulier
Comment 15 DrSlony 2014-09-01 06:05:16 UTC
1- How is it "badly encoded"?
2- digiKam uses exiv2, and exiv2 can extract the preview. That would be enough for the thumbnail to let us manage these files using digiKam.
Comment 16 DrSlony 2014-09-01 06:06:20 UTC
"Programs using a library like libkdcraw/libraw to fully read the raw image and obtain the preview. They fail, because libraw does not know how to read the float data. Since reading the data and the preview is performed in just one call to open(), the preview is not available when reading the data fails"
https://github.com/jcelaya/hdrmerge/issues/27#issuecomment-53459385
Comment 17 caulier.gilles 2014-09-01 06:09:26 UTC
Git commit 617f203acb9fec474c95c3dbd7660adb1dcc57f7 by Gilles Caulier.
Committed on 01/09/2014 at 06:05.
Pushed by cgilles into branch 'master'.

For dumy DNG file badbly encode and unsupported by libraw, try to use Exiv2 preview extraction to render thumbnail.
FIXED-IN: 4.3.0

M  +13   -1    libs/threadimageio/thumbnailcreator.cpp

http://commits.kde.org/digikam/617f203acb9fec474c95c3dbd7660adb1dcc57f7
Comment 18 caulier.gilles 2014-09-01 06:12:18 UTC
DrSlony,

The RAW Demosaicing problem still here. As HDRMerge do not provide a valid DNG file which can be processed with libraw, report this problem to HDRMerge team for future fixes. This problem cannot be solved in digiKam code. 

Gilles Caulier
Comment 19 DrSlony 2014-09-01 06:56:16 UTC
CG thank you VERY much for this patch! Seeing thumbs will make things so much easier :)
I don't need digiKam to read the raw data, just showing the thumbnails is enough.

ps. I've been collaborating with various teams for HDR-DNG thumbnail/preview support, and I sent the report links to Javier Celaya (the HDRMerge dev). I pasted the link in comment #16. As far as I am aware the DNG files are not "badly encoded", there is nothing wrong with them. The only problem is libraw does not supporting floating point data, and attempts to read both the raw data and the thumb in one call, so if reading the raw data fails, which it will, then reading the thumb will fail too, which it should not.