Bug 381967

Summary: Some lens metadata lost in converting CR2 into DNG
Product: [Applications] digikam Reporter: Octopuss <spm5243>
Component: Plugin-Bqm-DngConverterAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, mehmetgelisin
Priority: NOR    
Version: 7.0.0   
Target Milestone: ---   
Platform: openSUSE   
OS: Linux   
Latest Commit: Version Fixed In: 7.3.0
Sentry Crash Report:
Attachments: attach1

Description Octopuss 2017-07-03 19:39:57 UTC
Created attachment 106435 [details]
attach1

simular tiket: 255137

Hello,
After converting CR2 files into DNG, some metadata of lens are missing. It lost Lens manufacture so automatic lens corretion tools can't choose right lens automatically.

Please, pay attention to "Lens" in the file in attach. "Cannon " is missing in DNG.
Thanks in advance.
Comment 1 caulier.gilles 2017-07-03 19:46:59 UTC
The Lens description is given from Exiv2 from markernotes decoding. The markenotes from Canon is copied as well in DNG, but for obscur reasons, Exiv2 do not decode this as with CR2 file.

I recommend to ask to Exiv2 team, by providing both CR2 and DNG files to investigate.

In all cases, no DK code can be fixed to handle this metadata, as i know...

Gilles Caulier
Comment 2 Octopuss 2017-07-12 18:32:39 UTC
Hi,

Sorry for delay. I can add an example RAW(CR2) and the resulting DNG with the issue.
CR2: https://www.dropbox.com/s/y647nngjsonim5h/IMG_6963.CR2?dl=0
DNG: https://www.dropbox.com/s/ximzsacv82g9nz2/IMG_6963.dng?dl=0
Please, let me know when the files are not needed anymore.
Thanks in advance.
Comment 3 caulier.gilles 2019-12-24 14:15:50 UTC
digiKam 7.0.0-beta1 is out :

https://download.kde.org/unstable/digikam/

This problem still reproducible with this version which include Libraw engine version 0.20.0 ?

Thanks in advance and merry Christmas

Gilles Caulier
Comment 4 Octopuss 2019-12-24 23:53:54 UTC
Hello,

I've checked digikam-7.0.0-beta1-x86-64.appimage from 2019-12-22.
The problem is still here.

I've got a RAW file and open it in darktable(2.6.3) - lens correction module shows the correct lens (Cannon, Canon... bla-bla-bla)

Then I converted the RAW into DNG with the digikam(beta) and open the DNG in darktable - the lens model is not recognized (just blank)

Thanks
Comment 5 caulier.gilles 2020-08-04 07:04:39 UTC
digiKam 7.0.0 stable release is now published:

https://www.digikam.org/news/2020-07-19-7.0.0_release_announcement/

We need a fresh feedback on this file using this version.

Best Regards

Gilles Caulier
Comment 6 caulier.gilles 2021-05-11 06:03:09 UTC
octopus,

Which camera model did you use ?

Your CR2 file sample is not available anymore

Gilles Caulier
Comment 7 caulier.gilles 2021-05-11 11:06:51 UTC
I just tested with digiKam 7.3.0 + Exiv2 0.27.4 + DNG SDK to 1.5 + libraw snapshot 20210504. The problem is not reproducible.

See my Light Table screen shot : https://i.imgur.com/N0qAKgT.png

On the left, a CR2 raw file with Lens information
On the right the DNG file created with digiKam where you will found again the Lens information.

Adobe DNG SDK forward Lens info to Exif and XMP metadata.

CR2 File Sample taken from GDRive repository listed in bug #255137:

https://drive.google.com/folderview?id=0B0uPI0_kOOJXVTNWRXItVjNHZm8&usp=sharing

I close this file now.

Gilles Caulier
Comment 8 Octopuss 2021-05-14 08:30:30 UTC
Hi,

Just check with my current Digikam 7.2.0+libraw 0.21.0.
Source: CR2 file from Cannon EOS 60D
Dest: DNG 
Method: digicam batch converter

Result: "Lens model" is missing in the result DNG.
This info is used in Darktable for applying lens correction plugins.
Comment 9 caulier.gilles 2021-05-14 08:40:03 UTC
The changes are not in digiKam 7.2.0 but in next 7.3.0. You can test with the daily build bundles here :

https://files.kde.org/digikam/

Gilles Caulier
Comment 10 Octopuss 2021-05-14 08:54:15 UTC
Thanks!
Sorry, I can't test the latest version - I don't have a sandbox on my working machine(
Comment 11 caulier.gilles 2021-05-14 09:00:52 UTC
You don't need a sandbox. AppImage is already one, as the bundle include already all in place and don't install files on your system.

Just some tips :

1/ create a new account to run the bundle.
2/ import few files to convert to RAW.
3/ process files to DNG converter.
4/ import new DNG to darktable and check.

Very simple and safe. By this way you don't touch the prod user digiKam's settings and database...

Gilles Caulier
Comment 12 Octopuss 2021-05-14 13:50:35 UTC
Thanks for the trick. I've not thought about that)

I've done as y suggested and the problem is still here:

src: https://i.imgur.com/elZRZfv.png
dst: https://i.imgur.com/O2syDBx.png
Comment 13 caulier.gilles 2021-05-14 16:15:31 UTC
Ok, thanks. Please re-share the CR2 RAW files to test here...

Gilles Caulier
Comment 14 Octopuss 2021-05-14 16:40:59 UTC
Please find the original https://www.dropbox.com/s/axleuu83c33c4zn/IMG_9870.CR2
Comment 15 caulier.gilles 2021-05-15 06:22:43 UTC
Git commit 10c2c25bcf157b16c252ad7a90ef1f817ea044a7 by Gilles Caulier.
Committed on 15/05/2021 at 06:21.
Pushed by cgilles into branch 'master'.

Store makernotes to DNG private tag to be recognized by ExifTool.
Process maker notes by manufacturer through dedicated sub functions for better readability

M  +3    -4    core/libs/dngwriter/dngwriter_exif.cpp
M  +287  -237  core/libs/dngwriter/dngwriter_makernote.cpp
M  +15   -0    core/libs/dngwriter/dngwriter_p.h

https://invent.kde.org/graphics/digikam/commit/10c2c25bcf157b16c252ad7a90ef1f817ea044a7
Comment 16 caulier.gilles 2021-05-15 15:19:04 UTC
Git commit 5507cb4c7779f42321d751d3788ee18562df6c9d by Gilles Caulier.
Committed on 15/05/2021 at 15:14.
Pushed by cgilles into branch 'master'.

ExifToolParser: new command to copy tags from source file to a destination file.
Related: bug 134486, bug 170693, bug 219856, bug 237504, bug 264210, bug 309341, bug 325458, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286

M  +7    -0    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +64   -0    core/libs/metadataengine/exiftool/exiftoolparser_command.cpp
M  +17   -1    core/libs/metadataengine/exiftool/exiftoolprocess.h

https://invent.kde.org/graphics/digikam/commit/5507cb4c7779f42321d751d3788ee18562df6c9d
Comment 17 caulier.gilles 2021-05-16 05:34:35 UTC
Git commit 530c69a0d8619df6ed204bb472ce88fbbcf543cb by Gilles Caulier.
Committed on 16/05/2021 at 05:32.
Pushed by cgilles into branch 'master'.

DNGWriter: post process metadata with ExifTool. Now all metadata are preserved and restored, including makernotes, and lens information

M  +1    -0    core/libs/dngwriter/CMakeLists.txt
M  +3    -3    core/libs/dngwriter/dngwriter_backupraw.cpp
M  +11   -2    core/libs/dngwriter/dngwriter_convert.cpp
M  +3    -3    core/libs/dngwriter/dngwriter_import.cpp
M  +5    -5    core/libs/dngwriter/dngwriter_mosaic.cpp
M  +2    -2    core/libs/dngwriter/dngwriter_negative.cpp
M  +4    -0    core/libs/dngwriter/dngwriter_p.h
A  +59   -0    core/libs/dngwriter/dngwriter_postprocess.cpp     [License: GPL (v2+)]
M  +16   -0    core/libs/metadataengine/exiftool/exiftoolparser.cpp
M  +3    -1    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +4    -1    core/libs/metadataengine/exiftool/exiftoolparser_command.cpp
M  +7    -1    core/libs/metadataengine/exiftool/exiftoolparser_p.cpp
M  +1    -0    core/libs/metadataengine/exiftool/exiftoolparser_p.h
M  +2    -2    core/libs/metadataengine/exiftool/exiftoolprocess.h
M  +5    -0    core/tests/dngwriter/raw2dng.cpp

https://invent.kde.org/graphics/digikam/commit/530c69a0d8619df6ed204bb472ce88fbbcf543cb
Comment 18 caulier.gilles 2021-05-16 10:15:28 UTC
Git commit 8f332525788359d2c13c3e029bfb1b956dd5a4d2 by Gilles Caulier.
Committed on 16/05/2021 at 10:09.
Pushed by cgilles into branch 'master'.

DNGWriter : include ExifTool Error View in Settings Widget.
Fix API to check ExifTool program availability.
Factoring codes.
Related: bug 134486, bug 170693, bug 219856, bug 237504, bug 264210, bug 309341, bug 325458, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286

M  +29   -5    core/libs/dngwriter/dngsettings.cpp
M  +6    -0    core/libs/dngwriter/dngsettings.h
M  +3    -10   core/libs/metadataengine/exiftool/exiftoolparser.cpp
M  +4    -1    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +31   -21   core/libs/metadataengine/exiftool/exiftoolprocess.cpp
M  +2    -0    core/libs/metadataengine/exiftool/exiftoolprocess.h

https://invent.kde.org/graphics/digikam/commit/8f332525788359d2c13c3e029bfb1b956dd5a4d2
Comment 19 caulier.gilles 2021-05-17 04:01:48 UTC
Git commit 067a15c4ed14d75eb2ce3cc725fcba709131392e by Gilles Caulier.
Committed on 17/05/2021 at 03:57.
Pushed by cgilles into branch 'master'.

DPlugin interface; add new method to be able to open an host application setup dialog page from a plugin.
DNGWriter: add ExifTool error view to settings Widget. If ExifTool is not available, show error view and allow
Related: bug 134486, bug 170693, bug 219856, bug 237504, bug 264210, bug 309341, bug 325458, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286
to customize installation from configuration panel.

M  +1    -1    core/app/main/digikamapp.cpp
M  +1    -0    core/app/main/digikamapp_p.h
M  +28   -0    core/dplugins/bqm/convert/converttodng/converttodng.cpp
M  +1    -1    core/dplugins/bqm/convert/converttodng/converttodng.h
M  +20   -1    core/libs/database/utils/ifaces/dbinfoiface.cpp
M  +2    -0    core/libs/database/utils/ifaces/dbinfoiface.h
M  +5    -1    core/libs/dngwriter/dngsettings.cpp
M  +4    -4    core/libs/dngwriter/dngwriter_postprocess.cpp
M  +4    -1    core/libs/dplugins/iface/dinfointerface.cpp
M  +13   -0    core/libs/dplugins/iface/dinfointerface.h
M  +1    -1    core/libs/metadataengine/exiftool/exiftoolparser.cpp
M  +6    -6    core/libs/widgets/metadata/exiftool/exiftoolconfpanel.cpp
M  +1    -1    core/libs/widgets/metadata/exiftool/exiftoollistview.cpp
M  +1    -0    core/showfoto/CMakeLists.txt
M  +1    -1    core/showfoto/main/showfoto.cpp
M  +2    -1    core/showfoto/main/showfoto_p.h
A  +66   -0    core/showfoto/main/showfotoinfoiface.cpp     [License: GPL (v2+)]
A  +51   -0    core/showfoto/main/showfotoinfoiface.h     [License: GPL (v2+)]
M  +2    -2    core/tests/metadataengine/exiftool/exiftoolmulticore_cli.cpp
M  +14   -1    core/utilities/queuemanager/dplugins/dpluginbqm.cpp
M  +7    -0    core/utilities/queuemanager/dplugins/dpluginbqm.h
M  +1    -0    core/utilities/queuemanager/main/queuemgrwindow.h
M  +10   -0    core/utilities/queuemanager/manager/batchtoolsfactory.cpp
M  +3    -0    core/utilities/queuemanager/manager/batchtoolsfactory.h

https://invent.kde.org/graphics/digikam/commit/067a15c4ed14d75eb2ce3cc725fcba709131392e