Bug 219856

Summary: Exiv2 destroys RAW/DNG files when editing IPTC - Use ExifTool instead
Product: [Applications] digikam Reporter: hadmut
Component: Metadata-ExifToolAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles
Priority: NOR    
Version: 1.0.0   
Target Milestone: ---   
Platform: Ubuntu   
OS: Linux   
Latest Commit: Version Fixed In: 8.0.0
Sentry Crash Report:

Description hadmut 2009-12-23 18:37:40 UTC
Version:           1.0.0-beta5 (using KDE 4.3.2)
OS:                Linux
Installed from:    Ubuntu Packages

Hi,

I know, it's marked as experimental, just to let you know:

When editing the IPTC of RAW (DNG) images, the image file is destroyed/extremely shrinked. Looks as if digikam throws everything away thats not part of the preview image. 

regards
Comment 1 caulier.gilles 2009-12-23 18:49:10 UTC
It's an Exiv2 library issue. please make a file in Exiv2 bugzilla.

Also, please use 1.0.0 final relase, not beta5 !

Gilles Caulier
Comment 2 Mark Purcell 2009-12-23 20:25:40 UTC
Is it possible for the KDE bug tracker to check if there is a newer release available?

In the Debian reportbug(1) tool we cut down on a lot of these reports, but checking what the current release is and directing users to the newer release before they are able to file a report.
Comment 3 caulier.gilles 2011-11-03 12:21:22 UTC
This file still valid with last Exiv2 release ?

Gilles Caulier
Comment 4 caulier.gilles 2011-12-16 16:33:38 UTC
hadmut,

Do you see my comment #3 ?

Gilles Caulier
Comment 5 hadmut 2011-12-29 16:13:11 UTC
Sorry, can't tell. I'm not using that anymore, neither digikam nor DNG files. 

regards
Comment 6 caulier.gilles 2015-07-01 06:04:45 UTC
New digiKam 4.11.0 is available :

https://www.digikam.org/node/740

Can you reproduce the problem with this release ?
Comment 7 caulier.gilles 2016-07-14 13:05:38 UTC
It's an Exiv2 issue...
Comment 8 caulier.gilles 2021-04-25 08:36:43 UTC

*** This bug has been marked as a duplicate of bug 326408 ***
Comment 9 caulier.gilles 2021-05-08 16:50:52 UTC
Git commit 434f065e74282cbb9335c2cc4db256007ca2ae5d by Gilles Caulier.
Committed on 08/05/2021 at 16:46.
Pushed by cgilles into branch 'master'.

ExifTool support: add new method to get list of changes operated on metadata. The goal is to make a temporary EXV file to pass later to ExifTool to patch target file.
Related: bug 134486, bug 170693, bug 237504, bug 264210, bug 309341, bug 325458, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286

M  +162  -0    core/libs/metadataengine/engine/metaengine.cpp
M  +5    -0    core/libs/metadataengine/engine/metaengine.h
M  +0    -1    core/libs/metadataengine/engine/metaengine_p.cpp

https://invent.kde.org/graphics/digikam/commit/434f065e74282cbb9335c2cc4db256007ca2ae5d
Comment 10 caulier.gilles 2021-05-09 08:33:46 UTC
Git commit 2cb42fd246428932edd2049c59def826dcc9a420 by Gilles Caulier.
Committed on 09/05/2021 at 08:13.
Pushed by cgilles into branch 'master'.

ExifTool support: now export the list of tags changes from Exif, Iptc, and Xmp in a temporary EXV file.
We will pass later this file to ExifTool to patch target file.
Related: bug 134486, bug 170693, bug 237504, bug 264210, bug 309341, bug 325458, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286

M  +0    -186  core/libs/metadataengine/engine/metaengine.cpp
M  +8    -5    core/libs/metadataengine/engine/metaengine.h
M  +196  -0    core/libs/metadataengine/engine/metaengine_fileio.cpp
M  +4    -4    core/tests/metadataengine/CMakeLists.txt
R  +5    -4    core/tests/metadataengine/exportchanges_cli.cpp [from: core/tests/metadataengine/dmetadatadiff_cli.cpp - 082% similarity]

https://invent.kde.org/graphics/digikam/commit/2cb42fd246428932edd2049c59def826dcc9a420
Comment 11 caulier.gilles 2021-05-09 13:17:28 UTC
Git commit 2ef074a3f5c239bf4674c4cdda5bf99f511d8ae0 by Gilles Caulier.
Committed on 09/05/2021 at 13:04.
Pushed by cgilles into branch 'master'.

ExifTool support: add new method to apply list of changes operated on metadata.
A temporary EXV file constainer is computed with the list of changes to operated on Exif, Iptc, and Xmp.
This EXV file is processed with ExifTool to patch metadata.
Add new CLI test tool to try apply changes with EXV constainer. Here the file is patched with DMetadata::setImageDateTime()
which change or add new tags in Exif, Iptc, and Xmp

Exemple of changes processed to a CR3 raw image:

CR3 ORIGINAL:

exiftool -G:0:1:2:4 -exif:all -xmp:all 2020-09-22_21-27-23_0C4A1060.CR3_original
[EXIF:IFD0:Image] Image Width                   : 8192
[EXIF:IFD0:Image] Image Height                  : 5464
[EXIF:IFD0:Image] Bits Per Sample               : 8 8 8
[EXIF:IFD0:Image] Compression                   : JPEG (old-style)
[EXIF:IFD0:Camera] Make                         : Canon
[EXIF:IFD0:Camera] Camera Model Name            : Canon EOS R5
[EXIF:IFD0:Image] Orientation                   : Rotate 90 CW
[EXIF:IFD0:Image] Resolution Unit               : inches
[EXIF:IFD0:Author] Artist                       : kdmurray
[EXIF:IFD0:Author] Copyright                    :
[EXIF:ExifIFD:Image] Exposure Time              : 1/200
[EXIF:ExifIFD:Image] F Number                   : 10.0
[EXIF:ExifIFD:Camera] Exposure Program          : Manual
[EXIF:ExifIFD:Image] ISO                        : 400
[EXIF:ExifIFD:Image] Sensitivity Type           : Recommended Exposure Index
[EXIF:ExifIFD:Image] Recommended Exposure Index : 400
[EXIF:ExifIFD:Image] Exif Version               : 0231
[EXIF:ExifIFD:Time] Date/Time Original          : 2020:09:22 21:27:23
[EXIF:ExifIFD:Time] Offset Time                 : +10:00
[EXIF:ExifIFD:Time] Offset Time Original        : +10:00
[EXIF:ExifIFD:Time] Offset Time Digitized       : +10:00
[EXIF:ExifIFD:Image] Components Configuration   : Y, Cb, Cr, -
[EXIF:ExifIFD:Image] Shutter Speed Value        : 1/197
[EXIF:ExifIFD:Image] Aperture Value             : 9.9
[EXIF:ExifIFD:Camera] Flash                     : On, Fired
[EXIF:ExifIFD:Camera] Focal Length              : 90.0 mm
[EXIF:ExifIFD:Image] User Comment               :
[EXIF:ExifIFD:Time] Sub Sec Time                : 59
[EXIF:ExifIFD:Time] Sub Sec Time Original       : 59
[EXIF:ExifIFD:Time] Sub Sec Time Digitized      : 59
[EXIF:ExifIFD:Image] Flashpix Version           : 0100
[EXIF:ExifIFD:Image] Exif Image Width           : 8192
[EXIF:ExifIFD:Image] Exif Image Height          : 5464
[EXIF:ExifIFD:Camera] Focal Plane X Resolution  : 5773.079634
[EXIF:ExifIFD:Camera] Focal Plane Y Resolution  : 5769.799366
[EXIF:ExifIFD:Camera] Focal Plane Resolution Unit: inches
[EXIF:ExifIFD:Image] Custom Rendered            : Normal
[EXIF:ExifIFD:Camera] Exposure Mode             : Manual
[EXIF:ExifIFD:Camera] Scene Capture Type        : Standard
[EXIF:ExifIFD:Image] Serial Number              : 035021000124
[EXIF:ExifIFD:Image] Lens Info                  : 90mm f/0
[EXIF:ExifIFD:Image] Lens Serial Number         : 0000000000
[EXIF:GPS:Location] GPS Version ID              : 2.3.0.0
[EXIF:GPS:Location] GPS Altitude Ref            : Above Sea Level
[EXIF:GPS:Location] GPS Satellites              :
[EXIF:GPS:Location] GPS Status                  : Unknown ()
[EXIF:GPS:Location] GPS Measure Mode            : Unknown ()
[XMP:XMP-xmp:Image] Rating                      : 0

*EXV constainer generated:

exiftool -G:0:1:2:4 -exif:all -xmp:all 2020-09-22_21-27-23_0C4A1060_changes.exv
[XMP:XMP-x:Document] XMP Toolkit                : XMP Core 4.4.0-Exiv2
[XMP:XMP-exif:Time] Date/Time Original          : 2021:05:09 15:02:43
[XMP:XMP-exif:Time] Date/Time Digitized         : 2021:05:09 15:02:43
[XMP:XMP-tiff:Time] Date/Time Modified          : 2021:05:09 15:02:43
[XMP:XMP-xmp:Time] Create Date                  : 2021:05:09 15:02:43
[XMP:XMP-xmp:Time] Metadata Date                : 2021:05:09 15:02:43
[XMP:XMP-xmp:Time] Modify Date                  : 2021:05:09 15:02:43

*CR3 PATCHED:

exiftool]$ exiftool -G:0:1:2:4 -exif:all -xmp:all 2020-09-22_21-27-23_0C4A1060.CR3
[EXIF:IFD0:Image] Image Width                   : 8192
[EXIF:IFD0:Image] Image Height                  : 5464
[EXIF:IFD0:Image] Bits Per Sample               : 8 8 8
[EXIF:IFD0:Image] Compression                   : JPEG (old-style)
[EXIF:IFD0:Camera] Make                         : Canon
[EXIF:IFD0:Camera] Camera Model Name            : Canon EOS R5
[EXIF:IFD0:Image] Orientation                   : Rotate 90 CW
[EXIF:IFD0:Image] Resolution Unit               : inches
[EXIF:IFD0:Author] Artist                       : kdmurray
[EXIF:IFD0:Author] Copyright                    :
[EXIF:ExifIFD:Image] Exposure Time              : 1/200
[EXIF:ExifIFD:Image] F Number                   : 10.0
[EXIF:ExifIFD:Camera] Exposure Program          : Manual
[EXIF:ExifIFD:Image] ISO                        : 400
[EXIF:ExifIFD:Image] Sensitivity Type           : Recommended Exposure Index
[EXIF:ExifIFD:Image] Recommended Exposure Index : 400
[EXIF:ExifIFD:Image] Exif Version               : 0231
[EXIF:ExifIFD:Time] Offset Time                 : +10:00
[EXIF:ExifIFD:Time] Offset Time Original        : +10:00
[EXIF:ExifIFD:Time] Offset Time Digitized       : +10:00
[EXIF:ExifIFD:Image] Components Configuration   : Y, Cb, Cr, -
[EXIF:ExifIFD:Image] Shutter Speed Value        : 1/197
[EXIF:ExifIFD:Image] Aperture Value             : 9.9
[EXIF:ExifIFD:Camera] Flash                     : On, Fired
[EXIF:ExifIFD:Camera] Focal Length              : 90.0 mm
[EXIF:ExifIFD:Image] User Comment               :
[EXIF:ExifIFD:Time] Sub Sec Time                : 59
[EXIF:ExifIFD:Time] Sub Sec Time Original       : 59
[EXIF:ExifIFD:Time] Sub Sec Time Digitized      : 59
[EXIF:ExifIFD:Image] Flashpix Version           : 0100
[EXIF:ExifIFD:Image] Exif Image Width           : 8192
[EXIF:ExifIFD:Image] Exif Image Height          : 5464
[EXIF:ExifIFD:Camera] Focal Plane X Resolution  : 5773.079634
[EXIF:ExifIFD:Camera] Focal Plane Y Resolution  : 5769.799366
[EXIF:ExifIFD:Camera] Focal Plane Resolution Unit: inches
[EXIF:ExifIFD:Image] Custom Rendered            : Normal
[EXIF:ExifIFD:Camera] Exposure Mode             : Manual
[EXIF:ExifIFD:Camera] Scene Capture Type        : Standard
[EXIF:ExifIFD:Image] Serial Number              : 035021000124
[EXIF:ExifIFD:Image] Lens Info                  : 90mm f/0
[EXIF:ExifIFD:Image] Lens Serial Number         : 0000000000
[EXIF:GPS:Location] GPS Version ID              : 2.3.0.0
[EXIF:GPS:Location] GPS Altitude Ref            : Above Sea Level
[EXIF:GPS:Location] GPS Satellites              :
[EXIF:GPS:Location] GPS Status                  : Unknown ()
[EXIF:GPS:Location] GPS Measure Mode            : Unknown ()
[XMP:XMP-x:Document] XMP Toolkit                : Image::ExifTool 12.00
[XMP:XMP-exif:Time] Date/Time Digitized         : 2021:05:09 15:02:43           <====!
[XMP:XMP-exif:Time] Date/Time Original          : 2021:05:09 15:02:43           <====!
[XMP:XMP-photoshop:Time] Date Created           : 2021:05:09 15:02:43           <====!
[XMP:XMP-tiff:Time] Date/Time Modified          : 2021:05:09 15:02:43           <====!
[XMP:XMP-xmp:Time] Create Date                  : 2021:05:09 15:02:43           <====!
[XMP:XMP-xmp:Time] Metadata Date                : 2021:05:09 15:02:43           <====!
[XMP:XMP-xmp:Time] Modify Date                  : 2021:05:09 15:02:43           <====!
[XMP:XMP-xmp:Image] Rating                      : 0
Related: bug 134486, bug 170693, bug 237504, bug 264210, bug 309341, bug 325458, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286

M  +12   -2    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +32   -0    core/libs/metadataengine/exiftool/exiftoolparser_command.cpp
M  +6    -0    core/libs/metadataengine/exiftool/exiftoolparser_output.cpp
M  +1    -0    core/libs/metadataengine/exiftool/exiftoolprocess.h
M  +2    -1    core/tests/metadataengine/exiftool/CMakeLists.txt
C  +20   -19   core/tests/metadataengine/exiftool/exiftoolapplychanges_cli.cpp [from: core/tests/metadataengine/exiftool/exiftoolwrite_cli.cpp - 059% similarity]
M  +1    -1    core/tests/metadataengine/exiftool/exiftoolwrite_cli.cpp

https://invent.kde.org/graphics/digikam/commit/2ef074a3f5c239bf4674c4cdda5bf99f511d8ae0
Comment 12 caulier.gilles 2021-05-15 15:19:03 UTC
Git commit 5507cb4c7779f42321d751d3788ee18562df6c9d by Gilles Caulier.
Committed on 15/05/2021 at 15:14.
Pushed by cgilles into branch 'master'.

ExifToolParser: new command to copy tags from source file to a destination file.
Related: bug 134486, bug 170693, bug 237504, bug 264210, bug 309341, bug 325458, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286, bug 381967

M  +7    -0    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +64   -0    core/libs/metadataengine/exiftool/exiftoolparser_command.cpp
M  +17   -1    core/libs/metadataengine/exiftool/exiftoolprocess.h

https://invent.kde.org/graphics/digikam/commit/5507cb4c7779f42321d751d3788ee18562df6c9d
Comment 13 caulier.gilles 2021-05-16 10:13:51 UTC
Git commit 8f332525788359d2c13c3e029bfb1b956dd5a4d2 by Gilles Caulier.
Committed on 16/05/2021 at 10:09.
Pushed by cgilles into branch 'master'.

DNGWriter : include ExifTool Error View in Settings Widget.
Fix API to check ExifTool program availability.
Factoring codes.
Related: bug 134486, bug 170693, bug 237504, bug 264210, bug 309341, bug 325458, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286, bug 381967

M  +29   -5    core/libs/dngwriter/dngsettings.cpp
M  +6    -0    core/libs/dngwriter/dngsettings.h
M  +3    -10   core/libs/metadataengine/exiftool/exiftoolparser.cpp
M  +4    -1    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +31   -21   core/libs/metadataengine/exiftool/exiftoolprocess.cpp
M  +2    -0    core/libs/metadataengine/exiftool/exiftoolprocess.h

https://invent.kde.org/graphics/digikam/commit/8f332525788359d2c13c3e029bfb1b956dd5a4d2
Comment 14 caulier.gilles 2021-05-17 04:02:01 UTC
Git commit 067a15c4ed14d75eb2ce3cc725fcba709131392e by Gilles Caulier.
Committed on 17/05/2021 at 03:57.
Pushed by cgilles into branch 'master'.

DPlugin interface; add new method to be able to open an host application setup dialog page from a plugin.
DNGWriter: add ExifTool error view to settings Widget. If ExifTool is not available, show error view and allow
Related: bug 134486, bug 170693, bug 237504, bug 264210, bug 309341, bug 325458, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286, bug 381967
to customize installation from configuration panel.

M  +1    -1    core/app/main/digikamapp.cpp
M  +1    -0    core/app/main/digikamapp_p.h
M  +28   -0    core/dplugins/bqm/convert/converttodng/converttodng.cpp
M  +1    -1    core/dplugins/bqm/convert/converttodng/converttodng.h
M  +20   -1    core/libs/database/utils/ifaces/dbinfoiface.cpp
M  +2    -0    core/libs/database/utils/ifaces/dbinfoiface.h
M  +5    -1    core/libs/dngwriter/dngsettings.cpp
M  +4    -4    core/libs/dngwriter/dngwriter_postprocess.cpp
M  +4    -1    core/libs/dplugins/iface/dinfointerface.cpp
M  +13   -0    core/libs/dplugins/iface/dinfointerface.h
M  +1    -1    core/libs/metadataengine/exiftool/exiftoolparser.cpp
M  +6    -6    core/libs/widgets/metadata/exiftool/exiftoolconfpanel.cpp
M  +1    -1    core/libs/widgets/metadata/exiftool/exiftoollistview.cpp
M  +1    -0    core/showfoto/CMakeLists.txt
M  +1    -1    core/showfoto/main/showfoto.cpp
M  +2    -1    core/showfoto/main/showfoto_p.h
A  +66   -0    core/showfoto/main/showfotoinfoiface.cpp     [License: GPL (v2+)]
A  +51   -0    core/showfoto/main/showfotoinfoiface.h     [License: GPL (v2+)]
M  +2    -2    core/tests/metadataengine/exiftool/exiftoolmulticore_cli.cpp
M  +14   -1    core/utilities/queuemanager/dplugins/dpluginbqm.cpp
M  +7    -0    core/utilities/queuemanager/dplugins/dpluginbqm.h
M  +1    -0    core/utilities/queuemanager/main/queuemgrwindow.h
M  +10   -0    core/utilities/queuemanager/manager/batchtoolsfactory.cpp
M  +3    -0    core/utilities/queuemanager/manager/batchtoolsfactory.h

https://invent.kde.org/graphics/digikam/commit/067a15c4ed14d75eb2ce3cc725fcba709131392e
Comment 15 caulier.gilles 2022-04-07 04:58:00 UTC
Git commit 3a672f38cda7738356f19707a6a634ca91c269e6 by Gilles Caulier.
Committed on 07/04/2022 at 04:45.
Pushed by cgilles into branch 'master'.

Write metadata with ExifTool backend if Exiv2 fails to process, for exemple with RAW files or Video files.
Tested successfuly with a MP4 video to insert a comment.
For RAW and other king of files as MPO or JPEG with more than 64kB of Exif segement, Exiv2 must be bannned to uses in cases of writting operations as files can be corrupted.
This require more regression tests to validate ExifTool uses in place of Exiv2.
Related: bug 406540, bug 416516, bug 237504, bug 384092, bug 264210, bug 326408, bug 421464, bug 134486, bug 309341, bug 377622, bug 325458, bug 170693, bug 448729, bug 338075, bug 436876, bug 366348

M  +1    -0    core/libs/metadataengine/dmetadata/dmetadata.h
M  +43   -0    core/libs/metadataengine/dmetadata/dmetadata_exiftool.cpp
M  +49   -3    core/libs/metadataengine/dmetadata/dmetadata_fileio.cpp
M  +2    -2    core/libs/metadataengine/exiftool/exiftoolparser.h

https://invent.kde.org/graphics/digikam/commit/3a672f38cda7738356f19707a6a634ca91c269e6
Comment 16 caulier.gilles 2022-04-08 03:42:32 UTC
Git commit e9c34f1c78d1f38d7ced3c8da84cf7fa37f798cb by Gilles Caulier.
Committed on 08/04/2022 at 03:40.
Pushed by cgilles into branch 'master'.

Writing metadata to DNG is also unsafe with Exiv2. Use ExifTool backend instead.
FIXED-IN: 8.0.0

M  +9    -1    core/libs/metadataengine/dmetadata/dmetadata_exiftool.cpp
M  +1    -9    core/libs/metadataengine/engine/metaengine_p.cpp

https://invent.kde.org/graphics/digikam/commit/e9c34f1c78d1f38d7ced3c8da84cf7fa37f798cb