Bug 325458

Summary: Editing Exif information corrupts MPO 3D image with Exiv2 - Use ExifTool instead
Product: [Applications] digikam Reporter: gemlion
Component: Metadata-ExifToolAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: critical CC: caulier.gilles
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Kubuntu   
OS: Linux   
Latest Commit: https://invent.kde.org/graphics/digikam/commit/f490d0c7b62454a4935754be87e84985f0296115 Version Fixed In: 8.0.0
Sentry Crash Report:

Description gemlion 2013-09-30 10:16:15 UTC 
When editing metadata (especially caption and gps-coordinates) of mpo (multi picture object) images, these images cannot be displayed by the original camera or applications like geeqie, anymore.

The windows freeware (e.g. via wine) application stereophoto maker is able to restore them.

digiKam version 3.1.0

Reproducible: Always

Steps to Reproduce:
1. Take a 3D photo (e.g. with Fujifilm W3) or create one with StereoPhoto Maker
2. add GPS-Coordinates with digiKam
3. try to open photo with 3d camera or with geeqie in stereoscopic view
Actual Results:  
Fujifilm W3 displays error message: Read Error

Geeqie displays only left image

Expected Results:  
Fujifilm Camera should display image in 3d even when image was renamed.

Geeqie should display picture in stereoscopic mode (e.g. as anaglyph red-cyan)
Comment 1 caulier.gilles 2013-09-30 12:06:10 UTC 
This entry in not valid. Please report this problem to Exiv2 shared lib which is used by digiKam to manage metadata in file.

Gilles Caulier
Comment 2 gemlion 2013-09-30 12:58:46 UTC 
Alright. I'll do that.
Comment 3 caulier.gilles 2021-04-25 08:14:13 UTC 
Upstream bug not closed in 2021 yet:

https://dev.exiv2.org/issues/923
Comment 4 caulier.gilles 2021-04-25 13:46:48 UTC 
Git commit c701f2a84df48a7526903768778991190b51f776 by Gilles Caulier.
Committed on 25/04/2021 at 13:30.
Pushed by cgilles into branch 'master'.

ExifTool wrapper : introduce applyChanges() method to update existing tags or to add new tags.
The method take a TagsMap hash-table to host ExifTool tag names as key and tag values.
Typically, the key follow ExifTool CLI tool argument typo as EXIF:ImageDescription. The value is a UTF8 string.
This method is not yet used, unit tests must be written in first.
Related: bug 326408, bug 416516

M  +167  -47   core/libs/metadataengine/exiftool/exiftoolparser.cpp
M  +23   -4    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +39   -27   core/libs/metadataengine/exiftool/exiftoolprocess.cpp
M  +15   -6    core/libs/metadataengine/exiftool/exiftoolprocess.h

https://invent.kde.org/graphics/digikam/commit/c701f2a84df48a7526903768778991190b51f776
Comment 5 caulier.gilles 2021-05-08 16:51:09 UTC 
Git commit 434f065e74282cbb9335c2cc4db256007ca2ae5d by Gilles Caulier.
Committed on 08/05/2021 at 16:46.
Pushed by cgilles into branch 'master'.

ExifTool support: add new method to get list of changes operated on metadata. The goal is to make a temporary EXV file to pass later to ExifTool to patch target file.
Related: bug 134486, bug 170693, bug 219856, bug 237504, bug 264210, bug 309341, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286

M  +162  -0    core/libs/metadataengine/engine/metaengine.cpp
M  +5    -0    core/libs/metadataengine/engine/metaengine.h
M  +0    -1    core/libs/metadataengine/engine/metaengine_p.cpp

https://invent.kde.org/graphics/digikam/commit/434f065e74282cbb9335c2cc4db256007ca2ae5d
Comment 6 caulier.gilles 2021-05-09 08:33:48 UTC 
Git commit 2cb42fd246428932edd2049c59def826dcc9a420 by Gilles Caulier.
Committed on 09/05/2021 at 08:13.
Pushed by cgilles into branch 'master'.

ExifTool support: now export the list of tags changes from Exif, Iptc, and Xmp in a temporary EXV file.
We will pass later this file to ExifTool to patch target file.
Related: bug 134486, bug 170693, bug 219856, bug 237504, bug 264210, bug 309341, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286

M  +0    -186  core/libs/metadataengine/engine/metaengine.cpp
M  +8    -5    core/libs/metadataengine/engine/metaengine.h
M  +196  -0    core/libs/metadataengine/engine/metaengine_fileio.cpp
M  +4    -4    core/tests/metadataengine/CMakeLists.txt
R  +5    -4    core/tests/metadataengine/exportchanges_cli.cpp [from: core/tests/metadataengine/dmetadatadiff_cli.cpp - 082% similarity]

https://invent.kde.org/graphics/digikam/commit/2cb42fd246428932edd2049c59def826dcc9a420
Comment 7 caulier.gilles 2021-05-09 13:18:25 UTC 
Git commit 2ef074a3f5c239bf4674c4cdda5bf99f511d8ae0 by Gilles Caulier.
Committed on 09/05/2021 at 13:04.
Pushed by cgilles into branch 'master'.

ExifTool support: add new method to apply list of changes operated on metadata.
A temporary EXV file constainer is computed with the list of changes to operated on Exif, Iptc, and Xmp.
This EXV file is processed with ExifTool to patch metadata.
Add new CLI test tool to try apply changes with EXV constainer. Here the file is patched with DMetadata::setImageDateTime()
which change or add new tags in Exif, Iptc, and Xmp

Exemple of changes processed to a CR3 raw image:

CR3 ORIGINAL:

exiftool -G:0:1:2:4 -exif:all -xmp:all 2020-09-22_21-27-23_0C4A1060.CR3_original
[EXIF:IFD0:Image] Image Width                   : 8192
[EXIF:IFD0:Image] Image Height                  : 5464
[EXIF:IFD0:Image] Bits Per Sample               : 8 8 8
[EXIF:IFD0:Image] Compression                   : JPEG (old-style)
[EXIF:IFD0:Camera] Make                         : Canon
[EXIF:IFD0:Camera] Camera Model Name            : Canon EOS R5
[EXIF:IFD0:Image] Orientation                   : Rotate 90 CW
[EXIF:IFD0:Image] Resolution Unit               : inches
[EXIF:IFD0:Author] Artist                       : kdmurray
[EXIF:IFD0:Author] Copyright                    :
[EXIF:ExifIFD:Image] Exposure Time              : 1/200
[EXIF:ExifIFD:Image] F Number                   : 10.0
[EXIF:ExifIFD:Camera] Exposure Program          : Manual
[EXIF:ExifIFD:Image] ISO                        : 400
[EXIF:ExifIFD:Image] Sensitivity Type           : Recommended Exposure Index
[EXIF:ExifIFD:Image] Recommended Exposure Index : 400
[EXIF:ExifIFD:Image] Exif Version               : 0231
[EXIF:ExifIFD:Time] Date/Time Original          : 2020:09:22 21:27:23
[EXIF:ExifIFD:Time] Offset Time                 : +10:00
[EXIF:ExifIFD:Time] Offset Time Original        : +10:00
[EXIF:ExifIFD:Time] Offset Time Digitized       : +10:00
[EXIF:ExifIFD:Image] Components Configuration   : Y, Cb, Cr, -
[EXIF:ExifIFD:Image] Shutter Speed Value        : 1/197
[EXIF:ExifIFD:Image] Aperture Value             : 9.9
[EXIF:ExifIFD:Camera] Flash                     : On, Fired
[EXIF:ExifIFD:Camera] Focal Length              : 90.0 mm
[EXIF:ExifIFD:Image] User Comment               :
[EXIF:ExifIFD:Time] Sub Sec Time                : 59
[EXIF:ExifIFD:Time] Sub Sec Time Original       : 59
[EXIF:ExifIFD:Time] Sub Sec Time Digitized      : 59
[EXIF:ExifIFD:Image] Flashpix Version           : 0100
[EXIF:ExifIFD:Image] Exif Image Width           : 8192
[EXIF:ExifIFD:Image] Exif Image Height          : 5464
[EXIF:ExifIFD:Camera] Focal Plane X Resolution  : 5773.079634
[EXIF:ExifIFD:Camera] Focal Plane Y Resolution  : 5769.799366
[EXIF:ExifIFD:Camera] Focal Plane Resolution Unit: inches
[EXIF:ExifIFD:Image] Custom Rendered            : Normal
[EXIF:ExifIFD:Camera] Exposure Mode             : Manual
[EXIF:ExifIFD:Camera] Scene Capture Type        : Standard
[EXIF:ExifIFD:Image] Serial Number              : 035021000124
[EXIF:ExifIFD:Image] Lens Info                  : 90mm f/0
[EXIF:ExifIFD:Image] Lens Serial Number         : 0000000000
[EXIF:GPS:Location] GPS Version ID              : 2.3.0.0
[EXIF:GPS:Location] GPS Altitude Ref            : Above Sea Level
[EXIF:GPS:Location] GPS Satellites              :
[EXIF:GPS:Location] GPS Status                  : Unknown ()
[EXIF:GPS:Location] GPS Measure Mode            : Unknown ()
[XMP:XMP-xmp:Image] Rating                      : 0

*EXV constainer generated:

exiftool -G:0:1:2:4 -exif:all -xmp:all 2020-09-22_21-27-23_0C4A1060_changes.exv
[XMP:XMP-x:Document] XMP Toolkit                : XMP Core 4.4.0-Exiv2
[XMP:XMP-exif:Time] Date/Time Original          : 2021:05:09 15:02:43
[XMP:XMP-exif:Time] Date/Time Digitized         : 2021:05:09 15:02:43
[XMP:XMP-tiff:Time] Date/Time Modified          : 2021:05:09 15:02:43
[XMP:XMP-xmp:Time] Create Date                  : 2021:05:09 15:02:43
[XMP:XMP-xmp:Time] Metadata Date                : 2021:05:09 15:02:43
[XMP:XMP-xmp:Time] Modify Date                  : 2021:05:09 15:02:43

*CR3 PATCHED:

exiftool]$ exiftool -G:0:1:2:4 -exif:all -xmp:all 2020-09-22_21-27-23_0C4A1060.CR3
[EXIF:IFD0:Image] Image Width                   : 8192
[EXIF:IFD0:Image] Image Height                  : 5464
[EXIF:IFD0:Image] Bits Per Sample               : 8 8 8
[EXIF:IFD0:Image] Compression                   : JPEG (old-style)
[EXIF:IFD0:Camera] Make                         : Canon
[EXIF:IFD0:Camera] Camera Model Name            : Canon EOS R5
[EXIF:IFD0:Image] Orientation                   : Rotate 90 CW
[EXIF:IFD0:Image] Resolution Unit               : inches
[EXIF:IFD0:Author] Artist                       : kdmurray
[EXIF:IFD0:Author] Copyright                    :
[EXIF:ExifIFD:Image] Exposure Time              : 1/200
[EXIF:ExifIFD:Image] F Number                   : 10.0
[EXIF:ExifIFD:Camera] Exposure Program          : Manual
[EXIF:ExifIFD:Image] ISO                        : 400
[EXIF:ExifIFD:Image] Sensitivity Type           : Recommended Exposure Index
[EXIF:ExifIFD:Image] Recommended Exposure Index : 400
[EXIF:ExifIFD:Image] Exif Version               : 0231
[EXIF:ExifIFD:Time] Offset Time                 : +10:00
[EXIF:ExifIFD:Time] Offset Time Original        : +10:00
[EXIF:ExifIFD:Time] Offset Time Digitized       : +10:00
[EXIF:ExifIFD:Image] Components Configuration   : Y, Cb, Cr, -
[EXIF:ExifIFD:Image] Shutter Speed Value        : 1/197
[EXIF:ExifIFD:Image] Aperture Value             : 9.9
[EXIF:ExifIFD:Camera] Flash                     : On, Fired
[EXIF:ExifIFD:Camera] Focal Length              : 90.0 mm
[EXIF:ExifIFD:Image] User Comment               :
[EXIF:ExifIFD:Time] Sub Sec Time                : 59
[EXIF:ExifIFD:Time] Sub Sec Time Original       : 59
[EXIF:ExifIFD:Time] Sub Sec Time Digitized      : 59
[EXIF:ExifIFD:Image] Flashpix Version           : 0100
[EXIF:ExifIFD:Image] Exif Image Width           : 8192
[EXIF:ExifIFD:Image] Exif Image Height          : 5464
[EXIF:ExifIFD:Camera] Focal Plane X Resolution  : 5773.079634
[EXIF:ExifIFD:Camera] Focal Plane Y Resolution  : 5769.799366
[EXIF:ExifIFD:Camera] Focal Plane Resolution Unit: inches
[EXIF:ExifIFD:Image] Custom Rendered            : Normal
[EXIF:ExifIFD:Camera] Exposure Mode             : Manual
[EXIF:ExifIFD:Camera] Scene Capture Type        : Standard
[EXIF:ExifIFD:Image] Serial Number              : 035021000124
[EXIF:ExifIFD:Image] Lens Info                  : 90mm f/0
[EXIF:ExifIFD:Image] Lens Serial Number         : 0000000000
[EXIF:GPS:Location] GPS Version ID              : 2.3.0.0
[EXIF:GPS:Location] GPS Altitude Ref            : Above Sea Level
[EXIF:GPS:Location] GPS Satellites              :
[EXIF:GPS:Location] GPS Status                  : Unknown ()
[EXIF:GPS:Location] GPS Measure Mode            : Unknown ()
[XMP:XMP-x:Document] XMP Toolkit                : Image::ExifTool 12.00
[XMP:XMP-exif:Time] Date/Time Digitized         : 2021:05:09 15:02:43           <====!
[XMP:XMP-exif:Time] Date/Time Original          : 2021:05:09 15:02:43           <====!
[XMP:XMP-photoshop:Time] Date Created           : 2021:05:09 15:02:43           <====!
[XMP:XMP-tiff:Time] Date/Time Modified          : 2021:05:09 15:02:43           <====!
[XMP:XMP-xmp:Time] Create Date                  : 2021:05:09 15:02:43           <====!
[XMP:XMP-xmp:Time] Metadata Date                : 2021:05:09 15:02:43           <====!
[XMP:XMP-xmp:Time] Modify Date                  : 2021:05:09 15:02:43           <====!
[XMP:XMP-xmp:Image] Rating                      : 0
Related: bug 134486, bug 170693, bug 219856, bug 237504, bug 264210, bug 309341, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286

M  +12   -2    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +32   -0    core/libs/metadataengine/exiftool/exiftoolparser_command.cpp
M  +6    -0    core/libs/metadataengine/exiftool/exiftoolparser_output.cpp
M  +1    -0    core/libs/metadataengine/exiftool/exiftoolprocess.h
M  +2    -1    core/tests/metadataengine/exiftool/CMakeLists.txt
C  +20   -19   core/tests/metadataengine/exiftool/exiftoolapplychanges_cli.cpp [from: core/tests/metadataengine/exiftool/exiftoolwrite_cli.cpp - 059% similarity]
M  +1    -1    core/tests/metadataengine/exiftool/exiftoolwrite_cli.cpp

https://invent.kde.org/graphics/digikam/commit/2ef074a3f5c239bf4674c4cdda5bf99f511d8ae0
Comment 8 caulier.gilles 2021-05-15 15:19:35 UTC 
Git commit 5507cb4c7779f42321d751d3788ee18562df6c9d by Gilles Caulier.
Committed on 15/05/2021 at 15:14.
Pushed by cgilles into branch 'master'.

ExifToolParser: new command to copy tags from source file to a destination file.
Related: bug 134486, bug 170693, bug 219856, bug 237504, bug 264210, bug 309341, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286, bug 381967

M  +7    -0    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +64   -0    core/libs/metadataengine/exiftool/exiftoolparser_command.cpp
M  +17   -1    core/libs/metadataengine/exiftool/exiftoolprocess.h

https://invent.kde.org/graphics/digikam/commit/5507cb4c7779f42321d751d3788ee18562df6c9d
Comment 9 caulier.gilles 2021-05-16 10:14:08 UTC 
Git commit 8f332525788359d2c13c3e029bfb1b956dd5a4d2 by Gilles Caulier.
Committed on 16/05/2021 at 10:09.
Pushed by cgilles into branch 'master'.

DNGWriter : include ExifTool Error View in Settings Widget.
Fix API to check ExifTool program availability.
Factoring codes.
Related: bug 134486, bug 170693, bug 219856, bug 237504, bug 264210, bug 309341, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286, bug 381967

M  +29   -5    core/libs/dngwriter/dngsettings.cpp
M  +6    -0    core/libs/dngwriter/dngsettings.h
M  +3    -10   core/libs/metadataengine/exiftool/exiftoolparser.cpp
M  +4    -1    core/libs/metadataengine/exiftool/exiftoolparser.h
M  +31   -21   core/libs/metadataengine/exiftool/exiftoolprocess.cpp
M  +2    -0    core/libs/metadataengine/exiftool/exiftoolprocess.h

https://invent.kde.org/graphics/digikam/commit/8f332525788359d2c13c3e029bfb1b956dd5a4d2
Comment 10 caulier.gilles 2021-05-17 04:02:26 UTC 
Git commit 067a15c4ed14d75eb2ce3cc725fcba709131392e by Gilles Caulier.
Committed on 17/05/2021 at 03:57.
Pushed by cgilles into branch 'master'.

DPlugin interface; add new method to be able to open an host application setup dialog page from a plugin.
DNGWriter: add ExifTool error view to settings Widget. If ExifTool is not available, show error view and allow
Related: bug 134486, bug 170693, bug 219856, bug 237504, bug 264210, bug 309341, bug 326408, bug 377622, bug 384092, bug 406540, bug 416516, bug 421464, bug 436286, bug 381967
to customize installation from configuration panel.

M  +1    -1    core/app/main/digikamapp.cpp
M  +1    -0    core/app/main/digikamapp_p.h
M  +28   -0    core/dplugins/bqm/convert/converttodng/converttodng.cpp
M  +1    -1    core/dplugins/bqm/convert/converttodng/converttodng.h
M  +20   -1    core/libs/database/utils/ifaces/dbinfoiface.cpp
M  +2    -0    core/libs/database/utils/ifaces/dbinfoiface.h
M  +5    -1    core/libs/dngwriter/dngsettings.cpp
M  +4    -4    core/libs/dngwriter/dngwriter_postprocess.cpp
M  +4    -1    core/libs/dplugins/iface/dinfointerface.cpp
M  +13   -0    core/libs/dplugins/iface/dinfointerface.h
M  +1    -1    core/libs/metadataengine/exiftool/exiftoolparser.cpp
M  +6    -6    core/libs/widgets/metadata/exiftool/exiftoolconfpanel.cpp
M  +1    -1    core/libs/widgets/metadata/exiftool/exiftoollistview.cpp
M  +1    -0    core/showfoto/CMakeLists.txt
M  +1    -1    core/showfoto/main/showfoto.cpp
M  +2    -1    core/showfoto/main/showfoto_p.h
A  +66   -0    core/showfoto/main/showfotoinfoiface.cpp     [License: GPL (v2+)]
A  +51   -0    core/showfoto/main/showfotoinfoiface.h     [License: GPL (v2+)]
M  +2    -2    core/tests/metadataengine/exiftool/exiftoolmulticore_cli.cpp
M  +14   -1    core/utilities/queuemanager/dplugins/dpluginbqm.cpp
M  +7    -0    core/utilities/queuemanager/dplugins/dpluginbqm.h
M  +1    -0    core/utilities/queuemanager/main/queuemgrwindow.h
M  +10   -0    core/utilities/queuemanager/manager/batchtoolsfactory.cpp
M  +3    -0    core/utilities/queuemanager/manager/batchtoolsfactory.h

https://invent.kde.org/graphics/digikam/commit/067a15c4ed14d75eb2ce3cc725fcba709131392e
Comment 11 caulier.gilles 2022-04-07 04:58:41 UTC 
Git commit 3a672f38cda7738356f19707a6a634ca91c269e6 by Gilles Caulier.
Committed on 07/04/2022 at 04:45.
Pushed by cgilles into branch 'master'.

Write metadata with ExifTool backend if Exiv2 fails to process, for exemple with RAW files or Video files.
Tested successfuly with a MP4 video to insert a comment.
For RAW and other king of files as MPO or JPEG with more than 64kB of Exif segement, Exiv2 must be bannned to uses in cases of writting operations as files can be corrupted.
This require more regression tests to validate ExifTool uses in place of Exiv2.
Related: bug 406540, bug 416516, bug 237504, bug 384092, bug 264210, bug 326408, bug 421464, bug 134486, bug 309341, bug 219856, bug 377622, bug 170693, bug 448729, bug 338075, bug 436876, bug 366348

M  +1    -0    core/libs/metadataengine/dmetadata/dmetadata.h
M  +43   -0    core/libs/metadataengine/dmetadata/dmetadata_exiftool.cpp
M  +49   -3    core/libs/metadataengine/dmetadata/dmetadata_fileio.cpp
M  +2    -2    core/libs/metadataengine/exiftool/exiftoolparser.h

https://invent.kde.org/graphics/digikam/commit/3a672f38cda7738356f19707a6a634ca91c269e6
Comment 12 caulier.gilles 2022-04-10 03:51:40 UTC 
Git commit f490d0c7b62454a4935754be87e84985f0296115 by Gilles Caulier.
Committed on 10/04/2022 at 03:42.
Pushed by cgilles into branch 'master'.

ExifTool Support:

Add new option to write metadata to files using ExifTool nackend instead Exiv2.
This option replace all calls to Exiv2 to patch files by ExifTool as well.
This will prevent to corrupt files due to lack of stability support from Exiv2.
This includes large XMP section in JPEG and MPO files.
For RAW and DNG files 2 options still here but are depends of ExifTool availability.
We will never write at all to DNG or RAW files with Exiv2.
This new option to replace Exiv2 by ExifTool is turned off by default for the moment,
but if no side-effects is discovered in the future before official 8.0.0 release,
this option will be turned on by default to be safe in all cases.
Related: bug 448729

FIXED-IN: 8.0.0

M  +10   -8    NEWS
M  +1    -0    core/libs/metadataengine/dmetadata/dmetadata.cpp
M  +0    -63   core/libs/metadataengine/dmetadata/dmetadata_exiftool.cpp
M  +2    -48   core/libs/metadataengine/dmetadata/dmetadata_fileio.cpp
M  +14   -4    core/libs/metadataengine/engine/metaengine.cpp
M  +13   -4    core/libs/metadataengine/engine/metaengine.h
M  +171  -30   core/libs/metadataengine/engine/metaengine_p.cpp
M  +9    -2    core/libs/metadataengine/engine/metaengine_p.h
M  +1    -1    core/libs/metadataengine/engine/metaenginesettings.cpp
M  +65   -59   core/libs/metadataengine/engine/metaenginesettingscontainer.cpp
M  +1    -0    core/libs/metadataengine/engine/metaenginesettingscontainer.h
M  +36   -31   core/libs/widgets/metadata/exiftool/exiftoolconfpanel.cpp
M  +8    -0    core/libs/widgets/metadata/exiftool/exiftoolconfpanel.h
M  +3    -0    core/showfoto/setup/showfotosetupmetadata.cpp
M  +16   -30   core/utilities/setup/metadata/setupmetadata.cpp
M  +2    -1    core/utilities/setup/metadata/setupmetadata.h
M  +48   -17   core/utilities/setup/metadata/setupmetadata_behavior.cpp
M  +5    -1    core/utilities/setup/metadata/setupmetadata_p.cpp
M  +4    -0    core/utilities/setup/metadata/setupmetadata_p.h

https://invent.kde.org/graphics/digikam/commit/f490d0c7b62454a4935754be87e84985f0296115