Bug 451466

Summary: XMP with Google depth-map gets corrupted when metadata is modified in digiKam (likely exiv2 issue) - Use ExifTool instead
Product: [Applications] digikam Reporter: José Oliver-Didier <jose_oliver>
Component: Metadata-ExifToolAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, metzpinguin
Priority: NOR    
Version: 7.5.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed In: 8.0.0
Sentry Crash Report:
Attachments: XMP file before and after

Description José Oliver-Didier 2022-03-13 18:15:48 UTC
SUMMARY
***
An image file with XMP metadata for Google depth-map (Images taken in Portrait mode) gets corrupted when metadata is modified in Digikam (likely exiv2 issue)
***


STEPS TO REPRODUCE
1. Open image TestPortrait_original.jpg in digikam (Refer to attached sample files)
2. Perform an action which alters the file metadata such as adding a tag. For example add tag - "statue"
3. Extract altered XMP using exiftool ( exiftool TestPortrait_original.jpg -b -xmp > TestPortrait_digikam.xml)

OBSERVED RESULT
- Invalid XMP - XMP is missing the namespace -  xmlns:Camera='http://ns.google.com/photos/dd/1.0/camera/'
- XMP metadata is not read by other applications (Example: Windows Explorer among others)

EXPECTED RESULT
- Valid XMP, namespaces is preserved.

SOFTWARE/OS VERSIONS
Windows 11

ADDITIONAL INFORMATION
- Modifying metadata using exiftool works ok.
- Likely an issue with exiv2
Comment 1 José Oliver-Didier 2022-03-13 18:18:33 UTC
Adding link to reference document - Google Dynamic Depth Specification
https://developer.android.com/training/camera2/Dynamic-depth-v1.0.pdf
Comment 2 Maik Qualmann 2022-03-13 18:31:37 UTC
Do you have a sample image? The namespace should be called GCamera if you are sure with xmlns:Camera, Google has not registered anything under this namespace, you will find various other entries. 

Maik
Comment 3 Maik Qualmann 2022-03-13 18:34:23 UTC
This could be from Android: 

xmlns:camera="http://schemas.android.com/apk/res/com.android.camera"

Maik
Comment 4 José Oliver-Didier 2022-03-14 00:49:56 UTC
Created attachment 147484 [details]
XMP file before and after

Attaching files
TestPortrait_original.xml - XMP before adding tag in digikam
TestPortrait_digikam.xml - XMP after adding tag in digikam (namespace removed)
Comment 5 José Oliver-Didier 2022-03-14 02:22:31 UTC
(In reply to Maik Qualmann from comment #2)
> Do you have a sample image? The namespace should be called GCamera if you
> are sure with xmlns:Camera, Google has not registered anything under this
> namespace, you will find various other entries. 
> 
> Maik

Maik: I sent you a sample image file with the Google Depth Metadata via email given that it exceeds the allowed file size in bugzilla. 
The XMP extracts are attached to the file which can be of help as well.

From the original xmp you can observe the xmlns:Camera='http://ns.google.com/photos/dd/1.0/camera/'  which is then missing when the file metadata is modified in Digikam.

Refer to page 30 in the Google Dynamic Depth Specification document found at https://developer.android.com/training/camera2/Dynamic-depth-v1.0.pdf
Comment 6 Maik Qualmann 2022-03-14 07:05:02 UTC
The correct namespace is:
xmlns:GCamera="http://ns.google.com/photos/1.0/camera/"

Adding it would not be a problem, then digiKam can read the XMP metadata via Exiv2. However, there are still many error messages from Exiv2. This XML area is 2.1MB in size! It contains image data encoded as XML. I suspect either the Adobe XML parser or Exiv2 has a problem here due to the size. The image XMP metadata was written by ExifTool. An original sample from a device would be better to rule out that ExifTool has inserted an error here.

Ultimately, however, this problem would have to be reported to Exiv2 in order to support the namespace.

Maik
Comment 7 José Oliver-Didier 2022-03-14 13:41:51 UTC
Thanks for analysis Maik!

I opened issue 2150 in exiv2 github page - https://github.com/Exiv2/exiv2/issues/2150
Comment 8 Maik Qualmann 2022-03-14 21:25:49 UTC
Well, with the original image things look a little different. There will be a total of 9! new namespaces defined.

"http://ns.google.com/photos/dd/1.0/container/"    "Container"
"http://ns.google.com/photos/dd/1.0/item/"         "Item"
"http://ns.google.com/photos/dd/1.0/profile/"      "Profile"
"http://ns.google.com/photos/dd/1.0/camera/"       "Camera"
"http://ns.google.com/photos/dd/1.0/image/"        "Image"
"http://ns.google.com/photos/dd/1.0/depthmap/"     "DepthMap"
"http://ns.google.com/photos/dd/1.0/imagingmodel/" "ImagingModel"
"http://ns.google.com/photos/1.0/camera/"          "GCamera

ExifTool only takes "GCamera" from it, which I also think is a bug. I can register all these namespaces, I don't see the values, but I can now write metadata to the image. However, writing metadata to the image processed by ExifTool results in an error in the Adobe XML Validator.

In any case, you must provide the Exiv2 team with the original image. 

Maik
Comment 9 José Oliver-Didier 2022-03-14 22:54:43 UTC
Interesting.

I attached the original to the Exiv2 Github issue - https://github.com/Exiv2/exiv2/issues/2150
Comment 10 caulier.gilles 2022-04-10 04:50:30 UTC
ExifTool Support:

Add new option to write metadata to files using ExifTool nackend instead Exiv2.
This option replace all calls to Exiv2 to patch files by ExifTool as well.
This will prevent to corrupt files due to lack of stability support from Exiv2.
This includes large XMP section in JPEG and MPO files.
For RAW and DNG files 2 options still here but are depends of ExifTool
availability.
We will never write at all to DNG or RAW files with Exiv2.
This new option to replace Exiv2 by ExifTool is turned off by default
for the moment,
but if no side-effects is discovered in the future before official
8.0.0 release,
this option will be turned on by default to be safe in all cases.

BUGS: 325458
BUGS: 448729

FIXED-IN: 8.0.0

M  +10   -8    NEWS
M  +1    -0    core/libs/metadataengine/dmetadata/dmetadata.cpp
M  +0    -63   core/libs/metadataengine/dmetadata/dmetadata_exiftool.cpp
M  +2    -48   core/libs/metadataengine/dmetadata/dmetadata_fileio.cpp
M  +14   -4    core/libs/metadataengine/engine/metaengine.cpp
M  +13   -4    core/libs/metadataengine/engine/metaengine.h
M  +171  -30   core/libs/metadataengine/engine/metaengine_p.cpp
M  +9    -2    core/libs/metadataengine/engine/metaengine_p.h
M  +1    -1    core/libs/metadataengine/engine/metaenginesettings.cpp
M  +65   -59   core/libs/metadataengine/engine/metaenginesettingscontainer.cpp
M  +1    -0    core/libs/metadataengine/engine/metaenginesettingscontainer.h
M  +36   -31   core/libs/widgets/metadata/exiftool/exiftoolconfpanel.cpp
M  +8    -0    core/libs/widgets/metadata/exiftool/exiftoolconfpanel.h
M  +3    -0    core/showfoto/setup/showfotosetupmetadata.cpp
M  +16   -30   core/utilities/setup/metadata/setupmetadata.cpp
M  +2    -1    core/utilities/setup/metadata/setupmetadata.h
M  +48   -17   core/utilities/setup/metadata/setupmetadata_behavior.cpp
M  +5    -1    core/utilities/setup/metadata/setupmetadata_p.cpp
M  +4    -0    core/utilities/setup/metadata/setupmetadata_p.h

https://invent.kde.org/graphics/digikam/commit/f490d0c7b62454a4935754be87e84985f0296115