Bug 468830

Created attachment 158334 [details]
Display error message until exiftool can be used for all read/write operations?

What I am surmising here is that Digikam uses exiv2 to read the metadata and build a "tag file" which is then used by exiftool to write to the file. Prior to writing the tags, exiv2 is failing, hence the "Error #37", writes an empty tag file and as such exiftool erases the metadata and replaces it with the empty file.

I believe that if there is no immediate fix for this issue because of the exiv2 dependency this should produce an error to the user and not apply the changes to the files. Similar behavior can be observed in Microsoft Windows Photos app which displays the error message: "Description can't be saved."

However, Exiftool is able to read and write to the file.

Another workaround is to use exiftool itself to "re-build" the XMP metadata.

If I "fix" the XMP using exiftool using:

exiftool test_before.jpg -xmp:all= -tagsfromfile @ "-all:all<xmp:all" -overwrite_original

prior to applying the changes in Digikam, the changes are succesfully applied.

So, if the specified exiv2 error occurrs, Digikam could re-build the XMP and attempt one more time. If that attempt fails, then the changes should not be applied to avoid the risk of damaging the file and a message should be displayed to the user.

Writing with ExifTool is not native writing in digiKam-8.0.0 at the moment. We create a temporary EXV container with Exiv2 that is applied to the metadata with ExifTool. The problem occurs because Exiv2 does not yet support Exif metadata greater than 65535.

Maik

Git commit 3d71acc72593534c2a1398d2945992fbbf45935a by Gilles Caulier.
Committed on 16/05/2023 at 03:00.
Pushed by cgilles into branch 'master'.

Switch bundles from Exiv2 0.27 maintenance branch to new 0.28 release
Related: bug 386291, bug 386291, bug 452567

M  +11   -8    project/bundles/3rdparty/ext_exiv2/CMakeLists.txt

https://invent.kde.org/graphics/digikam/commit/3d71acc72593534c2a1398d2945992fbbf45935a

@José Olivier-didier,

digiKam 8.1.0 pre-release Windows bundle is now ported to Exiv2 0.28 which
come with a huge list of bugfixes :

https://github.com/Exiv2/exiv2/issues/2406#issuecomment-1529139799

Installer file is available here :

https://files.kde.org/digikam/

In case of Exiv2 bug fixed with this version, please give us a feedback.

Thanks in advance

Gilles Caulier

I was still able to reproduce the issue on digiKam 8.1.0 pre-release. :(

Seems I forgot to attach the sample file to this bug. I used the one from the original Bug 448729

Looking in the Exiv2 GitHub thread I could not find any indication to the 64K limit issue being finally fixed.

https://github.com/Exiv2/exiv2/issues/1549
https://github.com/Exiv2/exiv2/issues/2150

@José,

This problem still reproducible with the new digiKam 8.2.0 pre-release Windows
installer + ExifTool backend available at usual place:

https://files.kde.org/digikam/

This new bundle is based on last Qt framework 5.15.11 and KDE framework 5.110.

Thanks in advance

Gilles Caulier

Created attachment 163760 [details]
Example image file

Created attachment 163761 [details]
Example XMP sidecar file (big)

Created attachment 163762 [details]
Example XMP sidecar file (small)

I have encountered the same issue (digiKam deletes all metadata) and the core issue is the same (XMP segment is big), BUT the use case is slightly different and I feel like this is a problem with digiKam itself. I have attached 3 files:

  - A minimal jpg file
  - A "small" XMP sidecar file
  - A "big" XMP sidecar with 240Kb of data (minimal example of a file coming from Lightroom)

To reproduce the problem:

  - Save XMP sidecar files as "468830-small.xmp" and "468830-big.xmp"
  - Duplicate the jpg file into "468830-small.jpg" and "468830-big.jpg" to match both sidecar files.
  - Open both images in digiKam.
  - Change the rating (or any other metadata) for both files.
  - Synchronize/save the metadata.

What you will notice is that the "small" JPG file is fine (the metadata is stored), while the "big" JPG file is broken (metadata lost).

The interesting part is that the field that causes the problem ("crs:Table_...") is never actually stored within JPG by digiKam. This is fine, as the field does not make much sense in a JPG file, but despite not being saved it causes the metadata loss. My guess is that DigiKam passes all XMP data - including the sidecar file - to the backend, even if some data won't be stored in the JPG file.

If you think it's worth handling my use case separately from the initial bug, I can open a new ticket.

I have worked intensively with your example files and cannot reproduce any problems here with the current digiKam-8.3.0 developer version on Linux. A test on Windows will follow later. The "big" XMP file is not destroyed and metadata is added or changed. The crs:Table data remains unchanged.
Please post a screenshot of your current sidecar settings.

Maik

@Maik Thank you for looking into this. I should have been more specific: it is not the XMP file that is destroyed (here digiKam always works). It's the JPG file for which the sidecar exists that is destroyed. This is the setup:

- 468830-small.jpg + 468830-small.xmp
- 468830-big.jpg + 468830-big.xmp

Here are the settings you've requested

- ☑ Read from sidecar files
- ☑ Write to sidecar files; Write to XMP sidecar for read-only item only
- ☑ Sidecar file names are compatible with commercial programs

I also use: ☑ Behavior: Delegate to ExifTool backend all operations to write metadata to files

After changing the rating to "2" for both files, digiKam displays values from the sidecar file (rating 5). Inspect the JPG files manually to verify the problem:

$ exiftool -Rating 468830-big.jpg
<no result>
$ exiftool -Rating 468830-small.jpg
Rating                          : 2

And it's not only the rating that is missing for "big.jpg", it's the entire EXIF/XMP/IPTC metadata.

Some further notes:

- I've installed 8.2.0 after my comment yesterday and the issue is still reproducible.
- I'm on Windows 10.

As the workaround from José is not practicable for my use case (I would have to remove information from the XMP file), I will probably adjust my workflow. Right now I'm using "file.jpg" + "file.raf" + "file.xmp" (and hence the settings I've listed above), but I really only want to use XMP for the RAW file. As the (from my perspective) unintended JPG<>XMP connection causes the problem, I might use "file.jpg" + "file.raw.raf" + "file.raw.xmp" as a workaround, for which this bug does not occur. I hope this helps others with a similar setup.

Git commit 9e46012e63e5ebf6292d5798b6b197228971d2d1 by Maik Qualmann.
Committed on 03/12/2023 at 10:16.
Pushed by mqualmann into branch 'master'.

abort writing with ExifTool if Exiv2 has an error exception

M  +7    -1    core/libs/metadataengine/engine/metaengine_p.cpp

https://invent.kde.org/graphics/digikam/-/commit/9e46012e63e5ebf6292d5798b6b197228971d2d1

José,

The 8.3.0 pre-release installer is updated online with the last changes from
Maik.

Please install file and report: https://files.kde.org/digikam/

Best
Gilles caulier

(In reply to caulier.gilles from comment #15)
> José,
> 
> The 8.3.0 pre-release installer is updated online with the last changes from
> Maik.
> 
> Please install file and report: https://files.kde.org/digikam/
> 
> Best
> Gilles caulier

I tested this with 8.3.3 and observed that the metadata is not being erased as before (which is good), however any caption (description) text added is not written back to the file.

Created attachment 164495 [details]
Test Image File with large XMP

I tried today with digiKam-8.3.0-20240120T105916-Win64.exe

Metadata is no longer deleted, but keywords added (tags from the tags tree) are not longer written to the file.

Bug 481268 seems to be related or maybe the exact same: https://bugs.kde.org/show_bug.cgi?id=481268