SUMMARY *** Metadata is lost after write operation on certain image files when the size of XMP JPEG segment is larger than 65535 bytes. *** STEPS TO REPRODUCE 1. Launch Digikam. 2. Enable use of Exiftool for all backend opeations. 3. Open attached sample file "test_before.jpg". 4. In Digikam, add caption: "This is a test" and click to apply changes to file (write to metatada). OBSERVED RESULT - EXIF, XMP and IPTC metadata is erased. EXPECTED RESULT - Existing metadata preserved, and file modifications written to file. SOFTWARE/OS VERSIONS Windows 11 ADDITIONAL INFORMATION - Refer to attached DebugView log: Cannot export changes with Exiv2 backend: (Error # 37 : "Size of XMP JPEG segment is larger than 65535 bytes" 00000471 106.79980469 [44080] digikam.metaengine: ExifTool "Apply Changes EXV" "-xmp:all= -exif:all= -iptc:all= -TagsFromFile C:\\Users\\jmoli\\AppData\\Local\\Temp\\MetaEngine-HRkVhC-8187ffd8.exv -all:all -codedcharacterset=UTF8 -TagsFromFile @ -makernotes -overwrite_original D:\\Users\\Jose\\OneDrive\\Pictures\\Stagging\\2023-04-00 Spain-Greece\\02 Madrid\\Catedral\\test\\Test_before.jpg" 00000472 106.97925568 [44080] digikam.metaengine: ExifToolProcess::readOutput(): ExifTool command completed
Created attachment 158334 [details] Display error message until exiftool can be used for all read/write operations? What I am surmising here is that Digikam uses exiv2 to read the metadata and build a "tag file" which is then used by exiftool to write to the file. Prior to writing the tags, exiv2 is failing, hence the "Error #37", writes an empty tag file and as such exiftool erases the metadata and replaces it with the empty file. I believe that if there is no immediate fix for this issue because of the exiv2 dependency this should produce an error to the user and not apply the changes to the files. Similar behavior can be observed in Microsoft Windows Photos app which displays the error message: "Description can't be saved." However, Exiftool is able to read and write to the file.
Another workaround is to use exiftool itself to "re-build" the XMP metadata. If I "fix" the XMP using exiftool using: exiftool test_before.jpg -xmp:all= -tagsfromfile @ "-all:all<xmp:all" -overwrite_original prior to applying the changes in Digikam, the changes are succesfully applied. So, if the specified exiv2 error occurrs, Digikam could re-build the XMP and attempt one more time. If that attempt fails, then the changes should not be applied to avoid the risk of damaging the file and a message should be displayed to the user.
Writing with ExifTool is not native writing in digiKam-8.0.0 at the moment. We create a temporary EXV container with Exiv2 that is applied to the metadata with ExifTool. The problem occurs because Exiv2 does not yet support Exif metadata greater than 65535. Maik
Git commit 3d71acc72593534c2a1398d2945992fbbf45935a by Gilles Caulier. Committed on 16/05/2023 at 03:00. Pushed by cgilles into branch 'master'. Switch bundles from Exiv2 0.27 maintenance branch to new 0.28 release Related: bug 386291, bug 386291, bug 452567 M +11 -8 project/bundles/3rdparty/ext_exiv2/CMakeLists.txt https://invent.kde.org/graphics/digikam/commit/3d71acc72593534c2a1398d2945992fbbf45935a
@José Olivier-didier, digiKam 8.1.0 pre-release Windows bundle is now ported to Exiv2 0.28 which come with a huge list of bugfixes : https://github.com/Exiv2/exiv2/issues/2406#issuecomment-1529139799 Installer file is available here : https://files.kde.org/digikam/ In case of Exiv2 bug fixed with this version, please give us a feedback. Thanks in advance Gilles Caulier
I was still able to reproduce the issue on digiKam 8.1.0 pre-release. :( Seems I forgot to attach the sample file to this bug. I used the one from the original Bug 448729 Looking in the Exiv2 GitHub thread I could not find any indication to the 64K limit issue being finally fixed. https://github.com/Exiv2/exiv2/issues/1549 https://github.com/Exiv2/exiv2/issues/2150
@José, This problem still reproducible with the new digiKam 8.2.0 pre-release Windows installer + ExifTool backend available at usual place: https://files.kde.org/digikam/ This new bundle is based on last Qt framework 5.15.11 and KDE framework 5.110. Thanks in advance Gilles Caulier
Created attachment 163760 [details] Example image file
Created attachment 163761 [details] Example XMP sidecar file (big)
Created attachment 163762 [details] Example XMP sidecar file (small)
I have encountered the same issue (digiKam deletes all metadata) and the core issue is the same (XMP segment is big), BUT the use case is slightly different and I feel like this is a problem with digiKam itself. I have attached 3 files: - A minimal jpg file - A "small" XMP sidecar file - A "big" XMP sidecar with 240Kb of data (minimal example of a file coming from Lightroom) To reproduce the problem: - Save XMP sidecar files as "468830-small.xmp" and "468830-big.xmp" - Duplicate the jpg file into "468830-small.jpg" and "468830-big.jpg" to match both sidecar files. - Open both images in digiKam. - Change the rating (or any other metadata) for both files. - Synchronize/save the metadata. What you will notice is that the "small" JPG file is fine (the metadata is stored), while the "big" JPG file is broken (metadata lost). The interesting part is that the field that causes the problem ("crs:Table_...") is never actually stored within JPG by digiKam. This is fine, as the field does not make much sense in a JPG file, but despite not being saved it causes the metadata loss. My guess is that DigiKam passes all XMP data - including the sidecar file - to the backend, even if some data won't be stored in the JPG file. If you think it's worth handling my use case separately from the initial bug, I can open a new ticket.
I have worked intensively with your example files and cannot reproduce any problems here with the current digiKam-8.3.0 developer version on Linux. A test on Windows will follow later. The "big" XMP file is not destroyed and metadata is added or changed. The crs:Table data remains unchanged. Please post a screenshot of your current sidecar settings. Maik
@Maik Thank you for looking into this. I should have been more specific: it is not the XMP file that is destroyed (here digiKam always works). It's the JPG file for which the sidecar exists that is destroyed. This is the setup: - 468830-small.jpg + 468830-small.xmp - 468830-big.jpg + 468830-big.xmp Here are the settings you've requested - ☑ Read from sidecar files - ☑ Write to sidecar files; Write to XMP sidecar for read-only item only - ☑ Sidecar file names are compatible with commercial programs I also use: ☑ Behavior: Delegate to ExifTool backend all operations to write metadata to files After changing the rating to "2" for both files, digiKam displays values from the sidecar file (rating 5). Inspect the JPG files manually to verify the problem: $ exiftool -Rating 468830-big.jpg <no result> $ exiftool -Rating 468830-small.jpg Rating : 2 And it's not only the rating that is missing for "big.jpg", it's the entire EXIF/XMP/IPTC metadata. Some further notes: - I've installed 8.2.0 after my comment yesterday and the issue is still reproducible. - I'm on Windows 10. As the workaround from José is not practicable for my use case (I would have to remove information from the XMP file), I will probably adjust my workflow. Right now I'm using "file.jpg" + "file.raf" + "file.xmp" (and hence the settings I've listed above), but I really only want to use XMP for the RAW file. As the (from my perspective) unintended JPG<>XMP connection causes the problem, I might use "file.jpg" + "file.raw.raf" + "file.raw.xmp" as a workaround, for which this bug does not occur. I hope this helps others with a similar setup.
Git commit 9e46012e63e5ebf6292d5798b6b197228971d2d1 by Maik Qualmann. Committed on 03/12/2023 at 10:16. Pushed by mqualmann into branch 'master'. abort writing with ExifTool if Exiv2 has an error exception M +7 -1 core/libs/metadataengine/engine/metaengine_p.cpp https://invent.kde.org/graphics/digikam/-/commit/9e46012e63e5ebf6292d5798b6b197228971d2d1
José, The 8.3.0 pre-release installer is updated online with the last changes from Maik. Please install file and report: https://files.kde.org/digikam/ Best Gilles caulier
(In reply to caulier.gilles from comment #15) > José, > > The 8.3.0 pre-release installer is updated online with the last changes from > Maik. > > Please install file and report: https://files.kde.org/digikam/ > > Best > Gilles caulier I tested this with 8.3.3 and observed that the metadata is not being erased as before (which is good), however any caption (description) text added is not written back to the file.
Created attachment 164495 [details] Test Image File with large XMP
I tried today with digiKam-8.3.0-20240120T105916-Win64.exe Metadata is no longer deleted, but keywords added (tags from the tags tree) are not longer written to the file.
Bug 481268 seems to be related or maybe the exact same: https://bugs.kde.org/show_bug.cgi?id=481268