Created attachment 181029 [details] Sidecar file SUMMARY When adding a keyword, digikam fails and leaves the side car file corrupt. The bug is not reproducible by following the below steps STEPS TO REPRODUCE 1. Use app image digiKam-8.7.0-20250507T100042-Qt6-x86-64.appimage 2. Configure Digikam Settings => See attached images 3. Add the attached image 2025-05-02_131645.jpg and 2025-05-02_131645.jpg.xmp 4. Add a new tag to the image => "Keyword/Öl" OBSERVED RESULT Digikam fails writing the sidecar file, it will now only contain 1 line: <?xpacket begin="\ufeff" id="W5M0MpCehiHzreSzNTczkc9d"?> The debug output on the terminal indicates the problem as: "XMP Toolkit error 102: Unknown namespace prefix for qualified name" The expanded log is: Digikam::ExifToolParser::exifToolAvailable: Check ExifTool availability: true Digikam::ExifToolParser::Private::startProcess: ExifTool "Load Chunks" "-TagsFromFile /home/krikar/Pictures/2025-05-02_131645.jpg -all:all -icc_profile -o -.exv" Digikam::ExifToolProcess::Private::readOutput: ExifToolProcess::readOutput(): ExifTool command completed Digikam::ExifToolParser::cmdCompleted: ExifTool complete command for action "Load Chunks" with elasped time (ms): 92 Digikam::ExifToolParser::cmdCompleted: EXV chunk size: 65578 Digikam::ExifToolParser::cmdCompleted: ExifTool parsed command for action "Load Chunks" 1 properties decoded Digikam::ExifToolParser::Private::jumpToResultCommand: ExifTool complete "Load Chunks" for "/home/krikar/Pictures/2025-05-02_131645.jpg" Digikam::DMetadata::loadUsingExifTool: Metadata chunk loaded with ExifTool Digikam::DMetadata::load: Loading metadata with "ExifTool" backend from "/home/krikar/Pictures/2025-05-02_131645.jpg" Digikam::MetadataHub::writeTags: Writing tags Digikam::MetadataHub::writeTags: -------------------------- New Keywords QList("\Uffffffff", "Restaurante Casa del Mar") Digikam::MetaEngine::setIptcKeywords: "/home/krikar/Pictures/2025-05-02_131645.jpg" ==> New Iptc Keywords: QList("Restaurante Casa del Mar", "\Uffffffff") Digikam::MetaEngine::save: MetaEngine::metadataWritingMode 1 Digikam::MetaEngine::save: Will write XMP sidecar for file "2025-05-02_131645.jpg" Digikam::MetaEngine::Private::saveUsingExiv2: wroteComment: false Digikam::MetaEngine::Private::saveUsingExiv2: wroteEXIF: true Digikam::MetaEngine::Private::saveUsingExiv2: wroteIPTC: true Digikam::MetaEngine::Private::saveUsingExiv2: wroteXMP: true Digikam::MetaEngine::Private::printExiv2MessageHandler: Exiv2 ( 3 ) : XMP Toolkit error 102: Unknown namespace prefix for qualified name Digikam::MetaEngine::Private::printExiv2MessageHandler: Exiv2 ( 3 ) : Failed to encode XMP metadata. Digikam::MetaEngine::save: Metadata for file "2025-05-02_131645.jpg" written to XMP sidecar. Digikam::ExifToolParser::exifToolAvailable: Check ExifTool availability: true Digikam::ExifToolParser::Private::startProcess: ExifTool "Load Chunks" "-TagsFromFile /home/krikar/Pictures/2025-05-02_131645.jpg -all:all -icc_profile -o -.exv" Digikam::ExifToolProcess::Private::readOutput: ExifToolProcess::readOutput(): ExifTool command completed Digikam::ExifToolParser::cmdCompleted: ExifTool complete command for action "Load Chunks" with elasped time (ms): 81 Digikam::ExifToolParser::cmdCompleted: EXV chunk size: 65578 Digikam::ExifToolParser::cmdCompleted: ExifTool parsed command for action "Load Chunks" 1 properties decoded Digikam::ExifToolParser::Private::jumpToResultCommand: ExifTool complete "Load Chunks" for "/home/krikar/Pictures/2025-05-02_131645.jpg" Digikam::DMetadata::loadUsingExifTool: Metadata chunk loaded with ExifTool Digikam::DMetadata::load: Loading metadata with "ExifTool" backend from "/home/krikar/Pictures/2025-05-02_131645.jpg" Digikam::DImg::load: "/home/krikar/Pictures/2025-05-02_131645.jpg" : "JPEG" file identified EXPECTED RESULT The sidecar file to be updated with the new keyword "Öl" SOFTWARE/OS VERSIONS Digikam: digiKam-8.7.0-20250507T100042-Qt6-x86-64.appimage Operating System: NixOS 25.05 KDE Plasma Version: 6.3.4 KDE Frameworks Version: 6.13.0 Qt Version: 6.9.0 Kernel Version: 6.12.26 (64-bit) Graphics Platform: Wayland Processors: 32 × 13th Gen Intel® Core™ i9-13900KF Memory: 94.1 GiB of RAM Graphics Processor: NVIDIA GeForce RTX 3050 ADDITIONAL INFORMATION
Added attachments here: https://github.com/KristianKarl/digikam_bugs/tree/main/bug_503885
The cause is this Exiv2 bug: https://github.com/Exiv2/exiv2/issues/2932 We work around it by manually registering known namespaces, including the Google Container/Item used here. The issue was fixed in digiKam-8.6.0, and I can't reproduce it with your test image. Maik
Ah, good to know. However the problem is very real for me. Let me see if I can come up with a better repro. Kristian
I maybe have created a better repro for you. I forked digikam and digikam-test-data from https://invent.kde.org and created 2 commits that together reproduces the problem of getting the exiv2 errors: Exiv2 ( 3 ) : XMP Toolkit error 102: Unknown namespace prefix for qualified name Exiv2 ( 3 ) : Failed to encode XMP metadata. The 2 commits: Digikam: https://invent.kde.org/kristiankarl/digikam/-/commit/fe76e2291d31f4a40706e2d62bf942178488a3e3 digikam-test-data: https://invent.kde.org/kristiankarl/digikam-test-data/-/commit/ca63c6f4507ecbab1b0263d6256bb095624782ea Can you work with these 2 commits, or should I create PRs? When applying the 2 commits, I reproduced the problem by: krikar@UBUNTU-LTS-24-10:~/dev/digikam/build$ make -j8 && ctest --rerun-failed --output-on-failure -R digikam-usexmpsidecar_utest : : : QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.tests: File to process: "/home/krikar/dev/digikam/test-data/core/tests/metadataengine/BUG_503885.JPG" QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.tests: Temporary target file: "/tmp/digikam-/home/krikar/dev/digikam/build/bin/usexmpsidecar_utest-54710/BUG_503885.JPG" QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.tests: Temporary XMP target file: "/tmp/digikam-/home/krikar/dev/digikam/build/bin/usexmpsidecar_utest-54710/BUG_503885.JPG.xmp" QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: Loading metadata with "Exiv2" backend from "/tmp/digikam-/home/krikar/dev/digikam/build/bin/usexmpsidecar_utest-54710/BUG_503885.JPG" QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: "/tmp/digikam-/home/krikar/dev/digikam/build/bin/usexmpsidecar_utest-54710/BUG_503885.JPG" ==> New Iptc Keywords: ("\Uffffffff") QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: MetaEngine::metadataWritingMode 1 QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: Will write XMP sidecar for file "BUG_503885.JPG" QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: wroteComment: false QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: wroteEXIF: true QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: wroteIPTC: true QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: wroteXMP: true QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: Exiv2 ( 3 ) : XMP Toolkit error 102: Unknown namespace prefix for qualified name QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: Exiv2 ( 3 ) : Failed to encode XMP metadata. QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: Metadata for file "BUG_503885.JPG" written to XMP sidecar. QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.metaengine: Loading metadata with "Exiv2" backend from "/tmp/digikam-/home/krikar/dev/digikam/build/bin/usexmpsidecar_utest-54710/BUG_503885.JPG" FAIL! : UseXmpSidecarTest::testUseXmpSidecar() 'ret' returned FALSE. () Loc: [/home/krikar/dev/digikam/core/tests/metadataengine/usexmpsidecar_utest.cpp(177)] QDEBUG : UseXmpSidecarTest::testUseXmpSidecar() digikam.tests: Terminate ImageMagick PASS : UseXmpSidecarTest::cleanupTestCase() Totals: 2 passed, 1 failed, 0 skipped, 0 blacklisted, 290ms ********* Finished testing of UseXmpSidecarTest ********* digikam.metaengine: ExifToolProcess::shutDown(): send ExifTool shutdown command... digikam.metaengine: ExifTool process finished with code: 0 and status QProcess::NormalExit 0% tests passed, 1 tests failed out of 1 Total Test time (real) = 0.46 sec The following tests FAILED: 48 - digikam-usexmpsidecar_utest (Failed) Errors while running CTest
Looks like the CI caught my failing test: https://invent.kde.org/kristiankarl/digikam/-/pipelines/946628 I did not know that CI ran on forked project in KDE/Gitlab - very nice!
Sure. This is very useful...
The file BUG_503885.JPG must be hosted in the decated digiKam data repository. We separate data to run test has we uses a git-lfs stuff: https://invent.kde.org/graphics/digikam-test-data
I added the new test data (my 2nd commit) to my fork of digikam-test-data, but I guess you mean that the test job of the automatic pipeline used data from https://invent.kde.org/graphics/digikam-test-data, and not from my fork (https://invent.kde.org/kristiankarl/digikam-test-data)?
yes certainly.
It's clear where the problem comes from; I can see it in your sidecar. Unfortunately, ExifTool turns the Google "Container" and "Item" into "GContainer" and "GItem," with the same URI. I don't know why, but Phil must have a reason. However, Exif2 can't manage the same URI multiple times because the URI is stored as a key in a unique map. So we're using a trick and appending a counter to the URI. Now, "tmp0" seems to be appearing with the URI, either created by ExifTool or Exifv2. Let's see if we can find another solution; otherwise, Exif2 would have to fix the bug first. Maik
I can't create the "tmp0" container when starting with an empty sidecar. Various attempts, sometimes just reading with Exiv2 or ExifTool, sometimes just writing with Exiv2, etc., don't create a "tmp0" container for me. Do you use any other software to edit your images and create sidecars? Maik
No. No other software than Digikam for this image/sidecare. It was created by Digikam/Exfitool. Version of exiftool on my machine is 13.25
I think I have a repro for creating the occurrence of the "tmp0" container. 1 - Remove the sidecar file 2 - From Digikam select the image and from the menu do "Item -> Write Metadata to File" 3 - Now when the sidecar is created by Digikam, run the following exiftool command: exiftool -overwrite_original -m -XMP-digiKam:TagsList+="Location/Spanien/Comunitat Valenciana/ Alicante/Alacant /el Baix Vinalopó/Santa Pola/ Avenida de Fernando Pérez de Ojeda/Avinguda de Fernando Pérez de Ojeda" "BUG_503885.jpg.xmp" Check for the occurrence of the "tmp0" container: grep tmp BUG_503885.jpg.xmp xmlns:tmp0='http://ns.google.com/photos/1.0/container2/'> <tmp0:Item rdf:parseType='Resource'> </tmp0:Item> <tmp0:Item rdf:parseType='Resource'> </tmp0:Item> It seems that exiftool is creating the "tmp0" container is the side care already exists in this case. If no sidecar exists, running the exiftool command above will not generate the tmp0 container. I'm using exiftool in a script that uses geo positions in the sidecar files, then query https://nominatim.openstreetmap.org/reverse to get location tags that I add to the sidecar files. I know that the feature exists in Digikam as well, but my script is more customized to my specifics needs.
A workaround that seems to work for me is adding: "-xmp-gcontainer:all= " to my exiftool command in my script.
Git commit 03ff2ea7bb4e05c9e8d412174f847611164606e3 by Maik Qualmann. Committed on 12/05/2025 at 19:11. Pushed by mqualmann into branch 'master'. prevent Exiv2 from creating "Container" and "GContainer" in the XMP metadata FIXED-IN: 8.7.0 M +1 -1 NEWS M +16 -13 core/libs/metadataengine/engine/metaengine.cpp https://invent.kde.org/graphics/digikam/-/commit/03ff2ea7bb4e05c9e8d412174f847611164606e3
This commit only fixes future sidecar creations. I don't currently want to register "tmp0" as a namespace. Recreate the corresponding sidecar.s This prevents both "Container" and "GContainer" from being present in the sidecar and ExifTool from having to create a "tmp0" namespace. Here's why ExifTool converts Container to a GContainer: --- ExifTool uses the prefix 'GContainer' instead of 'Container' to avoid a conflict with the Google Device Container namespace. --- Maik
Thank you very much Maik /Kristian