Bug 509796

The cause is likely Bug 382645. It concerns the packet wrapper that both Exif2 and ExifTool write. This packet wrapper also exists in images with XMP metadata.
Darktable, on the other hand, writes pure XML without a packet wrapper.

One could also say that digiKam's sidecar is a pixel-free image with XMP metadata.
Darktable writes a configuration XML file.

There are also posts about this in the ExifTool forum, where they don't want to omit the packet wrapper either, since Adobe also apparently writes it.

A bug report would also have to be created for Darktable.

We can't fundamentally change anything about the creation of the sidecar; Exif2 does everything.

If there is a difference between macOS and Linux, we would need two identically processed files from each operating system to analyze the differences.

Alternatively, it would be necessary to clarify how Darktable behaves with sidecars created by ExifTool.

Maik

Thanks for the fast feedback. I’ve read the discussion on bug 382645 and together with your – very interesting - other explanations I tend to just forget the issue and do a workaround (let the XML be initialized from darktable and change to the digiKam stuff after). As far as I can see there are several parties that will likely never agree on the 'right way', so in view of my strained nerves and my finite lifespan I prefer the more passive, but much easier way.

Well, I've been playing around with Darktable-5.2.1 and digiKam-8.8.0 on Linux and can't find any issues. Darktable writes to XMP files created by digiKam without any problems, and vice versa.
I hope you're not using a Snap/Flatpak version of Darktable?

Otherwise send me an XMP file that Darktable doesn't want to write to.

Maik

Git commit 462090ca26a284b69c5403037aebcc7b82267d07 by Maik Qualmann.
Committed on 22/09/2025 at 19:38.
Pushed by mqualmann into branch 'master'.

update Darktable metadata profile file for better interaction with tags

M  +12   -12   core/data/metadata/darktable.dkamp

https://invent.kde.org/graphics/digikam/-/commit/462090ca26a284b69c5403037aebcc7b82267d07

Created attachment 185165 [details]
XMP made on Linux, not writable by darktable

Created attachment 185166 [details]
XMP made on Mac, writable by darktable on Linux

Created attachment 185167 [details]
XMP made on Mac and updated by darktable on Linux

Okay, the 3 attachments should  (hopefully) reveal all interesting stuff. And no, I don't use darktable as Snap or Flatpak but as AppImage too.

The reason why Darktable is not writing the sidecar file could be the "Xmp.acdsee.categories" entry.
It might depend on the length of this entry; shortening or removing this entry will cause Darktable to write the sidecar again.
However, further tests are necessary, which I'll do tomorrow.
In the Darktable metadata profile file, we don't write "Xmp.acdsee.categories."

Maik

Tests with sidecars converted by ExifTool show the same problem, although they are structured completely differently, and the cause is this entry:

<xmpMM:DerivedFrom
    stRef:documentID="03A87688A3E29E2257EEDD9EAC8D89BF"
    stRef:originalDocumentID="03A87688A3E29E2257EEDD9EAC8D89BF"/>

If this entry is removed, the sidecar is used under Linux. The macOS file also contains this entry and cannot be written by Darktable under Linux as long as it is present.
The entry does not come from digiKam, and the issue would need to be resolved with the Darktable team.

Maik

A look at the Darktable source code shows that "Xmp.xmpMM.DerivedFrom" is intended to be used as the filename. However, this may fail due to the existing content.

// The original file name
if(filename) xmpData["Xmp.xmpMM.DerivedFrom"] = filename;

Maik

Okay, thanks a lot, I can confirm that removing <xmpMM:DerivedFrom … /> makes the file writable to Darktable. So the problem is clearly narrowed down. 

But as you might have guessed I'm no developer and thus my understanding the interaction of the participating components is at best a rough outline.  To ask the Darktable team to change a behaviour I don’t really understand because it somehow doesn't harmonize with the behaviour the folks at digiKam, ExifTool and Adobe considerate as superior, doesn't sound too promising. So I guess I rather go the workaround path outlined in comment 2.

Nevertheless: Thank you for your time, efforts and outstanding patience.

Ralf

Sorry, just one more question: if <xmpMM:DerivedFrom is not coming from digiKam, who writes it then? ExifTool? As it lands in the XMP before Darktable has imported it, digiKam must have been at least somehow involved. Do you consider ExifTool as not being digiKam?

I suspect that xmpMM:DerivedFrom is already present in the DNG file and comes from the Adobe converter.
digiKam copies all XMP metadata from the image into the sidecar file when it is created. We could potentially remove it.

Another issue is that the actual XMP.xmpMM.DerivedFrom field is a structured metadata field (allowing multiple entries), whereas Darktable writes it as a simple string field.

Maik

Bingo! Your suspicion seems to be correct – I tested some original files from the cam (Fuji/RAF in this case) and there is no xmpMM:DerivedFrom in digiKam’s XMP. So Darktable will write to it without issues. To be honest: I dislike to use Adobe's DNG converter anyway and prefer digiKam's converter. Unfortunately it doesn't work for Fuji RAFs (approx. half of my photos, the other half Nikon NEFs which convert perfectly).

To remove xmpMM:DerivedFrom this little evil from the file would of course be great, although it looks af if it never bothered anyone but me. 

Otherwise the best workaround I found is to use digiKam first for culling, tagging and renaming, save the images, import them to Darktable and delete the XMPs immediately after to let Darktable write new ones that work in both directions.

Git commit a89d91cc1d6f888718b10bcc0aa1aa73d0973337 by Maik Qualmann.
Committed on 23/09/2025 at 15:10.
Pushed by mqualmann into branch 'master'.

remove Xmp.xmpMM.DerivedFrom from sidecar
darktable cannot write to the sidevar if this
metadata structure tag is present.
FIXED-IN: 8.8.0

M  +1    -1    NEWS
M  +12   -0    core/libs/metadataengine/engine/metaengine_p.cpp

https://invent.kde.org/graphics/digikam/-/commit/a89d91cc1d6f888718b10bcc0aa1aa73d0973337

Theoretically, this issue should be reported to darktable so that it can be fixed and we can remove our workaround. I hope no one will miss "Xmp.xmpMM.DerivedFrom" in the sidecar.

Maik

I would report it to darktable but don't really know what exactly to tell them and where to do it. I never had any contact to the Darktable team and don't know anything about their bug report system. Github I guess?

(In reply to Ralf S from comment #18)
> I would report it to darktable but don't really know what exactly to tell
> them and where to do it. I never had any contact to the Darktable team and
> don't know anything about their bug report system. Github I guess?

Okay, the issue is reported: https://github.com/darktable-org/darktable/issues/19429

Thanks for creating the bug report.

Maik

Darktable reports this Exiv2 exception in full debug mode:

49,5685 [exiv2] Fehler des XMP-Werkzeugsatzes 103: Structs and arrays can't have string values
49,5687 [dt_exif_xmp_write] /home/maik/Images/Darktable/img_linux_problem.dng.xmp: caught exiv2 exception '[xmp_write] failed to serialize xmp data'

Maik

Error message again in English:

5.5299 [exiv2] XMP Toolkit error 103: Structs and arrays can't have string values
5.5300 [dt_exif_xmp_write] /home/maik/Images/Darktable/img_linux_problem.dng.xmp: caught exiv2 exception '[xmp_write] failed to serialize xmp data'

Maik