Bug 264007

Well, I just found that the current digiKam's implementation is quite hostile to XMP keys created by other programs (in my case by darktable) and simply removes them.
Until the hostility is solved, it might be better not to follow up the bug reported above. (It is better to have two XMP files...)

Hm is there a standard about sidecar file naming? How is this supposed to be solved?

And please report the other issue, preferably with a sample sidecar file. Thanks!

I googled shortly and it looks like Adobe Lightroom intended XMP sidecars only for images, which cannot easily embed the xmp inside. That is: XMP would exist only for raw workflow to camera proprietary formats.

I.e. Adobe first created a flexible solution and then limited it to raw files.

Another trouble arise if a sidecar is shared for image.jpg and image.cr2: if the user adds tags to cr2 file, it gets written to image.xmp, however image.jpg would not get automatically updated for the new tags. As a consequence, if the user adds different tags to image.jpg, the image.xmp gets overwritten and original tags are lost...

So either digiKam disables writing sidecars for image types, which support embedding (which would be a pitty IMHO and would not solve the problem for situations like image.crw vs image.cr2 anyway), or better start using image.ext.xmp format.

The latter might of course create incompability with other applications...

Some reading from Adobe for various multimedia files and XMP:
http://www.adobe.com/content/dam/Adobe/en/devnet/xmp/pdfs/XMPSpecificationPart3.pdf

"For applications that need to find external XMP files, look in the same directory for a file with the same name as
the main document but with an .xmp extension. (This is called a sidecar XMP file.)"

Doesn't really help, one reader will assume the "name" is the basename, the other will assume the "name" is the complete file name.

Which path does Adobe and other software take? name.suf.xmp or name.xmp? In the latter case, they'll have the same problem.

MS Windows users do not know what a file extension is :-)

Google says, Adobe product creates sidecar xmp files only for proprietary raw files and names them "name.xmp". For other common file formats, the xmp is embedded and no sidecar file created (JPEG, DNG).
http://www.oreillynet.com/digitalmedia/blog/2007/04/xmp_sidecar_files_and_lightroo.html

Better someone actually confirms if it still is true nowadays...

P.S. Having read more about the matter, I would now prefer compatibility (name.xmp) to flexibility (name.ext.xmp).

Created attachment 57743 [details]
darktable sidecar file example

Andreas, 

Do you have any Exiv2 users feedback about XMP sidecar file naming convention ?

DarkTable use a naming notation based on complete name of the origin file with .xmp attached. With this we can distinguish between foo.jpg and foo.cr2 file where the sidecar files become foo.jpg.xmp and foo.cr2.xmp.

Do you know other photo management application which use the same way ? What do you think about ?

Gilles Caulier

> Do you have any Exiv2 users feedback about XMP sidecar file naming convention 

Maybe someone listening on the Exiv2 forum has an opinion: http://dev.exiv2.org/boards/3/topics/842

Geeqie places the files as .metadata/<complete filename>.gq.xmp, where <compelte filename> is e.g. "image.png".

I'm not sure whether the subdir is really covered by the standard, but it has at least the advantage of hiding the (possible loads of) sidecar files.

(Not sure why they've added the ".gq" part...

Created attachment 57912 [details]
Geequie XMP sidecar file example

(In reply to comment #10)
> Geeqie places the files as .metadata/<complete filename>.gq.xmp, where
> <compelte filename> is e.g. "image.png".
> 
> I'm not sure whether the subdir is really covered by the standard, but it has
> at least the advantage of hiding the (possible loads of) sidecar files.

IMHO hiding the files in an extra folder is not a good idea, because when a user copies the RAW file using a file manager to another folder, the copied file has lost all metadata. If sidecar files are visible, to user can easily see that there are extra files to take care of. In digikam we could hide the xmp files and transparently copy them along.

Michael

As we learn on the exiv2 forum, the mess is complete! now that some applications even use both conventions at the same time to have to different sidecar files...
The original specification is not precise, yet leans to the basename.xmp convention. This simply fails to handle the case in the original report here. The basename.suffix.xmp convention looks technically superior then.
It would be interesting to see what major competitors do when they are given such files to _read_ even if they write in the basename.xmp convention?

For reading, we can easily support both, giving priority to basename.suffix.xmp for example. For writing, I would tend to use basename.suffix.xmp unless that makes us incompatible with market leaders.

a) "Loosing" the meta-data is IMO _always_ a risk, whether you have it in a subdir or not. (Of course not if you move/copy/etc with "aware" programs).

b) Even if the original convention would be to use basename.xmp, I'd suggest to:

_write_ <fullname.xmp> ... (perhaps except a user - for compatibility reason - configures manually not to do so)

_read_ both styles (perhaps even with and without things like a .metadata-dir.
In cases where it's not clear which file is meant (e.g. when having basename.xmp) the metadata could simply be taken for all versions of that file.

This is my view-point about the lead subject of this entry : we must support a naming convention including original filename extension.

Typical case :

0/ Original file is RAW : myphoto.arw

1/ i apply digiKam properties in my workflow for ex. to identify good items in my album. arw is read only. digiKam create myphoto.arw.xmp

2/ I start to edit original file, and create a PNG copy as first refence of my work. I apply another digiKam properties to this new file, different than original. digiKam create myphoto.png.xmp

3/ from this PNG image i create a ready to publish JPEG. Same way than with PNG about digiKam properties. digiKam create myphoto.jpg.xmp.

To resume files below will be created in this ex. :

myphoto.arw            // original
myphoto.arw.xmp
myphoto.png            // lossless copy of modified image
myphoto.png.xmp
myphoto.jpg            // lossy copy ready to publish, print, export...
myphoto.jpg.xmp

For each item created, users can customize properties. We must to have a customized XMP file for each items created/managed in digiKam. Using file name extension is the right way to do it.

Gilles Caulier

Git commit 0a420804d83d6a1c33253b14626a502d7834c41b by Gilles Caulier.
Committed on 17/03/2011 at 14:42.
Pushed by cgilles into branch 'master'.

handle XMP sidecar files using full name of files with extention + ".xmp"
BUGS: 264007

M  +16   -5    libkexiv2/kexiv2.cpp     
M  +6    -0    libkexiv2/kexiv2.h     
M  +2    -4    libkexiv2/kexiv2_p.cpp     

http://commits.kde.org/libkexiv2/0a420804d83d6a1c33253b14626a502d7834c41b

If I understand this correctly, I'm afraid the new naming scheme will collide with that of Bibble, which also uses the basename.extension +".xmp" scheme, e.g., "DSC1234.arw.xmp".

How about basename.extension + ".dgk.xmp"?

@17: apps should be friendly to existing metadata in XMP and should not solve the problem of limited interoperability the "easy" way of creating its own sidecar. (Can you imagine the mess: mybestphoto.cr2.darktable.xmp, mybestphoto.cr2.digikam.xmp, mybestphoto.cr2.bibble.xmp,...)

Actually I would rather have the "mess" of different sidecars than trusting all applications to play nice.

We certainly dont wont to create proprietary sidecar files, one for each application. There is no sense in using a standard if only your own application will access it.

I'm totally agree with Marcel, metadata must be universal...

In XMP, each application can create a dedicated namespace. digiKam has one of this.

Gilles Caulier

Not reproducible using digiKam 7.3.0 + Exiv2 0.27.3