Bug 264007 - XMP sidecar file does not distinguish between images of the same name but with different extension
Summary: XMP sidecar file does not distinguish between images of the same name but wit...
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Metadata-Sidecar (show other bugs)
Version: 2.0.0
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-01-22 21:01 UTC by Milan Knížek
Modified: 2021-04-07 18:40 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In: 7.3.0


Attachments
darktable sidecar file example (3.46 KB, text/plain)
2011-03-07 12:17 UTC, Martin Fahrendorf
Details
Geequie XMP sidecar file example (1.14 KB, application/xml)
2011-03-13 02:51 UTC, Christoph Anton Mitterer
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Milan Knížek 2011-01-22 21:01:29 UTC
Version:           2.0.0 (using KDE 4.5.5) 
OS:                Linux

In digiKam 2.0.0 (trunk), the XMP sidecar files with metadata is saved with a pattern "image_base_name".xmp. This leads to situation, when image.crw, image.jpg, image.cr2, etc. have to share the same metadata.

Reproducible: Always



Expected Results:  
It would be better to use XMP filename "image_base_name.ext.xmp".

If someone argues that image.cr2 and image.jpg are variants of the same image, then there could be a switch in digiKam settings to allow both types of naming convention.
Comment 1 Milan Knížek 2011-01-23 18:55:17 UTC
Well, I just found that the current digiKam's implementation is quite hostile to XMP keys created by other programs (in my case by darktable) and simply removes them.
Until the hostility is solved, it might be better not to follow up the bug reported above. (It is better to have two XMP files...)
Comment 2 Marcel Wiesweg 2011-01-24 14:05:29 UTC
Hm is there a standard about sidecar file naming? How is this supposed to be solved?

And please report the other issue, preferably with a sample sidecar file. Thanks!
Comment 3 Milan Knížek 2011-01-24 20:18:24 UTC
I googled shortly and it looks like Adobe Lightroom intended XMP sidecars only for images, which cannot easily embed the xmp inside. That is: XMP would exist only for raw workflow to camera proprietary formats.

I.e. Adobe first created a flexible solution and then limited it to raw files.

Another trouble arise if a sidecar is shared for image.jpg and image.cr2: if the user adds tags to cr2 file, it gets written to image.xmp, however image.jpg would not get automatically updated for the new tags. As a consequence, if the user adds different tags to image.jpg, the image.xmp gets overwritten and original tags are lost...

So either digiKam disables writing sidecars for image types, which support embedding (which would be a pitty IMHO and would not solve the problem for situations like image.crw vs image.cr2 anyway), or better start using image.ext.xmp format.

The latter might of course create incompability with other applications...
Comment 4 Milan Knížek 2011-01-24 20:55:03 UTC
Some reading from Adobe for various multimedia files and XMP:
http://www.adobe.com/content/dam/Adobe/en/devnet/xmp/pdfs/XMPSpecificationPart3.pdf
Comment 5 Marcel Wiesweg 2011-02-05 13:18:19 UTC
"For applications that need to find external XMP files, look in the same directory for a file with the same name as
the main document but with an .xmp extension. (This is called a sidecar XMP file.)"

Doesn't really help, one reader will assume the "name" is the basename, the other will assume the "name" is the complete file name.

Which path does Adobe and other software take? name.suf.xmp or name.xmp? In the latter case, they'll have the same problem.
Comment 6 Milan Knížek 2011-02-05 18:45:27 UTC
MS Windows users do not know what a file extension is :-)

Google says, Adobe product creates sidecar xmp files only for proprietary raw files and names them "name.xmp". For other common file formats, the xmp is embedded and no sidecar file created (JPEG, DNG).
http://www.oreillynet.com/digitalmedia/blog/2007/04/xmp_sidecar_files_and_lightroo.html

Better someone actually confirms if it still is true nowadays...

P.S. Having read more about the matter, I would now prefer compatibility (name.xmp) to flexibility (name.ext.xmp).
Comment 7 Martin Fahrendorf 2011-03-07 12:17:02 UTC
Created attachment 57743 [details]
darktable sidecar file example
Comment 8 caulier.gilles 2011-03-07 12:49:45 UTC
Andreas, 

Do you have any Exiv2 users feedback about XMP sidecar file naming convention ?

DarkTable use a naming notation based on complete name of the origin file with .xmp attached. With this we can distinguish between foo.jpg and foo.cr2 file where the sidecar files become foo.jpg.xmp and foo.cr2.xmp.

Do you know other photo management application which use the same way ? What do you think about ?

Gilles Caulier
Comment 9 Andreas Huggel 2011-03-07 14:57:55 UTC
> Do you have any Exiv2 users feedback about XMP sidecar file naming convention 

Maybe someone listening on the Exiv2 forum has an opinion: http://dev.exiv2.org/boards/3/topics/842
Comment 10 Christoph Anton Mitterer 2011-03-13 02:48:33 UTC
Geeqie places the files as .metadata/<complete filename>.gq.xmp, where <compelte filename> is e.g. "image.png".

I'm not sure whether the subdir is really covered by the standard, but it has at least the advantage of hiding the (possible loads of) sidecar files.

(Not sure why they've added the ".gq" part...
Comment 11 Christoph Anton Mitterer 2011-03-13 02:51:08 UTC
Created attachment 57912 [details]
Geequie XMP sidecar file example
Comment 12 Michael G. Hansen 2011-03-13 11:53:57 UTC
(In reply to comment #10)
> Geeqie places the files as .metadata/<complete filename>.gq.xmp, where
> <compelte filename> is e.g. "image.png".
> 
> I'm not sure whether the subdir is really covered by the standard, but it has
> at least the advantage of hiding the (possible loads of) sidecar files.

IMHO hiding the files in an extra folder is not a good idea, because when a user copies the RAW file using a file manager to another folder, the copied file has lost all metadata. If sidecar files are visible, to user can easily see that there are extra files to take care of. In digikam we could hide the xmp files and transparently copy them along.

Michael
Comment 13 Marcel Wiesweg 2011-03-13 16:26:37 UTC
As we learn on the exiv2 forum, the mess is complete! now that some applications even use both conventions at the same time to have to different sidecar files...
The original specification is not precise, yet leans to the basename.xmp convention. This simply fails to handle the case in the original report here. The basename.suffix.xmp convention looks technically superior then.
It would be interesting to see what major competitors do when they are given such files to _read_ even if they write in the basename.xmp convention?

For reading, we can easily support both, giving priority to basename.suffix.xmp for example. For writing, I would tend to use basename.suffix.xmp unless that makes us incompatible with market leaders.
Comment 14 Christoph Anton Mitterer 2011-03-13 22:41:37 UTC
a) "Loosing" the meta-data is IMO _always_ a risk, whether you have it in a subdir or not. (Of course not if you move/copy/etc with "aware" programs).

b) Even if the original convention would be to use basename.xmp, I'd suggest to:

_write_ <fullname.xmp> ... (perhaps except a user - for compatibility reason - configures manually not to do so)

_read_ both styles (perhaps even with and without things like a .metadata-dir.
In cases where it's not clear which file is meant (e.g. when having basename.xmp) the metadata could simply be taken for all versions of that file.
Comment 15 caulier.gilles 2011-03-17 12:30:41 UTC
This is my view-point about the lead subject of this entry : we must support a naming convention including original filename extension.

Typical case :

0/ Original file is RAW : myphoto.arw

1/ i apply digiKam properties in my workflow for ex. to identify good items in my album. arw is read only. digiKam create myphoto.arw.xmp

2/ I start to edit original file, and create a PNG copy as first refence of my work. I apply another digiKam properties to this new file, different than original. digiKam create myphoto.png.xmp

3/ from this PNG image i create a ready to publish JPEG. Same way than with PNG about digiKam properties. digiKam create myphoto.jpg.xmp.

To resume files below will be created in this ex. :

myphoto.arw            // original
myphoto.arw.xmp
myphoto.png            // lossless copy of modified image
myphoto.png.xmp
myphoto.jpg            // lossy copy ready to publish, print, export...
myphoto.jpg.xmp

For each item created, users can customize properties. We must to have a customized XMP file for each items created/managed in digiKam. Using file name extension is the right way to do it.

Gilles Caulier
Comment 16 caulier.gilles 2011-03-17 14:44:39 UTC
Git commit 0a420804d83d6a1c33253b14626a502d7834c41b by Gilles Caulier.
Committed on 17/03/2011 at 14:42.
Pushed by cgilles into branch 'master'.

handle XMP sidecar files using full name of files with extention + ".xmp"
BUGS: 264007

M  +16   -5    libkexiv2/kexiv2.cpp     
M  +6    -0    libkexiv2/kexiv2.h     
M  +2    -4    libkexiv2/kexiv2_p.cpp     

http://commits.kde.org/libkexiv2/0a420804d83d6a1c33253b14626a502d7834c41b
Comment 17 Per Ångström 2011-03-26 18:01:35 UTC
If I understand this correctly, I'm afraid the new naming scheme will collide with that of Bibble, which also uses the basename.extension +".xmp" scheme, e.g., "DSC1234.arw.xmp".

How about basename.extension + ".dgk.xmp"?
Comment 18 Milan Knížek 2011-03-26 20:03:40 UTC
@17: apps should be friendly to existing metadata in XMP and should not solve the problem of limited interoperability the "easy" way of creating its own sidecar. (Can you imagine the mess: mybestphoto.cr2.darktable.xmp, mybestphoto.cr2.digikam.xmp, mybestphoto.cr2.bibble.xmp,...)
Comment 19 Per Ångström 2011-03-26 20:28:25 UTC
Actually I would rather have the "mess" of different sidecars than trusting all applications to play nice.
Comment 20 Marcel Wiesweg 2011-03-28 22:16:23 UTC
We certainly dont wont to create proprietary sidecar files, one for each application. There is no sense in using a standard if only your own application will access it.
Comment 21 caulier.gilles 2011-03-28 22:52:28 UTC
I'm totally agree with Marcel, metadata must be universal...

In XMP, each application can create a dedicated namespace. digiKam has one of this.

Gilles Caulier
Comment 22 caulier.gilles 2021-04-07 18:40:03 UTC
Not reproducible using digiKam 7.3.0 + Exiv2 0.27.3