Bug 220545

*** Bug 224784 has been marked as a duplicate of this bug. ***

If I understood Caulier, this should be implemented through libkexiv2, without any change in digiKam. If someone could give me a hint as to where to start in libkexiv2, I would be interested in getting my hands dirty...

Hmm... Looking through the KExiv2::load and save functions I think the support for XMP sidecar files might already be there, because Exiv2::XmpSidecar inherits from Exiv2::Image. So, by supplying a filename like photo.xmp, should read from, or write to, an XMP sidecar file.

I think what needs to be done is rather provide a means to tell digiKam to read from or write to XMP sidecar files, i.e. by supplying a filename that ends with '.xmp'.

The following are ways to do this (both should be made available, in my opinion):
1) An option in the metadata settings tab to have digikam always write XMP sidecar files, or perhaps only for certain image formats (e.g. raw formats). Not sure about reading...
2) Two commands in the menu (and contextual menu) "Read (write) metadata from (to) XMP sidecar file". This approach would be the easiest to implement, and the approach I would like to try first. The command would then just issue KExiv2::load(xmpSidecarFilename) and use the loaded metadata for the photo, or vice versa get the metadata from the photo and issue KExiv2::save(xmpSidecarFilename). If there is already some metadata associated with the photo, only the metadata that was loaded should be replaced.

What are your thoughts on this?

First, watch the little 'k' ;-)
You need to implement sidecar support in libkexiv2, using libexiv2. It's just our Qt/KDE layer on top of exiv2.

In digikam I see a number of different workflows, from ignoring all sidecars to using them exclusively. We'd need user input about the workflow needs here.
Also we should keep in mind that Sidecars carry only XMP; how to handle concurrently present Exif and IPTC in the picture itself? There are a few situations to think of.

Yeah, the little k is easy to miss... But I think in this case I have double checked: KExiv2::load and save use instances of Exiv2::Image obtained from Exiv2::ImageFactory::open(filename) for reading and writing. This is why it seems to me that it will also read and write XMP sidecar files. Have you experienced that this is not the case?

I think you're absolutely right about many different workflows with different uses of XMP sidecars. This should definitely be checked somehow with users.

As I understand XMP, everything in EXIF and IPTC can be put in XMP, using the corresponding namespaces. So this may not be a problem after all. But, as you say, metadata can be found both in the image file itself, and in an accompanying XMP sidecar file.

This is one reason to first implement a menu command solution, like the one I suggested in method 2), above.

>As I understand XMP, everything in EXIF and IPTC can be put in XMP, using the
>corresponding namespaces. So this may not be a problem after all. But, as you
>say, metadata can be found both in the image file itself, and in an
>accompanying XMP sidecar file.

Warning, one important part cannot be found in XMP : Exif makernotes. This is a huge problem there, because these private data cannot be translate to user readable xmp tags.

My proposal with Andreas Huggel (Exiv2 author, available in this room), is to set a single XMP tag to host whole binary Exif makernote tag content as well. He is not very convinced by this way, but at least, we not lost any import camera maker data in XMP.

Look like ExifTool do something like that :

http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/XMP.html

... search "Markernote" section...

Gilles Caulier

> Warning, one important part cannot be found in XMP : Exif makernotes. This is a
> huge problem there, because these private data cannot be translate to user
> readable xmp tags.

Ah, yes, I remember having read something about that at some point. What's in these makernotes?

Anyway, my main interest in getting XMP sidecar support (yes, I'm perhaps selfish) is to make it easy to share metadata with Bibble 5 (both ways), so the makernotes are of less importance to me.

I do realize that a perfect solution from the start would be the best, but maybe I could first get reading and writing XMP sidecar files supported from a menu command, and then we can see where we want to go from there?

I'm not sure where in digiKam to start coding, though...

Reading XMP sidecar files works in libkexiv2! I just did a quick test:

1. Select any photo.
2. From the menu: Image -> Metadata -> Import XMP...
3. Select a .xmp file with some tags

I imported a file with some exif:GPS*, lr:HierarchicalSubject, and dc:subject tags, and they were all imported into the XMP metadata of the selected photo.

Exiv2 (that without the little 'k' ;) converts a number of Exif tags and IPTC datasets to the XMP sidecar and back. Try it out with the exiv2 command line utility. 

Gilles is right about the makernote, that is lost. It's also true that I'm not keen to dump the makernote as a large binary blob into an XMP property as exiftool does. It is unlikely that any other application would ever be able to deal with such a blob properly. Instead I suggested to write the _known_ makernote tags as readable properties in new custom namespaces (e.g., like Xmp.Nikon3.WhiteBalance). But that's just an idea at this point, no such code exists yet.

> maybe I could first get reading and writing XMP sidecar files supported from a
> menu command, and then we can see where we want to go from there?

I think that should be possible. In digiKam the main concern is about the workflow and the options to control the handling of XMP sidecars. When it comes to the contents of the sidecar files, you'll need to turn to libexiv2. 

Andreas

Andreas, thanks for your reply!

(In reply to comment #9)
> Exiv2 (that without the little 'k' ;) converts a number of Exif tags and IPTC
> datasets to the XMP sidecar and back. Try it out with the exiv2 command line
> utility. 
Yes, it's a very handy tool. I guess you mean "... IPTC datasets _in image files_ ..."? In my simple test above, the XMP encoded Exif and IPTC stuff was returned in the XMP 'field' of the Exiv2::Image - not in the Exif and IPTC fields, respectively, as far as I could tell. How can this be accomplished? The case is different from what the command line utility does with the "-i X" option in that it _only_ reads the contents of the XMP sidecar, and doesn't have an image file to put write it to. Is it in the writing that the conversion happens?

I suppose that at least when I would do a "save" from libkexiv2 (which puts Exif, IPTC and XMP metadata in the Exiv2::Image using setExifData, setIPTCData, and setXMPData, respectively, and then calls writeMetaData) the Exif and IPTC data would get converted to XMP tags, and written to the XMP sidecar? Can you confirm this behaviour?
 
> Gilles is right about the makernote, that is lost. It's also true that I'm not
> keen to dump the makernote as a large binary blob into an XMP property as
> exiftool does. It is unlikely that any other application would ever be able to
> deal with such a blob properly. Instead I suggested to write the _known_
> makernote tags as readable properties in new custom namespaces (e.g., like
> Xmp.Nikon3.WhiteBalance). But that's just an idea at this point, no such code
> exists yet.

I must say that, personally, I have no opinion about these makernotes, as I do not know anything about them...
 
> > maybe I could first get reading and writing XMP sidecar files supported from a
> > menu command, and then we can see where we want to go from there?
> 
> I think that should be possible. In digiKam the main concern is about the
> workflow and the options to control the handling of XMP sidecars. 

Agreed.

> When it comes
> to the contents of the sidecar files, you'll need to turn to libexiv2. 

I'm not following... Are you talking about the issue with getting the XMP encoded Exif and IPTC tags converted into proper Exif and IPTC, i.e. the issue discussed at the top of this reply?

> 
> Andreas

about XMP sidecar support, see below my proposal :

1/ read XMP sidecar : 
 - must be used when metadata must be imported to database (scan at startup for ex.).
 - in first image metadata must be check, including makernotes. if nothing is find, XMP sidecar is the alternative.
 - this include all file format as images and video files.

2/ Write XMP sidecar :
 - must be used to sync image metadata with database contents.
 - in first metadata must be written in image file. If image format is read only or not supported by Exiv2, XMP sidecar must be created. This is the case of RAW files or video files for ex.
 
To manage these rules, 2 implementation must be fixed :

1/ libkexiv2, especially read and write method.
2/ digiKam MetadataHub which is the core metadata management from digiKam.
3/ digiKam metadata settings pannel where option must be added to play with XMP sidecar.

links to code :

1/ => http://websvn.kde.org/trunk/KDE/kdegraphics/libs/libkexiv2/libkexiv2/kexiv2.cpp?revision=1070289&view=markup
2/ => http://websvn.kde.org/trunk/extragear/graphics/digikam/digikam/metadatahub.cpp?revision=1050241&view=markup
3/ => http://websvn.kde.org/trunk/extragear/graphics/digikam/utilities/setup/setupmetadata.cpp?revision=1075046&view=markup

Important : only libkexiv2 and digiKam from trunk must be fixed in this way.

Marcel, i forget something there ?

Gilles Caulier

I forget a very important features : manage image file and xmp sidecar file as well :

1/ if image is removed, remove xmp sidecar too.
2/ if image is renamed, rename xmp sidecar too.
3/ if image is moved, move xmp sidecar too.

typically, xmp file name is image file name without image format extension, this one is replaced by xmp. ex : DSC12568.cr2 => DSC12568.xmp

Gilles Caulier

Re comment #10:

> I suppose that at least when I would do a "save" from libkexiv2 (which puts
> Exif, IPTC and XMP metadata in the Exiv2::Image using setExifData, setIPTCData,
> and setXMPData, respectively, and then calls writeMetaData) the Exif and IPTC
> data would get converted to XMP tags, and written to the XMP sidecar? Can you
> confirm this behaviour?

Yes, XmpSidecar::writeMetadata() converts certain Exif and IPTC tags from their respective Exiv2 containers as set in the XmpSidecar to XMP properties in the resulting file. So you'll find properties like exif:FNumber in the *.xmp file.

XmpSidecar::readMetadata converts (copies) these properties back into the Exiv2 Exif and IPTC containers. So you'll find the Exif tag Exif.Photo.FNumber in the ExifData container and the XMP property Xmp.exif.FNumber in the XmpData container after reading the sidecar.

>> When it comes
>> to the contents of the sidecar files, you'll need to turn to libexiv2. 
>
> I'm not following... Are you talking about the issue with getting the XMP
> encoded Exif and IPTC tags converted into proper Exif and IPTC, i.e. the issue
> discussed at the top of this reply?

Yes. I was referring to the conversion described above. That code in Exiv2 will be the place to tweak to tweak what Exif and IPTC metadata are converted and how they are stored in the sidecar file.

Andreas

(In reply to comment #11)
> about XMP sidecar support, see below my proposal :
> 
> 1/ read XMP sidecar : 
>  - must be used when metadata must be imported to database (scan at startup for
> ex.).
>  - in first image metadata must be check, including makernotes. if nothing is
> find, XMP sidecar is the alternative.

My opinion on this is that tags present in an associated XMP sidecar file should take precedence over the tags found in the image. In any case, an existing sidecar file would need to be read even if there is metadata in the image/video file itself, because it may contain other tags than what's found in the image/video.

>  - this include all file format as images and video files.
> 
> 2/ Write XMP sidecar :
>  - must be used to sync image metadata with database contents.
>  - in first metadata must be written in image file. If image format is read
> only or not supported by Exiv2, XMP sidecar must be created. This is the case
> of RAW files or video files for ex.

I think this should be set in preferences. I would rather _not_ write to the image, even if it is writable, but rather use the XMP sidecar.

> 
> To manage these rules, 2 implementation must be fixed :
> 
> 1/ libkexiv2, especially read and write method.
> 2/ digiKam MetadataHub which is the core metadata management from digiKam.
> 3/ digiKam metadata settings pannel where option must be added to play with XMP
> sidecar.
> 
> links to code :
> 
> 1/ =>
> http://websvn.kde.org/trunk/KDE/kdegraphics/libs/libkexiv2/libkexiv2/kexiv2.cpp?revision=1070289&view=markup
> 2/ =>
> http://websvn.kde.org/trunk/extragear/graphics/digikam/digikam/metadatahub.cpp?revision=1050241&view=markup
> 3/ =>
> http://websvn.kde.org/trunk/extragear/graphics/digikam/utilities/setup/setupmetadata.cpp?revision=1075046&view=markup
> 
> Important : only libkexiv2 and digiKam from trunk must be fixed in this way.

Can I use 1.2.0 as a base, though?

> 
> Marcel, i forget something there ?
> 
> Gilles Caulier

Jakob Malm

(In reply to comment #12)
> I forget a very important features : manage image file and xmp sidecar file as
> well :
> 
> 1/ if image is removed, remove xmp sidecar too.
> 2/ if image is renamed, rename xmp sidecar too.
> 3/ if image is moved, move xmp sidecar too.
> 
> typically, xmp file name is image file name without image format extension,
> this one is replaced by xmp. ex : DSC12568.cr2 => DSC12568.xmp
> 
> Gilles Caulier

Absolutely! This must be the final intention, but cannot be my first goal.

Just to let you know that I'm still working on this. I have some working code, but am right now thinking of how to make Exif and IPTC tags be written to the XMP sidecar. In the KExiv2::save function, before these tags are put in the Image object, it is checked whether the file has support for them. For XMP sidecar files, this is not the case and these fields are never set, and hence, information only found in Exif and IPTC tags are not written to the XMP sidecar. Perhaps I should just add a check to see if we're using XMP sidecar and write the fields anyway.

But it may be a question that actually is worth considering more generally. Can't we let exiv2 handle the issue of whether Exif and IPTC fields can be written to the file? In the XMP sidecar case, exiv2 will do the conversion. What about other cases? Will it convert, skip, or produce an error?

Allowing to use XMP sidecar file and to not touch the original files will also
improve backups: for now, each time I change any metadata (usually tags), the
file is backuped again and again. My backup program (backuppc) must store a
full new version of the image instead of using (hard)links to the previous
'version'. If XMP sidecar were used, the original image would not change,
avoiding similar but not exact copy in each backup. XMP files would be
duplicated but their size is a lot smaller than the original image.

A second thought: I see that you will use the same basename with the .xmp
extension. How will you deal with 'foo.jpg' and 'foo.avi' when both present ?
(ignore one ? which one in this case ?) Won't 'foo.jpg.xmp' and 'foo.avi.xmp'
be a better choice ?

A third thought: some other software (geeqie) allow to read/write xmp files in
another directory. This way, photos can be on a read-only medium (CD or ro NFS)
and xmp files (and so metadata) can still be edited and saved on the hard-disk.

I hope a support for using XMP files will soon be present. Many thanks for your
work on this part.

Regards,
  Vincent

(In reply to comment #17)
> Allowing to use XMP sidecar file and to not touch the original files will also
> improve backups: for now, each time I change any metadata (usually tags), the
> file is backuped again and again. My backup program (backuppc) must store a
> full new version of the image instead of using (hard)links to the previous
> 'version'. If XMP sidecar were used, the original image would not change,
> avoiding similar but not exact copy in each backup. XMP files would be
> duplicated but their size is a lot smaller than the original image.

Yes, this will indeed be a very nice effect. Thanks for your encouragement!

> A second thought: I see that you will use the same basename with the .xmp
> extension. How will you deal with 'foo.jpg' and 'foo.avi' when both present ?
> (ignore one ? which one in this case ?) Won't 'foo.jpg.xmp' and 'foo.avi.xmp'
> be a better choice ?

For me, personally, foo.xmp is necessary. Perhaps it should be configurable? Is foo.xmp a convention?

> A third thought: some other software (geeqie) allow to read/write xmp files in
> another directory. This way, photos can be on a read-only medium (CD or ro NFS)
> and xmp files (and so metadata) can still be edited and saved on the hard-disk.

Hmm... Interesting idea. In principle, this should be quite easy to implement. But I would eventually like digikam to be able to reread the XMP sidecar when it changes, and for this, it would need to be situated in a directory which belongs to an album, so that digikam will notice the change. This might complicate things a little bit.

Jakob

Ok, so I tried changing the checks mentioned in comment 16, and exif data is now also written to the XMP sidecar file. However, it seems that changing geolocation information alone doesn't initiate a metadata save. Also, XMP sidecar files for images without other tags set do not get written at all, even when issuing a "Write metadata to photo". Is this really the intended behaviour? 

In practice, this means that if you want geoinformation to be written to the file, you have to have some other tag as well, which is a bit odd in my mind.

Trying to get my head around this behaviour...

Comment, DateTime, Rating, Template, Tags

From metadatahub.cpp, these appear to be the only metadata that cause any metadata at all to actually be written to a file. Is this correct?

Well that depends from which place you choose to write the metadata.
Comment, DateTime, Rating, Template, Tags are currently supported by MetadataHub, which basically handles all metadata writing for us. Digikam does not write any other metadata (but "Template" includes a lot)
GPS metadata is not written at all by digikam currently but by the GPS correlator Kipi plugin.

Having tested my XMP sidecar fix a little bit now, I'm not sure it's the best solution to have sidecars written automatically. I experience quite som lag when editing the tags of several photos at once. Perhaps it would be better to just have this functionality as two menu commands: read / write XMP sidecar...

Do you mean UI hangs, or just the process taking time?
In the first case it's a problem, because with MetadataManager all metadata file writing should be done in a thread.
In the latter case I recommend benchmarking - is it a problem problem of CPU intensive work, callgrind could tell you.

Please, to handle XMP sidecar generation, add an option to digiKam Metadata control panel, to provide an homogenous interface.

I'm agree to have an entry in Image/Metadata menu entry to generate XMP sidecar with current selected album items. This entry is drived by MetadataEdit kipi-plugin, not digiKam itself.

Gilles Caulier

Created attachment 47754 [details]
patch to digikam 1.2.0

I thought I'd upload what I have so far. The code includes a Metadata setting "Read metadata from and write metadata to XMP sidecars (experimental)" option. It does not yet include menu items for reading / writing XMP sidecars.

One problem that needs to still be addressed is that the geolocator plugin only seems to writes to the sidecar if it already exists. I haven't looked into this yet.

(Oops, I thought I could attach the files together with this message...)

Created attachment 47755 [details]
patch to libkexiv2 svn revision 1111830

*** Bug 184638 has been marked as a duplicate of this bug. ***

*** Bug 192024 has been marked as a duplicate of this bug. ***

Patch looks fine for me so far. With KDE 4.5 in freeze, we cannot add it anyway so far, and will need #ifdefs for the libkexiv2 versions prior to this patch.

Jakob, is this implementing all your intended behavior, I mean, is this patch finished or do you have more plans?

Marcel,

I propose to make a copy of current libkexiv2 from trunk into digiKam GoSC branch and to apply patch against. Also i propose to fix digiKam code from this branch with this patch.

What do you think about ?

Gilles Caulier

Marcel, 

digiKam from GoSC branch need to be synchronized with trunk. I propose to only apply small fixes in trunk and concentrate all works in GoSC branch now. 

What do you think about ?

Gilles Caulier

I will in any case mostly work in the GSoc branch in the next weeks. Merge with trunk is under way.
I agree to apply the patches in the branch.

ok. so, i will branch libkexiv2 and apply patch

Gilles

Jakob,

In your libkexiv2 patch, test files "usexmpsidecar.cpp" and "readimagewritexmpsidecar.cpp" are missing.

Gilles Caulier

SVN commit 1142633 by cgilles:

apply patch #1111830 from Jakob Malm
Tests files are missing in this commit
CCBUGS: 220545


 M  +16 -0     CMakeLists.txt  
 M  +42 -4     libkexiv2/kexiv2.cpp  
 M  +9 -0      libkexiv2/kexiv2.h  
 M  +2 -0      libkexiv2/kexiv2_p.h  
 M  +8 -0      test/CMakeLists.txt  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1142633

SVN commit 1142678 by cgilles:

apply patch #47754 from Jakob Malm


 M  +1 -0      digikam/albumiconview.cpp
 M  +16 -0     digikam/albumsettings.cpp
 M  +3 -0      digikam/albumsettings.h
 M  +3 -0      digikam/metadatahub.cpp
 M  +1 -0      digikam/metadatahub.h
 M  +1 -0      digikam/metadatamanager.cpp
 M  +4 -0      utilities/kipiiface/kipiinterface.cpp
 M  +14 -0     utilities/setup/setupmetadata.cpp

Wow! That's a lot of activity today. Great to see that the patch is getting in. Did any of you try to use it?

Marcel: Yes, and no. Yes, this patch includes all the work that I will do in the near future. And no, it does not include all my intended functionality. I hope to be able to add functionality to write to / read from sidecar files from a menu command, as a possibility for those who do not want automatic reading/writing sidecars. But I will have to do this some other time, I think. Preferably on an already patched version.

Gilles: I will upload the test files. But I must say, I'm not very used to writing test files (though I really need to get into the habit of doing it), so they may need some refinement... Would you have a look?

Created attachment 48334 [details]
Test to read and write XMP sidecar. The test is incomplete!

I didn't realize this test was far from complete...

Created attachment 48336 [details]
Test to read metadata from image and write to XMP sidecar.

SVN commit 1143208 by cgilles:

add test pograms for XMP sidecar management
CCBUGS: 220545


 M  +6 -6      CMakeLists.txt  
 AM            readimagewritexmpsidecar.cpp   [License: GPL (v2+)]
 AM            usexmpsidecar.cpp   [License: GPL (v2+)]


WebSVN link: http://websvn.kde.org/?view=rev&revision=1143208

Jakob,

kipi-plugins need to be patched too, to support xmp sidecar as well. The most important tool is metadataedit. Can you take a look ?

Thanks in advance

Gilles Caulier

Jakob,

Your patches are now applied to digiKam 1.5.0 which is hosted to a dedicated branch for Google Summer of Code 2010. It's not in trunk where you will found digiKam 1.4.0.

digiKam 1.4.0 will be planed in one mounth, when KDE 4.5 will be out.
digiKam 1.5.0 will be planed later than september, after digiKam team reunion at end of august in France (http://techbase.kde.org/Projects/Digikam/CodingSprint2010).

digiKam google summer of code 2010 branches can be find there :

digikam : http://websvn.kde.org/branches/extragear/graphics/digikam/gsoc-2010/
libkipi : http://websvn.kde.org/branches/extragear/graphics/libkipi-gsoc-2010/
libkexiv2 : http://websvn.kde.org/branches/extragear/graphics/digikam/libkexiv2/

this version of digiKam need 2 new libraries to compile (under development) :

libkface : http://websvn.kde.org/branches/extragear/graphics/digikam/libkface/
libkmap : http://websvn.kde.org/branches/extragear/graphics/digikam/libkmap/

libkface is for face detection. It require libface and openCV libraries
libkmap is a wrapper around world-map services as googlemaps and openstreetmap using marble widget.

kipi-plugins is not yet branched there. I will do it.

Gilles Caulier

Gilles,

I may be able to take a look at kipi-plugins and metadataedit specifically sometime this week, but I can't promise...

/Jakob

Jakob,

forget all my urls given on #43. I managed whole dirs to be at the same place. You can now checkout all code directly using one svn call. Look there :

http://websvn.kde.org/branches/extragear/graphics/

... and take a look on README.GoSC2010

Thanks in advance for your help

Gilles Caulier

I have detected a bug in my implementation, but I'm having trouble finding the solution, probably due to my limited knowledge of C++...

The value d->useXMPSidecar in instances of DMetadata and KExiv2 is completely random when reading metadata. It does not seem to get set properly. This has the effect that the boolean is almost always true, which of course is a problem.

1.) KExiv2Priv::KExiv2Priv() should have an initialization useXMPSidecar = false;
2.) I don't know where to set the value of useXMPSidecar correctly. It currently gets set at the same places as writeRawFiles, but this means it is only set correctly for writing - not reading.

If you have an idea of how to deal with this, please let me know!

done with commit #1146000 :

http://websvn.kde.org/branches/extragear/graphics/libkexiv2/libkexiv2/kexiv2_p.cpp?revision=1146000&view=markup

It sound better ?

Gilles Caulier

Gilles,

Yes, this solves 1). Do you have an idea for 2)?

Why do you said that for reading this value is not set to false, as constructor switch off it as well ?

Gilles Caulier

Hmm, I'm not sure I understand your question... But let me try to answer.

Adding the initialization in the constructor of course solves the problem with useXMPSidecar having just some random value. But it doesn't get set to the value set in the settings window. So this way, instead of almost always reading from sidecar, it _never_ reads from sidecar.

What I need to know is where the proper place to set useXMPSidecar is, for it to be available in KExiv2IFace::KExiv2::load

As it is done for WriteRawFile stuff at 

http://websvn.kde.org/branches/extragear/graphics/digikam/digikam/metadatahub.cpp?revision=1144518&view=markup

line 476, you need to patch void MetadataHub::load(const DMetadata& metadata) method in the same way.

Marcel, there is another place to patch in digiKam to handle XMP sidecar with metadata registration in DB ?

Gilles Caulier

Yes, in metadatamanager.cpp is one occurrence of setWriteRawFiles. Nowhere else.

Marcel,

Yes in MetadataManager, but here we talk about to _read_ XMP sidecar and register data to DB. setWriteRAWFile() is only used to write/update metadata to file. It's a different case.

The new method about XMP sidecar management is setUseXMPSidecar(). The question is where we need to turn on/off this method in digiKam code.

Gilles Caulier

Oh, there are a lot of places where a DMetadata object is loaded. Grep for it, can be about 100 occurrences. And a lot of places where AlbumSettings is not available.

I suggest to set this setting / these settings centrally with a static method in DMetadata. Because DMetadata can be used very early after startup, it should even be able to read the setting directly from KConfig, with a method to trigger re-reading it when changed.

If there are going to be more settings than just one, we should consider setting up a MetadataSettings singleton.
We had similar problems and have come up with a similar solution for color management settings: See the IccSettings class.

*** Bug 228585 has been marked as a duplicate of this bug. ***

I have not progressed any further with this...

Marcel and Gilles, have you discussed the strategy concerning settings any further?

Jakob,

No more. Marcel as explained how to handle settings in #54 and give some vay to check.

I don't have yet tried it, but it sound good for me afetr to take a look in ICCSettings class as implementation example.

Note : XMP sidecar support must always be implemented in digiKam 2.0.0. Some changes have been done by me recently : the libkexiv2 required to build digiKam 2.0.0 have been moved back to kdegraphics/libs from trunk, since KDE 4.5.0 have been released.

We have planed to solve this entry for next coding sprint : 

http://techbase.kde.org/Projects/Digikam/CodingSprint2010

So, if you have time to code the last patch to solve it before the sprint, we will review it as well in time. This will simplify work team and we can concentrate effort for other pending task...

Gilles Caulier

Gilles Caulier

SVN commit 1165878 by cgilles:

introduce Metadata Settings class to manage XMP sidecar files. Not yet used


 M  +2 -0      CMakeLists.txt
 AM            libs/dmetadata/metadatasettings.cpp   [License: GPL (v2+)]
 AM            libs/dmetadata/metadatasettings.h   [License: GPL (v2+)]
 AM            libs/dmetadata/metadatasettingscontainer.cpp   [License: GPL (v2+)]
 AM            libs/dmetadata/metadatasettingscontainer.h   [License: GPL (v2+)]


--- branches/extragear/graphics/digikam/CMakeLists.txt #1165877:1165878
@@ -625,6 +625,8 @@
    SET_SOURCE_FILES_PROPERTIES(${libdimgfilters_SRCS} PROPERTIES COMPILE_FLAGS ${KDE4_ENABLE_EXCEPTIONS})

    SET(libdmetadata_SRCS
+        ${CMAKE_CURRENT_SOURCE_DIR}/libs/dmetadata/metadatasettings.cpp
+        ${CMAKE_CURRENT_SOURCE_DIR}/libs/dmetadata/metadatasettingscontainer.cpp
        ${CMAKE_CURRENT_SOURCE_DIR}/libs/dmetadata/metadatainfo.cpp
        ${CMAKE_CURRENT_SOURCE_DIR}/libs/dmetadata/photoinfocontainer.cpp
        ${CMAKE_CURRENT_SOURCE_DIR}/libs/dmetadata/dmetadata.cpp

SVN commit 1165986 by cgilles:

use everywhere new MetadataSettings class to manage all metadata write settings.
remove AlbumSettings metadata management methods
polish code when it's possible, especially set more private internal containers.
CCBUGS: 220545


 M  +0 -193    digikam/albumsettings.cpp  
 M  +0 -30     digikam/albumsettings.h  
 M  +5 -6      digikam/albumthumbnailloader.cpp  
 M  +8 -4      digikam/albumthumbnailloader.h  
 M  +4 -3      digikam/digikamview.cpp  
 M  +1 -1      digikam/digikamview.h  
 M  +7 -6      digikam/imagepreviewbar.cpp  
 M  +2 -2      digikam/imagepreviewbar.h  
 M  +7 -6      digikam/imagepreviewview.cpp  
 M  +1 -1      digikam/imagepreviewview.h  
 M  +3 -2      digikam/imagepreviewviewv2.cpp  
 M  +1 -1      digikam/imagepreviewviewv2.h  
 M  +4 -39     digikam/metadatahub.cpp  
 M  +6 -42     digikam/metadatahub.h  
 M  +4 -3      digikam/metadatamanager.cpp  
 M  +0 -4      libs/dmetadata/metadatasettings.cpp  
 M  +36 -3     libs/dmetadata/metadatasettingscontainer.cpp  
 M  +17 -2     libs/dmetadata/metadatasettingscontainer.h  
 M  +2 -2      utilities/batch/batchfacedetector.cpp  
 M  +4 -4      utilities/batch/fingerprintsgenerator.cpp  
 M  +2 -2      utilities/batch/fingerprintsgenerator.h  
 M  +7 -4      utilities/imageeditor/editor/imagewindow.cpp  
 M  +2 -2      utilities/kipiiface/kipiimageinfo.cpp  
 M  +9 -3      utilities/kipiiface/kipiinterface.cpp  
 M  +6 -5      utilities/lighttable/lighttablepreview.cpp  
 M  +4 -3      utilities/lighttable/lighttablewindow.cpp  
 M  +2 -1      utilities/queuemanager/queuemgrwindow.cpp  
 M  +46 -33    utilities/setup/setupmetadata.cpp  
 M  +2 -3      utilities/setup/setupmetadata.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1165986

Marcel,

Please make a double verification of my big commit #1165986.

NOTES: 

- Showfoto is not yet patched to read/save XMP sidecar.
- kipi-plugins are not yet patched to handle XMP sidecar. digiKam kipi-interface must export this settings of course.

Jakob, 

Please checkout digiKam from GoSC-2010 branch, recompile it and test with you image collections including XMP sidecar. Please, turn on the right option from Metadata settings panel to handle XMP sidecar.

Let's me hear if all work fine for you...

Gilles Caulier

Note : kipi-interface already export XMP sidecar settings, through Digikam::KipiInterface::line 308.

Gilles Caulier

Gilles,

I will hopefully find some time next week to test the GSoC-2010 branch. But I believe last time I tried, I had some problems with qt or kde dependencies... I will let you know how I fare.

SVN commit 1170226 by cgilles:

Add new wrapper for XMP sidecar support to provide a fine way to manage writting in file and in sidecar.
Separate implementation into sidecar and image file.
Look MetadataWritingMode enum description to know how to deal with metadata writting mode.
digiKam is not yet patched to compile with this API
Also, take a care, code is not yet fully tested in all conditions.
CCMAIL: mike@mghansen.de


 M  +58 -200   kexiv2.cpp
 M  +22 -4     kexiv2.h
 M  +238 -11   kexiv2_p.cpp
 M  +7 -1      kexiv2_p.h

SVN commit 1170286 by cgilles:

use new API from libkexiv2 from trunk to deal with XMP sidecar


 M  +3 -2      digikam/metadatahub.cpp
 M  +1 -1      digikam/metadatahub.h
 M  +2 -1      digikam/metadatamanager.cpp
 M  +13 -3     libs/dmetadata/metadatasettingscontainer.cpp
 M  +3 -1      libs/dmetadata/metadatasettingscontainer.h
 M  +6 -2      utilities/kipiiface/kipiinterface.cpp
 M  +29 -19    utilities/setup/setupmetadata.cpp


--- branches/extragear/graphics/digikam/digikam/metadatahub.cpp #1170285:1170286
@@ -203,9 +203,10 @@
    }
 }

-bool MetadataHub::load(const QString& filePath)
+bool MetadataHub::load(const QString& filePath, const MetadataSettingsContainer& settings)
 {
    DMetadata metadata;
+    metadata.setUseXMPSidecar4Reading(settings.useXMPSidecar4Reading);
    bool success = metadata.load(filePath);
    load(metadata); // increments count
    return success;
@@ -453,7 +454,7 @@
    bool dirty = false;

    metadata.setWriteRawFiles(settings.writeRawFiles);
-    metadata.setUseXMPSidecar(settings.useXMPSidecar);
+    metadata.setMetadataWritingMode(settings.metadataWritingMode);

 #if KEXIV2_VERSION >= 0x000600
    metadata.setUpdateFileTimeStamp(settings.updateFileTimeStamp);
--- branches/extragear/graphics/digikam/digikam/metadatahub.h #1170285:1170286
@@ -142,7 +142,7 @@
        (Uses DMetadata, QFileInfo)
        @returns True if the metadata could be loaded
    */
-    bool load(const QString& filePath);
+    bool load(const QString& filePath, const MetadataSettingsContainer& settings = MetadataSettingsContainer());

    // --------------------------------------------------

--- branches/extragear/graphics/digikam/digikam/metadatamanager.cpp #1170285:1170286
@@ -474,7 +474,8 @@
        DMetadata::ImageOrientation o = (DMetadata::ImageOrientation)orientation;
        metadata.setImageOrientation(o);
        metadata.setWriteRawFiles(MetadataSettings::instance()->settings().writeRawFiles);
-        metadata.setUseXMPSidecar(MetadataSettings::instance()->settings().useXMPSidecar);
+        metadata.setUseXMPSidecar4Reading(MetadataSettings::instance()->settings().useXMPSidecar4Reading);
+        metadata.setMetadataWritingMode(MetadataSettings::instance()->settings().metadataWritingMode);

        if (!metadata.applyChanges())
        {
--- branches/extragear/graphics/digikam/libs/dmetadata/metadatasettingscontainer.cpp #1170285:1170286
@@ -27,10 +27,17 @@

 #include <kconfiggroup.h>

+
+// LibKExiv2 includes
+
+#include <libkexiv2/kexiv2.h>
+
 // Local includes

 #include "metadatasettings.h"

+using namespace KExiv2Iface;
+
 namespace Digikam
 {

@@ -44,7 +51,8 @@
    saveTemplate        = false;
    saveTags            = false;
    writeRawFiles       = false;
-    useXMPSidecar       = false;
+    useXMPSidecar4Reading = false;
+    metadataWritingMode   = KExiv2::WRITETOIMAGEONLY;
    updateFileTimeStamp = false;
 }

@@ -61,7 +69,8 @@
    saveRating          = group.readEntry("Save Rating",           false);

    writeRawFiles       = group.readEntry("Write RAW Files",       false);
-    useXMPSidecar       = group.readEntry("Use XMP Sidecar",       false);
+    useXMPSidecar4Reading = group.readEntry("Use XMP Sidecar For Reading", false);
+    metadataWritingMode   = group.readEntry("Metadata Writing Mode",       (int)KExiv2::WRITETOIMAGEONLY);
    updateFileTimeStamp = group.readEntry("Update File Timestamp", false);
 }

@@ -78,7 +87,8 @@
    group.writeEntry("Save Rating",           saveRating);

    group.writeEntry("Write RAW Files",       writeRawFiles);
-    group.writeEntry("Use XMP Sidecar",       useXMPSidecar);
+    group.writeEntry("Use XMP Sidecar For Reading", useXMPSidecar4Reading);
+    group.writeEntry("Metadata Writing Mode",       metadataWritingMode);
    group.writeEntry("Update File Timestamp", updateFileTimeStamp);
 }

--- branches/extragear/graphics/digikam/libs/dmetadata/metadatasettingscontainer.h #1170285:1170286
@@ -66,8 +66,10 @@
    bool saveTags;

    bool writeRawFiles;
-    bool useXMPSidecar;
    bool updateFileTimeStamp;
+    bool useXMPSidecar4Reading;
+
+    int  metadataWritingMode;
 };

 }  // namespace Digikam
--- branches/extragear/graphics/digikam/utilities/kipiiface/kipiinterface.cpp #1170285:1170286
@@ -303,10 +303,14 @@
    {
        return (set.writeRawFiles);
    }
-    else if (settingName == QString("UseXMPSidecar"))
+    else if (settingName == QString("UseXMPSidecar4Reading"))
    {
-        return (set.useXMPSidecar);
+        return (set.useXMPSidecar4Reading);
    }
+    else if (settingName == QString("MetadataWritingMode"))
+    {
+        return (set.metadataWritingMode);
+    }
    else if (settingName == QString("FileExtensions"))
    {
        // do not save this into a local variable, as this
--- branches/extragear/graphics/digikam/utilities/setup/setupmetadata.cpp #1170285:1170286
@@ -48,6 +48,8 @@
 #include <ktoolinvocation.h>
 #include <kurllabel.h>
 #include <kvbox.h>
+#include <khbox.h>
+#include <kcombobox.h>

 // LibKExiv2 includes

@@ -61,6 +63,8 @@
 #include "metadatapanel.h"
 #include "metadatasettings.h"

+using namespace KExiv2Iface;
+
 namespace Digikam
 {

@@ -86,6 +90,7 @@
        resyncButton            = 0;
        tagsCfgPanel            = 0;
        tab                     = 0;
+        writingModeCombo        = 0;
    }

    bool           exifAutoRotateAsChanged;
@@ -106,6 +111,7 @@
    QCheckBox*     readFromNepomukBox;
    QToolButton*   resyncButton;

+    KComboBox*     writingModeCombo;
    KTabWidget*    tab;

    MetadataPanel* tagsCfgPanel;
@@ -174,35 +180,37 @@
    d->writeRawFilesBox->setWhatsThis( i18n("Turn on this option to write metadata into RAW TIFF/EP files. "
                                            "This feature requires the Exiv2 shared library, version >= 0.18.0. It is still "
                                            "experimental, and is disabled by default."));
-    d->writeRawFilesBox->setEnabled(KExiv2Iface::KExiv2::supportMetadataWritting("image/x-raw"));
+    d->writeRawFilesBox->setEnabled(KExiv2::supportMetadataWritting("image/x-raw"));

-    d->useXMPSidecarBox = new QCheckBox(commonGroup);
-    d->useXMPSidecarBox->setText(i18n("&Read metadata from and write metadata to XMP sidecar files"));
-    d->useXMPSidecarBox->setWhatsThis( i18n("Turn on this option to write metadata into XMP sidecar files, "
-                                            "and prefer metadata from XMP sidecar files when reading metadata."
-                                            "This feature requires the Exiv2 shared library, version >= 0.18.0. It is still "
-                                            "experimental, and is disabled by default."));
-    // TODO: Should perhaps do some testing for XMP sidecar support in libkexiv2
-    // d->useXMPSidecarBox->setEnabled(KExiv2Iface::KExiv2::supportXMPSidecar());
-
    d->updateFileTimeStampBox = new QCheckBox(commonGroup);
    d->updateFileTimeStampBox->setText(i18n("&Update file timestamp when metadata are saved"));
    d->updateFileTimeStampBox->setWhatsThis( i18n("Turn on this option to update file timestamps when metadata are saved."));

-#if KEXIV2_VERSION >= 0x000600
-    d->updateFileTimeStampBox->show();
-#else
-    d->updateFileTimeStampBox->hide();
-#endif
+    d->useXMPSidecarBox = new QCheckBox(commonGroup);
+    d->useXMPSidecarBox->setText(i18n("&Read metadata from XMP sidecar files"));
+    d->useXMPSidecarBox->setWhatsThis( i18n("Turn on this option to prefer metadata from XMP sidecar files when reading metadata."));
+    d->useXMPSidecarBox->setEnabled(KExiv2::supportXmp());

+    KHBox* hbox = new KHBox(commonGroup);
+    QLabel* writingModeLabel = new QLabel(i18n("Metadata Writting Mode:"), hbox);
+    writingModeLabel->setEnabled(KExiv2::supportXmp());
+    d->writingModeCombo      = new KComboBox(hbox);
+    d->writingModeCombo->addItem(i18n("Write to image only"),                           KExiv2::WRITETOIMAGEONLY);
+    d->writingModeCombo->addItem(i18n("Write to XMP sidecar only"),                     KExiv2::WRITETOSIDECARONLY);
+    d->writingModeCombo->addItem(i18n("Write to image and XMP Sidecar"),                KExiv2::WRITETOSIDECARANDIMAGE);
+    d->writingModeCombo->addItem(i18n("Write to XMP sidecar for read-only image only"), KExiv2::WRITETOSIDECARONLY4READONLYFILES);
+    d->writingModeCombo->setToolTip(i18n("Choose there how to write metadata."));
+    d->writingModeCombo->setEnabled(KExiv2::supportXmp());
+
    gLayout2->addWidget(d->saveTagsBox);
    gLayout2->addWidget(d->saveTemplateBox);
    gLayout2->addWidget(d->saveCommentsBox);
    gLayout2->addWidget(d->saveDateTimeBox);
    gLayout2->addWidget(d->saveRatingBox);
+    gLayout2->addWidget(d->updateFileTimeStampBox);
    gLayout2->addWidget(d->writeRawFilesBox);
    gLayout2->addWidget(d->useXMPSidecarBox);
-    gLayout2->addWidget(d->updateFileTimeStampBox);
+    gLayout2->addWidget(hbox);
    gLayout2->setMargin(KDialog::spacingHint());
    gLayout2->setSpacing(0);

@@ -231,7 +239,7 @@
                    "an older standard used in digital photography to store "
                    "photographer information in images.</p>"));

-    if (KExiv2Iface::KExiv2::supportXmp())
+    if (KExiv2::supportXmp())
        txt.append(i18n("<p><a href='http://en.wikipedia.org/wiki/Extensible_Metadata_Platform'>XMP</a> - "
                        "a new standard used in digital photography, designed to replace IPTC.</p>"));

@@ -387,7 +395,8 @@
    set.saveTags            = d->saveTagsBox->isChecked();
    set.saveTemplate        = d->saveTemplateBox->isChecked();
    set.writeRawFiles       = d->writeRawFilesBox->isChecked();
-    set.useXMPSidecar       = d->useXMPSidecarBox->isChecked();
+    set.useXMPSidecar4Reading = d->useXMPSidecarBox->isChecked();
+    set.metadataWritingMode   = d->writingModeCombo->currentIndex();
    set.updateFileTimeStamp = d->updateFileTimeStampBox->isChecked();
    mSettings->setSettings(set);

@@ -421,7 +430,8 @@
    d->saveTagsBox->setChecked(set.saveTags);
    d->saveTemplateBox->setChecked(set.saveTemplate);
    d->writeRawFilesBox->setChecked(set.writeRawFiles);
-    d->useXMPSidecarBox->setChecked(set.useXMPSidecar);
+    d->useXMPSidecarBox->setChecked(set.useXMPSidecar4Reading);
+    d->writingModeCombo->setCurrentIndex(set.metadataWritingMode);
    d->updateFileTimeStampBox->setChecked(set.updateFileTimeStamp);

 #ifdef HAVE_NEPOMUK

I see you've been hard at work!

I finally managed to build all the pieces of software needed for the GSoC branch. I've just done some very brief tests, but will return and report further. A few observations so far:

1. I like the way you've added more configuration options for how to write metadata, so that both image and sidecar can have the information.

2. Adding tags to an image makes it write metadata to sidecar when configured to do so, and otherwise not. Correct behaviour.

3. Selecting "Image > Write metadata to image" for some reason does NOT write to XMP sidecar. Incorrect behaviour.

4. I tried adding geolocation data, but I'm getting segmentation faults. Perhaps I didn't build libkmap correctly?

5. Selecting "Image > Read metadata from image again" (or something like that) does NOT update image keywords. Incorrect behaviour.

I will do some further tests, perhaps also with gdb to see what's going on, when I get time (unless you find the problems first).

Sidenote: Digikam crashes when I quit the application...
digikam(22856)/digikam (core) Digikam::StateSavingObjectPriv::getGroupFromObjectName: Object name for  Digikam::MapWidgetView(0xaae7ed8)  is empty. Returning the default config group 
ASSERT: "d" in file /usr/include/qt4/QtCore/qscopedpointer.h, line 112
KCrash: Application 'digikam' crashing...
sock_file=/home/malmjako/.kde/socket-lorenzo/kdeinit4__0
QSocketNotifier: Invalid socket 39 and type 'Read', disabling...

Git commit db70bd2c91af66cf34e23bcc49d337d0e085532f by Gilles Caulier.
Committed on 23/03/2011 at 10:16.
Pushed by cgilles into branch 'master'.

DMetadata : handle digiKam metadata settings in constructor to manage XMP sidecar files everywhere in core implementation
CCBUGS: 220545

M  +3    -1    digikam/main/digikamapp.cpp     
M  +2    -7    digikam/metadata/metadatahub.cpp     
M  +0    -3    digikam/metadata/metadatamanager.cpp     
M  +1    -8    libs/database/imagescanner.cpp     
M  +3    -5    libs/dimg/loaders/dimgloader.cpp     
M  +1    -1    libs/dimg/loaders/dimgloaderobserver.h     
M  +20   -0    libs/dmetadata/dmetadata.cpp     
M  +4    -0    libs/dmetadata/dmetadata.h     

http://commits.kde.org/digikam/db70bd2c91af66cf34e23bcc49d337d0e085532f

Marcel,

As you said in your comment #54, now metadata settings are handle in DMetadata constructors.

https://bugs.kde.org/show_bug.cgi?id=220545#c54

To complete XMP sidecar support, it still a lead problem to solve : manage image and sidecar files at the same in these cases :

1/ Image Editor : File Save/SaveAs : when temp image file is created, it's moved move to target file path. XMP sidecar still in temp place. Code relevant is there :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/imageeditor/editor/editorwindow.cpp#L2705

2/ Album GUI : When image files are moved, copied, renamed, deleted, sidecar files must be managed with the same rules at the same time. Code relevant is in DIO if i'm not too wrong :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/digikam/database/dio.cpp

3/ In BQM, when new file is created in a workflow, sidecar file must be managed at the the time. A temp file is created as in editor, it's the same problem. Code relevant is there :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/queuemanager/main/queuemgrwindow.cpp#L1116

4/ In CameraGUI, sidecar file must be managed during import. Code relevant is there :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/cameragui/controller/cameracontroller.cpp#L870

5/ In kipi-plugins, all tools which create new files must manage sidecar accordingly with digiKam metadata settings.

I hope to not forget another place where image files are managed like this...

Gilles Caulier

Response from Marcel by private mail:

I see the problem, and I'm not very sure how to solve it.

I think the sidecar should be moved along with each single file, ideally
atomically, so when an operation is cancelled, a clean situation remains.

If a file transfer fails, then usually the subsequent transfer of the sidecar
will fail as well. But I see we want to be sure.

I had the idea to implement it in the ioslave, but I'm not sure the copy()
method is always called. We'd need to double check that. If get() is called,
no other file operation can be managed.
For each file we'd need to check if a sidecar is available, and this can take
a long time when many files are moved. This means the task needs to be done
asynchronously. The ioslave would come really handy.

*** Bug 277574 has been marked as a duplicate of this bug. ***

Digikam 2.0.0-rc

When moving files from within Digikam there is a dialog saying "somefile.jpg.xmp is not in the database" and i can choose cancel or skip. If i choose "skip" is it safe to move the corresponding xpm files to the the new place outside of Digikam? There is also a hidden file .digikam-exifrotate-2284.jpg what is this file for and should it also be moved?

*** Bug 279159 has been marked as a duplicate of this bug. ***

(In reply to comment #68)
> For each file we'd need to check if a sidecar is available, and this can take
> a long time when many files are moved.

Is such a check really so slow? DigiKam is not quite as fast as e.g. Dolphin (not to speak of using the command line) in moving files anyway.

I just made a quick check, trying to move 292 non-existing xmp 'files' (from a directory with 292 jpg files) to "/tmp". It takes eleven seconds on my system, which is a little outdated already (a 1.3 GHz PPC from 2005, but with not such an old HDD).

Maybe there could be a database attribute like "has sidecar file" for each image, which could be checked much faster?

With digiKam's current behaviour of neither displaying, nor automatically moving sidecar files, this new feature is not very usable yet, in my opinion. I often first go through new images applying tags and ratings, and only then move them to different albums/ directories.

But XMP sidecar files would be excellent for my incremental backup system (instead of having only the complete database, or multiple copies of images modified from tagging).

*** This bug has been confirmed by popular vote. ***

*** Bug 281337 has been marked as a duplicate of this bug. ***

*** Bug 281784 has been marked as a duplicate of this bug. ***

*** Bug 264205 has been marked as a duplicate of this bug. ***

*** Bug 289845 has been marked as a duplicate of this bug. ***

Git commit 89b525576a53cf26db97e39f6730065d071ac7eb by Marcel Wiesweg.
Committed on 13/03/2012 at 18:55.
Pushed by mwiesweg into branch 'master'.

If a sidecar is read, implement merging of Xmp data to the main image's Xmp data

M  +14   -20   libkexiv2/kexiv2.cpp
M  +18   -0    libkexiv2/kexiv2_p.cpp
M  +6    -0    libkexiv2/kexiv2_p.h

http://commits.kde.org/libkexiv2/89b525576a53cf26db97e39f6730065d071ac7eb

Git commit ffe8c36d6a4090f590cecaf0a1c6f373afc63f3c by Marcel Wiesweg.
Committed on 13/03/2012 at 18:58.
Pushed by mwiesweg into branch 'master'.

Rewrite DIO: Before starting a job, check for sidecar files which need to be handled alongside.

The check involves disk access and can take longer, therefore it needs to be done in a thread.
That means all the DIO methods no longer return a KIO::Job, instead the DIO class now listens
for the result signal. This means code cleanup in the dependent classes.
DIO class has become a singleton, the thread operations are done with an internal WorkerObject.

M  +1    -19   digikam/album/albummodificationhelper.cpp
M  +0    -4    digikam/album/albummodificationhelper.h
M  +0    -14   digikam/album/albumtreeview.cpp
M  +0    -3    digikam/album/albumtreeview.h
M  +280  -66   digikam/database/dio.cpp
M  +51   -22   digikam/database/dio.h
M  +8    -24   digikam/dragdrop/albumdragdrop.cpp
M  +19   -36   digikam/dragdrop/imagedragdrop.cpp
M  +0    -3    digikam/items/digikamimageview.cpp
M  +11   -108  digikam/items/imageviewutilities.cpp
M  +0    -9    digikam/items/imageviewutilities.h
M  +3    -15   digikam/main/digikamapp.cpp
M  +0    -1    digikam/main/digikamapp.h
M  +5    -4    utilities/advancedrename/advancedrenameprocessdialog.cpp
M  +2    -35   utilities/imageeditor/editor/imagewindow.cpp

http://commits.kde.org/digikam/ffe8c36d6a4090f590cecaf0a1c6f373afc63f3c

Marcel,

With your last 2 commits, what the status of this important file to solve ?

Especially, i would to know which points from my comment #54 need to be fixed.

Gilles Caulier

Git commit 1d46b8a6ad3dca38ca890bdb4a668e64d56b8272 by Marcel Wiesweg.
Committed on 15/03/2012 at 21:13.
Pushed by mwiesweg into branch 'master'.

Handle moving sidecar files alongside main file when renaming the temp files in image editor

M  +45   -25   utilities/imageeditor/editor/editorwindow.cpp
M  +2    -1    utilities/imageeditor/editor/editorwindow.h

http://commits.kde.org/digikam/1d46b8a6ad3dca38ca890bdb4a668e64d56b8272

Git commit ff40e084119c05fe90b88cc83ed369a1e07ea17c by Marcel Wiesweg.
Committed on 15/03/2012 at 21:22.
Pushed by mwiesweg into branch 'master'.

Move sidecar file alongside main file in BQM

M  +10   -1    utilities/queuemanager/main/queuemgrwindow.cpp

http://commits.kde.org/digikam/ff40e084119c05fe90b88cc83ed369a1e07ea17c

Cases 1-4 in your comment #67 have been covered. Relevant places in kipi plugins need to be identified. What remains to be done is some testing.

Git commit 1140f2b3911fb8d03f2a4506a89d44b10e957c99 by Marcel Wiesweg.
Committed on 15/03/2012 at 21:33.
Pushed by mwiesweg into branch 'master'.

Move XMP sidecar file with main temp file which may have been created in postprocessing of Camera GUI

M  +9    -1    utilities/cameragui/controller/cameracontroller.cpp

http://commits.kde.org/digikam/1140f2b3911fb8d03f2a4506a89d44b10e957c99

Ok, i will identify where kipi-plugins need to be patched, and make a proposal to make a common technical solution into libkipiplugins...

Gilles Caulier

Marcel,

In kipi-plugins, i identified these block of code where we need to handle XMP sidecar properly :

1/ BatchProcessImage :

a) https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/entry/batchprocessimages/common/batchprocessimagesdialog.cpp#L410

This is the common code for all Batch Process Tools where target image is managed, excepted Batch Rename Tool.where code is here :

b) https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/entry/batchprocessimages/tools/renameimageswidget.cpp#L584

2/ DNGConverter

https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/entry/dngconverter/plugin/batchdialog.cpp#L435

3/ RawConverter

https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/entry/rawconverter/dialogs/batchdialog.cpp#L538

https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/entry/rawconverter/dialogs/singledialog.cpp#L464

Gilles

Git commit 3aa21783862c87f20027e76c8103374b161e99ea by Gilles Caulier.
Committed on 19/03/2012 at 15:29.
Pushed by cgilles into branch 'master'.

manage XMP sidecar file with DNGConvert kipi-plugin

M  +68   -59   dngconverter/dngwriter/dngwriter.cpp
M  +6    -0    dngconverter/dngwriter/dngwriter.h
M  +1    -0    dngconverter/dngwriter/dngwriter_p.cpp
M  +9    -7    dngconverter/dngwriter/dngwriter_p.h
M  +1    -0    dngconverter/plugin/actionthread.cpp
M  +10   -0    dngconverter/plugin/batchdialog.cpp

http://commits.kde.org/kipi-plugins/3aa21783862c87f20027e76c8103374b161e99ea

Git commit 737274918ae64067243d6778558ea6f041f2059a by Gilles Caulier.
Committed on 20/03/2012 at 11:43.
Pushed by cgilles into branch 'master'.

add XMP sidecar support to RawConverter kipi-plugin

M  +9    -0    common/libkipiplugins/tools/imageio/kpwriteimage.cpp
M  +13   -0    common/libkipiplugins/tools/kpmetadata.cpp
M  +4    -0    common/libkipiplugins/tools/kpmetadata.h
M  +10   -0    rawconverter/dialogs/batchdialog.cpp
M  +10   -0    rawconverter/dialogs/singledialog.cpp
M  +1    -2    rawconverter/manager/actionthread.cpp
M  +4    -8    rawconverter/manager/rawdecodingiface.cpp
M  +6    -2    rawconverter/manager/rawdecodingiface.h

http://commits.kde.org/kipi-plugins/737274918ae64067243d6778558ea6f041f2059a

Git commit 883bb43492307fcae33515966da8b2ae289498df by Gilles Caulier.
Committed on 20/03/2012 at 13:07.
Pushed by cgilles into branch 'master'.

add static method to move XMP sidecar into KPMetadata
Support XMP sidecar to BatchProcess kipiplugin

M  +4    -1    batchprocessimages/common/batchprocessimagesdialog.cpp
M  +7    -3    batchprocessimages/tools/renameimageswidget.cpp
M  +15   -0    common/libkipiplugins/tools/kpmetadata.cpp
M  +4    -0    common/libkipiplugins/tools/kpmetadata.h

http://commits.kde.org/kipi-plugins/883bb43492307fcae33515966da8b2ae289498df

Git commit e21c68245aec0f4ebf35d63a9aca1efa59ae66e5 by Gilles Caulier.
Committed on 20/03/2012 at 16:33.
Pushed by cgilles into branch 'master'.

MetadataEdit plugin : Exif Editor :
- support KIPI lock feature
- support KIPI metadata settings.
- support XMP sidecar.

M  +4    -4    metadataedit/exif/exifadjust.cpp
M  +9    -2    metadataedit/exif/exifadjust.h
M  +4    -4    metadataedit/exif/exifcaption.cpp
M  +9    -2    metadataedit/exif/exifcaption.h
M  +4    -4    metadataedit/exif/exifdatetime.cpp
M  +9    -2    metadataedit/exif/exifdatetime.h
M  +4    -4    metadataedit/exif/exifdevice.cpp
M  +9    -2    metadataedit/exif/exifdevice.h
M  +16   -19   metadataedit/exif/exifeditwidget.cpp
M  +1    -1    metadataedit/exif/exifeditwidget.h
M  +6    -4    metadataedit/exif/exiflens.cpp
M  +9    -2    metadataedit/exif/exiflens.h
M  +4    -4    metadataedit/exif/exiflight.cpp
M  +9    -2    metadataedit/exif/exiflight.h
M  +1    -1    metadataedit/iptc/iptckeywords.cpp
M  +1    -1    metadataedit/plugin/altlangstringedit.cpp
M  +1    -1    metadataedit/plugin/metadataedit.cpp
M  +1    -1    metadataedit/plugin/metadataedit.h
M  +1    -1    metadataedit/plugin/plugin_metadataedit.cpp
M  +1    -1    metadataedit/plugin/plugin_metadataedit.h

http://commits.kde.org/kipi-plugins/e21c68245aec0f4ebf35d63a9aca1efa59ae66e5

Git commit 763f818a273865f268b520dc1a94c0f230f47a7c by Gilles Caulier.
Committed on 21/03/2012 at 09:53.
Pushed by cgilles into branch 'master'.

MetadataEdit plugin : IPTC Editor :
- support KIPI lock feature
- support KIPI metadata settings.
- support XMP sidecar.

M  +5    -5    metadataedit/iptc/iptccategories.cpp
M  +10   -3    metadataedit/iptc/iptccategories.h
M  +5    -5    metadataedit/iptc/iptccontent.cpp
M  +10   -3    metadataedit/iptc/iptccontent.h
M  +5    -5    metadataedit/iptc/iptccredits.cpp
M  +10   -3    metadataedit/iptc/iptccredits.h
M  +24   -27   metadataedit/iptc/iptceditwidget.cpp
M  +1    -1    metadataedit/iptc/iptceditwidget.h
M  +5    -5    metadataedit/iptc/iptcenvelope.cpp
M  +10   -3    metadataedit/iptc/iptcenvelope.h
M  +5    -5    metadataedit/iptc/iptckeywords.cpp
M  +10   -3    metadataedit/iptc/iptckeywords.h
M  +5    -5    metadataedit/iptc/iptcorigin.cpp
M  +10   -3    metadataedit/iptc/iptcorigin.h
M  +5    -5    metadataedit/iptc/iptcproperties.cpp
M  +10   -3    metadataedit/iptc/iptcproperties.h
M  +9    -7    metadataedit/iptc/iptcstatus.cpp
M  +10   -3    metadataedit/iptc/iptcstatus.h
M  +5    -5    metadataedit/iptc/iptcsubjects.cpp
M  +10   -3    metadataedit/iptc/iptcsubjects.h

http://commits.kde.org/kipi-plugins/763f818a273865f268b520dc1a94c0f230f47a7c

Git commit 59cbaf9b94b6755cd5563496c37de4a14d1154c9 by Gilles Caulier.
Committed on 21/03/2012 at 10:07.
Pushed by cgilles into branch 'master'.

MetadataEdit plugin : XMP Editor :
- support KIPI lock feature
- support KIPI metadata settings.
- support XMP sidecar.

M  +5    -5    metadataedit/xmp/xmpcategories.cpp
M  +10   -3    metadataedit/xmp/xmpcategories.h
M  +5    -5    metadataedit/xmp/xmpcontent.cpp
M  +10   -3    metadataedit/xmp/xmpcontent.h
M  +5    -5    metadataedit/xmp/xmpcredits.cpp
M  +10   -3    metadataedit/xmp/xmpcredits.h
M  +22   -24   metadataedit/xmp/xmpeditwidget.cpp
M  +1    -1    metadataedit/xmp/xmpeditwidget.h
M  +5    -5    metadataedit/xmp/xmpkeywords.cpp
M  +10   -3    metadataedit/xmp/xmpkeywords.h
M  +5    -5    metadataedit/xmp/xmporigin.cpp
M  +10   -3    metadataedit/xmp/xmporigin.h
M  +5    -5    metadataedit/xmp/xmpproperties.cpp
M  +10   -3    metadataedit/xmp/xmpproperties.h
M  +5    -5    metadataedit/xmp/xmpstatus.cpp
M  +10   -3    metadataedit/xmp/xmpstatus.h
M  +5    -5    metadataedit/xmp/xmpsubjects.cpp
M  +9    -3    metadataedit/xmp/xmpsubjects.h

http://commits.kde.org/kipi-plugins/59cbaf9b94b6755cd5563496c37de4a14d1154c9

Git commit 5024b3c8b262ee6a9582f5cd71e7cb00662ce7c9 by Gilles Caulier.
Committed on 21/03/2012 at 10:10.
Pushed by cgilles into branch 'master'.

MetadataEdit plugin : Exif/IPTC/XMP Import :
- support KIPI lock feature
- support KIPI metadata settings.
- support XMP sidecar.

M  +6    -14   metadataedit/plugin/plugin_metadataedit.cpp

http://commits.kde.org/kipi-plugins/5024b3c8b262ee6a9582f5cd71e7cb00662ce7c9

What is the status of this work? I have the problem that sidecar files aren't being moved in 3.1.0 -- this is apparently item 2 in comment 67.

(In reply to comment #94)
> What is the status of this work? I have the problem that sidecar files
> aren't being moved in 3.1.0 -- this is apparently item 2 in comment 67.

Can't confirm. Tried drag-n-drop moving (3.1.0 on Ubuntu 12.10) and sidecar file was moved correctly along with the image. Even orphaned xmp are no longer a problem when moving.

I've done some further testing, and the result is: It works most of the time. Sometimes the sidecar file is left behind, but usually it's moved correctly. I have not yet found any specific situation that fails.

I'm using Fedora 18.