Bug 193232 - Add advanced input side-car files support (as .pto, .pp2, .pp3, .pano, thm, etc...)
Summary: Add advanced input side-car files support (as .pto, .pp2, .pp3, .pano, thm, e...
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Metadata-Sidecar (show other bugs)
Version: 4.6.0
Platform: Gentoo Packages Linux
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-05-19 12:56 UTC by DrSlony
Modified: 2017-05-12 23:19 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.6.0


Attachments
digiKam (47.70 KB, image/png)
2014-12-24 13:42 UTC, DrSlony
Details
Introduce additional sidecar types and refactor (32.27 KB, patch)
2017-05-07 05:43 UTC, Simon
Details
Introduce additional sidecar types and refactor - 2 (31.08 KB, patch)
2017-05-08 22:01 UTC, Simon
Details

Note You need to log in before you can comment on or make changes to this bug.
Description DrSlony 2009-05-19 12:56:44 UTC
Version:           0.10.0 (using KDE 4.2.3)
OS:                Linux
Installed from:    Gentoo Packages

I use RawTherapee for developing my files. RT (optionally) saves a sidecar pp2 text file alongside the demosaiced image, this file contains the settings RT applied. I wish there was a new album filter/mime type for these kinds of files, especially since many programs do that, not just RT. I currently had to add .pp2 to the "audio" group as a workaround, I wish there was a dedicated "text" group or "sidecar" group.

I see other people requesting the handling of RAW files and their JPEG counterparts as one item, which makes sense. It would be good if in addition to a new filter, you also included these sidecar files into the RAW+JPEG single item, so we could move/manage RAW+JPEG+sidecar as one unit.
Comment 1 DrSlony 2012-03-25 20:16:39 UTC
Bump, still not present in 2.5.0
Comment 2 caulier.gilles 2012-03-25 21:06:52 UTC
It's probably fixed by Marcel in current implementation from git/master (next 2.6.0-beta3)

Gilles Caulier
Comment 3 Marcel Wiesweg 2012-03-26 19:26:25 UTC
We handle standard XMP sidecars now; this is apparently about files with the suffix .pp2, so it falls into the "proprietary/non-standard sidecar" category, like the request to support .thm. Maybe we need to support these at least for copy/move as well.
Question here: for what purpose is it needed to have this filter? If they are text files, digikam is of not much use to access them.
Comment 4 DrSlony 2012-03-27 00:42:37 UTC
digiKam is a program designed to help people manage their photos. You have decided that it will also support audio and video files. I see no inconsistency why it should not allow the same people to manage sidecar files as well. In fact it already does so, except it limits the types of sidecar to just one: XMP. The fact is that people do use other software to develop their photos and would like to use digiKam to manage those photos.

By manage I mean two things:
1- to recognize sidecar files as such,
2- to let us use digiKam's MIME type filters on these sidecar files without having to do silly things like adding them to the "movie files" or "audio files" group,
3- and to let us easily move bundles around (or whatever you call a collection consisting of a photo, versions of that photo, and any other files directly related to that photo such as the sidecar file in XMP or any other format, which is one of the new features introduced in digiKam-2.0 http://scribblesandsnaps.wordpress.com/2011/03/28/new-features-in-digikam-2-0-versioning/ )

"But digiKam provides an elegant solution to the problem: you can combine the original and all its versions into a single group. To do this, select the original and its versions, right-click on the selection and choose the Group Selected Here command. This combines all the selected photos as one neat bundle"
If this group could also contain sidecar files, that would make managing them easier.

Just to be clear, I'm not asking for digiKam to understand the things contained in these non-XMP sidecar files, I would just find it useful if digiKam recognized these sidecar files, included a file type group for sidecar files, and let us bundle them with the photos each sidecar file corresponds to manually (via right-click + Group Selected) and/or automatically (pairing by matching filenames).
Comment 5 Marcel Wiesweg 2012-03-27 17:51:57 UTC
I see the need to move/copy them along with the main image, but why is it necessary to see the files in the album view? For me, it would be a nuisance to see files which we dont understand and do nothing with them
Comment 6 caulier.gilles 2012-03-27 19:39:50 UTC
I agree with Marcel. sidecar files are metadata, and a panel to right sidebar is dedicated to display metadata. There is nothing to see about sidecar file in Icon view.

Gilles Caulier
Comment 7 DrSlony 2012-03-28 18:31:29 UTC
Marcel: I need to see them to know which photos I've developed, or to move them with those photos, or to fire up a text editor and edit them, or to delete them. You're right that it would be a nuisance to see them all the time - I only want to see them when needed.

Caulier: well yes they are metadata, but as was stated they are metadata which digiKam does not understand. RawTherapee is switching from PP3 to XMP in the near future, but this XMP will only be partially compatible with digiKam: just the IPTC stuff. The rest - instructions for RawTherapee how to edit the photo - will obviously be Greek to digiKam. The metadata right panel will be of no help here. There are other raw converters which also use sidecar files, and they also don't and perhaps won't use XMP, or if they do use XMP then digiKam will only understand or share the tags and ratings, nothing else, and users of that software will also find it troublesome to move their photos with digiKam but then have to fire up a file manager to move the corresponding sidecar files.

Now imagine you rename photos with digiKam and your file manager doesn't handle batch renaming...

Adding these PP3/XMP/whatever sidecar files to a sidecar group optionally visible in digiKam would solve these problems.
Comment 8 DrSlony 2013-12-30 09:25:34 UTC
Bump, please look into this.
Adding support for a sidecar group with user-configurable formats (.pp3, .pano, .pto, .txt) is probably just a matter of copying existing code and should not take long to implement. All we want is to be able to see these files (MIME filters), move them together with the images and edit them in an external application.
Comment 9 DrSlony 2014-09-05 14:13:36 UTC
And to be able to rename them along with the images, so if I batch-rename the raws in a folder, the corresponding sidecar files, if they exist, get renamed too.
Comment 10 DrSlony 2014-09-05 14:43:57 UTC
Hmm, hijacking "Audio files" for sidecar file support used to work, I haven't used it for two years or more, and now I tried and I can't get it working.
I added "pp3 pano lhdr txt p2vr pef.pp3" to the audio extensions list, hit OK, re-scanned for new images, even restarted digiKam a few times, and it still won't show any of these files.
http://i.imgur.com/JoLIUCB.jpg
Comment 11 caulier.gilles 2014-09-05 14:49:26 UTC
There is a bug about audio management. Files are just ignored.

Gilles Caulier
Comment 12 DrSlony 2014-09-05 15:08:17 UTC
OK, it doesn't work for Movie Files or Image Files either.
Comment 13 caulier.gilles 2014-09-05 15:14:59 UTC
Movies files work well. Only Audio management is a problem. Look bug #319921

Gilles Caulier
Comment 14 DrSlony 2014-09-05 15:25:13 UTC
I added pp3;pano;p2vr;txt to Movie and Image, refreshed, restarted, and they do not show.
Comment 15 DrSlony 2014-09-17 12:20:18 UTC
Still does not work in 4.3.0. I tried adding "pp3" to Images, Movie and Audio Files, then re-scanning and restarting. PP3 files would never show.
Comment 16 DrSlony 2014-12-01 15:45:26 UTC
Please look into this. Without it one cannot move e.g. raw file + sidecar file to a different folder - I have to use an external program which defeats the purpose of using digiKam.
Comment 17 DrSlony 2014-12-24 12:26:31 UTC
In 4.6.0 adding custom extensions to Image/Movie/Audio Files in Configure > MIME Types finally works. Please add a fourth option, "Sidecar Files", so we don't have to hijack the Audio or Movie categories to be able to move our sidecar files along with our image files.
Comment 18 DrSlony 2014-12-24 13:38:33 UTC
Ignore my previous comment. In 4.6.0 is it still broken. I am absolutely certain digiKam showed me the .pp3 and .pano icons in the album view, but this must have been a one-time quirk of digiKam (one of many). I couldn't see then after restarting digiKam. I even deleted the database and every config file related to digiKam, re-added the collections, added pano;pp3;pto;p2vr to the MIME Types, and still can't get it to show me those files again.
Comment 19 DrSlony 2014-12-24 13:42:35 UTC
Created attachment 90116 [details]
digiKam

No active filters, pano;pp3;pto;p2vr added to MIME Types, Albums tree view correctly shows there are 4 items, but I only see the one JPEG.

 73K 2014-12-24 13:42:28 test_jpg.jpg
1.7M 2014-12-24 13:38:30 test_pano.pano
5.9K 2014-10-15 12:13:58 test_pp3.pp3
1.3K 2014-12-24 13:40:29 test_pto.pto
Comment 20 caulier.gilles 2015-05-16 13:17:07 UTC
*** Bug 288513 has been marked as a duplicate of this bug. ***
Comment 21 Matthieu Pupat 2015-09-12 20:34:47 UTC
Would it be possible for Digikam to read metadata from THM files when using "Reread metadata From Images"?
Comment 22 DrSlony 2016-02-03 20:30:31 UTC
Bump, please add support for digiKam-5 to treat user-specified file types as sidecar files so that when you move photos, these files get moved too.
Comment 23 Simon 2017-05-06 08:59:03 UTC
*** Bug 379511 has been marked as a duplicate of this bug. ***
Comment 24 Simon 2017-05-07 05:43:28 UTC
Created attachment 105376 [details]
Introduce additional sidecar types and refactor

This adds the new tab "Sidecars" to the metadata setup. It contains the existing settings about sidecars and a new field to enter sidecar extensions. These files will neither be displayed nor read from or written to. Only if the corresponding item is renamed/moved/copied/deleted, these files are transformed accordingly.

Screenshot of new metadata setup tab: http://i.imgur.com/5goLygr.png
Comment 25 DrSlony 2017-05-08 07:47:38 UTC
Thank you.
Comment 26 caulier.gilles 2017-05-08 09:43:41 UTC
DrSlony,

Did you have tested the Simon patch ? I do not do it yet myself.

Simon,

The setup panel is very good. Congratulations.

Gilles
Comment 27 DrSlony 2017-05-08 10:23:28 UTC
I have not tested it yet, I probably won't be able to do that for a few weeks, but the screenshot and description look and sound good.
Comment 28 Maik Qualmann 2017-05-08 18:58:57 UTC
I've tested it, working as expected. Simon, I find cleanUserFilterString () quite complicated. What do you think of this code, it is not tested:

QString filter = filterString.trimmed();
filter.remove(QLatin1String("-*."));
filter.remove(QLatin1String("*."));
filter.remove(QLatin1String("-."));
filter.remove(QLatin1Char('.'));
filter.replace(QLatin1Char(';'), QLatin1Char(' '));
QStringList filterList = filter.split(QLatin1Char(' '), QString::SkipEmptyParts);

Maik
Comment 29 caulier.gilles 2017-05-08 21:47:11 UTC
Simon,

Tested here too. It work fine for me...

Gilles
Comment 30 Simon 2017-05-08 22:01:32 UTC
Created attachment 105399 [details]
Introduce additional sidecar types and refactor  - 2

I agree, this function is too complex for what it does. Your code would delete the dot out of e.g. "out.pp3" and leading minuses must be preserved, so what is in the updated patch is a little less simple.

A question about the copyright notices: Should I add my name when I do (significant) changes?
Comment 31 caulier.gilles 2017-05-08 22:06:28 UTC
>A question about the copyright notices: Should I add my name when I do
>(significant) changes?

Of course, with pleasure (:=))...

Gilles
Comment 32 Simon 2017-05-12 23:19:35 UTC
Git commit 6485f23ed94757c31d4613fe8f45c983d847d8df by Simon Frei.
Committed on 12/05/2017 at 23:15.
Pushed by sfrei into branch 'master'.

The previous commit also included option to specify additional sidecar extensions

This commit does not contain the actual changes, refer to the previous one.
The newly specified sidecar files are not read from/written to, they are only
processed alongside the item they belong to (i.e. renamed, moved, deleted...)
FIXED-IN: 5.6.0

M  +2    -1    NEWS

https://commits.kde.org/digikam/6485f23ed94757c31d4613fe8f45c983d847d8df