Version: 0.8.0 (using KDE KDE 3.4.1) Compiler: gcc 3.3.5 OS: Linux I am using digikam for management of my digital photos. One thing I was missing was to export virtual folders i.e. images that share the same tags to the filesystem so that other applications (like JAlbum) can access such a virtual folder. Since I did not find a way in digikam to this, I wrote a small perl script that opens the digikam.db file and extracts all images that have been tagged. It then creates a subfolder for each tag name found and then places symbolic links to the original images inside these folders for the images with this tag. The script depends on the sqlite application to access the database. The script (vers 1.1) currently works with digikam 0.7x and 0.8.0 you only have to set the correct sqlite executable version (sqlite2, sqlite3)in the scripts head. The script can be downloaded from http://userpages.uni-koblenz.de/~krienke/ftp/unix/digikam/ I would happily provide this script for inclusion into digikam if you think that this would be a goog idea.
Brilliant! This was exactly what I was looking for as a user who wants to share photos with someone using iPhoto. I was thinking of coding an identical script myself. Rainer-- Well done. I think I'll be modifying the script for my personal use, however. (Rainer might be interested in this as well.) Since $photoRootDir is not constant across platforms, I felt that *relative* symbolic links would work best if I was sharing photos with friends. Thus I'm going to create the "$linktreeRootDir" inside "$photoRootDir/..", and employ relative links, so all I need to share my collection is to send a .tgz'ed subset of my photoRootDir. I vote that this be included as standard digiKam functionality, but where the linktree folder is just outside the root photo directory and uses symbolic links. I'll upload the modified version of the script with Rainer's permission. +20 points
Hmm, seems that maybe the script doesn't need to be modified to use relative paths...
I'm agree for an inclusion into trunk branch if all others team guys are agree. Normally all will work fine with 0.9.0 because the database structure is unchanged. The right place to put this tool in svn trunk is digikam/utilities/scripts (or something like that). The script need to be documented ( a man page is enough ) before to be included. Gilles
Am Dienstag, 25. Juli 2006 20:07 schrieb Mats Ahlgren: > I vote that this be included as standard digiKam functionality, but where > the linktree folder is just outside the root photo directory and uses > symbolic links. I'll upload the modified version of the script with > Rainer's permission. +20 points Relative links are a good thing I think. Don't know why I didn't use them right from the start. Can you send me a copy of the modified script? I would put this version into my download directory for those guys still referencing to my download URL. Thanks Rainer
Am Dienstag, 25. Juli 2006 21:34 schrieb Gilles Caulier: > The right place to put this tool in svn trunk is digikam/utilities/scripts > (or something like that). > > The script need to be documented ( a man page is enough ) before to be > included. > I'll write a man page for it. Can anyone tell me how to put the script as well as the man page into the correct place afterwards? Thanks Rainer
> I'll write a man page for it. Can anyone tell me how to put the script as well > as the man page into the correct place afterwards? Easy. Look the kipi-plugins MpegEncoder. I have written a backend script for mjpegtools named 'images2mpg'. There is a man page too. You just need to write a little Makefile.am like it's done in this tool. Look here : http://websvn.kde.org/trunk/extragear/libs/kipi-plugins/mpegencoder Gilles
Am Mittwoch, 26. Juli 2006 08:21 schrieb Gilles Caulier: > > I'll write a man page for it. Can anyone tell me how to put the script as > > well as the man page into the correct place afterwards? > > Easy. Look the kipi-plugins MpegEncoder. I have written a backend script > for mjpegtools named 'images2mpg'. There is a man page too. You just need > to write a little Makefile.am like it's done in this tool. Look here : > > http://websvn.kde.org/trunk/extragear/libs/kipi-plugins/mpegencoder > Thanks Gilles. The man page is written, the script is able to create relative or absolute links by setting an option and I tried to create a minimal Makefile.am according to the demo from above but have no real idea if what I wrote is correct for the framework it is used in. Now either someone with access to the svn repository could upload the files for me or perhaps someone can tell me how to get write access and in which directory to put the files then? The files can be accessed via: http://www.uni-koblenz.de/~krienke/digitaglinktree Thanks Rainer
SVN commit 566550 by cgilles: Add script from Rainer Krienke to export digiKam tags from database. BUG: 109820 A scripts (directory) A scripts/Makefile A scripts/Makefile.am A scripts/Makefile.in A scripts/digitaglinktree A scripts/digitaglinktree.1
Rainer, Done into svn : http://websvn.kde.org/trunk/extragear/graphics/digikam/utilities/scripts I have fixed all capital letters with all file name. If you want an svn acess to maintain this script directly, please take a look at this url : http://developer.kde.org/documentation/other/developer-faq.html#q1.8 http://developer.kde.org/policies/commitpolicy.html Gilles
I discovered unfortunately that relative symlinks do not work correctly due to a strange interaction with the hierarchicalness of the data folders. The only way to get any usefulness out of symlinks is to use ".." or "~"; .. doesn't work because links deeper in the tree need extra ..s to get out of the tree, and ~ doesn't work because "ln -s" auto-expands the ~ symbol before the symlink is ever created. Thus, there are two solutions I can see: 1) Augment the script to keep track of the depth of the tree while constructing the symlink's path, and prepend that number of "../"s before the symlink (give or take 1). 2) Taking inspiration from the UNIX filesystem, I designed a very cute indirection layer that works with your script. You create a symlink in /usr/local/, call it symlink-to-album1, and link it to your pictures. All symlinks in the tree structure are given the absolute path to /usr/local/symlink-to-album1, which is itself a symlink and resolved to the album. This is a very hackish and doesn't work well with multiple tag-trees, but allows people to share the tree of symlinks between computers while only having to run 2 bash commands and without having to recreate the absolute path. 3) Use hard links. If all you care about is tgz'ing your pictures to send to a friend, and don't mind there being duplicates, then that's one way to do it I think. Hard links also prevent symlinks from being messed up when you reorganize your digiKam album hierarchy, but a simple re-invocation of your script gets around that problem. I've come up with the perfect solution which is a blend of #1 and 2, but despite your code being wonderfully commented and clean, I'm not comfortable enough with Perl to make the changes, so I detail my suggestions below: -- Suggestions: Allow the script to take an "--export" argument, signifying the user wishes to make a file bundle they can email to a friend running any UNIX-based OS. Change the directory structure to be: > $linktreeRootDir/ > > README.txt > > compress-into-tar-gzip.sh ( only create if args contain "--export" ) > > hardlink-to-photos/ ( a hard link to $photoRootDir ) > > tag-tree/ > > > ( script puts all symlinks here ) README.txt : Your man page. I was in the process of writing up a doc to tell novice users what a symlink is, why they should leave everything inside the linktreeRootDir intact, why this only works with UNIX-based OSes like Linux and Mac, that changing their photo album hierarchy will mess up the links, and what anything else does (e.g. clean-links if you think it's worth including). compress-into-targzip.sh : This script will run "cd ..; tar -cvvzf $linktreeRootDir_archive.tgz $linktreeRootDir" hardlink-to-photos : This is an indirection layer. It seemed like the easiest way to do things; nomatter where you move your linktreeRootDirectory, the links will always remain in place. tag-tree : All symlink paths from the directory "tag1/tag2/tag3/.../tagN/" should be prefixed by (N+1)x"../" + "symlink-to-photos/" + "tag1/tag2/tag3/.../tagN/symlink" (I also had other ideas, like using a clean-links script to allow you to send only part of your album collection, but I thought this was the most efficient way.) Possible bug: Also, I didn't have a chance to go through your code fully, so I can't tell if this is a problem or not. It might be the case that your script doesn't correct handle two tags with the same name, such as /people/college/friends and /people/gradeschool/friends.
Correction to typo: tag-tree : All symlink paths from the directory "tag1/tag2/tag3/.../tagN/" should be prefixed by (N+1)x"../" + "HARDink-to-photos/" + "tag1/tag2/tag3/.../tagN/symlink" Also I did not see you had gotten the script 'working' by the time I wrote this. I hope you find these comments helpful anyway.
Yet another correction to typo: tag-tree : All symlink paths from the directory "tag1/tag2/tag3/.../tagN/" should be of the form target = (N+1)x"../" + "HARDink-to-photos/" + "whatever.ext"
Am Mittwoch, 26. Juli 2006 19:42 schrieb Mats Ahlgren: [bugs.kde.org quoted mail] Did you try the solution I found? Basically I think this one should work with relativ links provided that the File::Spec Perl module correctly constructs a relative path from an absolute one. In my limited testst it did. Your suggestion for the export option is not yet quite clear to me. What would you expect the script to do in this case. Just copy photos instead of linking them? Rainer
Rainer, have you see the message from Achim on devel ML ? There is no license into your script and you man page... Gilles
Quoting Rainer Krienke: > Did you try the solution I found? Basically I think this one should work with relativ links provided that the File::Spec Perl module correctly constructs a relative path from an absolute one. In my limited testst it did. Ah, Perl has a relative-path-constructing module? I haven't had a chance to see the new script in SVN, and may not have time until next week. > Your suggestion for the export option is not yet quite clear to me. What would you expect the script to do in this case. Just copy photos instead of linking them? I'm sorry, I'll try to be more clear. My suggestion is that currently your script has three problems, and I have an easy solution that will fix all of them: Problem 1: If I were to use your current script to make the linktree, and tar the linktree and email the tree to a friend, my friend would not be able to use it on his computer. Example: Photo directory: /home/mats/Pictures Relative linktree directory: /home/mats/Linktree [ I archive the linktree with my photos into a tgz file and email it to a friend, who extracts it onto her desktop. ] Friend's photo directory: /home/adria/Pictures Friend's linktree directory: /home/adria/Desktop/Linktree [ All the links will be broken on Adria's computer unless she moves it into her home directory. ] This can become alot worse if I had my Photo directory as /home/mats/Creative/Pictures; then Adria would have had to create a folder structure somewhere called /Creative/Pictures and put her Linktree folder in just the right place... etc. Problem 2: Every time you move the linktree directory, your links will break. Problem 3: Every time you move around your albums in digiKam, or move pictures between albums, your links will break. Solution: Create a folder (user gets to name it). That folder contains two directories: one named Linktree and one named Photos. The Photos directory is an exact clone of your actual Photo root directory, but all the image files are hard links ("ln" not "ln -s") to your regular files. Then, when creating the linktree, make the links using the Photos directory I just described as the $PhotoRootDir in your script. This fixes all three problems I think. (My friend will be able to untar the directory on her computer, and she'll have exactly one copy of each file and a working linktree without having to create strange new directories. Additionally, your links will rarely break on your own computer if you move your linktree directory, or if you move your photos around in digiKam.) Bonus: This has the added bonus that you can "rm" any entries in the Photos directory before tarring and emailing it to friends, rather than having to send ALL your photos. You can even include a script to remove the symlinks in the tagtree that have become dead after rm-ing. P.S.: (One may think an easy solution is to turn all the links in the linktree to hard links. However, this can drastically increase the amount data you'd have to send if you want to tar and email your tagtree to a friend (consider the case where most of your pictures are all tagged with dozens of the same tags).
In this case I'm not suggesting an export option anymore; I'm suggesting a tweak to the design.
Am Donnerstag, 27. Juli 2006 09:59 schrieb Gilles Caulier: [bugs.kde.org quoted mail] I added a license (GPL) text to the script now and committed it to svn. Thanks for the hint. I missed Achims Mail. Rainer
Am Freitag, 28. Juli 2006 02:06 schrieb Mats Ahlgren: > Problem 1: > If I were to use your current script to make the linktree, and tar the > linktree and email the tree to a friend, my friend would not be able to use > it on his computer. > ... Ok now I understand what you want to do. You are right at the moment exporting the tag structure is bound to the path of the original photos because the script symbolically links to them. Your suggestion to create an option for let me call it "archiving" the tag structure in an extra directory containing a Linktree and a Photos subdirectory and using hard links will solve the problems for this scenario. However one limitation to this solution is that hard links are bound to filesystems, so the archive directory has to be on the same filesystem as the original photos because hardlins can only be created in one filesystem and not across filesystem boundaries. I think the best way is to create an option for this kind of use say -A <archivedir> by which the user can specify where he wants to place the archive directory. However as said, this directory has to stay on the same filesystem like the photos in digikam. Rainer
> Your suggestion to create an option for let me call it "archiving" the tag > structure in an extra directory containing a Linktree and a Photos > subdirectory and using hard links will solve the problems for this scenario. Yes, exactly. Just to be sure we're on the same page: {linktree} <--[symbolic links]--> {photo directory} <--[hard links]--> {digiKam collection} This scheme (called an indirection layer) will solve the problem. > However one limitation to this solution is that hard links are bound to > filesystems, so the archive directory has to be on the same filesystem as the > original photos because hardlins can only be created in one filesystem and > not across filesystem boundaries. Not exactly; it still works beautifully (I have tested it infact). Here's a proof of concept: >> Step 1) Create a fake DigiKam photos directory. mats@moonfish:~ $ mkdir Pictures mats@moonfish:~ $ echo "test: 123" > Pictures/example.jpg >> Step 2) Simulate proposed modifications to your script. mats@moonfish:~ $ mkdir DigiKamExport mats@moonfish:~ $ mkdir DigiKamExport/linktree mats@moonfish:~ $ mkdir DigiKamExport/photos mats@moonfish:~ $ ln Pictures/example.jpg DigiKamExport/photos/hard-link-to-example.jpg mats@moonfish:~ $ cd DigiKamExport/linktree/ mats@moonfish:~/DigiKamExport/linktree $ ln -s ../photos/hard-link-to-example.jpg soft-relative-link-to-hard-link-to-example.jpg mats@moonfish:~/DigiKamExport/linktree $ cat soft-relative-link-to-hard-link-to-example.jpg test: 123 > What the test setup looks like... mats@moonfish:~ $ ls -CFRl DigiKamExport/ DigiKamExport/: total 8 drwxr-xr-x 2 mats mats 4096 2006-08-02 14:55 linktree/ drwxr-xr-x 2 mats mats 4096 2006-08-02 14:52 photos/ DigiKamExport/linktree: total 0 lrwxrwxrwx 1 mats mats 34 2006-08-02 14:55 soft-relative-link-to-hard-link-to-example.jpg -> ../photos/hard-link-to-example.jpg DigiKamExport/photos: total 4 -rw-r--r-- 2 mats mats 10 2006-08-02 14:52 hard-link-to-example.jpg >> Step 3) Create a tarball-gzip archive of DigiKamExport to export to a friend mats@moonfish:~/DigiKamExport/linktree $ cd ../.. mats@moonfish:~ $ tar -cvvzf DigiKamExport.tgz DigiKamExport/ drwxr-xr-x mats/mats 0 2006-08-02 14:51:12 DigiKamExport/ drwxr-xr-x mats/mats 0 2006-08-02 14:55:50 DigiKamExport/linktree/ lrwxrwxrwx mats/mats 0 2006-08-02 14:55:43 DigiKamExport/linktree/soft-relative-link-to-hard-link-to-example.jpg -> ../photos/hard-link-to-example.jpg drwxr-xr-x mats/mats 0 2006-08-02 14:52:05 DigiKamExport/photos/ -rw-r--r-- mats/mats 10 2006-08-02 14:52:49 DigiKamExport/photos/hard-link-to-example.jpg >> Step 4) Simulate sending file to another computer, and extract it. mats@moonfish:~ $ mkdir AnotherComputer mats@moonfish:~ $ mv DigiKamExport.tgz AnotherComputer/ mats@moonfish:~ $ cd AnotherComputer/ mats@moonfish:~/AnotherComputer $ tar -xvvzf DigiKamExport.tgz drwxr-xr-x mats/mats 0 2006-08-02 14:51:12 DigiKamExport/ ...[snip]... >> Step 5) Proof that the new hard link is an entirely new file, but the symbolic relative link still works. mats@moonfish:~/AnotherComputer $ cat DigiKamExport/linktree/soft-relative-link-to-hard-link-to-example.jpg test: 123 mats@moonfish:~/AnotherComputer $ echo "test2: 456" > DigiKamExport/photos/hard-link-to-example.jpg > Relative symlink still works on "AnotherComputer". mats@moonfish:~/AnotherComputer $ cat DigiKamExport/linktree/soft-relative-link-to-hard-link-to-example.jpg test2: 456 mats@moonfish:~/AnotherComputer $ cat ../DigiKamExport/ linktree/ photos/ > The files ~/DigiKamExport/photos/hard-link-to-example.jpg and ~/AnotherComputer/DigiKamExport/photos/hard-link-to-example.jpg are different files now. mats@moonfish:~/AnotherComputer $ cat ../DigiKamExport/linktree/soft-relative-link-to-hard-link-to-example.jpg test: 123
Am Mittwoch, 2. August 2006 21:16 schrieb Mats Ahlgren: > Yes, exactly. Just to be sure we're on the same page: > {linktree} <--[symbolic links]--> {photo directory} <--[hard links]--> > {digiKam collection} No, not exactly:-) I want to implement this: {linktree} <--[hard links]--> {photo directory} <--[hard links]-->{digiKam collection} Without hard links in the "export"-directory linking from "export/linktree" to "export/photo directory" you cannot move the linktree to somewhere else because this would break (relativ) soft links. However this was "Problem 2" you described in your Mail from 2006-07-28 (Comment #15). Absolute soft links are no good because of "Problem 1". > > This scheme (called an indirection layer) will solve the problem. > > > However one limitation to this solution is that hard links are bound to > > filesystems, so the archive directory has to be on the same filesystem as > > the original photos because hardlins can only be created in one > > filesystem and not across filesystem boundaries. > > Not exactly; it still works beautifully (I have tested it infact). Here's a proof of concept: > >> Step 1) Create a fake DigiKam photos directory. It cannot work if the directory for "digikam collection" (A) and the directory for "photo directory" (B) are on different filesystems simply because hard linking files from (A) to (B) is technically impossible in this case. In your examples for the proof of concept they were on the very same filesystem, both in your HOME directory. This of course works. I hope to have a prototype soon, then we can test things and see if it works as expected. Rainer
Am Freitag, 28. Juli 2006 02:06 schrieb Mats Ahlgren: [bugs.kde.org quoted mail] I completed a new prototype test version. You can get it under: http://www.uni-koblenz.de/~krienke/digitaglinktree This version now has an -A directory option. The script will then archive all of digikams photos and the tag structure (both by using hard links) into the given directory. In this directory the photos are placed in a subfolder called Photos and the tag structure in a subfolder called Links. Because hard links are used the given directory has to be on the same filesystem like digikams root directory. A call to archive photos and tehir tag structure might look like this: digitaglinktree -r /home/krienke/photos -d /home/krienke/photos/digikam3.db \ -A /home/krienke/export Please test it. Thanks Rainer
> Without hard links in the "export"-directory linking from "export/linktree" to "export/photo directory" > you cannot move the linktree to somewhere else because this would break (relativ) soft links. > However this was "Problem 2" you described in your Mail from 2006-07-28 (Comment #15). > Absolute soft links are no good because of "Problem 1". That's true; you can't. But that isn't an issue. You instead move the "Export" directory. =) The Export directory can be moved to anywhere. That's the point of keeping the Linktree and Photo in this movable directory. > It cannot work if the directory for "digikam collection" (A) and the directory for "photo directory" (B) > are on different filesystems simply because hard linking files from (A) to (B) is technically > impossible in this case. In your examples for the proof of concept they were on the very same > filesystem, both in your HOME directory. This of course works. The very last line of my simulation proves that this does indeed work: >> The files ~/DigiKamExport/photos/hard-link-to-example.jpg and ~/AnotherComputer/DigiKamExport/photos/hard-link-to-example.jpg are different files now. > mats@moonfish:~/AnotherComputer > $ cat ../DigiKamExport/linktree/soft-relative-link-to-hard-link-to-example.jpg test: 123 When tar uncompresses files, the hard links are no longer bound to the same file. If the output was "test2: 234", then I'd be wrong. I've emailed you the .tgz file in my experiment. If you extract it you'll see it works.
Am Donnerstag, 3. August 2006 19:07 schrieb Mats Ahlgren: > The very last line of my simulation proves that this does indeed work: > >> The files ~/DigiKamExport/photos/hard-link-to-example.jpg and > >> ~/AnotherComputer/DigiKamExport/photos/hard-link-to-example.jpg are > >> different files now. I think there was a misundertanding. The problems I described were problems that arise when creating the photo linktree structure, not when extracting such a structure from a tar archive. This will of course work. Did you Mats test the script I provided under http://www.uni-koblenz.de/~krienke/digitaglinktree It should do the trick now. For me it now works fine. Please test it. Thanks Rainer
Rainer-- I did a quick test of the script. 1) suggestion: Amendment to help: -A <arDir> [add: "Alternative to -l."] Create selfcontained... ["root and arDir have to be on the *same* filesystem!" -- I don't understand what you mean] 2) In regular mode, moving the linktree folder still destroys the symlinks; why not just make them hardlinks in regular mode? 3) > No, not exactly:-) I want to implement this: > {linktree} <--[hard links]--> {photo directory} <--[hard links]-->{digiKam collection} > > Without hard links in the "export"-directory linking from "export/linktree" to "export/photo directory" > you cannot move the linktree to somewhere else because this would break (relativ) soft links. > However this was "Problem 2" you described in your Mail from 2006-07-28 (Comment #15). > Absolute soft links are no good because of "Problem 1". By the way, I still don't understand why you're using hard links rather than symlinks in Archive Mode? It defeats the purpose of my original suggestion for an -A option, which was to send tarballs to friends via email. (One might as well just ignore the Photo directory and make the linktree out of hardlinks, but that creates a problem where the archive size can potentially be up to T times larger than necessary, where T = the number of tags, perhaps 100.) I'm just abit confused why you implemented an Archive option if not for sending tarballs... If there's a problem with symlinks... > The problems I described were problems that arise when creating the photo linktree structure ... then I'm still confused as to what it is, if you'd be so kind as to elucidate.
Am Samstag, 5. August 2006 05:16 schrieb Mats Ahlgren: > I did a quick test of the script. > > 1) suggestion: Amendment to help: > -A <arDir> [add: "Alternative to -l."] Create selfcontained... > ["root and arDir have to be on the *same* filesystem!" -- I don't > understand what you mean] I changed the help text a little. If this is not ok, then please explain exactly what is unclear for you? > > 2) In regular mode, moving the linktree folder still destroys the symlinks; > why not just make them hardlinks in regular mode? Well symbolic links can go across different filesystems. So your linktree does not have to be on the same filesystem as digikams root directory. This is more flexible and this is the way I use the script myself. > By the way, I still don't understand why you're using hard links rather > than symlinks in Archive Mode? It defeats the purpose of my original > suggestion for an -A option, which was to send tarballs to friends via > email. (One might as well just ignore the Photo directory and make the > linktree out of hardlinks, but that creates a problem where the archive > size can potentially be up to T times larger than necessary, where T = the > number of tags, perhaps 100.) I'm just abit confused why you implemented an > Archive option if not for sending tarballs... If there's a problem with > symlinks... Well I used hardlinks because this offers best flexibility for the purpose you wanted it for. Using hardlinks in the Linktree pointing to the Photo directory you can tar the whole archive directory (containing Phots and Links subdirs) and send it to a friend. This does work. Compared to symbolic links your friend can then even move the linktree directory out of the archive directory and things will still work. In the current version I changed the default to symlinks (I personally prefer them too) but added an Option -H if you want to use hardlinks instead. So now you are free to choose. > > > The problems I described were problems that arise when creating the photo > > linktree structure > > ... then I'm still confused as to what it is, if you'd be so kind as to > elucidate. Say digikams root directory is on /home/krienke/digikam and say I wand to create an archive (using -A ...) on /var/tmp/krienke. Moreover /home shall physically be stored on disk /dev/hda2 and /var/tmp shall be on /dev/hda3. So /home/* and /var/tmp/* are now on different filesystems. In this situation calling digitaglinktree -r /home/krienke/digikam -d /home/krienke/digikam/digikam.db -A /var/tmp/krienke will fail, because the script will try to *hard* link files from /var/tmp/krienke/Photos/* to /home/krienke/digikam/*. But since both directories are on different filesystems and hardlinks cannot go across filesystem borders the link system call will fail. Thats why I said that when creating the link structure the root directory of digikam as well as the archive directory have to be on the very same filesystem! In the current version to be downloaded under http://www.uni-koblenz.de/~krienke/digitaglinktree/ I changed the default behaviour for -A in that the photodirectory containing hard links to photos in digikams root directory will contain only photos that have tags set. If you for whatever reason want this photodirectory to contain links to all photos in digikams root directory you can set option -C. Rainer