Bug 427183 - Use database (and thumbs) for 'volatile' local storage (like a git-annex repository)
Summary: Use database (and thumbs) for 'volatile' local storage (like a git-annex repo...
Status: REPORTED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Thumbs (show other bugs)
Version: unspecified
Platform: Debian stable Linux
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-09-30 21:27 UTC by al.t
Modified: 2021-01-05 16:29 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description al.t 2020-09-30 21:27:14 UTC
Dear digiKam-team, 

thanks a lot for your work on digiKam - it's an awesome piece of software!  Never the less I'd love an additional feature to be implemented (or maybe it is and I just wasn't able to find it - in this case please point me to a how-to).


SUMMARY
I have a lot of images.  I have them inside a git-annex repository (see https://git-annex.branchable.com/) which allows me to back them up easily and at the same time keep the links and the structure inside my folder system, since git-annex allows to replace actual binary files (read: images) with symlinks to an empty file (the concept is strange, but extremely powerful, see e.g. https://www.youtube.com/watch?v=qgVXg0eRws4).

When using digiKam and the images are there, everything works fine.  But as soon as I replace the images with symlinks, I'm no longer able to see my image thumbnails - and I assume that this should be possible since the thumbnails are stored within the SQlite database.


STEPS TO REPRODUCE
1. point a 'local collection' to a git-annex repository
2. remove the local files (so that only the symlinks are kept)
3. open digiKam


OBSERVED RESULT
The files are missing from the collection.

EXPECTED RESULT
The thumbnails should be available (since they are stored in the local database.


SOFTWARE/OS VERSIONS
Linux: Debian Stable 'buster' using the Kernel 4.19.118-2+deb10u1 (actually 'PureOS 9 amber', which is based upon Debian Stable)
KDE Plasma Version: ??? (I'm using i3 as my WM)
KDE Frameworks Version: 5.54.0
Qt Version: 5.11.3 (built against 5.11.3)
Comment 1 Maik Qualmann 2020-10-01 11:21:00 UTC
Without having read the entire git-annex documentation. I understand correctly that digiKam only sees a 0-byte file via the symlink? Well, then it is clear that when the scan for new items at start runs, it will re-scan these 0-byte images and generate an empty thumbnail. We could certainly ignore 0-byte files when scanning, but Exiv2 would no longer display any metadata either. There would be a lot more problems with digiKam in normal operation with 0-byte images.

Maik
Comment 2 al.t 2020-10-01 19:44:26 UTC
(In reply to Maik Qualmann from comment #1)
> I understand correctly that digiKam only sees a 0-byte file via the symlink? 

Yes, that is correct: git-annex replaces the files with (0-byte) symlinks and allows for a smart way to handle the deletion of parts of the repo.

> Well, then it is clear that when the scan for new items at start runs, it will re-scan these 0-byte images and generate an empty thumbnail. 

So it is intended that the (correct, old) thumbnails get overwritten with an empty thumbnail? 

> We could certainly ignore 0-byte files when scanning, but Exiv2 would no longer display any metadata either. 

Just for my understanding: the Exiv2-metadata is not stored in a database but read again each time?

> There would be a lot more problems with digiKam in normal operation with 0-byte images.

Hmmm... out of curiosity: do you have any other solution to my problem?  

I assume that my situation is not that uncommon: I have more images than my laptop can handle, so I have stored all them remotely (nfs or external hdd) and only parts of them locally. But I want digiKam to not forget what it had already seen (especially the folder structure, the metadata and the thumbnails). 

Is there a good solution?  I thought git-annex is probably the smartest one, but I'm open to alternatives.
Comment 3 Maik Qualmann 2020-10-01 20:33:06 UTC
Right, digiKam can manage an offline collection. But then nothing should be available anymore. Yes, the metadata sidebar is always re-read. Digikam also keeps some metadata in the database, all of which are possible in the advanced search. So camera model, lens, exposure time, tags, date, etc.

Maik
Comment 4 Maik Qualmann 2020-10-01 20:36:08 UTC
We have a similar request for git-annex with regard to sidecars with bug 415949.

Maik
Comment 5 al.t 2021-01-05 16:29:47 UTC
> Yes, the metadata sidebar is always re-read. 

Is there an option to disable the automatic re-reading of the metadata?  It would be great if that were an option - especially, if that could be managed on a per-collection basis.