Bug 297922

We plan to implement DLNA support in digiKam, through a new kipi-plugins. This will not a solution to your problem to share collections on local network ?

Gilles Caulier

my problem is the internet -.-''
we don't have a LAN connection, therefore I want to store the images and thumbnails locally (because they don't need too much updating, yet are big) and the rest on my server (accessible over the internet) to keep it in sync

If the DLNA support allows me to specify a different location for my images AND thumbnails it would be a solution

In principle the thumbnail location is not bound to the main db location. I'm not sure though if with the configuration UI it's currently possible. For SQLite, editing the config file would probably be sufficient, I'm not sure about MySQL.

I would like to support this wishlist feature for a different reason: separating the location of the main database from the thumbnails database would allow users to place the thumbnail database on fast SSD storage, and the main database on slow, redundant storage.

This would also be a very valuable feature for me, Are there any news on this?

Thanks for an awesome Software and best regards.

*** Bug 366092 has been marked as a duplicate of this bug. ***

The workaround mentioned in Comment 3 to set this manually in the config file does not work in current digiKam version (5.1.0 devel). See duplicate bug:
https://bugs.kde.org/show_bug.cgi?id=366092
Digikam automatically resets a different thumbnail database location to main database location.

As far as I can gather, thumbnails appear to be the major bottleneck when attempting to use Digikam with a "remote" MySQL/MariaDB database server, and images on a network drive -- basically, any connection to the network other than LAN by ethernet cable is unusable for me, including re-visiting folders with previously generated thumbnails, with the possible exception of perusing folders with no more than a dozen or so images. While it's possible I'm wrong, my assumption is that with a thumbnails database stored on each wifi/remotely connected computer running Digikam, but the core/metadata database on a single server, wifi access could become a workable reality. I'm not sure what value there is in requiring only a single thumbnails database in the same location as the core, faces, etc., but there may be technical(?) or use-case specific reasons. There just seems to be no way to test or accomplish/workaround this currently.

The code for storing thumbnails on disk is present in digiKam and is also used by showFoto. As a test I have already activated this in digiKam and the increase of the performance with MySQL is considerable, even if the DB runs locally over network. Although the PGF format is a very data-saving format, many thousands of thumbnails become very slow over the network. It would be possible that we always use a local SQlite database for the thumbnails.

Maik

Maik,

It sound logic to store thumbnails in local database with a remote mysql server make several improvements. But why not to use a local mysql database for the thumbnails, to be homogeneous ? This will prevent to use 2 different Qt sql plugins in memory at the same time.

But i'm not sure about PGF. The compress in wavelets and very optimum. There is a problem to use this format ?

Gilles

We are talking only about the thumbnail DB. The problem is that SQlite (on SSD) is even much faster than a local MySQL DB over network. Only an internal MySQL server (also on SSD) that communicates via a socket file is faster than SQlite. I would be more inclined to allow the user to save the thumbnails to the Freedesktop standard. So just in the local file system. The PGF format is a good and data-saving format.

Maik

Maik,

Warning: the FreeDesktop standard do not support PGF. Only PNG. If you use it you will explode the home directory.

Also, with DK, if you store thumbs as freedesktop, the way to show items from disconnected removable media will not be possible.

This is why FreeDesktop is only used in Showfoto, because it do not use a database.

Q: 1/ It's not possible to mix a local and a remote DB, first one for thumbs, second one for the rest of data ?

2/ You said that thumbs data take a while to transfert through the network with a remote database. What's about face recognition histogram, and similarity fingerprints ? there are also non negligible sizes... The problem will be the same, at least to register data and to perform searches.

3/ Why not to use a local database as cache to host all data and to sync with a remote one when time permit ?

Gilles

Since I have a slow wifi card on one of my computers, I also thought about how to reduce digiKam's wifi usage, and figured out that having thumbnails on the local machine only would be the best way. I actually found a way to do this, without modifying digiKam itself (by using ProxySQL): https://www.vlado-do.de/pc-tips/digikam/

Still I think it would be great if digiKam itself could already offer individual database configurations for the three different databases, allowing to use a different host for each. That would greatly simplify the configuration for the users (compared to setting up ProxySQL for that purpose).

Certainly ProxySQL could also be used for more advanced purposes, like caching, as Gilles was talking about introducing... but I'm trying to keep things as simple as possible. Like one of my colleagues always says: there are only two problems in software development: naming, caching, and off by one error... ;-)

*** Bug 373479 has been marked as a duplicate of this bug. ***

*** This bug has been confirmed by popular vote. ***

*** Bug 443900 has been marked as a duplicate of this bug. ***

I strongly support the idea of a local thumbnail database! Currently my use case is a seafile database for images connected to the client via seadrive and a mysql database on the same server. I want to use digikam with it on several clients, so that image ratings, comments etc. are shared. However the mysql database over wan is very slow even though I have a pretty fast connection (500 mbit down/100 mbit up). Over lan its very fast and over 5 ghz wlan its fast. One more problem I just realized is that the image location is also saved absolutely in the database? So ideally the whole folder structure and thumbnails could be saved locally, while only additional image data has to be saved in the mysql database. I guess every image is identified by UUID/hash?

Git commit 6853d49d4d64d0f6fedae329e32601e3acfb23a1 by Maik Qualmann.
Committed on 29/10/2021 at 17:50.
Pushed by mqualmann into branch 'master'.

add code to use a local SQLite DB for thumbnails when using MySQL

M  +12   -0    core/libs/database/thumbsdb/thumbsdbaccess.cpp

https://invent.kde.org/graphics/digikam/commit/6853d49d4d64d0f6fedae329e32601e3acfb23a1

Git commit de444bac53aa65e50908bdf3b1ebb42f908b0343 by Maik Qualmann.
Committed on 29/10/2021 at 20:16.
Pushed by mqualmann into branch 'master'.

add GUI option to configure a local SQLite Thumbnail DB folder
FIXED-IN: 7.4.0

M  +1    -1    NEWS
M  +1    -1    core/libs/database/thumbsdb/thumbsdbaccess.cpp
M  +38   -17   core/libs/database/utils/widgets/dbsettingswidget.cpp
M  +1    -0    core/libs/database/utils/widgets/dbsettingswidget.h
M  +4    -2    core/libs/database/utils/widgets/dbsettingswidget_p.h

https://invent.kde.org/graphics/digikam/commit/de444bac53aa65e50908bdf3b1ebb42f908b0343

Can you tell us where to find the option to configure a separate location for the thumbnails-digikam.db file (to separate it from the other 3 db files)? I saw that this was marked as "completed"...but looked through all the settings dialog & couldn't find the option anywhere.

Thanks in advace

https://docs.digikam.org/en/setup_application/database_settings.html#the-mysql-remote-server

Look the Browse button on the remote Mysql database settings.

I'm using sqlite. Is there no way to do this with sqlite - to separate the thumbnails db file from the other 3?

This bug report is about having the thumbnail database as a local SQLite database if the connection to a MySQL database is slow. With SQLite it makes no sense to have the thumbnail database in a different location. If they really want that, they could try a symbolic link.

Maik

> This bug report is about having the thumbnail database as a local SQLite database if the connection to a MySQL database is slow

Ah I see. I interpreted the title "different database locations for thumbnails" to just mean different locations, not limited only to when using that specific combination of db types.

> With SQLite it makes no sense to have the thumbnail database in a different location.

...I'm not really sure how it "makes no sense." The main database is essential and is small, the thumbnail database is transient and extremely large. Most software (including Lightroom) separates cache-like data from the essential data that should be i.e. synced or backed up. I'd assume that most people want their main database backed up, but don't need to backup a big multi-gig set of thumbnails each time the software is used & the file is touched. Requiring them all to be in the same folder means any backup software will need an explicit filename exception just for this one application - rather than simply placing thumbs/cache in a separate thumb/cache location.

> If they really want that, they could try a symbolic link.

Doesn't work - just as Digikam will follow a symlink, so too would backup software.