Bug 283062 - THUMBDB : limit the size of thumbnail database
Summary: THUMBDB : limit the size of thumbnail database
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Thumbs (show other bugs)
Version: 2.2.0
Platform: Fedora RPMs Linux
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-09-30 06:16 UTC by info
Modified: 2017-02-22 15:10 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.5.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description info 2011-09-30 06:16:32 UTC
Version:           1.9.0 (using KDE 4.6.5) 
OS:                Linux

There should be an option to limit the size of the thumbnail database. Mine is several hundred MB already, and this is a problematic issue e.g. for backups because the database also changes all the time. It's also a problem on network drives.

Alternatively give users the option to use KDE's .thumbnail folder instead, which is there anyway.

Reproducible: Always



Expected Results:  
Saves disk space
Comment 1 caulier.gilles 2011-09-30 08:29:53 UTC
>There should be an option to limit the size of the thumbnail database. Mine is
>several hundred MB already, and this is a problematic issue e.g. for backups
>because the database also changes all the time.

About size this is not true. .thumbnail is bigger than digiKam DB...

>It's also a problem on network drives.

Why ? .thumbnail cannot manage removable media as network and usb drives.

>Alternatively give users the option to use KDE's .thumbnail folder instead,
>which is there anyway.

This option exist in cmake compilation setup :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/README#L118

But .thumbnail way is not a solution to fix size amount used by thumbs : It use PNG image (LZW compression). digiKam DB use PGF which use wavelets compression. There is a ratio of 2~3 between.

Gilles Caulier
Comment 2 Marcel Wiesweg 2011-10-01 10:22:52 UTC
There is need for a garbage collector if you have a troughput of images (getting many new ones _and_ delete old ones). If would expect that most here dont delete old images.

Apart from garbage collection, limiting the size would mean deleting some (random) entries when new ones are added. So consequently: If you want a small thumbnail db, delete the file and start with 0, all will be regenerated... As for backup, the indication would be to preserve the processing time invested in creating the thumbnails. There is no extra info to be backed up, backup is needed for the main database and the image files.
Comment 3 info 2011-10-03 09:43:45 UTC
Hi, perhaps I wasn't quite clear.

The problem with backups is that the thumbnail-db is one large file which changes every time. So as soon as I do something in digikam, my backup needs to upload the whole several hundred MB to my backup medium again.

This is not the case with the .thumbnails folder. There only a few files change, so only the actual changes need to be backed up.

My comment about network drives: If home directory is on a network, the whole db file has to move through the network. With .thumbnails the network traffic would be much less, because again only the actual changes, not the whole folder, need to be moved through the network.

I understand that .thumbnails is larger than the db. But the issue is that the thumbnails are already there, if you use other programs like konqueror to look at the images. So if you were able to re-use the .thumbnails folder in digikam, there would be no *additional* space necessary.

It's nice that there is a compilation option, but most users don't compile...
Comment 4 caulier.gilles 2011-10-03 09:54:25 UTC
The reasons why we use a dedicated DB for thumbnails are :

- size of thumbs divided by 2~3 due to use wavelets compression (instead LZW through PNG)
- The way to thumbnalize un-moutable media as DVD/CD/USB key. You can search items through unavailable media and show thumbs of items use DB. It's not possible to do with .thumbnails specification from free-desktop paper.

Anyway, you can turn off thumbs DB and use .thumbnails instead. Look Cmake compilation option as i explain previously.

Gilles Caulier
Comment 5 Marcel Wiesweg 2011-10-03 11:40:03 UTC
Btw, face thumbnails (thumbnails for a detail of a picture) won't work with the .thumbnails standard
Comment 6 info 2011-10-04 05:38:22 UTC
Hello again, let me please clarify something again. Using .thumbnail is not the main point of my featuer request (just an alternative suggestion).

The main point is: please provide an option to limit the size of the thumbnails database. There are many reasons why users may not want a file of several hundred MBs. 

Comment #2 says you can delete the db and it gets recreated in a smaller size. Yes, I've figured that out and done that several times. Later I just symlinked it to /tmp. But this is a power user perspective.

If the file is not strictly needed and the necessary info can be recreated easily, why can't you make it easier for the casual user who does not have the time to figure out what is taking up so much diskspace/backup bandwidth but just assumes that programmes are efficient with their ressources?

It seems to me that this is an easy and fair request to ask that a programme doesn't create huge databases that are not strictly needed, or at least give the user more control over the process.
Comment 7 Peter Potrowl 2012-09-12 08:42:50 UTC
My question may be stupid but all my pictures have an embedded thumbnail. I never use digiKam with unmountable media. So, why do I need those thumbnails to be saved again (in a DB or in .thumbnails)?

Maybe we could save disk space by storing the thumbnails in the DB only when the original files does not contain one?
Comment 8 caulier.gilles 2012-09-12 08:46:42 UTC
To be able to display thumbnail from removable media which are been disconnected from computer. The idea : searching items through DB with visible thumbnails must be possible...

Gilles Caulier
Comment 9 Mario Frank 2017-02-22 15:10:47 UTC
Git commit 2f8ddd42ef62d7aea9e490cdb05ffcc644810c81 by Mario Frank.
Committed on 22/02/2017 at 15:05.
Pushed by mfrank into branch 'master'.

Merged the current state of the garbage collection branch which improves the database cleanup stage of the maintenance
and improves the reactiveness of the maintenance overall. We ported the way items are processed to a queue based method
that can use the CPUs more effectively and does not create thousands of threads.
Related: bug 216895, bug 374225, bug 351658, bug 362023, bug 329353
FIXED-IN: 5.5.0

M  +17   -12   NEWS

https://commits.kde.org/digikam/2f8ddd42ef62d7aea9e490cdb05ffcc644810c81