| Summary: | It should be possible to limit the amount of disk space used by thumbnails (.thumbnails directory) | ||
|---|---|---|---|
| Product: | [Frameworks and Libraries] kio-extras | Reporter: | Michał Kosmulski <michal> |
| Component: | Thumbnails and previews | Assignee: | Plasma Bugs List <plasma-bugs> |
| Status: | RESOLVED DOWNSTREAM | ||
| Severity: | wishlist | CC: | 1lukas1, alessandro.ufms, bryce2, bugs.kde.org, bugs, cfeck, de.techno, dion, faure, finex, gnemmi, grey_earl, halphaz, info, jrsmeed, jtamate, j__n, katyaberezyaka, kdebugs, michel.poleur, misc-kdeorg, nate, peter017, richih-kde, romain.perier, ssameer+bugs, sven.burmeister, ultr |
| Priority: | NOR | ||
| Version: | 18.04.2 | ||
| Target Milestone: | --- | ||
| Platform: | Slackware | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | ||
| Sentry Crash Report: | |||
|
Description
Michał Kosmulski
2004-04-19 12:24:36 UTC
mauve:~$ du -Hs .thumbnails/* 33k .thumbnails/fail 436M .thumbnails/large 1.1G .thumbnails/normal This is simply ridiculous. I know this is an exact implementation of the freedesktop.org thumbnails spec[1], but it's broken, quite frankly. I suggest KDE moves to update this spec or allow for extensions to it. a) 128x128 is too large to be the smallest available size (64x64 would be better - just over a quarter of the size, while loading faster because KDE wouldn't need to resize them down) b) Thumbnails aren't being pruned according to the recommendations in the spec. c) Files should be pruned by access time over and above the considerations mentioned in the spec. d) Konqueror should have a 'Clear Thumbnails Cache' button in its settings window. [1] http://triq.net/~jens/thumbnail-spec/index.html thibs@simbad:~$ du .thumbnails/* 508 .thumbnails/large 103176 .thumbnails/normal thibs@simbad:~$ I support the comments made above. It would be great if we could be able to limit the amount of disk space used by thumbnails and also specify a "time limit". For example, we could ask to automatically delete every file older than X days. I still have images from January 2004 in my .thumbnails directory. I found this (bug? feature?) too, and must say that I igree. Perfect solution would be: 1) Way to limit the size of the thumbnail directory 2) Possibility to set a time limit. IE: Thumbs older than "x" days are deleted. 3) A buton to clean the thumbnail's cache. Maybe in the Control Center? I'm using Conectiva 10, with KDE 3.2.3 same here, please!, limit the size of that folder, I've just found that my .thumbnails was 2.4GB!!, that's just not right... So will this bug get some attention, please :) Just had to delete 7 GB of thumbnails! (had to do this from command line, because there were too many files in the thumbnails directory for konqueror. I notice that the first report was Slackware 2004 package - mine is from a SuSE 2005 distribuition (SuSE 9.2 - Konqueror 3.3) - is anybody paying attention to this? I am now only running Konqueror from a shell script that deletes thumbnails on exit. I support all the above comments - Ernesto Gomez It is not only konqueror that writes to that directory, but also Gimp, e.g. when opening pictures, which makes this a general bug as it seems. This bug still exists. Recently I tried to find out where my space gone, And found that there are ~4GB in .thumbnails folder usually older than a few months. This is a security problem too, since some data, like previews of internal company documents or even worse, documents with passwords, could be cached and kept for future. I deleted 1,8 GB of thumbnails: that's ridiculous!!! Moreover it's a hidden folder!!! So normal users will never know where their free space has gone. I hope someone solve this as soon as possible. Severity: wish-list??? Are you joking? I't probably an error ... This is not a "whish-list" kind of bug .. neither a should it have a "normal" priority unless _security_ is not a KDE concern (in a simple scenario like: only one partition, any user could fill the whole disk obliterating acces to the system)... BTW, this is neither a "Platform: Slackware Packages" bug nor a "OS: Linux" as it affects every single KDE implementation I've ever used since 2004 .. on Linux and FreeBSD as well. Hope this will be fixed soon as this is not only a longstandig bug but also a shame in view of all the radical changes that KDE went through in all this time with this bug still standing. Regards What about some sort KDE disk cleanup/Privacy keeper daemon? It could be set to delete files from specified directories matching specified rules. (User, of course, can customize them) E.g. dir:~/.thumbnails/*; filter:older than 7 days dir: /tmp/*; filter:older than 2 days && does NOT match *yast* || older than 5 days dir:/flash off line storage/; filter:older than 1 day && domain is not in bookmarks Such solution would be global and software independent, but also a great tool for advanced user to get more of their PC's ;) Even if the last thing I need is KDE SC running yet another daemon, at this point in time (6 years from the original report), any solution is better to what we have ... *** Bug 145301 has been marked as a duplicate of this bug. *** TODO: check what http://jens.triq.net/thumbnail-spec/delete.html says about this :) (In reply to comment #14) > TODO: check what http://jens.triq.net/thumbnail-spec/delete.html says about > this :) I'm not sure that any of the things on that page help for this problem. My thumbnails are just .png files named for a hash of the file (e.g. 00227fc2be18e17f7d33d529640619be.png) -- I've looked through them and these files do not contain any URI's. So, periodic cleanup based on "does the file still exist?" is highly impractical -- the only way to determine that is to hash every file on the filesystem and see if you can find one that has the same hash. I think "toss thumbnails older than x days" is probably the best approach. ~Felix. (In reply to comment #15) > I've looked through them and these > files do not contain any URI's. > afaik, these files contain URI and metainfo (using QImage::setText()), see PreviewJobPrivate::slotThumbData() in kdelibs/kio/kio/previewjob.cpp:572. Or just open any of these thumbnails with emacs and type C-c C-c to view the image as text. see also http://jens.triq.net/thumbnail-spec/creation.html (In reply to comment #16) > > afaik, these files contain URI and metainfo (using QImage::setText()), see > PreviewJobPrivate::slotThumbData() in kdelibs/kio/kio/previewjob.cpp:572. > Or just open any of these thumbnails with emacs and type C-c C-c to view the > image as text. > Interesting -- I just looked through a larger sample and some of them do contain URI's. Looks to be about half that do, half that don't. So, some program on my system is probably ignoring that part of the spec. ~Felix. In fact, some of them are saved by PreviewJob (which saves metainfo) and other by kioslave thumbnail (for example, thumbnail for a whole directory) but in this case metainfo aren't saved at all. I've just removed over a gig of junk from this folder. Consuming disk space without limit is a bad bug, this bug is also heavily voted. Is it even being worked on ? Work arounds involving cron jobs are not as elegant as, for instance, what happens with the Wastebin folder. I had a major data loss tonight cause of this bug. "No space left on device", the root device i.e. Setting to critical as per https://bugs.kde.org/page.cgi?id=fields.html#bug_severity Although there are obvious security and privacy implications, please don't add yet another deamon to KDE when you could instead borrow whatever code the Wastebin is using. This would land the fix a lot sooner and with less work. There is no need for any daemon if the process generating new thumbnails takes care of cleaning up, as well. As thumbnails do not expire, this has no downsides at all while making sure that the clean-up is as lean as possible. I think it'll be a good idea to make kio_thumbnail see the amount of free space left in the directory, if there's less than x (user defined), old thumbnails will be deleted to make space for new ones. Everyone has their own opinions on this issue without subjectively looking at it. I am just amused at those of you that call this a security issue. How exactly is this a security issue ? Or for that matter a privacy one ? If you do not want previews (aka thumbnails) to be generated you can disable them in dolphin or konqueror. The only thing for which there is no GUI option is the favorite icons, but those too can be disabled by adding Now back to the issue of limiting the number images that can be put in the thumbnails directory. Since that directory is a non-desktop specific directory shared by any application that adheres to the freedesktop standard, how is KDE supposed to manage or enforce the limit ? Anyhow, there is already a tool that cleans up this and many other KDE caches. It is called sweeper and is part of kdeutils. You can use that to clean up all the caches in KDE, not just the thumbnail cache. Oh and for the record, the tiny (16x16) favorite icons downloaded when you visit a web page are not saved in the .thumbnails directory! They are saved under /var/tmp/kdecache-<username>/favicons/. Just came to my mind. What if temporary files and caches would be linked into trash:/ as some sort of magic folder? Need more space, just open Trash and delete least important files. What to limit the size of thumbnails or other temp files, just enable "Delete oldest files when Trash reaches 10% of my partition size". This's a security issues cause the users do not know that a small copy of their private images are in ~thumbnail. (In reply to comment #28) > This's a security issues cause the users do not know that a small copy of their > private images are in ~thumbnail. That still is not a security issue. You are confusing security with privacy. How does having a small copy of your private images in a hidden directory on your own private home folder automatically or otherwise expose your information to others ? It does not. Even then none of the file management, i.e. dolphin and konqueror, generate an preview only for directories by default. In dolphin that is true so long as you do not enable the information panel. If you do hovering over the file will create a preview of the image to display in the information panel. However, that panel too is not displayed in the default KDE configuration either. Most importantly however neither one of these applications generate thumbnail images for remote. That is the default setup in KDE. What each distribution does is up to them. They might decide to enable any of these things by default for convenience. Other application such as image viewers have their own needs. Some of them show thumbnails of pictures and as such automatically store a thumbnail of those images into the .thumbnail directory. However, that is the prerogative of each application developer and the issue needs to be taken up with them as does the issue of changing the default KDE settings needs to be discussed with your preferred distro. Regardless this is not a security issue. If you can show having thumbnail images somehow makes it easy for someone to compromise your system, then I will accept your premise of this being a security issue. Yes, this's a privacy issue. Mounting an encrypted partition with pictures and enabling thumbnail view to view the same for convenience will make a copy of the confidential pictures in ~/.thumbnail which can be found by a simple find operation. Then it's a common problem of the filesystem being filled with a bottomless list of thumbnails, afterwards opening a file from that FS in applications like nano causes data loss (my make.conf and fstab got emptied as a consequence while I was diagnosing the reason why my system was not booting which happen to be lack of free space.). + who knows dolphin generates thumbnails in ~/.thumbnails? It'll come as a surprise to most users. This might not be a security issue but it is still a critical one. I manipulate a lot of pictures and yesterday, I had more than 3 GB of thumbnails on my 10 GB partition. So, this can cause data loss (when you try to write on a filesystem with 0 bytes left) or a system crash (if your home folder is on the / partition). Of course, KDE programs are not the only ones that might totally fill your partition, but the problem here is that "normal" users are not aware of the existence of this folder and so, they won't delete it when it's too big nor make a cron task for deleting old thumbnails... Greetings.
On Monday 20 June 2011, dE wrote:
> Yes, this's a privacy issue. Mounting an encrypted partition with
> pictures and enabling thumbnail view to view the same for convenience
> will make a copy of the confidential pictures in ~/.thumbnail which can
> be found by a simple find operation.
It seems to me this is a problem with transparent disk encryption
generally, and not with kdelibs in particular. Potentially any application
(web browsers, image editors, word processors) using the data on such a
disk could cache it on unencrypted temporary storage.
Regards,
Tristan
One possible solution: Using the same kded4 daemon that checks for disk space, to have the ability to launch also kleansweep to clean the .thumbnails directory. Or to create another kded4 daemon that only checks the .thumbnails directory. (In reply to comment #34) > One possible solution: > Using the same kded4 daemon that checks for disk space, to have the ability to > launch also kleansweep to clean the .thumbnails directory. > Or to create another kded4 daemon that only checks the .thumbnails directory. OMG. It's ugly solution. Personal daemon for regular monitoring .thumbnails directory's size? Yes, why not? I think it is as ugly as a personal daemon to check for disk space. :-) It could be used to check the disk usage of other directories also. (some programs, for example, generate log files that grow and grow and are never deleted until they fill the disk and you have to do it manually). Also, the "normal" user does not have to know about the directory, nor how to create a cron task to clean it. For example, I remember the first time I saw a kdirstat of my home directory to have more than 2 GiB in that directory. "who knows dolphin generates thumbnails in ~/.thumbnails? It'll come as a surprise to most users." What is surprising about that ? You cannot possibly be assuming that it generates these images on demand over and over again and keeps them all in memory, are you ? Or did you assume it stored them somewhere other than ~/.thumbnails ? ----- "So, this can cause data loss (when you try to write on a filesystem with 0 bytes left) or a system crash (if your home folder is on the / partition)." I am sorry, but to me that is a configuration issue. There is no reason why you should experience this if the /home directory has its own partition, which is the recommended and long standing configuration for linux/unix systems. Most systems even come preconfigured with disk usage quota for users so they do not eat up all the disk space in /home. ----- "the problem here is that "normal" users are not aware of the existence of this folder and so, they won't delete it when it's too big nor make a cron task for deleting old thumbnails..." That is a valid point, but it is not something that can be easily addressed. Even if we fix KDE and all KDE applications such that they honor some sort of size limitation for ~/.thumbnails directory, that fix won't apply to non-KDE applications. That means KDE application will stop generating and showing proper thumbnail if the size limit is reached because a non-KDE application used it all up. IOW, the ~/.thumbnails directory is a fdo standard used by most, if not all, KDE and non-KDE applications alike and as such the standard needs to be fixed to address this issue so that everyone could abide by it. ----- "Using the same kded4 daemon that checks for disk space, to have the ability to launch also kleansweep to clean the .thumbnails directory. Or to create another kded4 daemon that only checks the .thumbnails directory." The problem with that is which file should get deleted when the limit is reached ? You could say, start removing the oldest files, but that would assume those files are currently not in use or required by a running application. Since the average user is like to use multimedia related application that generate and use these thumbnails, deleting them in such manner becomes an issue. The only solution to deal with this issue right now is to either manually delete the thumbnails periodically, tell the application you use to remove the thumbnails it created upon exit if it has such configuration (IIRC digikam allows you to do this ??) or to configure the "sweeper" application to only delete the thumbnails cache and add it to a cron job with its "--automatic" command line option. " configure the "sweeper" application to only delete the thumbnails cache and add it to a cron job with its "--automatic" command line option." Hmm, this may be better than the current hacked together script posted on this bug. Is KDE suggesting we have to petition each KDE distributor individually to have this cron job added, rather than have whatever checks the Trash size also check the thumbnails size ? Hmm. 'Sweeper' in a cron job doesn't appear to be appropriate. It deletes everything in the cache, rather than (say) only the ones unused for N days or oldest until the size goes below N meg (gig !). Either of the later would be the correct solution. There is also no command line argument to Sweeper to tell it to only clear the cache, so it wipes a whole load of other stuff too. The thumbnails folder is meant to be cache, not permanent store, therefore it must be purged regularly, and applications will cope fine with this. >The problem with that is which file should get deleted when the limit is
>reached ? You could say, start removing the oldest files, but that would assume
>those files are currently not in use or required by a running application.
>Since the average user is like to use multimedia related application that
>generate and use these thumbnails, deleting them in such manner becomes an
>issue.
This is left to the user. kded4 could just show a notification of type "size of
the directory x exceeded. Open the directory with a file manager or execute the
"sweeper" application or a custom application.
" kded4 could just show a notification of type "size of the directory x exceeded." That would work, but the experience is fairly terrible. Why should a user have to care ? KDE should ship with sane defaults (i.e. different to the current 'write until disk full') and provide a way (such as in system settings) for users to alter this default if they want to. Most people wont. Hi, I'm the original reporter of this issue. What Tom Chiverton said very much reflects my point of view - KDE should ship with sane defaults. I've been using Linux with KDE for quite a few years, but still if one day my system doesn't allow me to log in (as was the case when I first reported this issue), even I am confused and angry, not thrilled to be given the opportunity to learn about my system's internals in order to find out what happened and be able to log in. Another true story: when I was backing up my home dir one time and it didn't fit on the medium I had always used, it took me some time to find out .thumbnails had grown and that I had to exclude it from backups. A regular user who wants to get his or her job done will not know what to do, they'll say "Linux is stupid" and happily go back to whatever system they were using before. If a user has a 1 TB /home and doesn't perform backups, they may never run into trouble. But many people have netbooks whose drives are not at all that large, and if they install Linux side by side with Windows, and they have a separate /home and /, the size of /home may well be 1/4th of the drive size or less, perhaps just 10 GB or so. On my desktop, ~/.thumbnails right now takes up 5 GB, and I have cleaned it on several occasions. While I do keep digital photos on this machine, I don't think my use of graphics is much above average. So, in brief: default settings should not cause a user's home directory to grow by several gigabytes without his knowing. And: expecting users to know about such behavior, to take manual action in order to prevent bad things from happening and to not curse KDE/Linux/developers/dolphins if something bad does happen, is absolutely not realistic. Even in server software defaults should be sane and safe, much more so in desktop software for the masses. Regards, Michał (In reply to comment #33) > Greetings. > > On Monday 20 June 2011, dE wrote: > > Yes, this's a privacy issue. Mounting an encrypted partition with > > pictures and enabling thumbnail view to view the same for convenience > > will make a copy of the confidential pictures in ~/.thumbnail which can > > be found by a simple find operation. > > It seems to me this is a problem with transparent disk encryption > generally, and not with kdelibs in particular. Potentially any application > (web browsers, image editors, word processors) using the data on such a > disk could cache it on unencrypted temporary storage. > > Regards, > Tristan Web browsers, mailing clients, yes. But not file managers... no one expects a FM to store copies of confidential files. Other apps can be easily run as a separate user and ~ of this user can be deleted on periodic basis. (In reply to comment #37) > "who knows dolphin generates thumbnails in ~/.thumbnails? It'll come as a > surprise to most users." > > What is surprising about that ? You cannot possibly be assuming that it > generates these images on demand over and over again and keeps them all in > memory, are you ? Or did you assume it stored them somewhere other than > ~/.thumbnails ? I'm an admin so I know. I deploy desktops systems. But do the windows migrants know? > "So, this can cause data loss (when you try to write on a filesystem with 0 > bytes left) or a system crash (if your home folder is on the / partition)." > > I am sorry, but to me that is a configuration issue. There is no reason why you > should experience this if the /home directory has its own partition, which is > the recommended and long standing configuration for linux/unix systems. Most > systems even come preconfigured with disk usage quota for users so they do not > eat up all the disk space in /home. These are good ideas. Honestly I was too wondering about these. BTW in my case I've changed the ownership of .thumbnails so it doesn't generate thumbs anymore. > "the problem here is that "normal" users are not aware of the existence of this > folder and so, they won't delete it when it's too big nor make a cron task for > deleting old thumbnails..." > > That is a valid point, but it is not something that can be easily addressed. > Even if we fix KDE and all KDE applications such that they honor some sort of > size limitation for ~/.thumbnails directory, that fix won't apply to non-KDE > applications. That means KDE application will stop generating and showing > proper thumbnail if the size limit is reached because a non-KDE application > used it all up. IOW, the ~/.thumbnails directory is a fdo standard used by > most, if not all, KDE and non-KDE applications alike and as such the standard > needs to be fixed to address this issue so that everyone could abide by it. If non KDE apps fill it, KDE apps don't have to. This might be a feature which other apps don't have. > "Using the same kded4 daemon that checks for disk space, to have the ability to > launch also kleansweep to clean the .thumbnails directory. Or to create another > kded4 daemon that only checks the .thumbnails directory." > > The problem with that is which file should get deleted when the limit is > reached ? You could say, start removing the oldest files, but that would assume > those files are currently not in use or required by a running application. > Since the average user is like to use multimedia related application that > generate and use these thumbnails, deleting them in such manner becomes an > issue. > > > The only solution to deal with this issue right now is to either manually > delete the thumbnails periodically, tell the application you use to remove the > thumbnails it created upon exit if it has such configuration (IIRC digikam > allows you to do this ??) or to configure the "sweeper" application to only > delete the thumbnails cache and add it to a cron job with its "--automatic" > command line option. As a system admin, I think cron is the ultimate solution to this. For distro providers, they may too do it automatically. (In reply to comment #37) > "So, this can cause data loss (when you try to write on a filesystem with 0 > bytes left) or a system crash (if your home folder is on the / partition)." > > I am sorry, but to me that is a configuration issue. There is no reason why you > should experience this if the /home directory has its own partition, which is > the recommended and long standing configuration for linux/unix systems. Most > systems even come preconfigured with disk usage quota for users so they do not > eat up all the disk space in /home. Everything is a configuration issue because you can work around most issues with some config. Yet fact is, most distros do not set a quota for /home, so that argument is not valid because KDE should have sensible defaults and not rely on some config that forces it to not fill-up a partition. Second, it does not make sense that this issue does not occur only because /home has its own partition unless you know of some magic hard disk that has endless disk-space. Any partition will fill-up and run out of disk space. And "others don't so why should KDE?" is a really bad argument. If KDE uses .thumbnail is should have a mechanism to manage it, i.e. it's size. Blaiming others who also use it and perform as bad is just bailing out. Web-Browsers have cache management since ages – why? Simply rely on the quota… (In reply to comment #44) > (In reply to comment #37) > > "So, this can cause data loss (when you try to write on a filesystem with 0 > > bytes left) or a system crash (if your home folder is on the / partition)." > > > > I am sorry, but to me that is a configuration issue. There is no reason why you > > should experience this if the /home directory has its own partition, which is > > the recommended and long standing configuration for linux/unix systems. Most > > systems even come preconfigured with disk usage quota for users so they do not > > eat up all the disk space in /home. > > Everything is a configuration issue because you can work around most issues > with some config. Yet fact is, most distros do not set a quota for /home, so > that argument is not valid because KDE should have sensible defaults and not > rely on some config that forces it to not fill-up a partition. > > Second, it does not make sense that this issue does not occur only because > /home has its own partition unless you know of some magic hard disk that has > endless disk-space. Any partition will fill-up and run out of disk space. Did you even bother to read the context upon which that answer was given ??? It was a specific answer to a specific issue raid, not a general answer. > And "others don't so why should KDE?" is a really bad argument. If KDE uses > .thumbnail is should have a mechanism to manage it, i.e. it's size. Blaiming > others who also use it and perform as bad is just bailing out. Again. Read the entire exchange the next time around. I did not say the solution was quotas or separate partitions. That answer was specific to the issue of data loss raised not only in comment #32, but also comment #30 and comment #20. As I explicitly stated this issue needs to be resolved in the specification because piece meal fixes won't address the situation for the reasons stated in my previous response. But you somehow chose to ignore that completely. > Web-Browsers have cache management since ages – why? Simply rely on the quota… A browser does not have to deal with every application in the world screwing around with its cache management. Each browser has its own separate cache management. If that was the case with thumbnail caching, then this bug report won't exist and we would not be having this discussion. As such, stating a browser has coped with this "since ages" is meaningless in this case. (In reply to comment #45) > Did you even bother to read the context upon which that answer was given ??? It > was a specific answer to a specific issue raid, not a general answer. Did you read your own quotation? I even left it in there to make sure you could see what you quoted and refered to and what my posting refers to. If your answer does not relate to what you quote, don't blame me. Current behaviour can lead to data-loss and even not being able to log into KDE. Calling that a configuration issue is simply wrong. Sorry mate. > > And "others don't so why should KDE?" is a really bad argument. If KDE uses > > .thumbnail is should have a mechanism to manage it, i.e. it's size. Blaiming > > others who also use it and perform as bad is just bailing out. > > Again. Read the entire exchange the next time around. I did not say the > solution was quotas or separate partitions. That answer was specific to the > issue of data loss raised not only in comment #32, but also comment #30 and > comment #20. Yep and mine relates to exactly that bit and your answer. So please quote what you are referring to or maybe don't use ---- to separate your answers – just skip whatever makes you confuse your own text. > As I explicitly stated this issue needs to be resolved in the specification > because piece meal fixes won't address the situation for the reasons stated in > my previous response. But you somehow chose to ignore that completely. I stick to what you quote and your answer to that. > > Web-Browsers have cache management since ages – why? Simply rely on the quota… > > A browser does not have to deal with every application in the world screwing > around with its cache management. Each browser has its own separate cache > management. If that was the case with thumbnail caching, then this bug report > won't exist and we would not be having this discussion. As such, stating a > browser has coped with this "since ages" is meaningless in this case. Neither does KDE have to deal with where uncritical data such as thumbnails came from. If KDE is set to keep the thumbnails folder to size x or remove files older than x days it does not matter what app puts them in there. Sweeper does not care about that and users deleting .thumbnail don't either. Both of which you mentioned. Please don't get into arguments over tangential details. I think most people will agree that, in the absence of a solution from freedesktop, kde should provide a way to prevent the thumbnails folder from growing indefinitely, and activate it by default. That is the main issue. Let's find a way to solve it :) This issue can also be blamed on the distros (except ones like Gentoo and arch). You may add a script at startup to check free space and then delete the thumbnails. This problem is also with downloaded package of package managers, the script may be used to take care of that too. Also I think it'll be a good idea to make generation of thumbnails optional, if done so, the thumbnails will be placed in /tmp. Placing the thumbnails in another place doesn't help. That'll just fill up too. The distros could ship with a cron job of some sort, but this is a work around for a bad bug in KDE itself that should just be fixed. I'm not sure what you mean about "This problem is also with downloaded package of package managers, the script may be used to take care of that too." but it sounds of topic for this bug. I don't think distros or package managers have anything to do with it. IMO it is specifically a "desktop" issue, therefore the desktop environment should deal with it. You can run a machine without X and never have to deal with thumbnails. Btw, a certain popular operating system places thumbnails in a hidden file in every directory with files that are "thumbnailed", rather than in a centralized place. Of course, that doesn't work for read-only locations, but other than that, it solves the space problem - thumbnails automatically come and go with the original files. But I guess that's not an option for Linux, it has already been decided to use ~/.thumbnails, we just need a way to clean it up automatically. I think that the same thing that manages the trash could take care of the thumbnails too (and I'm not the first one). Is there any argument against it? (In reply to comment #49) > Placing the thumbnails in another place doesn't help. That'll just fill up too. /tmp gets wiped out on every reboot. > > I'm not sure what you mean about "This problem is also with downloaded package > of package managers, the script may be used to take care of that too." but it > sounds of topic for this bug. I think the free Desktop standards need to change, I find aditsu's advice good. Anyone notified about this to freedesktop? (In reply to comment #51) > (In reply to comment #49) > > Placing the thumbnails in another place doesn't help. That'll just fill up too. > > /tmp gets wiped out on every reboot. Just because it does on your system, doesn't mean that happens everywhere. Also, for those of us who have uptimes of over a day or so, this isn't going to help. The standard needs to be updated, but not with some half-baked workaround. We need a solution that works for everyone and should be well thought out and tested. If putting it in /tmp works for you, you can just link .thumbnails there for the time being. Or you can just make a separate, small partition and link all users' .thumbnails in there. There are enough workarounds for everybody to choose from, until the standard gets updated. What has FreeDesktop.org got to do with it ? KDE decided to store a thumbnail cache and forgot to ever flush the cache, KDE apps are the only ones who use it afaik. My C is rusty, but if anyone knows where in the KDE source the Trash chek is invoked I might be able to copy what that does. (In reply to comment #53) > What has FreeDesktop.org got to do with it ? KDE decided to store a thumbnail > cache and forgot to ever flush the cache, KDE apps are the only ones who use it > afaik. Wrong. Read the whole bugthread for more details. > My C is rusty, but if anyone knows where in the KDE source the Trash chek is > invoked I might be able to copy what that does. Unlike .thumbnails, trash is not just a simple directory (see ~/.local/share/Trash/). I agree that using a similar structure in the standard would be an idea. I'd just like to point out that I'm not affiliated with KDE, and know only what I picked up over the years and in this thread and I know way better than you. Maybe a little reading instead of storming ahead with your incorrect assumptions might help. The spec hasn't seen any action since 2004. The spec is broken. It should be an easy fix for KDE to duplicate the 'Trash' code while working on a revised spec. if that's even required. The spec says (at http://jens.triq.net/thumbnail-spec/delete.html) that "a thumbnail should be deleted if the orginial[sic] file doesn't exist anymore ... consider the following rules ... local file ... check if the original file exists. If it doesn't exist anymore the program should delete the thumbnail .... For all internet related schemes (like http: or ftp:) delete the thumbnail if it hasn't been accessed within a certain user defined time period (can default to 30 days)." So KDE is technically and practically in breach of that spec. I don't see an appropriate topic in https://bugs.freedesktop.org/enter_bug.cgi so assume KDE is not using a shared fd.o component to manage it's thumbnails so here is the right place to lodge a bug. (In reply to comment #54) > I'd just like to point out that I'm not affiliated with KDE, and know only what > I picked up over the years and in this thread and I know way better than you. > Maybe a little reading instead of storming ahead with your incorrect > assumptions might help. Actually that's wrong as well in a practical sense. :) Have a look at the date this bug was filed and at the number of votes. Waiting for some spec to tell KDE people how to approach this did not really help, did it? There are two things one can do. 1) Keep on waiting for some coincidental spec change. (Didn't happen over the last 7 years) 2) Act and solve the issue for KDE leaving it to those that did not care for the last 7 years whether they take over that approach or keep on not caring. If people from other DEs or apps cared, the spec would have changed or improved over the years. Didn't happen. This issue and its solution not being part of the spec does not forbid KDE to solve the issue. It's not like solving it would act against the spec or some common good. What you suggest is to just wait and you see where that lead to. It is a KDE issue and KDE has to solve it, whether it implements a solution because it is described in some spec or whether it implements it because it is a viable solution does not make any difference. In fact, for most people it might actually be true that only KDE-apps use that folder because they only use KDE-apps. (In reply to comment #56) > (In reply to comment #54) > > I'd just like to point out that I'm not affiliated with KDE, and know only what > > I picked up over the years and in this thread and I know way better than you. > > Maybe a little reading instead of storming ahead with your incorrect > > assumptions might help. > > Actually that's wrong as well in a practical sense. :) If I was planning to do something about it, yes, obviously. Which is my point, somebody offering to help should maybe research the issue. At least to the point where they're on par with someone who's only superficially familiar with it. > Have a look at the date this bug was filed and at the number of votes. Waiting > for some spec to tell KDE people how to approach this did not really help, did > it? > > There are two things one can do. 1) Keep on waiting for some coincidental spec > change. (Didn't happen over the last 7 years) 2) Act and solve the issue for > KDE leaving it to those that did not care for the last 7 years whether they > take over that approach or keep on not caring. I'd consider both approaches suboptimal. Anybody wanting to fix this should do what should have been done quite a while ago: go, take the spec and fix it. > If people from other DEs or apps cared, the spec would have changed or improved > over the years. I call bull on this argument. You can just as easily say, if KDE cared, the spec would have been changed. > Didn't happen. This issue and its solution not being part of > the spec The issue *is* part of the spec (as in the bug is caused by the spec and therefore in the spec). > does not forbid KDE to solve the issue. It's not like solving it would > act against the spec or some common good. I'm pretty sure that depends on the solution. If it can be worked around in KDE without affecting anybody else, then great, and I don't think anybody would really mind it (in fact I assume it would be welcomed by most). > What you suggest is to just wait and you see where that lead to. No, it isn't. What I'm suggesting is do this properly, not wait for somebody to do it properly. I personally don't have the time to, so I hope somebody else will take care of it in the meantime, but that has nothing to do with how the issue is approached in general. > It is a KDE > issue and KDE has to solve it, whether it implements a solution because it is > described in some spec or whether it implements it because it is a viable > solution does not make any difference. > > In fact, for most people it might actually be true that only KDE-apps use that > folder because they only use KDE-apps. That is *so* not relevant. Any solution/workaround has to be general enough so that other applications aren't affected by it, and that means any solution outside the spec will cause so much breakage and need so much testing with programs outside of KDE that solving it in the spec is simply more convenient. Also, consider that I won't be the only one getting really annoyed if suddenly some of my more frequently used programs (say the GTK file dialog) stop working properly because KDE decided to implement some incompatible change. (In reply to comment #57) > I'd consider both approaches suboptimal. Anybody wanting to fix this should do > what should have been done quite a while ago: go, take the spec and fix it. So tell me how this is different from simply thinking about a solution that works and writing that down as spec? Right, no talking to all those that did not care in the last seven years, including KDE. > I call bull on this argument. You can just as easily say, if KDE cared, the > spec would have been changed. Thanks for agreeing. If KDE people care, they fix it and thus provide some solution to document, i.e. write down as part of the spec. Talking does not change anything. Changing the spec does not change anything. Implementing a solution in KDE does change something – and from there can be taken to the spec and those that do not have a solution yet and did not bother to come up with one in the first place. In theory you might be right that one should talk first, but in reality there is nobody to talk to and those that do, set the pace. > The issue *is* part of the spec (as in the bug is caused by the spec and > therefore in the spec). The solution is not and I hope you don't want KDE to implement some issue that is part of a spec. There is no solution in the spec only a problem. There would be a solution in KDE and that solution can then become part of the spec. The other way around did not work. (Maybe it would if we wait another 7 years...) > I'm pretty sure that depends on the solution. If it can be worked around in KDE > without affecting anybody else, then great, and I don't think anybody would > really mind it (in fact I assume it would be welcomed by most). > > > What you suggest is to just wait and you see where that lead to. > > No, it isn't. What I'm suggesting is do this properly, not wait for somebody to > do it properly. I personally don't have the time to, so I hope somebody else > will take care of it in the meantime, but that has nothing to do with how the > issue is approached in general. You miss the point. If there would be somebody willing to implement a solution that person would be the only person interested in a solution AND willing to work on it. All others are either not interested or not willing to work on it thus there is no solution yet. So the person that implements it has nobody to talk to other than people not willing to work on it/spend time on it and hence have no right to tell him how to do it. Anything else will just end-up in endless debates by those that don't have to actually do anything but just talk. As me and you. So I'm saying, whoever implements a solution is right as long as it respects the bits of the spec which do exist. > That is *so* not relevant. Any solution/workaround has to be general enough so > that other applications aren't affected by it, and that means any solution > outside the spec will cause so much breakage and need so much testing with > programs outside of KDE that solving it in the spec is simply more convenient. How come you and anybody else not actually implementing this set the rules? > Also, consider that I won't be the only one getting really annoyed if suddenly > some of my more frequently used programs (say the GTK file dialog) stop working > properly because KDE decided to implement some incompatible change. Sorry, but an app which stops to work or fails because a thumbnail is missing is – let's say it nicely – not well thought through. Especially since no thumbnail should exist in .thumbnails anyway whose source file does not exist anymore. (In reply to comment #52) > (In reply to comment #51) > > (In reply to comment #49) > > > Placing the thumbnails in another place doesn't help. That'll just fill up too. > > > > /tmp gets wiped out on every reboot. > > Just because it does on your system, doesn't mean that happens everywhere. This's standard FHS stuff... if you're system doesn't do that, it doesn't follow standards. > Also, for those of us who have uptimes of over a day or so, this isn't going to > help. > > The standard needs to be updated, but not with some half-baked workaround. We > need a solution that works for everyone and should be well thought out and > tested. > If putting it in /tmp works for you, you can just link .thumbnails there for > the time being. Or you can just make a separate, small partition and link all > users' .thumbnails in there. There are enough workarounds for everybody to > choose from, until the standard gets updated. I've changed the ownership of ~.thumbnails to root... no thumbnails for me.(In reply to comment #53) > What has FreeDesktop.org got to do with it ? KDE decided to store a thumbnail > cache and forgot to ever flush the cache, KDE apps are the only ones who use it > afaik. > > My C is rusty, but if anyone knows where in the KDE source the Trash chek is > invoked I might be able to copy what that does. KDE is written in C++ (In reply to comment #55) > The spec hasn't seen any action since 2004. The spec is broken. > It should be an easy fix for KDE to duplicate the 'Trash' code while working on > a revised spec. if that's even required. > > The spec says (at http://jens.triq.net/thumbnail-spec/delete.html) that "a > thumbnail should be deleted if the orginial[sic] file doesn't exist anymore ... > consider the following rules ... local file ... check if the original file > exists. If it doesn't exist anymore the program should delete the thumbnail > .... For all internet related schemes (like http: or ftp:) delete the thumbnail > if it hasn't been accessed within a certain user defined time period (can > default to 30 days)." > So KDE is technically and practically in breach of that spec. > I think the thumnails should only be deleted if the files are too using dolphin/konquerer. > I don't see an appropriate topic in https://bugs.freedesktop.org/enter_bug.cgi > so assume KDE is not using a shared fd.o component to manage it's thumbnails so > here is the right place to lodge a bug. I'll talk to them about this.(In reply to comment #56) > In fact, for most people it might actually be true that only KDE-apps use that > folder because they only use KDE-apps. I've also deployed a lot of Gnome systems, and none of them complained about it, although this problem personally did occur a long time back. (In reply to comment #52) > (In reply to comment #51) > > (In reply to comment #49) > > > Placing the thumbnails in another place doesn't help. That'll just fill up too. > > > > /tmp gets wiped out on every reboot. > > Just because it does on your system, doesn't mean that happens everywhere. This's standard FHS stuff... if you're system doesn't do that, it doesn't follow standards. > Also, for those of us who have uptimes of over a day or so, this isn't going to > help. > > The standard needs to be updated, but not with some half-baked workaround. We > need a solution that works for everyone and should be well thought out and > tested. > If putting it in /tmp works for you, you can just link .thumbnails there for > the time being. Or you can just make a separate, small partition and link all > users' .thumbnails in there. There are enough workarounds for everybody to > choose from, until the standard gets updated. I've changed the ownership of ~.thumbnails to root... no thumbnails for me.(In reply to comment #53) > What has FreeDesktop.org got to do with it ? KDE decided to store a thumbnail > cache and forgot to ever flush the cache, KDE apps are the only ones who use it > afaik. > > My C is rusty, but if anyone knows where in the KDE source the Trash chek is > invoked I might be able to copy what that does. KDE is written in C++ (In reply to comment #55) > The spec hasn't seen any action since 2004. The spec is broken. > It should be an easy fix for KDE to duplicate the 'Trash' code while working on > a revised spec. if that's even required. > > The spec says (at http://jens.triq.net/thumbnail-spec/delete.html) that "a > thumbnail should be deleted if the orginial[sic] file doesn't exist anymore ... > consider the following rules ... local file ... check if the original file > exists. If it doesn't exist anymore the program should delete the thumbnail > .... For all internet related schemes (like http: or ftp:) delete the thumbnail > if it hasn't been accessed within a certain user defined time period (can > default to 30 days)." > So KDE is technically and practically in breach of that spec. > I think the thumnails should only be deleted if the files are too using dolphin/konquerer. > I don't see an appropriate topic in https://bugs.freedesktop.org/enter_bug.cgi > so assume KDE is not using a shared fd.o component to manage it's thumbnails so > here is the right place to lodge a bug. I'll talk to them about this.(In reply to comment #56) > In fact, for most people it might actually be true that only KDE-apps use that > folder because they only use KDE-apps. I've also deployed a lot of Gnome systems, and none of them complained about it, although this problem personally did occur a long time back. (In reply to comment #52) > (In reply to comment #51) > > (In reply to comment #49) > > > Placing the thumbnails in another place doesn't help. That'll just fill up too. > > > > /tmp gets wiped out on every reboot. > > Just because it does on your system, doesn't mean that happens everywhere. This's standard FHS stuff... if you're system doesn't do that, it doesn't follow standards. > Also, for those of us who have uptimes of over a day or so, this isn't going to > help. > > The standard needs to be updated, but not with some half-baked workaround. We > need a solution that works for everyone and should be well thought out and > tested. > If putting it in /tmp works for you, you can just link .thumbnails there for > the time being. Or you can just make a separate, small partition and link all > users' .thumbnails in there. There are enough workarounds for everybody to > choose from, until the standard gets updated. I've changed the ownership of ~.thumbnails to root... no thumbnails for me.(In reply to comment #53) > What has FreeDesktop.org got to do with it ? KDE decided to store a thumbnail > cache and forgot to ever flush the cache, KDE apps are the only ones who use it > afaik. > > My C is rusty, but if anyone knows where in the KDE source the Trash chek is > invoked I might be able to copy what that does. KDE is written in C++ (In reply to comment #55) > The spec hasn't seen any action since 2004. The spec is broken. > It should be an easy fix for KDE to duplicate the 'Trash' code while working on > a revised spec. if that's even required. > > The spec says (at http://jens.triq.net/thumbnail-spec/delete.html) that "a > thumbnail should be deleted if the orginial[sic] file doesn't exist anymore ... > consider the following rules ... local file ... check if the original file > exists. If it doesn't exist anymore the program should delete the thumbnail > .... For all internet related schemes (like http: or ftp:) delete the thumbnail > if it hasn't been accessed within a certain user defined time period (can > default to 30 days)." > So KDE is technically and practically in breach of that spec. > I think the thumnails should only be deleted if the files are too using dolphin/konquerer. > I don't see an appropriate topic in https://bugs.freedesktop.org/enter_bug.cgi > so assume KDE is not using a shared fd.o component to manage it's thumbnails so > here is the right place to lodge a bug. I'll talk to them about this.(In reply to comment #56) > In fact, for most people it might actually be true that only KDE-apps use that > folder because they only use KDE-apps. I've also deployed a lot of Gnome systems, and none of them complained about it, although this problem personally did occur a long time back. (In reply to comment #52) > (In reply to comment #51) > > (In reply to comment #49) > > > Placing the thumbnails in another place doesn't help. That'll just fill up too. > > > > /tmp gets wiped out on every reboot. > > Just because it does on your system, doesn't mean that happens everywhere. This's standard FHS stuff... if you're system doesn't do that, it doesn't follow standards. > Also, for those of us who have uptimes of over a day or so, this isn't going to > help. > > The standard needs to be updated, but not with some half-baked workaround. We > need a solution that works for everyone and should be well thought out and > tested. > If putting it in /tmp works for you, you can just link .thumbnails there for > the time being. Or you can just make a separate, small partition and link all > users' .thumbnails in there. There are enough workarounds for everybody to > choose from, until the standard gets updated. I've changed the ownership of ~.thumbnails to root... no thumbnails for me.(In reply to comment #53) > What has FreeDesktop.org got to do with it ? KDE decided to store a thumbnail > cache and forgot to ever flush the cache, KDE apps are the only ones who use it > afaik. > > My C is rusty, but if anyone knows where in the KDE source the Trash chek is > invoked I might be able to copy what that does. KDE is written in C++ (In reply to comment #55) > The spec hasn't seen any action since 2004. The spec is broken. > It should be an easy fix for KDE to duplicate the 'Trash' code while working on > a revised spec. if that's even required. > > The spec says (at http://jens.triq.net/thumbnail-spec/delete.html) that "a > thumbnail should be deleted if the orginial[sic] file doesn't exist anymore ... > consider the following rules ... local file ... check if the original file > exists. If it doesn't exist anymore the program should delete the thumbnail > .... For all internet related schemes (like http: or ftp:) delete the thumbnail > if it hasn't been accessed within a certain user defined time period (can > default to 30 days)." > So KDE is technically and practically in breach of that spec. > I think the thumnails should only be deleted if the files are too using dolphin/konquerer. > I don't see an appropriate topic in https://bugs.freedesktop.org/enter_bug.cgi > so assume KDE is not using a shared fd.o component to manage it's thumbnails so > here is the right place to lodge a bug. I'll talk to them about this.(In reply to comment #56) > In fact, for most people it might actually be true that only KDE-apps use that > folder because they only use KDE-apps. I've also deployed a lot of Gnome systems, and none of them complained about it, although this problem personally did occur a long time back. Sorry about the duplicates. The KDE sites seems super-slow. Before replying, I'd like to point out that the main thing I'm trying to achieve here is for people to recognise that this isn't a problem that can be fixed with some quick hack. That is one of the reasons why this bug has been around for so long. Another is that with proper measures, .thumbnails can be restricted in size quite easily, so this isn't likely to become a problem for developers anytime soon. A few possible workarounds have been discussed (e.g. the watchdog program that keeps the directory's size under a certain limit), and I think the short-term solution lies with one of them or something similar. People whining about an issue that isn't such a huge problem isn't going to encourage any developer to work on it, so it'd be kinda cool if we could focus on a solution and have less "waaah, nobody's working on this" kind of comments. (In reply to comment #58) > In theory you might be right that one should talk first, but in reality there > is nobody to talk to and those that do, set the pace. That is a sure recipe for chaos. If somebody does this in an unstable branch and it doesn't get included in a stable release until it has been formalised into a standard, fine. That's one way to develop a standard, and not a bad one IMO. > > The issue *is* part of the spec (as in the bug is caused by the spec and > > therefore in the spec). > > The solution is not and I hope you don't want KDE to implement some issue that > is part of a spec. Here, the issue is that some kind of control mechanism for max. directory size is missing in the spec. So adding one to the spec (and implementing it, obviously) is a solution in the spec. > You miss the point. If there would be somebody willing to implement a solution > that person would be the only person interested in a solution AND willing to > work on it. All others are either not interested or not willing to work on it > thus there is no solution yet. So the person that implements it has nobody to > talk to other than people not willing to work on it/spend time on it and hence > have no right to tell him how to do it. That is a really dangerous point of view. If I went ahead and did anything I pleased to mitigate the issue just because nobody tried changing the spec, that would likely end up doing more harm than good. Also, I might have missed it, but I haven't seen anybody actually try to change the spec. I prefer this way because then at least others will have a reference if push comes to shove (IOW nobody talks and somebody just changes the spec anyway they want, which is pretty much the same thing as you're suggesting but with a formal reference for everybody else). > So I'm saying, whoever implements a solution is right as long as > it respects the bits of the spec which do exist. They also have to make sure any spec-compliant program will still work with the changes, and that is the bit that is difficult to do cleanly. > How come you and anybody else not actually implementing this set the rules? I'm not setting any rules, I'm arguing for a solution I can live with. > Sorry, but an app which stops to work or fails because a thumbnail is missing > is – let's say it nicely – not well thought through. Especially since no > thumbnail should exist in .thumbnails anyway whose source file does not exist > anymore. (little aside: that bit of the spec is pretty difficult to implement in itself, which is why they're often simply left there) That is not exactly what I'm worried about. To make any decent kind of change, you need some sort of control mechanism, and the logical place to put whatever files that mechanism needs is in .thumbnails. Now there are two obvious way a third-party app can fuck this up: - it somehow can't deal with the additional files/directories - it doesn't find the proper sources and deletes the "superfluous" files *That* is why I think changing the spec first is imperative. (In reply to comment #62) > (In reply to comment #52) > > (In reply to comment #51) > > > /tmp gets wiped out on every reboot. > > > > Just because it does on your system, doesn't mean that happens everywhere. > > This's standard FHS stuff... if you're system doesn't do that, it doesn't > follow standards. Last time I checked (admittedly several years ago), it simply said that applications need to cope with files in /tmp vanishing. It didn't say anything about *having* to empty it at every boot. Has that changed? (If so, that is a really stupid change, but also another issue.) Anyway, it doesn't address the fact that my system stays on long enough to fill up /tmp on its own every now and then, and placing .thumbnails there isn't really going to help this. > I've changed the ownership of ~.thumbnails to root... no thumbnails for me. All you needed to do is remove all permissions for the directory. Has worked for my size-constrained university account for years... ;-) > (In reply to comment #55) > > consider the following rules ... local file ... check if the original file > > exists. If it doesn't exist anymore the program should delete the thumbnail > > .... For all internet related schemes (like http: or ftp:) delete the thumbnail > > if it hasn't been accessed within a certain user defined time period (can > > default to 30 days)." > > So KDE is technically and practically in breach of that spec. > > I think the thumnails should only be deleted if the files are too using > dolphin/konquerer. I have yet to see a file that uses dolphin/konqueror (except maybe shell scripts). If you mean files that dolphin can handle (i.e. show a preview of), that's pretty short-sighted and also looking at it the wrong way. I can have both dolphin-based stuff and GTK-based apps access these (e.g. pictures), and if KDE assumes it is solely responsible for them, that'll create chaos. C or C++ is much of a muchness. If I can find the code that deals with Trash I can probably figure out how to do the same for thumbnails. To the KDE guys -- https://bugs.freedesktop.org/show_bug.cgi?id=38663 Gnome has this feature. There's no reason why KDE shouldn't have. The Gnome impl. follows what KDE does already for Trash : http://git.gnome.org/browse/gnome-settings-daemon/tree/plugins/housekeeping/gsd-housekeeping-manager.c - I believe the main entry point is purge_thumbnail_cache() Also I noticed gwenview already has this feature. As you can see, what gwenview does is to remove all the files in the .thumbnail directory before exiting. I think it is not the same as to limit the space used. What I've seen in the gnome code is that they clean the .thumbnail directory at session entry, every 2 minutes (like a cron entry) and/or at session exit, deleting the oldest entries until the space used is bellow some limit. This can be done with a kded4 daemon. Yes, precisely. And ALL the thumbnails should go into the ".cashe" directory, to make it easier to exclude from backup files that can readily be recreated. Using different directories is out of scope, and really breaks the spec., as opposed to fixing it. Using a cache directory allows a system-scope process to clean out entries, as opposed to something hand-rolled only for image thumbnails in KDE. It also allows users to skip backing data that is derived, saving bandwidth and dollars. The name and location of the cache doesn't matter for the purposes of this bug (unconstrained disk usage until crash). You can already exclude this 'backing data that is derived'. Why has the version changed to unspecified ? This is a known-crasher data-loss bug in the current version of KDE !?! FYI: I'm trying to fix this bug with a kded daemon. See https://git.reviewboard.kde.org/r/102083/ Awesome James, much better than what I (didn't!) write yet. One thing though, the README says "shows a warning dialog when it uses" but I see in the code it supports 'just clear it up automatically' without raising the dialogue box, is that right ? Which is the default (I hope the Just Work version !) ? Unfortunately, there is no other way (yet) to configure a kded daemon (or I haven't figured how to). We have to show the dialog, at least one time. But one can enable the clear automatically to not see it anymore. I think first the standards should change, then this fix can be made. Till then you guys can pressurize that freedesktop bug - https://bugs.freedesktop.org/show_bug.cgi?id=38663 By casting your vote and commenting. @dE - I disagree. Everyone knows the spec is broken in this regard, and this issue is causing problems now, not at some point in the future. Let's do both ! The spec actually proposes (as one of several possibilities) running a daemon to clean up the cache (http://jens.triq.net/thumbnail-spec/delete.html: ... 2. A daemon runs in the background which cleans up the cache in certain intervalls. ...) It doesn't specify completely when to clean up the cache, only mentioning that thumbnails should be deleted if the original file doesn't exist anymore. IMHO, this at least implies that a daemon may also clean up old thumbnails. This is actually proposed for http:// and ftp:// thumbnails and thumbnails for removable media files. To not interrupt the operation of running programs, the daemon should IMHO not delete a thumbnail if there is a process holding a handle to it, but otherwise I don't see any problems with this approach, especially as Gnome also does it. (In reply to comment #80) > @dE - I disagree. Everyone knows the spec is broken in this regard, and this > issue is causing problems now, not at some point in the future. Let's do both ! Why not ignore the software patent laws -- everyone knows this law's broken. Not following standards will always produce some problem or the other; we should amend the standard first. Off topic, but you can't patent software in my country :-p The standard is bound to be corrected to fill the gap but lets at least try and lead by example here and fix the serious problem. If you see a man drowning, but the law doesn't say that you must save him, do you let him drown (for fear of disturbing the fish), until you can get the legislators to enact a law that says you must save him if you can? Besides, the standard ALREADY suggests a daemon for deleting thumbnails, as explained in comment #81. What else do you need? Just pointing that the spec URI referenced widely on this bug report now is a 403. However, another copy can be found here: http://people.freedesktop.org/~vuntz/thumbnail-spec-cache/ Also, on the freedesktop.org wiki, the Thumbnail spec is listed under the header "Draft specifications that are new and not yet widely used, though they may be used by one or more desktops or desktop applications". Considering that this spec is still considered "draft" after more than 7 years of inactivity, I believe KDE should drop this and take a particular approach to care about thumbails, which could also be very well documented and publicized so others can adhere to it if they want. Or look for another already published specification used by a third party and implement the same functionality. IMHO blaming a 2004 draft spec for a bug still open in 2011 is pointless. And even the 2004 draft spec had a solution for this problem. Anyone needs a script which deletes these thumbs after checking for free space? I, for instance, use find ~/.thumbnails -atime +9 -delete This does not set an upper limit, but aviods recreation of the same thumbnails again and again while purging the ones less used, either due to deleted files, less visited folders or external drives no longer available. Of course, the date range may vary to suit any need. A little more than a week is useful to keep both the workday and weekend use, but will purge everything after a long holiday away from computer. Also, more days does not always mean more disk used but more delay on one-time thumbnails purge and also less recreation of same thumbnails seen less frequently. For users with a lot of different once-seen-and-forgotten folders, a short period is more desirable as the cache can grow quite fast; however for those who use a large image collection once in a while, every recreation can be a loss of time to compute and will reclaim the saved disk space. The size limit may be more sensible, but meanwhile I'm using this approach. Dawit Alemayehu: > How exactly is this a security issue? > Or for that matter a privacy one? Storing private data in a unencrypted form without even noticing the user about this fact. It just is, face it. > you can disable them in dolphin or konqueror. But not in Gwenview and probably many other applications. > If you do not want previews (aka thumbnails) to be generated I want them to be generated. I just don't want them stored on the harddrive, even temporarily. Or if stored, 'wipe' should be used to remove it. I want them to be kept in RAM only. I know this is paranoid, but that's how the security and privacy protection works. WORKAROUND: Delete/wipe your ~/.thumbnails directory and then create a file with the same name as the directory. Thumbnails will be generated on the fly and won't be stored on the harddrive. (In reply to comment #89) > Dawit Alemayehu: > > > How exactly is this a security issue? > > Or for that matter a privacy one? > > Storing private data in a unencrypted form without even noticing the user about > this fact. It just is, face it. > > > you can disable them in dolphin or konqueror. > > But not in Gwenview and probably many other applications. > > > If you do not want previews (aka thumbnails) to be generated > > I want them to be generated. > I just don't want them stored on the harddrive, even temporarily. Or if stored, > 'wipe' should be used to remove it. I want them to be kept in RAM only. > > > I know this is paranoid, but that's how the security and privacy protection > works. > > > WORKAROUND: > Delete/wipe your ~/.thumbnails directory and then create a file with the same > name as the directory. Thumbnails will be generated on the fly and won't be > stored on the harddrive. Using ~ as an unencrypted partition is not a good idea when you want security. You know, a lot of configs lie there. Using a single directory and cryptic file names for thumbnails has created this problem. Because they are all in one place it is easy to measure the space used by thumbnails, unlike windows XP, where there is a thumbnail file in each indexed directory. The thumbnail directory on my computer is far too large to be efficient. If more than, (5000 files. a guess), are in a single directory access times rise significantly. My guess is that 63,000 files would cause a massive slow down in access time. A better solution is a thumbnail file with the name of the indexed folder, this is how windows does it. This has two benefits, firstly this file can be compressed, secondly unwanted thumbnail files, can be easily removed. I expect that this would improve loading time for indexed directories. The name of the files, the cache or (to an extent) it's location is out of scope of this bug, which is to do with the cache growing without limit until the disk is full. Changing the name convention wouldn't solve this more serious problem, only help in (automate) clean ups. (In reply to comment #92) > The name of the files, the cache or (to an extent) it's location is out of > scope of this bug, which is to do with the cache growing without limit until > the disk is full. > Changing the name convention wouldn't solve this more serious problem, only > help in (automate) clean ups. You have missed the main point of my comment that is, if thumbnails are used an extremely large number of files is created, all of which reside in the thumbnail directory. This creates the same problem that windows has with its central registry, bloat. Using zipped files to store thumbnails will reduce the number of entries in the thumbnail directory, and will save space and should enable faster indexing. If there was no central thumbnail directory and a zipped thumbnail file resided in each indexed directory this would not, be seen a problem. I agree that changing the storage convention alone will not solve the problem, however I feel it is a vital first step, that makes cleanup possible. All programs that need to load huge directories, have problems, either slow loading, or worse crash. Most programs cannot allocate enough memory for a directory as big as that of a large thumbnail directory. This is probably why cleanup has not been attempted. Generating thumbnails on the fly is slow but might be faster than finding thumbnails in a bloated thumbnail directory. A large, ever-changing zip file would create even worse wearout on flash media than thousands of tiny files already do. Limiting the number of files/filesize is the way to go. Maybe even moving the .thumbnail dir into RAM until the system is shut down. (In reply to comment #94) > A large, ever-changing zip file would create even worse wearout on flash media > than thousands of tiny files already do. Limiting the number of files/filesize > is the way to go. Maybe even moving the .thumbnail dir into RAM until the > system is shut down. You could be right however my tests have revealed that there is little or no speed gain, compared with creating thumbnails on the fly. Definitely not enough to justify the space used. My computer is slow compared with yours. (About 7 years old and has a Celeron processor.) Moving the thumbnail dir into memory is equivalent to what windows does with its central directory, (a badly flawed concept). Both are very bad ideas. My Verdict is that there is no justification for a central thumbnail directory. Scrap it!! It seems that this similar in effect to loading part of MS Office into memory so that it loads faster. The supposed gains in speed prove to be mostly illusory, and are a total waste of space. (In reply to comment #95) > [...] While I definitely don't disagree with you on the whole bunching all thumbnails together thing, this is still a different bug and you should open a new one for that particular issue. (And yes, these issues have some common ground, but not enough to pollute this bug report even further with an only slightly related issue.) I think it's about time this bug should be fixed. In the mean time, I made a cleanup script for Debian - #! /bin/bash # from /etc/rc.local make this script run. #Estimate free space from df temp=$(df | grep --word-regexp /) temp=$(echo $temp | cut -d ' ' -f4) # If space is less than 500 MB..... if [[ $temp -le 512000 ]] then echo "Making free space, cleaning up redundant files....." aptitude clean cd /home DIRs="$(ls -d */ -1)" while [[ $DIRs != "" ]] do cur_dir="$(echo "$DIRs" | head -1)" rm -R "$cur_dir".thumbnails/* rm -R "$cur_dir".cache/* DIRs="$(echo "$DIRs" | grep --line-regexp --invert-match "$cur_dir")" done echo "Done freeing up." fi This'll basically remove /home/*/.thumbnails/* and /home/*/.thumbnails/* i.e. in each directory inside home it'll remove .thumbnails/* and .cache/* If you're on another distro, just remove 'aptitude clean'. This's intended to be run at startup. Why not just do rm -rf /home/*/.thumbnails/* and /home/*/.cache/* rather than the convoluted while loop ? (In reply to comment #98) > Why not just do > rm -rf /home/*/.thumbnails/* and /home/*/.cache/* > rather than the convoluted while loop ? Hum... I didn't know that. Bash was smarted than I expected. Thanks! This will delete any thumbnails that were last accessed more than 90 days ago: find ~/.thumbnails/ -atime +90 -delete This reduced my thumbnail cache from 2.4G to 58M We know it's easy to fix with a single line shell command. It should never get to this state in the first place, which is what this bug is about. Not that KDE seem to care about filling peoples disks with crap. Do most people have atime enabled? I thought it was off by default on most KDE systems. (In reply to comment #101) > We know it's easy to fix with a single line shell command. It should never > get to this state in the first place, which is what this bug is about. Not > that KDE seem to care about filling peoples disks with crap. And that's why I voted for the bug! The workaround will be useful for people till the problem is fixed. (In reply to comment #102) > Do most people have atime enabled? I thought it was off by default on most > KDE systems. The default mount option is 'relatime', which updates atime in about a day after the file was accessed. https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Power_Management_Guide/Relatime.html This is sufficient for our purpose. Maybe it might make sense to have a background job automatically remove any thumbnail not accessed in the last year or something. Five year old bug. KDE doesn't care. Ten year old bug. KDE doesn't care. Actually, it's a 14 year-old bug. :) KDE cares a lot, but we have thousands of other bugs too, and not enough developer resources to address every bug in as timely a manner as folks would prefer. Not being addressed yet is probably because this is not a visible bug that causes data loss, and doesn't seem trivial to fix. That doesn't mean it's not important, of course! But most of us (including me) don't get paid to do this. I'm currently on vacation, yet here I am, triaging bugs. We would welcome patches from anyone who happens by and thinks of a clever method to do this. Here's our documentation on the subject: - https://community.kde.org/Get_Involved/development - https://community.kde.org/Infrastructure/Phabricator Then help us to help you. There are multiple simple scripts listed here, any of which would fix the issue. But we can't ask every distribution to include a cron or systemd unit file to do this. How can we easily get KDE's core to execute a script as a service ? There are several already listed in the System Settings app, for instance... I consulted with some other KDE developers on this. The thumbnail system we have is not KDE-specific; it an implementation of a cross-desktop standard promulgated by FreeDesktop.org, and the thumbnails in the cache are re-used across a plethora of software. For example thumbnails created in KDE Dolphin are displayed in GNOME's Nautilus, and vice versa. As a result, it doesn't seem appropriate for anything in KDE Plasma or KIO to clean up old thumbnails, because this would affect more than just KDE software. It's important to fix problems in the right place. If we add this cleanup script to KIO, it is inevitable that we'll receive bug reports saying "KDE is deleting my thumbnails, but GNOME doesn't do this!" Instead, I recommend both of the following next steps: - Propose to the FreeDesktop folks that the thumbnail spec (https://specifications.freedesktop.org/thumbnail-spec/thumbnail-spec-latest.html) be amended to address the issue of the indefinitely growing thumbnail cache - In the meantime, propose that individual distros adopt a time-based thumbnail cleanup script like the ones discussed here, or a maximum thumbnail cache size, or something else (depending on their preferences). I understand that this may disappoint some people, but I hope the above explanation helps people understand why this isn't the right place to fix the issue, and why I recommend proposing the fixes to distros and the FreeDesktop folks--both of which are agnostic of desktop environments that implement the spec. Here's an email thread I started about this with the FreeDesktop folks: https://lists.freedesktop.org/archives/xdg/2018-July/014065.html Here's the request I've filed for Ubuntu: https://bugs.launchpad.net/ubuntu/+source/thumbnailer/+bug/1782613 Users don't switch DE often. It's a cache - if it's removed not too much of deal. |