For (understandable) security and privacy reasons, Dolphin will not cache thumbnails for files that are located on a separate encrypted container. While I support this as a default behavior, and feel it is a responsible decision, there are two reasons why I suggest there should be an option that a user can override to allow thumbnail caching from separate encrypted containers: 1. The user's $HOME directory is already located on a encrypted container, and thus an external drive that is also using encryption needn't worry about "spillover" to a non-encrypted device. 2. Performance. External and/or separate devices need to re-generate their thumbnails every time the user browses to this folder. This causes extra reads and CPU cycles, and usually on slower devices in comparison to their primary drive. Not to mention the fact that reading thumbnails from cache is insanely faster than re-generating them every time. Thus, while the default should remain as is (which errs on the side of security and privacy) there should definitely be an option to allow thumbnail caching on separate encrypted devices. I suppose this also applies to KDialog, and not just Dolphin? --- Just for fun, here is a third reason: 3. The current implementation is inconsistent from a security and privacy standpoint. Network shares accessed via CIFS (SMB) have their thumbnails cached, even though (unknown to KDE) they are encrypted on the server. This is a case of "spillover" of potentially sensitive thumbnails, where KDE does not know whether or not the separate location is "encrypted", and thus whether or not to forgo thumbnail caching. --- For reference, here is a bug report that addresses the previous problem: https://bugs.kde.org/show_bug.cgi?id=411919 I'm not suggesting a reversal; only an *option* to override this for the reasons explained above. In certain cases, the drawbacks outweigh the benefits, which is why an option for the user makes the most sense.
For clarity, my examples use LUKS on external drives. I haven't yet tested with other encryption alternatives, but from what I've read, it's the same concept / rules with forgoing caching thumbnails in KDE.
For any users or developers reading this, here is a summarized version of how things stand now, as of version 21.08.1 default behaviour: * Local folder on same filesystem, no encryption Cached thumbnails? Yes! * Local folder on same filesystem, LUKS encryption Cached thumbnails? Yes! * External USB drive, no encryption Cached thumbnails? Yes! * Remote location (CIFS/SMB) Cached thumbnails? Yes! * External USB drive, LUKS encryption Cached thumbnails? NO! It is the very last example in which an option to override the default behaviour is important.
(In reply to flan_suse from comment #2) > For any users or developers reading this, here is a summarized version of > how things stand now, as of version 21.08.1 default behaviour: > > * Local folder on same filesystem, no encryption > Cached thumbnails? Yes! > > * Local folder on same filesystem, LUKS encryption > Cached thumbnails? Yes! > If you don't mind me asking, what is your partition scheme? I have a LVM on LUKS setup with Home,Root & Swap subvolumes in openSUSE Leap (Argon). However, the thumbnail cache doesn't get created at all! '~/.cache/thumbnails' doesn't exist and thumbnails are painstakingly re-generated every time I open '~/Pictures' with Dolphin, Gwenview or the KDE File Picker.
(In reply to Lemuel Simon from comment #3) > If you don't mind me asking, what is your partition scheme? Gladly. I'll keep it simple and laid out, so as to make it easier to follow and compare. Just FYI, I do not use LVM whatsoever. The following sources have multimedia (picture and video) that support thumbnail previews. SRC1, Single local partition (entire system, /boot, /, and /home) | encrypted (LUKS) SRC2, Single local partition (external USB) | encrypted (LUKS) SRC3, Single local partition (external USB) | plain SRC4, Network share (CIFS) | encrypted (ZFS) --- When using Dolphin to browse images/videos in SRC1, SRC3, and SRC4, thumbnails are generated and CACHED under ~/.cache/thumbnails With subsequent re-visits of these folders, the thumbnails are QUICKLY displayed (because they're loaded from cache) Examples including ~/Pictures, ~/Videos, a CIFS share mounted via "cifs" from a server, a USB stick with pictures, et al. Because /home resides within an encrypted partition, there is no risk of "spillover" of sensitive thumbnails jumping from an encrypted source to a plain (non-encrypted) ~/.cache However, if my /home actually lived in a plain (non-encrypted) partition, SRC4 is an example of "spillover" of sensitive thumbnails being cached to a non-encrypted ~/.cache (yikes!), whereas the original photos are stored encrypted (on the server). --- When using Dolphin to browse images/videos in SRC2, thumbnails are NOT cached under ~/.cache/thumbnails With subsequent re-visits of these folders, the thumbnails must be re-generated all over again... every... single... time... --- This bug report (feature request) is to give users, like you and me, an option to toggle this behavior. Keep the default as it is, but give us the option to allow thumbnail caching ALWAYS (regardless if the source multimedia lives in an encrypted space.) * It would be much faster (rather than always re-generating the thumbnails from scratch) * Very important when dealing with numerous files in large folders * It's not necessary to protect from "spillover" if your /home lives in its own encrypted space anyways (such as /home in LUKS) * It's not even implemented properly anyways, as seen with the example of SRC4 above
Can't this be implemented in one of two simple ways? 1) Simply give the user the option to "Allow caching thumbnails from encrypted and remote locations" Enabling this option would disable/ignore the "safety check" on whether or not caching thumbnails is allowed. Or... 2) If the user's $HOME is located in an encrypted location (such as LUKS), then the "safety check" should be disabled/ignored. --- As it currently stands, KDE currently has implemented a "halfway" safety measure, since it *does not consider*... * ...the user's preference * ...if the user's $HOME is encrypted anyways * ...if the remote network share (e.g., SMB, NFS) is encrypted on the server itself
Created attachment 146051 [details] Toggle to enable/disable thumbnail caching of media that lives on encrypted source Is there anyone looking into this? Folders with many image files can really make the CPU crank up the heat, especially when they are WEBP images. And this happens *every time* I open the folder, since nothing is ever cached. :'( My $HOME is encrypted, as are many other users' setups. Which means preventing thumbnails from being cached only slows down the workflow without any added security benefits. A simple "bypass" option or toggle would suffice, that allows thumbnail caching from any folder: remote, encrypted, plain, doesn't matter The option could look something like so: Prevent thumbnail caching of media from encrypted sources [X] It can be enabled by default, but at least the user has an option to completely disable ("uncheck") this feature. I implore the devs to seriously consider this. While the original intentions were good, it is causes more issues than benefits for users who either have encrypted $HOME and/or don't want to sacrifice performance for security (not even a security issue for them.) What's more, it's not even implemented correctly, since as I explain above, network shares (SMB, NFS) in which the media is stored encrypted on a server, KDE treats it like it's non-encrypted and will cache the thumbnails *anyways*. Please see my attachment of a mock idea of how this could look in the GUI.
Pretty please, can this be considered? KDE is slamming my CPU every time I browse a directory with many images, as it has to re-generate the thumbnails Every. Single. Time. And they are never cached. See my attachment of how this option could easily be toggled on or off. This decision to outright disable caching of thumbnails (without a means to re-enable them) from an encrypted source, causes more issues than solutions. Please, I humbly beg of you. The current implementation makes NO SENSE. If the user's home folder is encrypted, it makes no sense. If the user is view images on a network share, it makes no sense. If the user prefers cache'd thumbnails, speed, and cooler CPU's, and doesn't care it's from an encrypted source, it makes no sense. Please, I'm begging. I don't understand why this change was abruptly made with Dolphin / KIO without considering the most common real-world scenarios.
I can confirm this is the same problem I encounter every day with the same setup, also having luks encrypted drives connected, also my linux drive is fully encrypted and I have to agree it doesn't make sense to me either, please can this be considered
(In reply to Alexander Wipperfürth from comment #8) > I can confirm this is the same problem I encounter every day with the same > setup, also having luks encrypted drives connected, also my linux drive is > fully encrypted and I have to agree it doesn't make sense to me either, > please can this be considered Imagine having folders with lots of WEBP images. It's bad enough with JPEG and PNG, having to re-generate the thumbnails every single time, but it REALLY cooks the CPU when it's WEBP images. I laid out the case. I even created a mock screenshot of what such an option could look like in Dolphin. (see the attachment) Not sure what else we can do at this point. Out of desperation, I'm trying to figure out how to forcefully cache thumbnails in a folder manually, if I knew what command to issue. :(
I don't have to imagine, I have thousands of video files on my drives, it's not cool that this doesn't work properly, I thought about some stuff we could do and this would be to fork dolphin and patch it so it doesn't regenerate the thumbnails and instead use the cached files instead, this would require searching through the code and identify where this is and we can build our own version of Dolphin, or someone from the dev team gets the same issue and gets tired of waiting for the previews to finish regenerate and patch it. Currently I don't have the time to look into this very deeply but I will when I have some spare time and maybe we can somehow find a fix for this. On 08.03.22 17:58, flan_suse wrote: > https://bugs.kde.org/show_bug.cgi?id=443806 > > --- Comment #9 from flan_suse <windows2linux@zoho.com> --- > (In reply to Alexander Wipperfürth from comment #8) >> I can confirm this is the same problem I encounter every day with the same >> setup, also having luks encrypted drives connected, also my linux drive is >> fully encrypted and I have to agree it doesn't make sense to me either, >> please can this be considered > Imagine having folders with lots of WEBP images. It's bad enough with JPEG and > PNG, having to re-generate the thumbnails every single time, but it REALLY > cooks the CPU when it's WEBP images. > > I laid out the case. I even created a mock screenshot of what such an option > could look like in Dolphin. (see the attachment) > > Not sure what else we can do at this point. > > Out of desperation, I'm trying to figure out how to forcefully cache thumbnails > in a folder manually, if I knew what command to issue. > > :( >
I would also like a option to keep the generated thumbnails. Only a few weeks ago I encrypted my drives to LUKS. My main SSD + 3 internal HDD's + 1 External HDD. I was confused why my thumbnails weren't being saved, until I found this thread. Now, I get the reason why it doesn't save them by default, but I would love a option to keep the thumbnails anyway. I have many folders with videos and pictures and now they are constantly being regenerated every time I go to those folders.
Is this bug report just not reaching the right people? Should I change the component or CC a developer in particular?
We already have Bug 411919 about the general issue; we don't need another one proposing a different solution. Let's use Bug 411919 to continue the discussion so we don't get confused by having the discussion split off into two places. *** This bug has been marked as a duplicate of bug 411919 ***
Reopening this bug because of miscommunication
*** Bug 448302 has been marked as a duplicate of this bug. ***
An ideal solution! (In reply to Marcin Gurtowski) > The previous policy, in regards to encrypted $HOME was: > "If encrypted file for which thumbnail is generated is on the same drive as > ~/.cache/thumbnails, do create cache for this file.". > > Should we change it to: > "If ~/.cache/thumbnails is on encrypted device, create cache for thumbnails > of encrypted files."? This comment was taken from another bug report, but it basically solves the issue in this bug report. I'm assuming the same logic will be used across-the-board, regardless if such encrypted files live on a LUKS device or are encrypted on a network share. If the above is implemented, it solves this issue. No need for any extra toggles or options. However, implementing the above fix will not exclude the possibility for additional toggles or options in the future. It's a win-win-win! :)
Any update on this front? It's been thoroughly discussed with solutions laid out. It's getting ridiculous that on KDE my CPU has to scream and blast the fans just to browse folders with many images. Even if I re-open the folder after having just viewed it.
I can only think the reason that this isn't a higher priority is that I went through a dozen different threads to find this one with a lot of users unaware for the cause of slow thumbnails. Having just updated from Kubuntu 20.04 to 22.04 this is (thankfully!) the only glaring bug I've found. It really does make using dolphin wearisome unless you just accept you can't use icons without unreasonable delay. Using encrypted drives are so prevalent I was surprised this issue is from 2021. Of the solutions the simplest would seem to be to just check if $HOME is encrypted and if so then allow the thumbnail cache for other encrypted drives. The toggle also makes sense, as would simply having the thumbnail directory itself encrypted, but my vote would be for whatever gets done the quickest.
There is actually another workaround you guys could try, symlink the folders you want to access on a regular basis and put the link into your ~/ this somehow helped me and the thumbnails were loading faster, but this is still not the best solution. On 27.03.23 19:22, bugzilla_noreply@kde.org wrote: > https://bugs.kde.org/show_bug.cgi?id=443806 > > toroidalaxioms@gmail.com changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > CC| |toroidalaxioms@gmail.com > > --- Comment #18 from toroidalaxioms@gmail.com --- > I can only think the reason that this isn't a higher priority is that I went > through a dozen different threads to find this one with a lot of users unaware > for the cause of slow thumbnails. Having just updated from Kubuntu 20.04 to > 22.04 this is (thankfully!) the only glaring bug I've found. It really does > make using dolphin wearisome unless you just accept you can't use icons without > unreasonable delay. Using encrypted drives are so prevalent I was surprised > this issue is from 2021. > > Of the solutions the simplest would seem to be to just check if $HOME is > encrypted and if so then allow the thumbnail cache for other encrypted drives. > The toggle also makes sense, as would simply having the thumbnail directory > itself encrypted, but my vote would be for whatever gets done the quickest. >
(In reply to Alexander Wipperfürth from comment #19) > There is actually another workaround you guys could try, symlink the > folders you want to access on a regular basis and put the link into your > ~/ Sadly, this isn't feasible for myself (and probably many users), as we have different folders all over our drive. The issue needs to be resolved by upstream KDE. :( It's a matter of waiting now? I'm not even sure if this is being worked on. It used to work on earlier versions of KDE, until it was "fixed", and now we have our thumbnails on encrypted drives constantly being regenerated every time which costs time and CPU.
I ran into this bug on my own machine. I'm new to KDE development but will be taking up this bug as the fix is relatively straightforward. I'll be sending out a merge request within the next few days.
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/kio/-/merge_requests/1283
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/kio/-/merge_requests/1284
A possibly relevant merge request was started @ https://invent.kde.org/system/dolphin/-/merge_requests/548
Merge requests: https://invent.kde.org/system/dolphin/-/merge_requests/548 https://invent.kde.org/frameworks/kio/-/merge_requests/1284
Because I can't create a KDE Identity account (my email gets flagged as "spam"), I'll reply in here to Nate. On the merge request Nate wrote: "Personally I think the ideal way to fix this would be to store the thumbnails for files located on an encrypted drive on that drive itself, rather than in the user's home directory." Please don't do this. 1. It breaks from standards 2. External drives are slower, especially with laptops that use NVMe and SSD internal drives 3. It's incongruent with network shares 4. It creates extra (slower) writes on an external USB If the user's home directory lives on an encrypted space, this should be a non-issue from the very beginning. Store cached thumbnails in the standard directory. If the user's home directory does not live on an encrypted space, give the user's a choice (such as a simple option of "Allow storing thumbnails of images from encrypted source in the user's cache". You can even have this option regardless... Let's not make this more complicated than it needs to be. Encrypted home? No problems. Store thumbnails like they have been for years and years. Plain home? Default to a "secure" method, but give the user the option to change this.
This isn't about home directories (which experience no issues, as you point out) but rather other encrypted volumes, such as Plasma Vaults and encrypted external disks.
But anyway, this isn't the right place to discuss it; it would be be either the GitLab issue, or in Bug 411919. If you need help getting an account created, please contact sysadmin@kde.org. Thanks!
Just a reminder that 411919 is not the same issue and that has already been discussed. Please discuss on gitlab, not on that issue (or this one, but especially not that one).
(In reply to Nate Graham from comment #27) > This isn't about home directories (which experience no issues, as you point > out) but rather other encrypted volumes, such as Plasma Vaults and encrypted > external disks. This bug report and the entire discussion has always been about external drives / locations. As was my latest comment. If the user's home is encrypted, everything else is moot: *always* store thumbnails in the standard cache directory I have to repeat this again: If the user's home is encrypted, everything else is moot: *always* store thumbnails in the standard cache directory There is no "problem to solve". The user's home is encrypted. (Currently, KDE does not behave in this way, and hence the user complaints and this bug report.) What if the user's home is *not* encrypted AND the files live on an external drive / location that *IS* encrypted? There should be an *option* to continue with the normal standard: store thumbnails in the standard cache directory It's that simple. There are many reasons why someone would want to continue to store their thumbnails locally, even if their home is not encrypted. (It is *THEIR* computer, after all. They have full access and rights. Let's not try to subject them to some out-of-scope security hand-holding if they happen to encrypt their USB drives.) The lists I include in my comments are important and have real-world consequences based on being longtime a KDE user. 1. STANDARDS: Don't randomly break from standards. Other DEs use this home cache directory 2. PERFORMANCE: The user's home almost always lives on a much faster device (NVMe, SSD) 3. PERFORMANCE: Ejecting an external USB flushes the buffers from RAM; meanwhile the home drive is still snappier with buffers in RAM 4. CONSISTENCY: KDE is not aware of encrypted network shares or SED-encrypted drives. Any "protection" for files on an external USB drive will not apply to encrypted files from a network share or SED external drive. 5. WORKFLOW: Writing and updating thumbnails to external drives will change the "last modified time" on the individual folders if the thumbnails are stored folder-by-folder. (This can ruin certain tasks and workflows.) (I understand #5 might not apply.) I humbly ask that we don't start writing and updating thumbnails to external drives. (Currently we do not, so we should offer the user an option if they understand the risks involved if their home is not encrypted.) Scenario A: Home encrypted = *Always* store thumbnails in the standard cache directory Scenario B: Home *not* encrypted = Default to *NOT* storing thumbnails from encrypted external locations, but offer the user an *OPTION* if they understand the risks. What about Plasma Vaults? Sure, since they are KDE-specific, you can have them store thumbnails within the container itself. But please don't conflate Plasma Vaults with standard universal solutions like LUKS and network shares. Done. Simple.
I think the reason for the disconnect here is that there are two different use cases: Some people use encrypted drives in a static fashion. As in, it's always attached to their machine. Other people use encrypted drives in a more "moveable" fashion. As in, it's either an encrypted volume that they can move around (as in KDE Vaults), or it's a portable drive they take with them to other machines. For the first group, if the home directory is encrypted, then there's no downside to storing the thumbnails in the usual location in the homedir. However, there would be a downside to storing them on the separate drive, for the reasons listed above. However, for the second group, it's the exact opposite. For people who use their encrypted volumes in a portable manner, storing the thumbnails on the drive itself has no downsides. If the freedesktop spec was updated to permit it, then any other compliant machine they take their drive to would check on the drive itself to look for thumbnails. However, storing them in the homedir would have a downside, as the thumbnails would need to get regenerated every time the drive is used on a new machine. The problem is that based on the thumbnail spec provided to me by Nate, what the first group is advocating for is compliant with the existing thumbnail spec. I suggest that we go ahead with the fix I proposed on my merge request. Namely, for any encrypted volume whatsoever, if the homedir is encrypted, store the thumbnails in the homedir per the spec. Then, if we want to pursue a change to the spec, we can have that conversation separately. In the meantime, I don't think it's fair for kde users nor is it a good look for kde to remain noncompliant with the spec and leave users without basic functionality for so long.
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/kio/-/merge_requests/1286
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/kio/-/merge_requests/1287
Git commit 231668fb984fb1e7269a313bb7688c7473497f47 by Payton Quinn. Committed on 22/05/2023 at 18:15. Pushed by ngraham into branch 'master'. Enable thumbnail caching if thumbnail directory is on an encrypted volume M +8 -1 src/gui/previewjob.cpp https://invent.kde.org/frameworks/kio/commit/231668fb984fb1e7269a313bb7688c7473497f47
Git commit c76abd288938011860ac32032a2a66018736c402 by Nate Graham, on behalf of Payton Quinn. Committed on 22/05/2023 at 18:55. Pushed by ngraham into branch 'kf5'. Enable thumbnail caching if thumbnail directory is on an encrypted volume (cherry picked from commit 231668fb984fb1e7269a313bb7688c7473497f47) M +8 -1 src/widgets/previewjob.cpp https://invent.kde.org/frameworks/kio/commit/c76abd288938011860ac32032a2a66018736c402
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/kio/-/merge_requests/1293
Fixed in frameworks 5.107 onwards