Bug 411919 - Store thumbnails for files inside an encfs or cryFS encrypted container somewhere inside the encrypted container itself, not ~/.cache/thumbnails, or generate them but don't store them at all
Summary: Store thumbnails for files inside an encfs or cryFS encrypted container somew...
Status: CONFIRMED
Alias: None
Product: kio-extras
Classification: Frameworks and Libraries
Component: Thumbnails and previews (show other bugs)
Version: 19.08.1
Platform: Other Linux
: HI normal
Target Milestone: ---
Assignee: Marcin Gurtowski
URL:
Keywords:
: 419183 420130 430202 444380 (view as bug list)
Depends on:
Blocks:
 
Reported: 2019-09-14 19:09 UTC by Marcin Gurtowski
Modified: 2024-01-01 14:13 UTC (History)
11 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marcin Gurtowski 2019-09-14 19:09:56 UTC
SUMMARY
While it used to work in the past, now Dolphin doesn't show previews for files in encfs encrypted directories. It goes for both Vaults and directories mounted manually.

STEPS TO REPRODUCE
1. Enable previews
2. Open encfs encrypted directory containing files that should generate previews

OBSERVED RESULT
No previews are shown


SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Linux (x86_64) release 5.0.0-27-generic
(available in About System)
KDE Plasma Version: 5.16.5
KDE Frameworks Version: 5.61.0
Qt Version: 5.12.3
Comment 1 Marcin Gurtowski 2019-09-15 08:13:40 UTC
Changed title as the same happens for cryFS encryption
Comment 2 Nate Graham 2019-09-15 22:11:05 UTC
This is deliberate, because right now, the only place that thumbnails are stored are outside of the encrypted container in ~/.cache/thumbnails, which represents a security leak that could compromise the reason why you're using an encrypted behavior in the first place.

We can turn this back on once the thumbnails for files in an encrypted container are stored in the container itself.
Comment 3 Marcin Gurtowski 2019-09-16 14:43:16 UTC
(In reply to Nate Graham from comment #2)
> This is deliberate, because right now, the only place that thumbnails are
> stored are outside of the encrypted container in ~/.cache/thumbnails, which
> represents a security leak that could compromise the reason why you're using
> an encrypted behavior in the first place.
> 
> We can turn this back on once the thumbnails for files in an encrypted
> container are stored in the container itself.

This makes sense. Maybe previews should be generated, but not stored?
Comment 4 Nate Graham 2019-09-16 18:52:46 UTC
That's not a bad idea either, yeah.
Comment 5 Nate Graham 2019-09-16 19:46:51 UTC
If you want to try your hand at doing that, here's the related commit that turned them off entirely: https://cgit.kde.org/kio.git/commit/?id=bc42a1b2f9138558eb0d3d33f70e0e416a5225ca

Might be easy to adapt that to only avoid saving the thumbnails if it's an encrypted container rather than avoiding generating them entirely.
Comment 6 Nate Graham 2020-03-24 15:51:47 UTC
*** Bug 419183 has been marked as a duplicate of this bug. ***
Comment 7 Marcin Gurtowski 2020-03-24 17:26:38 UTC
I've taken this bug, but since it might take me bit longer to fix I've decided to share where I'm at.

This is my first time working on KDE code, so it takes extra time. Also I only work on it when I have free time, which isn't much.

I decided to resolve problem by generating previews without saving them as I hope it will be easier to do. Someone might go back to it and add catching later.

Currently I have working solution for all files except of directories, which seem to have some different mechanism for creating and catching thumbnails. Haven't found code for it yet.

Feel free to contact me if you want to help me or take this over.
Comment 8 Christoph Feck 2020-03-24 20:45:54 UTC
Directory thumbnails are created at https://cgit.kde.org/kio-extras.git/tree/thumbnail/thumbnail.cpp#n466
Comment 9 Marcin Gurtowski 2020-03-24 20:48:51 UTC
(In reply to Christoph Feck from comment #8)
> Directory thumbnails are created at
> https://cgit.kde.org/kio-extras.git/tree/thumbnail/thumbnail.cpp#n466

Thanks! Hopefully this will be a big help. Also, no wonder I couldn't find it as I didn't look in kio-extras repository.
Comment 10 Nate Graham 2020-04-13 16:03:44 UTC
Git commit ae0beded1fd9533f0526e06ee52e892558051410 by Nate Graham, on behalf of Marcin Gurtowski.
Committed on 13/04/2020 at 16:03.
Pushed by ngraham into branch 'master'.

Show previews on encrypted filesystems

Summary:
Instead of skipping generating previews on encrypted filesystems, do create them but don't cache.
FIXED-IN: 5.70

Reviewers: ivan, #frameworks, dfaure, ngraham

Reviewed By: ivan, ngraham

Subscribers: ngraham, kde-frameworks-devel

Tags: #frameworks

Differential Revision: https://phabricator.kde.org/D28746

M  +6    -8    src/widgets/previewjob.cpp

https://commits.kde.org/kio/ae0beded1fd9533f0526e06ee52e892558051410
Comment 11 Nate Graham 2020-04-15 18:06:23 UTC
*** Bug 420130 has been marked as a duplicate of this bug. ***
Comment 12 Nate Graham 2020-12-10 16:03:54 UTC
*** Bug 430202 has been marked as a duplicate of this bug. ***
Comment 13 David Faure 2021-05-29 08:51:03 UTC
Git commit e56893b087ef021706ee4623dcadfc495fba528c by David Faure, on behalf of Marcin Gurtowski.
Committed on 29/05/2021 at 08:50.
Pushed by dfaure into branch 'master'.

Updating caching logic for thumbnails, to detect encrypted devices

M  +1    -0    src/widgets/CMakeLists.txt
M  +110  -16   src/widgets/previewjob.cpp

https://invent.kde.org/frameworks/kio/commit/e56893b087ef021706ee4623dcadfc495fba528c
Comment 14 Nate Graham 2021-06-07 12:54:15 UTC
One more by Marcin Gurtowski takes care of the rest of this:

https://invent.kde.org/network/kio-extras/-/commit/6ac8fd72edbb67d4a32cdfc455df8633db9ea2e0
Comment 15 Nate Graham 2021-10-25 22:07:24 UTC
*** Bug 444380 has been marked as a duplicate of this bug. ***
Comment 16 David 2021-10-25 22:38:57 UTC
I still get previews generated under ~/.chache/thumbnails in 21.08.2.
Comment 17 Nate Graham 2021-10-25 22:46:11 UTC
:(
Comment 18 Marcin Gurtowski 2021-10-26 15:50:22 UTC
(In reply to David from comment #16)
> I still get previews generated under ~/.chache/thumbnails in 21.08.2.

I can confirm. I'm getting previews generated now. I'll look into it this weekend.
Comment 19 Marcin Gurtowski 2021-10-30 11:17:14 UTC
It looks like my hard drive just broke, so I won't be able to work on the problem this weekend. Sorry about that. Let me know if someone else will be looking at this. If not, I should get around to it eventually.
Comment 20 flan_suse 2022-04-05 13:50:37 UTC
This "fix" is creating more problems than solutions.

Please see my bug report for a more sensible way to approach this. As others have confirmed (and as it is clearly evident), this was NOT the right approach.

https://bugs.kde.org/show_bug.cgi?id=443806

It's getting really bad. Outright disabling the caching of thumbnails from an encrypted container makes no sense. I lay out the reasons why in my bug report. I highly encourage others to read it and vote on the issue.

There are better ways to approach this than the nuclear option.
Comment 21 Nate Graham 2022-04-07 15:52:33 UTC
*** Bug 443806 has been marked as a duplicate of this bug. ***
Comment 22 Nate Graham 2022-04-07 15:53:42 UTC
I agree that the current situation is not ideal. Marcin, if you aren't able to fix the issue, and nobody else can either, we may have to revert the commits.

Flan, Would you be able to submit a merge request to implement your idea? Caching thumbnails in ~ when ~ is encrypted is reasonable. However I'm still in favor of one of the original ideas which is to cache the thumbnails inside the encrypted volume itself. I think that solves all potential problems, if it's technically feasible.
Comment 23 Marcin Gurtowski 2022-04-07 16:01:26 UTC
(In reply to Nate Graham from comment #22)
> I agree that the current situation is not ideal. Marcin, if you aren't able
> to fix the issue, and nobody else can either, we may have to revert the
> commits.

It depends. I'm not great with time currently, but should be able to find enough to at least check if it's a simple or more difficult fix.

Anyone knows what might be the reason it broke?
Comment 24 Nate Graham 2022-04-07 16:21:01 UTC
It's possible that this is suffering from the same issue as Bug 451408 now.
Comment 25 flan_suse 2022-04-08 18:39:50 UTC
(In reply to Nate Graham from comment #22)
> Flan, Would you be able to submit a merge request to implement your idea?

How and where do I submit a merge request? I'm just an end-user that believes in practicality and usability. :) I'll do it, just need to know the proper method.


----------


> Caching thumbnails in $HOME when $HOME is encrypted is reasonable.

This makes a lot of sense, but see below where I will try to break down this "problem". This is not so much a technical issue (though it requires a technical solution in the future), but rather this is a practical issue, with some psychological and use-case elements involved.


----------


>However I'm still
> in favor of one of the original ideas which is to cache the thumbnails
> inside the encrypted volume itself. I think that solves all potential
> problems, if it's technically feasible.

This might not be ideal, and I would argue against it, for four reasons:

1) The user's $HOME is almost always going to be on the faster media (SSD, NVMe, Optane, etc), whereas an encrypted volume is commonly on a slower media (secondary HDD, external USB drive, USB stick, etc).

2) The user's $HOME is where we expect cache to be consolidated, and it's the main "driver" of daily work. I'd assume most users that utilize encrypted volumes use them nearly exclusively on their *own* system, rather than on someone else's system. It can complicate things if KDE is an outlier in that it caches thumbnails on the folder/volume itself (where the media resides), while other desktop environments continue to use $HOME/.cache consistently.

3) This can be bad for those who use NAS servers or network shares. Not only is KDE *unaware* of whether or not the photos/videos live on an encrypted volume ("Is the SMB / NFS share residing on an encrypted location on the server? No idea!"), but if it were to assume it does, it's now storing cache'd thumbnails on a network share? We're now losing performance and I advise against trying to write on a network share or NAS server "under-the-hood" like that, especially for media archives, etc.

4) It's expected that $HOME/.cache will always have read-write access for the user account. However, offloading the caching of thumbnails to an *external location* gets even more complicated if it is: write-protected; or mounted read-only; or does not have write permissions; etc.

I'm not trying to poo-poo any idea nor come off as a pessimist. I'm just trying to cut off user experience pitfalls *ahead of time*. If it sounds like the above reasons make it seem like there are too many complexities involved, that's because there *are*. Once again, pursuing such a workaround might risk more problems than solutions.


----------


Here is the original problem: "As a user, I want to safeguard against sensitive photos/videos dumping their thumbnails from an encrypted storage onto a non-encrypted storage."

This should not be confused with a friend or adversary in possession of the device. Once it is in their hands, none of this stuff about caching thumbnails matters.

This should not be confused with using the encrypted device on *someone else's* computer. Why? Because the friend/stranger could be using *any* OS, distro, or desktop environment on their own computer.

This really only applies to the *owner's computer* running KDE; in which they do not want "spillover" of encrypted data onto a non-encrypted volume.

There are three common places where encrypted photos/videos reside that are under the owner's control: secondary volumes; external drives; network shares (encrypted on the server).

KDE is only aware of the encryption state of secondary volumes and external drives. KDE is *not* aware of the encryption state on network shares (on a server / NAS).

Therefor, to treat this as "How should KDE handle thumbnail caching of encrypted volumes?" falls short of its *own goal*.

It should be split into *two* approaches:

1) "How should KDE handle thumbnail caching of encrypted volumes?"
2) "How should KDE handle thumbnail caching of network shares / remote locations?"

In my opinion, you can simplify this by yielding control to the end-user with *two different toggles*. Plain and simple:

* Cache thumbnail previews for multimedia from encrypted volumes? Yes/No
* Cache thumbnail previews for multimedia from network shares? Yes/No

Simple as that.

Here is the expected behavior, which is very clean and unambiguous, and let's the user choose:

Cache thumbnail previews for multimedia from encrypted volumes? YES!
- Regardless of the external storage or secondary volume using encryption, thumbnails will be cache'd to $HOME/.cache/thumbnails

Cache thumbnail previews for multimedia from encrypted volumes? NO!
- Previews will be generated for multimedia from external storage or secondary volumes that use encryption, but no thumbnails will be cache'd to $HOME/.cache/thumbnails

Cache thumbnail previews for multimedia from network shares? YES!
- From any network share (SMB, NFS), thumbnails will be cache'd to $HOME/.cache/thumbnails

Cache thumbnail previews for multimedia from network shares? NO!
- Previews will be generated for multimedia from network shares (SMB, NFS), but no thumbnails will be cache'd to $HOME/.cache/thumbnails

No need for any special workarounds or clever analysis, or "exceptions" based on whether or not $HOME lives in an encrypted space. In a sense, this will be "business as usual", except now the user has two additional toggles that they may use for enhanced security (if they so desire).

This satisfies nearly *everyone* with the most common scenarios:

* People who have encrypted $HOME. These toggles do not remove nor *force* anything. They needn't worry about "spillover" either way. These can be toggles that they "leave alone" or decide to revisit later.

* People who have non-encrypted $HOME. These toggles are a blessing! They can choose to go the route of "I don't want ANY thumbnails to be cache'd if they are from encrypted volumes or network shares!" Or they might  choose the route of "I don't care about spillover! I want speed and performance!"


----------


I hope I was able to lay this out as clearly as possible.

I'm not an engineer nor developer. So I view this from the perspective of someone who uses KDE everyday, from basic tasks to dealing with tons of images and videos.
Comment 26 Marcin Gurtowski 2022-04-08 20:43:31 UTC
(In reply to flan_suse from comment #25)
> How and where do I submit a merge request? I'm just an end-user that
> believes in practicality and usability. :) I'll do it, just need to know the
> proper method.

https://community.kde.org/Infrastructure/GitLab#Submitting_a_merge_request is a good guide. If you also need information on development, I used https://community.kde.org/Get_Involved/development when I looking into this problem for the first time


> > Caching thumbnails in $HOME when $HOME is encrypted is reasonable.

To clarify, that's not exactly how the problem was previously solved. Kio would save thumbnail for files on encrypted partition if $HOME was on the same partition. If both were encrypted, but were different devices, thumbnail would be created, but not saved. 

> 2) The user's $HOME is where we expect cache to be consolidated, and it's
> the main "driver" of daily work. I'd assume most users that utilize
> encrypted volumes use them nearly exclusively on their *own* system, rather
> than on someone else's system. It can complicate things if KDE is an outlier
> in that it caches thumbnails on the folder/volume itself (where the media
> resides), while other desktop environments continue to use $HOME/.cache
> consistently.

If someone uses both unencrypted $HOME and some encrypted devices, it implies they don't care as much about data from $HOME being leaked as they do about from encrypted media. Catching data from encrypted drive gives access to data that shouldn't be given.
 
I'll try to look into the problem and fix previous solution (create thumbnails, but don't save them), but if you want to  add extra options to this, is sounds fine to me.
Comment 27 flan_suse 2022-04-08 22:05:50 UTC
(In reply to Marcin Gurtowski from comment #26)

> > 2) The user's $HOME is where we expect cache to be consolidated, and it's
> > the main "driver" of daily work. I'd assume most users that utilize
> > encrypted volumes use them nearly exclusively on their *own* system, rather
> > than on someone else's system. It can complicate things if KDE is an outlier
> > in that it caches thumbnails on the folder/volume itself (where the media
> > resides), while other desktop environments continue to use $HOME/.cache
> > consistently.
> 
> If someone uses both unencrypted $HOME and some encrypted devices, it
> implies they don't care as much about data from $HOME being leaked as they
> do about from encrypted media. Catching data from encrypted drive gives
> access to data that shouldn't be given.

But that once again stresses the importance of user-choice. If KDE *hard-locks* a user to one method or the other, based on the encryption (or lack thereof) of their $HOME, then the user is stuck with the developer's decisions and has to either (1) "live with it", or (2) find a hacky workaround, or (3) file bug reports and wait for the changes to be reverted/fixed upstream.

Some user's *without* encrypted $HOME may prefer performance over spillover prevention. While some might not even care. While yet some still might want zero spillover.

Guess what solves all of the above? An option that can be toggled. ;)

As for users *with* encrypted $HOME? Providing the same toggles does no harm, and it still gives them a choice if they so desire.

"Oh, but what about network shares and NAS servers?"

Since KDE is unaware of whether or not the files are encrypted on the server, there should be no fancy workarounds or guessing. Once again, an option that can be toggled resolves this issue. No one needs to ask "How can we determine if the files are encrypted on the server? Maybe we should assume they are or are not?"

A simple toggle solves this.

No one is hard-locked into anything. As for the "default" setting? Err on the side of caution. Have the toggles default to *DISABLED*, and let the user decide if they want to enable them.

No need to deviate from the other desktop environments and OSes by writing thumbnails to external media and network shares.

For the record, I would *NOT* want KDE to store thumbnails on my external drives and *DEFINITELY NOT* try to store them on my network shares.

If KDE tries to go the route of "We'll just save thumbnails on the filesystem from where the images/videos reside", it can really open up a can of worms down the line...

I listed out *four* independent issues of concern if pursuing this approach. (I even thought of *more* reasons off the top of my head just now.)

  
> I'll try to look into the problem and fix previous solution (create
> thumbnails, but don't save them), but if you want to  add extra options to
> this, is sounds fine to me.

I truly, truly believe the two toggles mentioned above would resolve nearly all user scenarios.
Comment 28 Nate Graham 2022-04-11 16:45:08 UTC
Choice is good, but it's even more important that we solve the common use cases by default. I don't object, as long as we make sure that people get their thumbnails in Plasma vaults shown again without having to configure anything.
Comment 29 flan_suse 2022-04-11 17:52:47 UTC
(In reply to Nate Graham from comment #28)
> Choice is good, but it's even more important that we solve the common use
> cases by default. I don't object, as long as we make sure that people get
> their thumbnails in Plasma vaults shown again without having to configure
> anything.

If this is about Plasma Vaults, then how come it inadvertently bled into non-KDE storage?

This affects user of LUKS (internal and external drives), as well as SMB and NFS shares. No Plasma Vaults involved, and yet thumbnails are not being cached inside $HOME/.cache/thumbnails
Comment 30 flan_suse 2022-04-11 17:57:36 UTC
(In reply to flan_suse from comment #29)
> (In reply to Nate Graham from comment #28)

> This affects user of LUKS (internal and external drives), as well as SMB and
> NFS shares. No Plasma Vaults involved, and yet thumbnails are not being
> cached inside $HOME/.cache/thumbnails

To clarify: Thumbnails are not being cached inside $HOME/.cache/thumbnails if the files live on an external or internal LUKS partition. Yet, thumbnails will be saved in the cache if the files are accessed via an SMB or NFS share (even though they are encrypted on the server itself.)

That's the reason I made the other bug report. There are no Plasma Vaults involved, whatsoever.
Comment 31 Nate Graham 2022-04-11 19:16:24 UTC
As I understand it, it's a part of the general problem of certain filesystem types being detected as "non-local" and that being assumed to be slow, and thumbnails not being shown for slow locations.

I think think we need much more discussion here; someone just needs to sit down and fix it.
Comment 32 flan_suse 2022-04-14 01:05:57 UTC
(In reply to Nate Graham from comment #31)
> As I understand it, it's a part of the general problem of certain filesystem
> types being detected as "non-local" and that being assumed to be slow, and
> thumbnails not being shown for slow locations.
> 
> I think think we need much more discussion here; someone just needs to sit
> down and fix it.

Trying to solve a dilemma in KDE (whereas "Plasma Vaults" are unique to KDE) actually introduced more issues for KDE users who *don't* even use this particular feature of "Plasma Vaults".

This is a wide-sweep that is causing headaches for LUKS users, with no means of bypassing it. There's no option, no toggle.

Hence, why I think the toggle option is the best short-term (and possibly long-term) approach. No one will be forced into one way or the other, and this is not a minor issue. This directly involves security and performance. *Notable* performance impacts.

---

As for Plasma Vaults, specifically? Sure, cache the thumbnails directly into the vault itself. Any user who goes out of there way to specifically use Plasma Vaults understands its benefits and implications.

However, users of LUKS, SMB, and NFS were caught in the "crossfire" as uninvolved bystanders, and now we're hoping to have our thumbnails cache once again in $HOME/.cache/thumbnails/

(Many reasons listed above in and the other bug  report.)
Comment 33 Marcin Gurtowski 2022-04-14 17:18:02 UTC
Is seems like thumbnails for files on encrytped devices aren't shown, because of

```
skipCurrentItem = !d->ignoreMaximumSize && size > d->maximumRemoteSize;
```
in `PreviewJob::slotResult`.

I'll see what's exactly the problem, but I'm leaving for holidays, so I won't be able to work on this for almost a week.
Comment 34 Marcin Gurtowski 2022-04-19 15:57:10 UTC
Ok, to be more specific, previews aren't currently showed, because when we check if file is local, we also exclude "slow" devices

`if ((itemUrl.isLocalFile() || KProtocolInfo::protocolClass(itemUrl.scheme()) == QLatin1String(":local")) && !d->currentItem.item.isSlow())`

This causes kio to use logic for remote files. Once we change dolphin settings and set a limit for remote previews, previews for encrypted storage will also become visible.

What behavior are we expecting here? Should I just remove `!d->currentItem.item.isSlow())` for this check or do we actually want to treat slow devices same as remote ones?
Comment 35 Nate Graham 2022-04-20 00:59:35 UTC
Seems like there are a few issues:
- Vaults are marked as slow despite being backed by fast local storage
- Dolphin UI talks about setting a limit for previews for "remote files" but does not mention that slow local files are similarly affected

I think first we should fix whatever is causing vaults to be marked as slow, as that's just not accurate at all. Then we should fix the UI in dolptin to clarify that the remote preview control affects things marked as slow too, and then we should add a little text below it telling the user what "slow" means in this context.
Comment 36 Nate Graham 2022-04-20 01:00:14 UTC
Number 1 is probably an instance of https://bugs.kde.org/show_bug.cgi?id=451408.
Comment 37 flan_suse 2022-04-20 02:07:01 UTC
(In reply to Nate Graham from comment #35)

Slow or not, *displaying* previews are one thing. But what about the over-arching issue of *caching* thumbnails?
Comment 38 Nate Graham 2022-04-20 02:34:51 UTC
I still think each mounted volume's thumbnails should be stored *on* that mounted volume. It solves basically all the problems.
Comment 39 flan_suse 2022-04-20 02:50:23 UTC
(In reply to Nate Graham from comment #38)
> I still think each mounted volume's thumbnails should be stored *on* that
> mounted volume. It solves basically all the problems.

I think it might lead into new issues down the line.

I listed the reasons why.

And I absolutely don't want thumbnails being cache'd on a network share / volume. 

Not to mention some people might mount an external drive as read-only, or not have sufficient permissions.

Not to mention that external media is usually slower than where someone's $HOME resides.

And finally, it means that KDE will stand out as the odd-ball desktop environment, straying for other standards. If one switches between desktops (Xfce, GNOME, Mate, KDE), then with only KDE will it try to write and read thumbnails from each separate volume, while on the others they will write and read  thumbnails directly from $HOME.

We've been using KDE with no problems until this "fix" was implemented, and now we don't have thumbnails being cached. And now thumbnails might in fact attempt to write outside of home? What does that mean for network shares? That would be a step backwards.
Comment 40 Nate Graham 2022-04-21 15:01:42 UTC
Maybe we could only do it for encrypted volumes, and for everything else, the thumbnails could live in the normal location.

Basically the problem to be solved here is that storing thumbnails for files located on an encrypted volume in an unencrypted location elsewhere is a data leak. Not generating thumbnails at all is a nuclear option that reduces usĭlity, so that's no good. Generating thumbnails but storing them *inside* the encrypted volume solves the data leak and preserves the thumbnails feature.

The current spec doesn't consider the case of an encrypted volume at all, so we have to break the spec somehow unless we want to propose a formal change to it. Which is possible. But it can be politically challenging.
Comment 41 flan_suse 2022-04-21 16:02:50 UTC
(In reply to Nate Graham from comment #40)
> Maybe we could only do it for encrypted volumes, and for everything else,
> the thumbnails could live in the normal location.

Nate, from the other bug report I created, all the commenters, as well as myself, are using an encrypted $HOME (which is very common for users who go out of there way to use encryption on external USB, network shares, and secondary drives.)

This comes off as a "solution in search of a problem" for such users.

We get hit with a performance penalty, sluggish browsing, and overworked CPUs (not an exaggeration, I hear my laptop fans spin up every time I browse a folder with many WEBP images.)

Our $HOME is encrypted. There is no risk of spillover.

But once again, I must re-emphasize, your philosophy about reducing or eliminating spillover sounds good, but it's not even feasible when you consider network shares. With ubiquitous home solutions of NAS (TrueNAS, Synology, QNAP, etc), and even just SMB / NFS shares in general, KDE generates and *stores* thumbnails from *encrypted* sources. (They are encrypted on the NAS / server.) This is *spillover* if the user's $HOME is not encrypted.

The solution should NOT be to read and write thumbnails to a network share. That would be a hard block for me, and I would have to jump to another desktop environment if that ever happens. (Too many reasons to list in here.)

------

So for users that have an encrypted $HOME, there's zero reason to use any sort of sophistication of attempting spillover prevention.

No reason to check whether or not the source images and videos live in an encrypted space.

No reason to determine the "speed" of the device.

Just let thumbnails be cache'd under $HOME/.cache/thumbnails/, like normal. We're using encrypted $HOME for a reason, after all.

------

For those with plain $HOME, you've got an entirely different problem.

Try to prevent spillover by forcing the writing and reading of thumbnails to the source folder?

But...

What happens if there are permission issues?

What if the device is read-only (to view, but not modify the images / videos)?

Save the thumbnails in each separate folder? Or save them under the root directory? What if the root directory does not have sufficient permissions? (The common r-xr-x--- for root:wheel)

Should we try to write and read thumbnail files over a network share? (That's a can of worms...)

------

What I can say confidently for now this is:

*** The immediate resolution for users with encrypted $HOME is to remove any such spillover protection. ***

Using our computers is becoming a bottleneck when it involves browsing many images and videos.
Comment 42 Nate Graham 2022-04-21 16:08:30 UTC
Let's try to focus on one thing at a time rather than generalizing, increasing the scope of the issue, and writing walls of text. :) 

I have a Plasma Vault which mounts an encrypted volume when unlocked. I want thumbnails when I browse its files. It lives in my home folder, which is not encrypted. Hence, if its thumbnails are stored in my unencrypted home folder, there is a data leak.

Hopefully we can agree that this specific narrow use case/issue is valid.
Comment 43 Alexander Wipperfürth 2022-04-21 16:25:00 UTC
(In reply to Nate Graham from comment #42)
> Let's try to focus on one thing at a time rather than generalizing,
> increasing the scope of the issue, and writing walls of text. :) 
> 
> I have a Plasma Vault which mounts an encrypted volume when unlocked. I want
> thumbnails when I browse its files. It lives in my home folder, which is not
> encrypted. Hence, if its thumbnails are stored in my unencrypted home
> folder, there is a data leak.
> 
> Hopefully we can agree that this specific narrow use case/issue is valid.

well this is one way I guess, but are the files you are trying to preview on the same drive as your vault?
We have the problem that we have files on a different drive and thumbnails are being regenerated everytime the directory is changed or reloaded, if the files that we want to preview are on the same drive where the thumbnails are being stored its no problem at all, it just works, but not for external drives with encryption enabled on both external and internal drive, I agree that what you stated is valid, are you able to reproduce our issue to verify it?
Comment 44 flan_suse 2022-04-21 16:36:51 UTC
(In reply to Nate Graham from comment #42)
> Let's try to focus on one thing at a time rather than generalizing,
> increasing the scope of the issue, and writing walls of text. :) 

The last part of my post was the "non wall of text" summary. ;)

> What I can say confidently for now this is:
> 
> *** The immediate resolution for users with encrypted $HOME is to remove any such spillover protection. ***
> 
> Using our computers is becoming a bottleneck when it involves browsing many images and videos.


(In reply to Nate Graham from comment #42)
> Hopefully we can agree that this specific narrow use case/issue is valid.

For Plasma Vaults? Sure, I agree.

But the reason this is getting "out of scope" is because trying to resolve the issue around a KDE-specific technology (Plasma Vaults) affects users *without* Plasma Vaults. We use LUKS on external media and/or secondary drives. (LUKS is desktop-agnostic.)

So I circle back around to this point, which I believe will *immediately* resolve one of the most impactful regressions from these recent changes to KDE:

> *** The immediate resolution for users with encrypted $HOME is to remove any such spillover protection. ***
Comment 45 Nate Graham 2022-04-21 16:38:10 UTC
Your issue is valid too, but it's off-topic for this bug report, which is purely about the issue with thumbnails for an encfs/cryfs (etc) encrypted volume. It's not about thumbnails for network mounts or external disks.

If your issue doesn't fall under Bug 451408, can you please file a new bug report for it? Thanks!
Comment 46 flan_suse 2022-04-21 16:40:36 UTC
(In reply to Nate Graham from comment #45)
> Your issue is valid too, but it's off-topic for this bug report, which is
> purely about the issue with thumbnails for an encfs/cryfs (etc) encrypted
> volume. It's not about thumbnails for network mounts or external disks.
> 
> If your issue doesn't fall under Bug 451408, can you please file a new bug
> report for it? Thanks!

If my issue is off-topic, then why'd you mark my other bug report as a "duplicate" of this one?

And no, the issue does not fall under Bug 451408
Comment 47 Nate Graham 2022-04-21 16:41:46 UTC
That must have been a mistake on my behalf then; feel free to un-dupe the bug report you filed, and we can either continue discussing there, or dupe it to a more appropriate one.
Comment 48 Alexander Wipperfürth 2022-04-21 16:42:42 UTC
(In reply to Nate Graham from comment #47)
> That must have been a mistake on my behalf then; feel free to un-dupe the
> bug report you filed, and we can either continue discussing there, or dupe
> it to a more appropriate one.

Alright, I was actually wondering really hard right now since you told us to come here to discuss this further
Comment 49 Marcin Gurtowski 2022-04-22 15:05:38 UTC
The previous policy, in regards to encrypted $HOME was:
"If encrypted file for which thumbnail is generated is on the same drive as ~/.cache/thumbnails, do create cache for this file.".

Should we change it to:
"If ~/.cache/thumbnails is on encrypted device, create cache for thumbnails of encrypted files."?
Comment 50 Nate Graham 2022-04-22 17:55:38 UTC
Would that fix the Vaults use case? The vault gets mounted, but it is itself encrypted. We want to store the cached thumbnails for its files inside of it, not outside of it.
Comment 51 Marcin Gurtowski 2022-04-22 21:05:53 UTC
(In reply to Nate Graham from comment #50)
> Would that fix the Vaults use case? The vault gets mounted, but it is itself
> encrypted. We want to store the cached thumbnails for its files inside of
> it, not outside of it.

It solves problem with data leak and lack of thumbnails.
As for using extra resources, it only makes difference if $HOME is encrypted. For now, my plan is to go back into solution that displays thumbnails on encrypted devices and depending on $HOME it either saves it or not.

Two options I have is when file in encrypted, to either
1. Save cache only if it's on the same device as $HOME. It will fix problem mentioned by @flan_suse, but only when user has whole partition encrypted and assuming that viewed files are on the same partition
2. Save cache if $HOME is encrypted regardless of if source file in encrypted on the same device or not. It solves problem mentioned by @flan_suse without any extra assumptions. As for problem with data leak, it's solved as long as we don't mind saving thumbnails on different device when this different device is encrypted as well.

Neither of those solutions would save cache for encrypted files if $HOME is unencrypted. Anything beyond that (like caching on encrypted device) I consider to be outside of scope of this story, as I won't be able to take care of it, but I'm not in any way against it.
Comment 52 Marcin Gurtowski 2022-04-23 09:13:37 UTC
Continuing work. It turns out, reason why `isSlow()` returns `true`, for encrypted media is that `KFileSystemType::fileSystemType` returns Nfs for them – which is wrong.  I'll try to see why exactly that is, but it might be a tricky problem to solve, so any help is appreciated.
Comment 53 Nate Graham 2022-04-25 15:42:02 UTC
If $HOME Is not encrypted, then generating thumbnails for vault files but not caching them anywhere would be better than what we currently have, since at least we would see thumbnails again. But caching them *inside* the vault would be even better, since then we wouldn't have to waste resources endlessly regenerating thumbnails all the time.

I still think this is what we should do even if $HOME is encrypted. Because even if it is, there is still a  data leak of cached content from one encrypted location to a location outside of it. Yes, $HOME being encrypted protects against thieves who steal your laptop, but it does not protect against other local users with physical access (e.g. when you step away to use the bathroom, or whatever), which is currently a thing you can use Vaults to protect against. But if cached thumbnails are leaked outside of the vault, that protection is diminished.
Comment 54 Contijn Buijs 2022-04-26 06:36:12 UTC
Just adding my opinion on this.

I have all my drives encrypted to LUKS a few short months ago and I also got this issue.

I think storing the thumbnails on the encrypted drive itself is a good option.

Looking forward to a solution!
Comment 55 Marcin Gurtowski 2022-04-30 09:54:06 UTC
Another update.

The exact source of bug is in kcoreaddons/src/lib/io/kfilesystemtype.cpp
static KFileSystemType::Type determineFileSystemTypeImpl(const QByteArray &path)

When we use switch to check the file system type, one of cases for Nfs is

case FUSE_SUPER_MAGIC: // TODO could be anything. Need to use statfs() to find out more.

If we can delegate this to someone more familiar with topic, it would be great. If not, I can try to figure it out myself, but it will probably take more time.
Comment 56 Marcin Gurtowski 2022-04-30 09:55:12 UTC
Alternatively, I could just change it so FUSE_SUPER_MAGIC returns Other instead of Nfs.
Comment 57 Marcin Gurtowski 2022-04-30 10:39:40 UTC
The merge request: https://invent.kde.org/frameworks/kcoreaddons/-/merge_requests/216
Comment 58 Marcin Gurtowski 2022-04-30 11:05:43 UTC
And fix for not showing thumbnails in trash also: https://invent.kde.org/frameworks/kio/-/merge_requests/829
Comment 59 Pedro V 2024-01-01 14:13:19 UTC
Generally it seems like there's no one size fits all solution, but there's where KDE's configurability should help which is lacking here.

I believe that some improvements should make it possible to completely avoid saving thumbnails for at least local storage devices, but I found out that there isn't even just one global thumbnail generation strategy, so compared Gwenview and Dolphin just to find out that they are lacking in different ways.

Gwenview:
- Makes it possible to disable saving thumbnails, but "punishes" the user by using low quality previews in this case, and regenerates them when they get into the viewport
- Doesn't do lazy preview generation so eventually it has a preview for all files even without user interaction
- Generates/loads preview strictly in sorting order, so if the user doesn't adjust sorting to fit the viewing order, then even saved thumbnails aren't exactly helpful as they still take a long time to load

Dolphin:
- Apparently no option to adjust the thumbnail-saving behavior
- Does lazy preview generation so scrolling to a new area always involves waiting for previews to be generated / loaded
- Due to lazy preview generation there's at least no need to wait for a ton of other previews to be generated first

Both:
- Only a single CPU core is used for preview generation which is both slow and power inefficient
- User interaction isn't taken into account too well as previews should be generated/loaded before they are needed but without making the user wait for them unnecessarily

I won't butt into the slow remote storage topic where local thumbnails could be actually important for the people desiring fast previews in such cases, but as local storage is just getting faster and CPUs keep on getting more cores, I really believe that saving thumbnails really shouldn't be forced anymore as generating previews on-demand just keeps on getting more and more sensible (and viable).