Bug 486549 - filelight: doesn't correctly process OneDrive cloud files
Summary: filelight: doesn't correctly process OneDrive cloud files
Status: RESOLVED FIXED
Alias: None
Product: filelight
Classification: Applications
Component: general (show other bugs)
Version: 23.04.0
Platform: Other Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: Unassigned bugs mailing-list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-05-04 08:58 UTC by fuyutsuki98_r
Modified: 2024-05-23 22:44 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In: 24.05.1


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description fuyutsuki98_r 2024-05-04 08:58:56 UTC
STEPS TO REPRODUCE
1. Open Filelight
2. Try to scan OneDrive directory.

OBSERVED RESULT
Even though some listed files does not occupy space on disk (they are stored on the cloud), Filelight still includes their size in the graph.

EXPECTED RESULT
Filelight ignores the files that are stored only on the cloud but not on local machine.

SOFTWARE/OS VERSIONS
Windows: 11 22631.3527
(available in About System)
KDE Frameworks Version: 5.105.0
Qt Version: 5.15.9

ADDITIONAL INFORMATION
The latest version on Microsoft Store is 23.04.0. Have also tried out the latest CI build version (24.07.70), same behavior.
Comment 1 Harald Sitter 2024-05-07 11:01:50 UTC
It's curious that both the store version and the CI version have this problem. bug #473024 seems very related to the problem at hand, but should be fixed in the CI build.

Specifically https://learn.microsoft.com/en-us/windows-hardware/drivers/ifs/placeholders says placeholders are FILE_ATTRIBUTE_SPARSE_FILE which we explicitly handle differently as per the bug I've mentioned, in CI builds. And we do so using a function that is documented as "Retrieves the actual number of bytes of disk storage used to store a specified file." which you would expect should return 1KB as per https://learn.microsoft.com/en-us/windows/win32/cfapi/build-a-cloud-file-sync-engine#placeholder-files ("Sync engines can create placeholder files that consume only 1 KB of storage for the filesystem header, and that automatically hydrate into full files under normal use conditions.").

Needs some inspection what exactly goes on there with onedrive placeholders. I'd like to avoid having to special case onedrive.
Comment 2 Harald Sitter 2024-05-07 12:08:43 UTC
As it turns out processes aren't by default privy to see onedrive files as spares files. That has the advantage that most apps simply don't need to care about what's what. One would need to set RtlSetProcessPlaceholderCompatibilityMode to change that and I am not sure is a good idea because it is NT API and could break things elsewhere in the stack (e.g. in Qt).

Unhydrated files seem to have the FILE_ATTRIBUTE_UNPINNED attribute set though, so we could just hold on to that in absence of other information. Supposedly an unpinned file would only ever be ephemerally present, if at all.
Comment 3 Bug Janitor Service 2024-05-22 20:03:15 UTC
A possibly relevant merge request was started @ https://invent.kde.org/utilities/filelight/-/merge_requests/124
Comment 4 Harald Sitter 2024-05-23 13:11:45 UTC
Git commit 06990b8205d8ad3b270a4fed3c130d837a1ecac7 by Harald Sitter.
Committed on 23/05/2024 at 13:09.
Pushed by sitter into branch 'master'.

windows: treat unpinned files as compressed

an unpinned file is not present on disk and so we don't actually care
about its theoretical size, but rather its actual on-disk size.

this in particular covers onedrive files that haven't been pulled onto
disk

M  +1    -1    src/windowsWalker.cpp

https://invent.kde.org/utilities/filelight/-/commit/06990b8205d8ad3b270a4fed3c130d837a1ecac7
Comment 5 Harald Sitter 2024-05-23 13:14:22 UTC
Git commit 5d345574db3f5a142ef00fcfd575b93ce89353d3 by Harald Sitter.
Committed on 23/05/2024 at 13:12.
Pushed by sitter into branch 'release/24.05'.

windows: treat unpinned files as compressed

an unpinned file is not present on disk and so we don't actually care
about its theoretical size, but rather its actual on-disk size.

this in particular covers onedrive files that haven't been pulled onto
disk


(cherry picked from commit 06990b8205d8ad3b270a4fed3c130d837a1ecac7)

M  +1    -1    src/windowsWalker.cpp

https://invent.kde.org/utilities/filelight/-/commit/5d345574db3f5a142ef00fcfd575b93ce89353d3