Bug 399885 - Support indexing NTFS disks
Summary: Support indexing NTFS disks
Status: CONFIRMED
Alias: None
Product: frameworks-baloo
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: 5.51.0
Platform: Neon Linux
: HI wishlist
Target Milestone: ---
Assignee: baloo-bugs-null
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-16 16:26 UTC by S. Christian Collins
Modified: 2023-09-12 09:18 UTC (History)
9 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description S. Christian Collins 2018-10-16 16:26:21 UTC
SUMMARY
I have two NTFS partitions mounted in my home folder via the following lines in /etc/fstab:
UUID=10156E860DB794A2 /home/chris/Personal ntfs-3g defaults,auto,noatime,big_writes,uid=1000,gid=1000,umask=022,windows_names,locale=en_US.utf8 0 0
UUID=0F14CDF22D70C77E /home/chris/Storage ntfs-3g defaults,auto,noatime,big_writes,uid=1000,gid=1000,umask=022,windows_names,locale=en_US.utf8 0 0

Both of these locations appear in "System Settings" -> "Search" -> "File Search" -> "Do not search in these locations". If I remove the folders from the list then close and restart System Settings, they are back in the list.

~/.config/baloofilerc contains no reference to either folder (there is no "exclude folders[$e]=" line) and ~/.config/baloorc doesn't exist.

STEPS TO REPRODUCE
1. I assume that meeting conditions similar to the above description will cause the issue for others as well, but I don't have a similar setup on any other PC to test this with.

OBSERVED RESULT
NTFS mount points appear in excludeFolders and can't be removed.

EXPECTED RESULT
NTFS folders are indexed.

SOFTWARE VERSIONS
KDE Plasma Version: 5.14.0
KDE Frameworks Version: 5.51.0
Qt Version: 5.11.1

ADDITIONAL INFORMATION
If I try to remove one of the folders using "balooctl config remove excludeFolders '/home/chris/Personal'", two things happen:
1. "baloorc" is created containing "exclude folders[$e]=$HOME/Storage" under "[General]"
2. "baloofilerc" is updated adding "exclude folders[$e]=$HOME/Storage" under "[General]"

Running "balooctl config list excludeFolders" now shows:
/home/chris/Personal
/home/chris/Storage
/home/chris/Storage

If I run "balooctl config remove excludeFolders '/home/chris/Personal'" a second time, both "baloorc" and "baloofilerc" now show "exclude folders[$e]=$HOME/Storage,$HOME/Storage"

Running "balooctl config list excludeFolders" now shows:
/home/chris/Personal
/home/chris/Storage
/home/chris/Storage
/home/chris/Storage

If I now run "balooctl config remove excludeFolders '/home/chris/Storage'", both "baloorc" and "baloofilerc" now show "exclude folders[$e]=$HOME/Personal"
Comment 1 S. Christian Collins 2018-10-16 16:30:39 UTC
I filed this report at the request of Nate Graham on bug #375370. Linking here for reference.
Comment 2 Stefan Brüns 2018-10-19 01:29:58 UTC
Do you have by any chance a
/home/chris/.kde4/share/config/baloofilerc
file? If yes, can you provide the contents?
Comment 3 S. Christian Collins 2018-10-19 05:21:10 UTC
(In reply to Stefan Brüns from comment #2)
> Do you have by any chance a
> /home/chris/.kde4/share/config/baloofilerc
> file? If yes, can you provide the contents?

I don't have a ~/.kde4 folder. I do have a ~/.kde folder, but "baloofilerc" doesn't exist anywhere in that path. I do have a copy of "baloofilerc" in /home/chris/.config/. Here are its contents:
--------------------------
[Basic Settings]
Indexing-Enabled=true

[General]
dbVersion=2
exclude filters=.moc,*.gb,*.po,*.tmp,*.pyc,*.rcore,*.fasta,*.qmlc,*.swap,lost+found,.xsession-errors*,*.init,*.fna,.npm,*.fq,ui_*.h,Makefile.am,*.pc,.obj,*.part,*.m4,.bzr,core-dumps,CVS,_darcs,po,*~,*.class,config.status,*.csproj,confstat,CMakeTmp,*.gmo,.yarn,.uic,*.faa,moc_*.cpp,.yarn-cache,*.a,*.pyo,*.so,.histfile.*,CMakeCache.txt,*.rej,nbproject,conftest,CTestTestfile.cmake,*.map,*.la,.git,*.omf,*.db,node_modules,*.qrc,autom4te,*.ini,node_packages,*.loT,*.fastq,*.o,.svn,*.moc,.pch,*.jsc,litmain.sh,qrc_*.cpp,libtool,*.vm*,CMakeTmpQmake,.hg,*.lo,lzo,CMakeFiles,__pycache__,*.orig,*.nvram,cmake_install.cmake,*.elc,*.aux,*.gbff,confdefs.h
exclude filters version=4
first run=false
folders[$e]=$HOME/
only basic indexing=false
Comment 4 Kai Uwe Broulik 2018-10-19 06:23:39 UTC
Judging from the code this is by design: all mounts other than /home are ignored. This was mostly done so it doesn't start indexing a USB thumb drive that may only be around for so long. Perhaps a "isRemovable" check needs to be added.
Comment 5 S. Christian Collins 2019-01-03 17:48:48 UTC
I've solved the issue by disabling hot plugging for SATA devices in my motherboard BIOS. My NTFS mounts no longer appear in the "do not search" list, and Baloo is now busy indexing them.
Comment 6 Stefan Brüns 2019-06-11 16:35:27 UTC
More importantly, external drives do not provide a stable ID. This is 2-part:

1. The device-id changes (may change) when a device is unmounted and later mounted again. This is mostly an issue for external drives, e.g. USB. The kernel uses the bus number to create the device id, so this typically works for SATA drives (unless you change the port or add an additional controller).

2. The inode may change. For ext4, btrfs this is not an issue, but for network mounts it is (the inode is stable as long as a drive stays mounted, but may be different on next mount). NTFS optionally provides unique IDs for file objects, but I am not sure how ntfs-3g creates the inode numbers.

For (1.), this can only be solved by having separate "namespaces" for each drive/filesystem, and using some different method to identify the drive (LABEL, UUID, ...). For the home partition, "same device as where the DB is stored" suffices.

This is significantly more work than just changing some config parameters.
Comment 7 skierpage 2019-08-07 02:53:04 UTC
FWIW I have never seen NTFS mount points appear in .baloofilerc in "exclude folders" or "Do not search in these locations". But I don't add them under my home folder.

To access and index my Windows files on a dual-boot machine, I:
1. Mount my Windows 10 NTFS partition. My ancient /etc/fstab line is
   UUID=F402E20B02E1D2A2   /media/Windows  ntfs-3g defaults,windows_names,locale=en_US.utf8        0 0
2. Make symlinks in my home folder for subdirectories I want to point to NTFS, e.g. $HOME/Pictures to /media/Windows/Users/spage/Pictures , $HOME/Documents/Win10 to /media/Windows/Users/spage/My\ Documents/ , etc.

3. Baloo does not follow these symlinks when indexing my home directory, so I explicitly tell Baloo to index specific paths on the NTFS drive by changing the line under [General] in $HOME/.config/baloofilerc to:
  folders[$e]=$HOME/,/media/Windows/Users/spage/Desktop/,/media/Windows/Users/spage/Documents/,/media/Windows/Users/spage/Downloads/, etc. etc.

that has worked fairly well for me and I mention the technique in https://community.kde.org/Baloo/Configuration#Include_Folders . If there are issues and other workarounds with NTFS disks and UUIDs please document them on that page (or tell me and I will try).