Bug 406506

Probably it should just index NTFS disks...

Or index temporarily (without history) all the files loaded in dolphin when it is not in an indexed path.
Or short-circuit baloo and extract metadata the same sort of way KPreviewJob does.

Volatile storage/temporary storage is a common use case, we could offer metadata there as well.

Ntfs file system of internal hard disks is just an example.
Currently Dolphin also does not show metadata of files stored
in removable devices formatted with any file system.

Created attachment 119527 [details]
both info panel and  "Details" tab of "Properties" window show the metadata

As we can see in my screenshot taken on neon dev unstable, both info panel and "Details" tab of "Properties" window show the metadata even when the file is not indexed.

why was this bug closed?

I did because I understood your last comment suggested the issue at hand does not exist.
And I checked myself.
Do you think it should stay open ?

This report is about Dolphin. As we can see in the screenshot from comment 4, Dolphin is still not showing the metadata.
I attached the screenshot for comparison purposes.

I didn't get that was about the detail view column in the first place.
Thank you for correcting me.

(In reply to Méven Car from comment #8)
> I didn't get that was about the detail view column in the first place.
> Thank you for correcting me.

Me neither ...

@Patrick - next time, please be more specific.

I did some investigation to fix this.

The limitation is currently in kbaloorolesprovider 
In QHash<QByteArray, QVariant> KBalooRolesProvider::roleValues(const Baloo::File& file,
[...]
// The baloo roles provider calls :
    QMapIterator<KFileMetaData::Property::Property, QVariant> it(file.properties());

But has no other patch when baloo does not respond anything.
Contrary to the InformationPanel that uses baloo-widgets FileMetaDataWidget widget that uses FileMetaDataProvider that uses IndexedDataRetriever when baloo has not find metadata.
However IndexedDataRetriever calls directly the executable baloo_filemetadata_temp_extractor in a process to retrieve metadata for a file, and this is not a great idea to reuse the IndexedDataRetriever in a loop as the detail view needs, since for each file a process would need to get spawned, creating some overhead.
And the dolphin view is filled synchronously in KFileItemModelRolesUpdater, making the UI freeze when the metadata are retrieved.

So the solution would be either reuse IndexedDataRetriever for the detail view and pay the price, or implement an API in Baloo to call do what baloo_filemetadata_temp_extractor does asynchronously and make dolphin able to load metadata asynchronously.

(In reply to Nate Graham from comment #1)
> Probably it should just index NTFS disks...

I'm facing the same issue as the OP, & am definitely not after a solution of indexing NTFS disks. Being able to view media columns on removable USB drives, SD cards, network drives, etc is also important - & those shouldn't be indexed.  In my case, I actually have indexing disabled entirely, as I don't use search & thus don't need the overhead.  But losing those columns when moving over from Nemo (which supports them via extension) definitely felt like a bit of a regression.

(In reply to Méven Car from comment #10)
> I did some investigation to fix this.
> 
> The limitation is currently in kbaloorolesprovider 
> In QHash<QByteArray, QVariant> KBalooRolesProvider::roleValues(const
> Baloo::File& file,
> [...]
> // The baloo roles provider calls :
>     QMapIterator<KFileMetaData::Property::Property, QVariant>
> it(file.properties());
> 
> But has no other patch when baloo does not respond anything.
> Contrary to the InformationPanel that uses baloo-widgets FileMetaDataWidget
> widget that uses FileMetaDataProvider that uses IndexedDataRetriever when
> baloo has not find metadata.
> However IndexedDataRetriever calls directly the executable
> baloo_filemetadata_temp_extractor in a process to retrieve metadata for a
> file, and this is not a great idea to reuse the IndexedDataRetriever in a
> loop as the detail view needs, since for each file a process would need to
> get spawned, creating some overhead.
> And the dolphin view is filled synchronously in KFileItemModelRolesUpdater,
> making the UI freeze when the metadata are retrieved.
> 
> So the solution would be either reuse IndexedDataRetriever for the detail
> view and pay the price, or implement an API in Baloo to call do what
> baloo_filemetadata_temp_extractor does asynchronously and make dolphin able
> to load metadata asynchronously.

`baloo_filemetadata_temp_extractor` is in `extractor.cpp` whose code could be turned into a KJob, that dolphin could use.

*** Bug 426154 has been marked as a duplicate of this bug. ***

*** Bug 429740 has been marked as a duplicate of this bug. ***

A possibly relevant merge request was started @ https://invent.kde.org/system/dolphin/-/merge_requests/157

(In reply to Metal450 from comment #11)

> ... those shouldn't be indexed.

You could possibly imagine a future where an index is built and held on the remote/removeable filesystem and is queried when that filesystem is online? Remote/removeable systems have their own wastebaskets, could they not have their own indexes?

Not seriously suggesting that this is on anyone's workplan. But, wouldn't that future be nice?

(In reply to tagwerk19 from comment #16)
> (In reply to Metal450 from comment #11)
> 
> > ... those shouldn't be indexed.
> 
> You could possibly imagine a future where an index is built and held on the
> remote/removeable filesystem and is queried when that filesystem is online?
> Remote/removeable systems have their own wastebaskets, could they not have
> their own indexes?
> 
> Not seriously suggesting that this is on anyone's workplan. But, wouldn't
> that future be nice?

That wouldn't be my preference.  Far better if metadata just worked without needing to store indexes everywhere (as it does on Windows).  It does take a second or two for large directories to populate all the meta fields, but I'd much rather wait ~2 seconds than have indexes all over the place.

(In reply to Patrick Silva from comment #0)
> It would be very nice if Dolphin could show metadata of non-indexed files
> too.
Having looked at Bug 434565

If baloo is enabled, as per 
    Indexing-Enabled=true
in baloofilerc, then dolphin displays the tags, ratings, comment (xattr metadata) read from the filesystem

With
    Indexing-Enabled=false
dolphin doesn't display the attributes.

    Neon Testing
    Dolphin : 20.12.3
    Plasma : 5.21.3
    Frameworks : 5.81.0

(In reply to tagwerk19 from comment #18)
> (In reply to Patrick Silva from comment #0)
> > It would be very nice if Dolphin could show metadata of non-indexed files
> > too.
> Having looked at Bug 434565
> 
> If baloo is enabled, as per 
>     Indexing-Enabled=true
> in baloofilerc, then dolphin displays the tags, ratings, comment (xattr
> metadata) read from the filesystem
> 
> With
>     Indexing-Enabled=false
> dolphin doesn't display the attributes.
> 

Indeed it depends on baloo metadata indexing currently.

But it does not support path not indexed by baloo such as external drives.
Also we can implement this without baloo as a fallback.

My patch suite https://invent.kde.org/system/dolphin/-/merge_requests/157 stuck in review limbo does that.

*** Bug 445067 has been marked as a duplicate of this bug. ***

(In reply to Nate Graham from comment #1)
> Probably it should just index NTFS disks...
As a marker...

Tested a Fedora 35 system with the new Paragon ntfs3 driver. I created an NTFS drive with KDE Partition manager and attached as "/home2" (that is, listed in /etc/fstab with some "magic dust" for selinux).

I was able to create a user on that filesystem, copy files across, see that they were being picked up and indexed (with balooctl monitor), tag them, add a "tags" column in Dolphin and jump back and forth between the "tag folders". I think that's not so bad...

This is very much a "Toe In The Water" test but there's no immediate evidence of calamity (device numbers and inodes _do_ seem stable and the baloo index is living on the same drive). However I don't know enough to say whether ntfs3 is stable and when you could trust it with real data.

The Magic Dust I used was:

    UUID=XXXXXXXXXXXXXXXX   /home2   ntfs3   rw,relatime,context="system_u:object_r:home_root_t:s0" 0 0

A possibly relevant merge request was started @ https://invent.kde.org/network/kio-extras/-/merge_requests/141

*** Bug 481420 has been marked as a duplicate of this bug. ***

*** Bug 495592 has been marked as a duplicate of this bug. ***