Bug 417170

Summary: Option in Baloo to index file metadata (tags, ratings, comments, etc) without enabling full text indexing
Product: [Frameworks and Libraries] frameworks-baloo Reporter: tagwerk19
Component: generalAssignee: Stefan Brüns <stefan.bruens>
Status: RESOLVED FIXED    
Severity: wishlist CC: bugseforuns, metal450, nate, tagwerk19
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description tagwerk19 2020-02-04 23:33:25 UTC
SUMMARY:

    Baloo should have an option to index file metadata (such as file name, tags, ratings and comments).

    This would allow Dolphin to support tag folders if Baloo is configured not to perform free text indexing. At the moment Baloo has the option:

        [General]
        only basic indexing=true 

    that indexes to only filenames (confirmation on this would be nice). If "Basic indexing" is selected tag folders in Dolphin don't work. There seems to be many problems reported with free text indexing, an index of metadata would allow a minimum solution.

SOFTWARE/OS VERSIONS:

    Dolphin 19.12.2
    Baloo 5.67.0
    from Neon Testing

    KDE Plasma 5.17.90
    KDE Frameworks 5.67.0
    Qt 5.13.2

ADDITIONAL INFORMATION:

    There's an old Bug 337026 that refers to Dolphin displaying the metadata rather than searching for it.

    Dolphin displays tags, rating and comments when showing the properties of a file but not in columns added to a panel. Maybe this is a separate bug.
Comment 1 Nate Graham 2020-02-05 21:36:14 UTC
So you would like to have Baloo insed filenames and metadata but not full text content? May I ask why? What's the use case for this? Are you have problems with full-text indexing?
Comment 2 tagwerk19 2020-02-05 22:54:25 UTC
I noticed that if baloo indexing was disabled Dolphin stopped displaying metadata in the (filesystem) folders; if you have added columns for "tags", "ratings", they'd be unpopulated. It also stopped offering tag folders.

Google shows quite a few threads on baloo and CPU load, including advice how to disable it.

Personal experience is I've have one instance of a baloo index corrupting (no problem once you realise it, purge and rebuild) but I've also noticed a fair load on the system when renaming folders under Pictures, to the extent that logging out and back in again is a worthy workround.

If you are indexing images (with tags, ratings and comments), I feel you only really need the metadata.
Comment 3 Nate Graham 2020-02-05 23:00:59 UTC
Fair enough.
Comment 4 Stefan Brüns 2020-02-05 23:10:26 UTC
This is definitely on my Todo list.

Even just keeping the file names - baloo not finding files is a recurrent complaint.
Comment 5 Stefan Brüns 2020-03-24 20:39:51 UTC
Git commit 7498b41a1979b084d05b99a51d3561eeef68e974 by Stefan Brüns.
Committed on 24/03/2020 at 20:39.
Pushed by bruns into branch 'master'.

[Baloo KCM] Complete overhaul of the include/exclude folder list

Summary:
The current "Excluded folders" list in the KCM is quite awkward:

1. It tries to mimic baloos automatic exclusion of external drives,
   but fails doing so and adds almost any external drive even when not
   mounted below an indexable path.

2. Deleting an autogenerated entry actually **adds** it to the included
   folder list, and then hides it.

3. There is no way to show the included folder list, or add any entries
   to it.

Remove the custom "excluded mounts" heuristic from the KCM and retrieve
the additional (not explicitly configured) excluded ones from baloo.

Replace the "excluded list" with a common list for included and excluded
folders, and flag its state. This also makes it easy to add additional
properties later.

Create a new UI delegate for each config list item, allowing to enable
and disable indexing for each entry. Make the "delete" actually always
delete a config entry, and make the control inline. Move the "Add" button
to the *right* bottom of the list (in accordance with UI guidelines) and
add some text to it.

Depends on D28024

Solves the following parts of T9879:
[x] Allow adding of includeFolders in the KCM
[x] Correctly show excludeFolders below explicit (non-$HOME) includeFolders in KCM
Related: bug 417763, bug 417762, bug 366521

Reviewers: #baloo, #vdg, ngraham, mart

Reviewed By: #baloo, #vdg, ngraham

Subscribers: plasma-devel

Tags: #plasma

Differential Revision: https://phabricator.kde.org/D28025

M  +119  -113  kcms/baloo/filteredfoldermodel.cpp
M  +18   -12   kcms/baloo/filteredfoldermodel.h
M  +76   -19   kcms/baloo/package/contents/ui/main.qml

https://commits.kde.org/plasma-desktop/7498b41a1979b084d05b99a51d3561eeef68e974
Comment 6 spiesant 2020-05-27 08:01:18 UTC
I found my way to this bug while trying to figure out why all the metadata columns in Dolphin were blank.  I (deliberately) have my search index disabled, but even if it were enabled, that wouldn't solve the issue as viewing media on external hard drives or networked drives - which shouldn't be part of the index either way - would never work.

Should that be a separate issue from this one?

As mentioned in the OP, Dolphin is definitely able to extract the metadata without Baloo (as can be seen via the file properties dialog), so it seems like a bug that it would leave all the columns blank. I'm not clear, though, if this issue is only asking for metadata to be indexed, or for the columns to be fixed regardless of the index, as it seems to sort of address both.
Comment 7 Patrick Silva 2020-05-27 08:14:07 UTC
displaying metadata in Dolphin regardless baloo indexing was already requested via bug 406506
Comment 8 spiesant 2020-05-27 08:15:48 UTC
Perfect, thanks for the quick reply :)
Comment 9 tagwerk19 2020-05-27 08:28:38 UTC
(In reply to Metal450 from comment #6)

> ... I'm not clear, though, if this issue is only asking for metadata to be indexed ...

The original bug was specifically about baloo indexing and whether there could be an option that indexed "tags" even if the full text indexing was off.

The wished for results being that you could do a "baloosearch tags:mytag" and that dolphin would show you a "tags:/mytag" folder.

Yes, there are parallel issues of dolphin not populating columns with xattr info in different situations. I've met it in Bug 418590 (where it applies to dolphin listing a "tag" folder)
Comment 10 spiesant 2020-05-27 08:48:18 UTC
Gotcha. #406506 is definitely what I'm after - having the columns function without the need for Baloo.  Bummer it hasn't gotten any love in over a year, but at least I've got something to keep my eye on :)
Comment 11 tagwerk19 2021-01-18 09:23:25 UTC
Somewhere along the line ...

... this has been fixed for indexing tags. Thank you!

    balooctl purge
    Stopping the File Indexer .... - done
    Deleted the index database
    Restarting the File Indexer

    balooctl config list contentindexing
    no

    getfattr -d Documents/testfile.txt
    # file: Documents/testfile.txt
    user.xdg.tags="mytag"

    baloosearch tags:mytag
    /home/bug417170/Documents/testfile.txt
    Elapsed: 1,55229 msecs

I'll flag as Resolved Fixed.
Comment 12 tagwerk19 2021-01-18 16:25:20 UTC
For completeness ...

... still without needing contentindexing:

    getfattr -d Documents/testfile.txt
    # file: Documents/testfile.txt
    user.xdg.comment="this is myworld"

    baloosearch userComment:myworld
    /home/bug417170/Documents/testfile.txt
    Elapsed: 1,44166 msecs

and, having given the file 3 stars:

    getfattr -d Documents/testfile.txt
    # file: Documents/testfile.txt
    user.baloo.rating="6"

    baloosearch 'rating>4'
    /home/bug417170/Documents/testfile.txt
    Elapsed: 1,69197 msecs

The "What to search for" keywords taken from:

    https://api.kde.org/frameworks/baloo/html/searching.html