Bug 468256 - inconsistent search results for newer files in Dolphin: OK from "My files", empty from specific folder
Summary: inconsistent search results for newer files in Dolphin: OK from "My files", e...
Status: REPORTED
Alias: None
Product: dolphin
Classification: Applications
Component: search (show other bugs)
Version: 22.04.3
Platform: Kubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: Dolphin Bug Assignee
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-04-07 16:53 UTC by brazzmonkey
Modified: 2025-01-23 11:38 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description brazzmonkey 2023-04-07 16:53:43 UTC
SUMMARY
Search features in Dolphin is inconsistent.
When new files are added into new folders, the do not appear in Dolphin search results from these newer folders ("from here").

STEPS TO REPRODUCE
1. in Dolphin, create new folders, subfolders
2. in Dolphin, put new files in these newly created folders (JPG files for instance)
3. in Dolphin, launch search box using 'Ctrl+F', search location "from here" for files named '*.jpg' 

OBSERVED RESULT
No files matching search criteria found.

EXPECTED RESULT
Files matching search criteria listed as results.

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Kubuntu 22.04, KDE Plasma 5.25.5
KDE Plasma Version: 5.25.5
KDE Frameworks Version: 5.98.0
Qt Version: 5.15.3

ADDITIONAL INFORMATION
- Baloo indexing seems to be working fine, as 'balooctl monitor' detects new files and indexes them.

- When "your files" is selected as a search location instead of "from here", the newest files do appear in search results as expected (but among the many other files from other folders complying to search criteria...).

- When the same search is done later (I'm talking several hours at least, maybe days), search results are consistent even using "from here" location.
Comment 1 tagwerk19 2024-07-07 08:19:41 UTC
(In reply to brazzmonkey from comment #0)
> ... Baloo indexing seems to be working fine, as 'balooctl monitor' detects new
> files and indexes them ...
So, the indexing is going on as it should (you are indexing content and 'balooctl monitor' is showing you the progress)

> ... When "your files" is selected as a search location instead of "from here",
> the newest files do appear in search results as expected (but among the many
> other files from other folders complying to search criteria...) ...
Sounds like:

    Doubling up of search results, Bug 475919, might catch them unawares
    (but that is something that is not "out of the ordinary" where your search
    results contains files with the same name)

    Confusing "empty" results when searching from a particular directory (as
    opposed to "Your Files"). That's the Bug 474973

There were issues with openSUSE and latterly Fedora. The systems were using BTRFS and there were patches to solve them:
    https://invent.kde.org/frameworks/baloo/-/merge_requests/131
for KF6 and here:
    https://invent.kde.org/frameworks/baloo/-/merge_requests/169
for KF5. These are now a while back 

What is your filesystem? I'd be particularly interested if it is BTRFS or a FUSE filesystem. If BTRFS it would also be interesting if you are doing snapshots (and Baloo might be indexing them).
Comment 2 brazzmonkey 2024-07-08 08:49:46 UTC
Yes baloo works and seems to index fine, even when newer files and folders are added.
I insist on the fact that we're talking newer files and folder being indexed by baloo, with dolphin not being able to provide results when a search is being done in these newer folders. It gives proper results after a while, though (hours or days later).

I don't know if this is related to those 2 bugs you mentioned, nor if this is  a baloo or dolphin issue.
I do have multiple search results when dolphin searches in "Your files" (I'm talking 12-13 identical search results here, not just 2).
But results are fine when I search "From Here" (no multiple results).

For filesystem I use ext4 only.
Comment 3 tagwerk19 2024-07-08 11:41:35 UTC
(In reply to brazzmonkey from comment #2)
> ... I do have multiple search results when dolphin searches in "Your files" (I'm
> talking 12-13 identical search results here, not just 2) 
> But results are fine when I search "From Here" (no multiple results) ...
The bits of the jigsaw really do not fit together here :-/

Maybe check and see if Baloo has multiple records for the file, 12 to 13 identical results is *a* *lot*. Try:

    $ baloosearch -i one-of-your-files.txt
    
and see if it gives you multiple results (the "-i" asks for Baloo's internal DocID).

A different possibility is that you have symlinks from folder to folder, I know I had doubled up results when I has a symlink from my Desktop to my Documents (so in effect I had a ~/Documents and a ~/Desktop/Documents folder). This wouldn't affect Baloo though, Baloo deliberately avoids symlinks.
Comment 4 brazzmonkey 2024-07-08 13:26:04 UTC
Yes I do get 12 times the same result when using baloo -i [some file].
I don't think of any symlink that would be responsible for this.

I'm no expert, but I don't think this is related to the original bug.
(Personally I'd rather get 12 times more search results than expected, than no search result at all)
Comment 5 brazzmonkey 2024-07-08 13:49:31 UTC
I tried something: I moved a directory, then search a file inside this directory using baloo -i.
The output is the previous location, not the new one.
Regarding your previous post, it looks like baloo's internal DocID isn't instantly updated.
Comment 6 tagwerk19 2024-07-08 14:41:22 UTC
(In reply to brazzmonkey from comment #4)
> Yes I do get 12 times the same result when using baloo -i [some file].
> I don't think of any symlink that would be responsible for this.
OK, we've pinned it down then...

Although we don't know why Baloo might have seen the file 12 times (it effectively thinks it has seen the file on 12 different discs. Up until now I'd only seen that with BTRFS).
    
There's no easy way of solving this apart from reindexing:

    $ systemctl --user stop kde-baloo
    $ balooctl purge
    $ systemctl --user start kde-baloo

and exercise patience. Baloo will however index a *lot* faster than you were seeing before
Comment 7 brazzmonkey 2024-07-08 14:53:08 UTC
I've already re-indexed in the past.
I will try again and let you know.
Comment 8 tagwerk19 2024-07-08 16:16:41 UTC
(In reply to brazzmonkey from comment #7)
> I've already re-indexed in the past.
> I will try again and let you know.
Find one of your files that you are indexing and note down what "stat" says about it:

    $ stat one-of-your-files.txt

you want to note the line that look like:

    Device: fd01h/64769d    Inode: 1316783     Links: 1

Try again after a reboot. These values ought to be stable from reboot to reboot, maybe they are not...

You said at the top of the thread:
    KDE Frameworks Version: 5.98.0
I think the fix that prevents "multiple entries" arrived in 5.111
Comment 9 brazzmonkey 2024-07-08 16:31:05 UTC
Indeed, re-indexing suppressed multiple search results.
But the original bug report still applies.
Comment 10 tagwerk19 2024-07-08 19:33:19 UTC
(In reply to brazzmonkey from comment #9)
> But the original bug report still applies.
Wierd ... I think I caught a glimpse of a problem but it's gone again.

Thank you for your persistance, I will see if I can replicate. It may take a while...
Comment 11 brazzmonkey 2024-07-11 17:15:40 UTC
FYI
since we've mentioned multiple search results (although not related yo the original bug report), after a few days I realised I get multiple search results again (3 at the moment). And I suppose the number increases over time.
Comment 12 tagwerk19 2024-07-11 17:26:33 UTC
(In reply to brazzmonkey from comment #11)
> FYI
> since we've mentioned multiple search results (although not related yo the
> original bug report), after a few days I realised I get multiple search
> results again (3 at the moment). And I suppose the number increases over
> time.
Can you check the results you get from "stat" (as per Comment 8)? 

If you have the option to update (so you run version 5.111 or later), that would be advisable....
Comment 13 brazzmonkey 2024-07-11 17:44:57 UTC
$ stat capt0000.jpg
  File: capt0000.jpg
  Size: 2583222         Blocks: 5048       IO Block: 4096   regular file
Device: 10307h/66311d   Inode: 72750303    Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1001/     myuser)   Gid: ( 1001/     myuser)
Access: 2024-07-10 09:31:12.309942491 +0200
Modify: 2021-12-14 19:55:48.000000000 +0100
Change: 2021-12-14 19:55:48.492820918 +0100
 Birth: 2021-12-14 19:55:48.492820918 +0100

This is my daily production workstation running kubuntu 22.04 LTS.
I tried backports in the past and encountered serious bugs affecting my workflow (especially with Kontact). So I won't mess again with it and will upgrade only when Kubuntu 24.04.1 will be released.
Anyhow I don't think 5.111 is available in Ubuntu prior 23.10, which I skip for not being LTS.

Once again, this side bug is no big deal for me. I can wait until I upgrade to 24.04.1 which features KDE framework 5.115.
I just hope this will solve the original bug too.
Comment 14 tagwerk19 2024-07-11 18:05:17 UTC
(In reply to brazzmonkey from comment #13)
> Device: 10307h/66311d   Inode: 72750303    Links: 1
I'd be interested  in whether this changes after a reboot - and whether the Device number of the inode changes. The fix aimed at BTRFS meant that Baloo based its calculations on the "FileSystem ID" (which should be invariant) rather than the mounted device number.

If  it is the device number of changing with ext4, then something unusual is happening when you mount your disks...

> ... Once again, this side bug is no big deal for me. I can wait until I upgrade
> to 24.04.1 which features KDE framework 5.115.
> I just hope this will solve the original bug too ...
I hope so too ...
Comment 15 brazzmonkey 2024-07-11 19:44:37 UTC
yes it does, for Device only:
$ stat capt0000.jpg
  File: capt0000.jpg
  Size: 2583222         Blocks: 5048       IO Block: 4096   regular file
Device: 1030ch/66316d   Inode: 72750303    Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1001/     myuser)   Gid: ( 1001/     myuser)
Access: 2024-07-10 09:31:12.309942491 +0200
Modify: 2021-12-14 19:55:48.000000000 +0100
Change: 2021-12-14 19:55:48.492820918 +0100
 Birth: 2021-12-14 19:55:48.492820918 +0100
Comment 16 tagwerk19 2024-07-11 21:38:09 UTC
(In reply to brazzmonkey from comment #15)
> Device: 1030ch/66316d   Inode: 72750303    Links: 1
So Device Number has changed, inode stayed the same... 

That explains the gradually increasing number of results in the search. Worth watching to see if the pattern continues...

So the question is why. Do you do anything out of the ordinary with the disk mounts? Are they mounted through /etc/fstab or somehow else?
Comment 17 brazzmonkey 2024-07-11 21:44:14 UTC
(In reply to tagwerk19 from comment #16)
> (In reply to brazzmonkey from comment #15)
> > Device: 1030ch/66316d   Inode: 72750303    Links: 1
> So Device Number has changed, inode stayed the same... 
> 
> That explains the gradually increasing number of results in the search.
> Worth watching to see if the pattern continues...
> 
> So the question is why. Do you do anything out of the ordinary with the disk
> mounts? Are they mounted through /etc/fstab or somehow else?

My /home is on a SSD partition, mounted through fstab using UUID.
Comment 18 tagwerk19 2024-07-11 21:56:11 UTC
(In reply to brazzmonkey from comment #17)
> My /home is on a SSD partition, mounted through fstab using UUID.
That ought to be plain and simple, nothing like systemd-homed?
Comment 19 brazzmonkey 2024-07-11 22:31:34 UTC
(In reply to tagwerk19 from comment #18)
> That ought to be plain and simple, nothing like systemd-homed?
Never heard of this service.
I have:
$ sudo systemctl | grep home
  home.mount                                                                     loaded active mounted   /home

I have no systemd-homed.service to be found.
I don't know if this helps. That's as far as I can go (I basically hate systemd, but it's hard to avoid it these days).
Comment 20 tagwerk19 2024-07-12 05:37:37 UTC
(In reply to brazzmonkey from comment #19)
> I have:
> $ sudo systemctl | grep home
>   home.mount                                                                 loaded active mounted   /home
I have the same on a Tumbleweed box.

That seems to have a home.mount; BTRFS though but ext4 ought to be simpler. Baloo seems stable on it.
Comment 21 brazzmonkey 2025-01-13 15:19:59 UTC
I get a similar behaviour with Kubuntu 24.04 (KDE 23.08.5).