Bug 392793 - Renamed files are not updated in the index
Summary: Renamed files are not updated in the index
Status: RESOLVED DUPLICATE of bug 433116
Alias: None
Product: frameworks-baloo
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: baloo-bugs-null
URL:
Keywords:
: 453278 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-04-06 09:33 UTC by pmargeti34
Modified: 2023-07-10 16:13 UTC (History)
13 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
A fix? (2 bytes, patch)
2021-03-16 22:14 UTC, Ignacio Serantes
Details
Log of adding the test repo.... (12.21 KB, text/x-log)
2021-03-26 11:34 UTC, tagwerk19
Details
A fix for directories? (1.14 KB, patch)
2021-03-31 08:11 UTC, Ignacio Serantes
Details

Note You need to log in before you can comment on or make changes to this bug.
Description pmargeti34 2018-04-06 09:33:17 UTC
This is an updated https://bugs.kde.org/show_bug.cgi?id=312888

Steps to reproduce:
- open dolphin
- create a file "A"
- activate find in dolphin and search for "A"

    case 1:
    - rename "test" to "B"
    - find view now displays result "B" which doesn't match find criteria
    expected result: A is removed from the find results list

    case 2:
    - select "A" and delete it
    - find view still displays result "A" which no longer exists
    expected result: A is removed from the find results list

Summary:
Find view should be updated on filesystem changes such as delete and rename. It is possible I'm forgetting to include other file operations here which lead to same innacurate find result, but rename and delete are most common uses of find for me. Keep an open mind about other possible use cases which haven't been included in this bug report.

Reproducible on:
Any version of Dolphin and BalooWidgets
Comment 1 pmargeti34 2018-04-06 09:37:54 UTC
Whoopsie. Made a mistake:
 case 1:
    - rename "test" to "B"
should read
 case 1:
    - rename "A" to "B"
Comment 2 Nate Graham 2020-11-13 16:16:32 UTC
Let's track the deletion case with Bug 429006 and use this for the rename case.
Comment 3 tagwerk19 2020-11-21 17:07:17 UTC
Recognise this from Fedora 32, indexing is working but then 'stops' at some point. A "balooctl check" triggers a reindex and there is a scurry to catch up with the files that need reindexing.

I can reproduce when renaming a parent folder

In Neon testing (Plasma 5.20.3, Frameworks 5.77, Qt 5.15.1, Filesystem Ext4, New install)

    balooctl purge

    cd Documents
    mkdir folder1
    echo "Test File" > folder1/filea.txt 

    baloosearch Test 

gives what is should:

    /home/bug392793/Documents/folder1/filea.txt

If you rename the folder:

    mv folder1 folder2 

this pops up an error (nicely visible if you have previously done a "balooctl purge")

    replace called with invalid arguments, docId: 7330929353751553 url: "/home/bug392793/Documents/folder2/"

and

    baloosearch Test

shows the "old results" (*):

    /home/bug392793/Documents/folder1/filea.txt

It has not seen the change. If you nudge baloo check:

    balooctl check

You get:

    replace called with invalid arguments, docId: 5647955008748545 url: "/home/bug392793/"
    7330929353751553 "/home/bug392793/Documents/folder2" renaming "folder1" to "folder2"
    Started search for unindexed files

and it updates/corrects the index. A search:

    baloosearch Test

then gives the correct results:

    /home/bug392793/Documents/folder2/filea.txt

It is noticeable that if you have renamed a folder (as per the "*" above), you can create/write/change all sorts files in the newly renamed folder without these being indexed

Installed inotify-tools and ran inotifywait in a separate shell:

    inotifywait -mr Documents/

iNotify saw the folder rename:

    Documents/ MOVED_FROM,ISDIR folder1
    Documents/ MOVED_TO,ISDIR folder2
    Documents/folder2/ MOVE_SELF
Comment 4 Ignacio Serantes 2021-03-15 22:29:35 UTC
Hi,

Maybe I'm wrong but I'm was revising the code and seems the problem is git commit bd1041ea "Make renames a DB only operation".

I apply add next few lines to MetadataMover::updateMetadata() method in "metadatamover.cpp" file and renaming works again for me.

     Document doc;
     doc.setId(id);
     doc.setParentId(parentId);
     doc.setUrl(toPath);
+    QFileInfo fileInfo(to);
+    doc.addFileNameTerm("F" + fileInfo.baseName().toUtf8());
+    doc.addFileNameTerm("F" + fileInfo.suffix().toUtf8());
     tr->replaceDocument(doc, DocumentUrl | FileNameTerms);

Please consider this code a simple proof of concept and not a repair attempt because I'm not a C++ developer.
Comment 5 tagwerk19 2021-03-16 16:47:56 UTC
I'm even less of a developer but if someone picks up the suggestion and something gets into Neon Unstable, I will think of tests to try...

This post is mainly to say "Thanks!" :-)
Comment 6 Ignacio Serantes 2021-03-16 22:14:12 UTC
Created attachment 136755 [details]
A fix?

This is a proper fix I guessed surfing in Baloo code. Don't know it is the right solution but works for me and maybe for others.
Comment 7 Ignacio Serantes 2021-03-16 22:15:39 UTC
Comment on attachment 136755 [details]
A fix?

diff --git a/src/file/metadatamover.cpp b/src/file/metadatamover.cpp
index abb22dd1..ccd4e4e9 100644
--- a/src/file/metadatamover.cpp
+++ b/src/file/metadatamover.cpp
@@ -10,6 +10,7 @@
 #include "database.h"
 #include "transaction.h"
 #include "baloodebug.h"
+#include "termgenerator.h"
 
 #include <QFile>
 
@@ -102,6 +103,9 @@ void MetadataMover::updateMetadata(Transaction* tr, const QString& from, const Q
     doc.setId(id);
     doc.setParentId(parentId);
     doc.setUrl(toPath);
+    const QByteArray fileName = toPath.mid(lastSlash + 1);
+    TermGenerator tg(doc);
+    tg.indexFileNameText(fileName);
     tr->replaceDocument(doc, DocumentUrl | FileNameTerms);
 
     // Possible scenarios
Comment 8 Nate Graham 2021-03-18 20:46:51 UTC
*** Bug 433116 has been marked as a duplicate of this bug. ***
Comment 9 Ignacio Serantes 2021-03-23 22:20:00 UTC
If someone is interested I created patched packages for openSUSE using 5.80.0 version.

Packages are available at https://download.opensuse.org/repositories/home:/serantes:/branches:/openSUSE:/Factory/openSUSE_Factory/

I intend to maintain these packages until the problem if fixed.
Comment 10 tagwerk19 2021-03-26 11:34:43 UTC
Created attachment 137080 [details]
Log of adding the test repo....

(In reply to Ignacio Serantes from comment #9)
> Packages are available at
> https://download.opensuse.org/repositories/home:/serantes:/branches:/
> openSUSE:/Factory/openSUSE_Factory/
Installed a new tumbleweed and added your repo. Log attached so you can see if I've missed anything (no guarantee I've got it right :-) Rebooted after the update.

If I test baloo 'noticing' that a file's been moved, I'm not seeing it. After a 'balooctl check', it works.

echo "Hello Penguin" > file1.txt

balooshow -x file1.txt

    d1600000036 54 3350 file1.txt [/home/test/Documents/file1.txt]
            Mtime: 1616757267 2021-03-26T12:14:27
            Ctime: 1616757267 2021-03-26T12:14:27

    Internal Info
    Terms: Mplain Mtext T5 T8
    File Name Terms: Ffile1 Ftxt
    XAttr Terms:

baloosearch file1.txt

    /home/test/Documents/file1.txt
    Elapsed: 0.280144 msecs

mv file1.txt file2.txt

balooshow -x file2.txt

    d1600000036 54 3350 file2.txt [/home/test/Documents/file1.txt]
            Mtime: 1616757267 2021-03-26T12:14:27
            Ctime: 1616757267 2021-03-26T12:14:27

    Internal Info
    Terms: Mplain Mtext T5 T8
    File Name Terms: Ffile1 Ftxt
    XAttr Terms:

baloosearch file2.txt

    Elapsed: 0.242654 msecs

balooctl check

balooshow -x file2.txt

    d1600000036 54 3350 file2.txt [/home/test/Documents/file2.txt]
            Mtime: 1616757267 2021-03-26T12:14:27
            Ctime: 1616757272 2021-03-26T12:14:32

    Internal Info
    Terms: Mplain Mtext T5 T8
    File Name Terms: Ffile2 Ftxt
    XAttr Terms:

baloosearch file2.txt

    /home/test/Documents/file2.txt
    Elapsed: 0.286758 msecs
Comment 11 tagwerk19 2021-03-28 08:14:20 UTC
Re, fix in:
    Baloo 5.81.0
mentioned:
    https://bugs.kde.org/show_bug.cgi?id=433116#c10

On Neon Unstable:

    I see the test as per Comment 10 working
    No change with the test of renaming a parent folder, Comment 3
Comment 12 Ignacio Serantes 2021-03-28 20:50:15 UTC
(In reply to tagwerk19 from comment #10)
> Created attachment 137080 [details]
> Log of adding the test repo....
> 
> (In reply to Ignacio Serantes from comment #9)
> > Packages are available at
> > https://download.opensuse.org/repositories/home:/serantes:/branches:/
> > openSUSE:/Factory/openSUSE_Factory/
> Installed a new tumbleweed and added your repo. Log attached so you can see
> if I've missed anything (no guarantee I've got it right :-) Rebooted after
> the update.
> 
> If I test baloo 'noticing' that a file's been moved, I'm not seeing it.
> After a 'balooctl check', it works.
> 
> echo "Hello Penguin" > file1.txt
> 
> balooshow -x file1.txt
> 
>     d1600000036 54 3350 file1.txt [/home/test/Documents/file1.txt]
>             Mtime: 1616757267 2021-03-26T12:14:27
>             Ctime: 1616757267 2021-03-26T12:14:27
> 
>     Internal Info
>     Terms: Mplain Mtext T5 T8
>     File Name Terms: Ffile1 Ftxt
>     XAttr Terms:
> 
> baloosearch file1.txt
> 
>     /home/test/Documents/file1.txt
>     Elapsed: 0.280144 msecs
> 
> mv file1.txt file2.txt
> 
> balooshow -x file2.txt
> 
>     d1600000036 54 3350 file2.txt [/home/test/Documents/file1.txt]
>             Mtime: 1616757267 2021-03-26T12:14:27
>             Ctime: 1616757267 2021-03-26T12:14:27
> 
>     Internal Info
>     Terms: Mplain Mtext T5 T8
>     File Name Terms: Ffile1 Ftxt
>     XAttr Terms:
> 
> baloosearch file2.txt
> 
>     Elapsed: 0.242654 msecs
> 
> balooctl check
> 
> balooshow -x file2.txt
> 
>     d1600000036 54 3350 file2.txt [/home/test/Documents/file2.txt]
>             Mtime: 1616757267 2021-03-26T12:14:27
>             Ctime: 1616757272 2021-03-26T12:14:32
> 
>     Internal Info
>     Terms: Mplain Mtext T5 T8
>     File Name Terms: Ffile2 Ftxt
>     XAttr Terms:
> 
> baloosearch file2.txt
> 
>     /home/test/Documents/file2.txt
>     Elapsed: 0.286758 msecs

Seems not exactly the problem I fixed because in your case seems like renaming notification it not yet processed. I'm guessing that because file name terms are not emptied a has old values.

Maybe you have a problem with iNotify.
Comment 13 Ignacio Serantes 2021-03-28 21:00:07 UTC
(In reply to tagwerk19 from comment #11)
> Re, fix in:
>     Baloo 5.81.0
> mentioned:
>     https://bugs.kde.org/show_bug.cgi?id=433116#c10
> 
> On Neon Unstable:
> 
>     I see the test as per Comment 10 working
>     No change with the test of renaming a parent folder, Comment 3

Confirmed. My patch only fixed filename renaming and not parent folders renaming. I will try to guess the reason.
Comment 14 tagwerk19 2021-03-28 22:06:12 UTC
(In reply to Ignacio Serantes from comment #12)
> Maybe you have a problem with iNotify.
I will check that...

Heads up though that you and Stefan are now chasing this/these bugs. Don't know if independently or together. Happy to run tests and will try to make sure I'm clear about what I'm testing :-/

My Comment 11 was for Neon Unstable and baloo 5.81.0 (Stefan)
Comment 15 Ignacio Serantes 2021-03-29 08:53:20 UTC
(In reply to tagwerk19 from comment #14)
> (In reply to Ignacio Serantes from comment #12)
> > Maybe you have a problem with iNotify.
> I will check that...
> 
> Heads up though that you and Stefan are now chasing this/these bugs. Don't
> know if independently or together. Happy to run tests and will try to make
> sure I'm clear about what I'm testing :-/
> 
> My Comment 11 was for Neon Unstable and baloo 5.81.0 (Stefan)

I'm independent. I look for a fix because this bug is really annoying and was not fixed for a long time.
Comment 16 Ignacio Serantes 2021-03-31 08:11:45 UTC
Created attachment 137190 [details]
A fix for directories?

This is my first attempt to fix the directories part. If I'm not sure about my file rename fix with this one is worst.

Seems to work for me but sometimes I get some debug DB warnings so probably I'm missing something or doing something wrong. More test needed to confirm I am not breaking the DB.

When I have time I will update my Tumbleweed packages with this patch for the brave.
Comment 17 tagwerk19 2022-05-02 14:54:17 UTC
*** Bug 453278 has been marked as a duplicate of this bug. ***
Comment 18 Stefan Brüns 2023-07-10 16:05:55 UTC

*** This bug has been marked as a duplicate of bug 433116 ***