Bug 368557

Summary: Improve: Error handling on lmdb errors
Product: [Frameworks and Libraries] frameworks-baloo Reporter: Christoph Cullmann <christoph>
Component: generalAssignee: Pinak Ahuja <pinak.ahuja>
Status: RESOLVED WORKSFORME    
Severity: crash CC: christoph, nate, olaf.the.lost.viking, poprocks, stefan.bruens, tagwerk19
Priority: HI    
Version: unspecified   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Christoph Cullmann 2016-09-10 15:30:05 UTC
e.g. try:

# first: let it index some dir with / at end => triggers assert

cullmann@ene:/local/cullmann/kde-5/bin$ ./balooctl index /local/cullmann/kdesrc/taglib/tests/data/
Indexing /local/cullmann/kdesrc/taglib/tests/data/
ASSERT: "!url.endsWith('/')" in file /local/cullmann/kdesrc/baloo/src/engine/documenturldb.cpp, line 46
Aborted

# now: any index will fail :/

cullmann@ene:/local/cullmann/kde-5/bin$ ./balooctl index balooctl 
ASSERT failure in Transaction: "MDB_PANIC: Update of meta page failed", file /local/cullmann/kdesrc/baloo/src/engine/transaction.cpp, line 54
Aborted


Reproducible: Always
Comment 1 Christoph Cullmann 2016-09-10 15:31:57 UTC
Btw., at akademy have I taken a look at the code.
IMHO the error handling is below any standard we want.
We have this asserts everywhere but zero real error handling.
(which is not mean to insult, I understand that it was written in a hurry)

See my change:

https://git.reviewboard.kde.org/r/128865/

(but even with that, only complete corruption at startup is handled, it is still a mystery for me, why we need to kill the lmdb lock and how that should not lead to random corruption).
Comment 2 Christoph Cullmann 2016-09-12 11:33:20 UTC
Actually, any transaction will fail after that, until you kill + restart baloo_file/**
Comment 3 Christoph Cullmann 2016-09-12 11:33:44 UTC
*** Bug 356769 has been marked as a duplicate of this bug. ***
Comment 4 Nate Graham 2017-10-27 19:31:23 UTC
Looks like your changes have made a big difference for this particular issue, at least:

$ balooctl index /home/dev/
Indexing /home/dev/
File(s) indexed
Comment 5 Nate Graham 2018-08-09 20:22:57 UTC
*** Bug 397255 has been marked as a duplicate of this bug. ***
Comment 6 Justin Zobel 2022-09-22 12:29:27 UTC
Thank you for reporting this crash in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the crash with a recent software version?

If you can reproduce the issue, please change the status to "CONFIRMED" when replying. Thank you!
Comment 7 Nate Graham 2022-09-22 13:52:44 UTC
Still an issue.
Comment 8 tagwerk19 2022-10-11 21:16:58 UTC
(In reply to Nate Graham from comment #7)
> Still an issue.
I'd be interested to see if I can get a failure :-/

I've seen similar situations after renaming a parent folder (Bug 392793) and there are times when baloo stops noticing changes but without me noticing what might lead to it. I'm not able to reproduce the issue as per the "Steps to Reproduce" on Neon Unstable.

What I see with debugging is:

  $ balooctl index testfiles/
     ...
    kf.filemetadata: No extractor for "inode/directory"
    Indexing /home/test/testfiles/
    File(s) indexed

  $ balooshow -x testfiles
    kf.baloo.engine: DocumentDB::get 6081089575648257 MDB_NOTFOUND: No matching key/data pair found
    159ab80000fc01 64513 1415864 testfile [/home/test/testfiles]
            Mtime: 1665416144 2022-10-10T17:35:44
            Ctime: 1665416144 2022-10-10T17:35:44

    Internal Info
    Terms: Mdirectory Minode T9
    File Name Terms: Ftestfiles
    XAttr Terms:

Then indexing a file:

  $ balooctl index testfile/test.txt
     ...
    kf.filemetadata: Fetching extractors for "text/plain"
    Indexing /home/test/testfile/test.txt
    File(s) indexed

  $ balooshow -x testfile/test.txt
    kf.baloo.engine: DocumentDB::get 5622241039547393 MDB_NOTFOUND: No matching key/data pair found
    13f9660000fc01 64513 1309030 testfile/test.txt [/home/test/testfile/test.txt]
            Mtime: 1665416144 2022-10-10T17:35:44
            Ctime: 1665416144 2022-10-10T17:35:44
            Cached properties:
                    Line Count: 1

    Internal Info
    Terms: Mplain Mtext T5 T8 X20-1 hello penguin
    File Name Terms: Ftest Ftxt
    XAttr Terms:
    lineCount: 1

A few more debug statements would be nice ...
Comment 9 Christoph Cullmann 2024-05-01 19:12:21 UTC
I close this bug now, we did improve the locking issues in the last release, better just look at concrete new crash reports to fix more stuff.