Bug 360946 - Baloo should handle lmdb corruption - Query::exec() should be fail-safe (and all things called inside)
Summary: Baloo should handle lmdb corruption - Query::exec() should be fail-safe (and ...
Status: RESOLVED FIXED
Alias: None
Product: frameworks-baloo
Classification: Frameworks and Libraries
Component: Engine (show other bugs)
Version: unspecified
Platform: Chakra Linux
: NOR crash
Target Milestone: ---
Assignee: Pinak Ahuja
URL:
Keywords:
: 353342 354705 356009 357344 358657 359488 359667 359692 359839 359996 362200 362621 363491 363511 364013 364360 364520 364521 366005 366011 366038 366293 366437 367769 368442 368446 368519 368699 368761 368809 368946 368991 369076 369092 370347 370622 371813 372549 373006 375293 376403 376525 376932 377311 379788 380882 381213 382610 384505 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-03-24 17:46 UTC by Fabian
Modified: 2020-03-04 13:49 UTC (History)
52 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
New crash information added by DrKonqi (47.08 KB, text/plain)
2017-10-27 20:20 UTC, John Aylward
Details
New crash information added by DrKonqi (27.79 KB, text/plain)
2017-10-27 20:28 UTC, John Aylward
Details
New crash information added by DrKonqi (5.20 KB, text/plain)
2020-03-04 13:49 UTC, Yury Gubich
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Fabian 2016-03-24 17:46:54 UTC
When ~/.local/share/baloo/index, baloo will crash. The reason for this is that in both positiondb.cpp and postingdb.cpp mdb_get is expected to either return MDB_SUCCESS (0) or  to return MDB_NOTFOUND. However, in case of database corruption mdb_get returns MDB_PAGE_NOTFOUND, as can be seen in the backtrace below. Note that silently ignoring this error will not work, as further database operations will yield MDB_BAD_TXN (Transaction cannot recover - it must be aborted).


ASSERT failure in PositionDB::iter: "MDB_PAGE_NOTFOUND: Requested page not found", file /home/fabian/projects/baloo/src/engine/positiondb.cpp, line 173

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffc00aa700 (LWP 15679)]
0x00007ffff457f5f8 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff457f5f8 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff4580a7a in abort () from /usr/lib/libc.so.6
#2  0x00007ffff4d07391 in qt_message_fatal (context=..., message=<synthetic pointer>) at global/qlogging.cpp:1610
#3  QMessageLogger::fatal (this=this@entry=0x7fffc00a8b60, 
    msg=msg@entry=0x7ffff4fb5fa0 "ASSERT failure in %s: \"%s\", file %s, line %d") at global/qlogging.cpp:784
#4  0x00007ffff4d02721 in qt_assert_x (where=<optimized out>, what=<optimized out>, file=<optimized out>, 
    line=<optimized out>) at global/qglobal.cpp:3002
#5  0x00007fffc02df51c in Baloo::PositionDB::iter (this=0x7fffc00a8c60, term=...)
    at /home/fabian/projects/baloo/src/engine/positiondb.cpp:173
#6  0x00007fffc02e84fa in Baloo::Transaction::postingIterator (this=0x7fffc00a98b0, query=...)
    at /home/fabian/projects/baloo/src/engine/transaction.cpp:312
#7  0x00007fffc07449f3 in Baloo::SearchStore::constructQuery (this=0x7fffc00a99c0, tr=0x7fffc00a98b0, term=...)
    at /home/fabian/projects/baloo/src/lib/searchstore.cpp:283
#8  0x00007fffc0743cc7 in Baloo::SearchStore::constructQuery (this=0x7fffc00a99c0, tr=0x7fffc00a98b0, term=...)
    at /home/fabian/projects/baloo/src/lib/searchstore.cpp:163
#9  0x00007fffc0743cc7 in Baloo::SearchStore::constructQuery (this=0x7fffc00a99c0, tr=0x7fffc00a98b0, term=...)
    at /home/fabian/projects/baloo/src/lib/searchstore.cpp:163
#10 0x00007fffc074346d in Baloo::SearchStore::exec (this=0x7fffc00a99c0, term=..., offset=0, limit=10, 
    sortResults=true) at /home/fabian/projects/baloo/src/lib/searchstore.cpp:75
#11 0x00007fffc0731466 in Baloo::Query::exec (this=0x7fffc00a9b40)
    at /home/fabian/projects/baloo/src/lib/query.cpp:210
#12 0x00007fffc0963710 in SearchRunner::match (this=this@entry=0x10aa080, context=..., type=..., category=...)
    at /chakra/core/plasma-workspace/src/plasma-workspace-5.6.0/runners/baloo/baloosearchrunner.cpp:99
#13 0x00007fffc09644c9 in SearchRunner::match (this=0x10aa080, context=...)
    at /chakra/core/plasma-workspace/src/plasma-workspace-5.6.0/runners/baloo/baloosearchrunner.cpp:164
#14 0x00007fffd802b2d8 in Plasma::AbstractRunner::performMatch (this=0x10aa080, localContext=...)
    at /chakra/core/krunner/src/krunner-5.20.0/src/abstractrunner.cpp:131
#15 0x00007fffcfdeffd0 in ThreadWeaver::Executor::run(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () from /usr/lib/libKF5ThreadWeaver.so.5
#16 0x00007fffcfdeeb70 in ThreadWeaver::Job::execute(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*) () from /usr/lib/libKF5ThreadWeaver.so.5
#17 0x00007fffcfdee67a in ThreadWeaver::Thread::run() () from /usr/lib/libKF5ThreadWeaver.so.5
#18 0x00007ffff4d1f238 in QThreadPrivate::start (arg=0x10bbef0) at thread/qthread_unix.cpp:340
#19 0x00007ffff2884464 in start_thread () from /usr/lib/libpthread.so.0
#20 0x00007ffff463512d in clone () from /usr/lib/libc.so.6


Reproducible: Always

Steps to Reproduce:
1. Corrupt your database
2. Cause a query
3.

Actual Results:  
In debug mode, the assertion is triggered. In release mode,  the cast will be UB. This manifests in a traceback similar to #355280, as the data is corrupted.

Expected Results:  
Baloo handles the crash gracefully. Ideally, it would try to repair the database or at least evict the bad data.
Comment 1 Fabian 2016-03-24 20:05:49 UTC
I should add that starting konsole from krunner is a very easy way to trigger that code path, which might be an explanation for those bugs (360754, 360727, etc.)
Comment 2 Christoph Cullmann 2016-09-11 18:21:54 UTC
*** Bug 364360 has been marked as a duplicate of this bug. ***
Comment 3 Christoph Cullmann 2016-09-11 18:25:10 UTC
*** Bug 359488 has been marked as a duplicate of this bug. ***
Comment 4 Christoph Cullmann 2016-09-11 18:43:47 UTC
*** Bug 362200 has been marked as a duplicate of this bug. ***
Comment 5 Christoph Cullmann 2016-09-11 20:58:49 UTC
*** Bug 366437 has been marked as a duplicate of this bug. ***
Comment 6 Christoph Cullmann 2016-09-11 20:58:54 UTC
*** Bug 362621 has been marked as a duplicate of this bug. ***
Comment 7 Christoph Cullmann 2016-09-11 20:59:00 UTC
*** Bug 359667 has been marked as a duplicate of this bug. ***
Comment 8 Christoph Cullmann 2016-09-11 20:59:26 UTC
*** Bug 356009 has been marked as a duplicate of this bug. ***
Comment 9 Christoph Cullmann 2016-09-11 21:32:51 UTC
*** Bug 353342 has been marked as a duplicate of this bug. ***
Comment 10 Christoph Cullmann 2016-09-11 21:33:16 UTC
*** Bug 366005 has been marked as a duplicate of this bug. ***
Comment 11 Christoph Cullmann 2016-09-11 21:33:27 UTC
*** Bug 359996 has been marked as a duplicate of this bug. ***
Comment 12 Christoph Cullmann 2016-09-11 21:34:01 UTC
*** Bug 359839 has been marked as a duplicate of this bug. ***
Comment 13 Christoph Cullmann 2016-09-11 21:34:12 UTC
*** Bug 359692 has been marked as a duplicate of this bug. ***
Comment 14 Christoph Cullmann 2016-09-11 21:34:33 UTC
*** Bug 366011 has been marked as a duplicate of this bug. ***
Comment 15 Christoph Cullmann 2016-09-11 21:34:53 UTC
*** Bug 366038 has been marked as a duplicate of this bug. ***
Comment 16 Christoph Cullmann 2016-09-11 21:35:09 UTC
*** Bug 367769 has been marked as a duplicate of this bug. ***
Comment 17 Christoph Cullmann 2016-09-11 21:35:33 UTC
*** Bug 368446 has been marked as a duplicate of this bug. ***
Comment 18 Christoph Cullmann 2016-09-11 21:36:04 UTC
*** Bug 357344 has been marked as a duplicate of this bug. ***
Comment 19 Francisco Cribari 2016-09-20 03:17:33 UTC
I believe this bug is related to https://bugs.kde.org/show_bug.cgi?id=366022 and https://bugs.kde.org/show_bug.cgi?id=366190. See also this video: https://www.youtube.com/watch?v=u-aGfU-whhA
Comment 20 Christoph Cullmann 2016-10-07 16:16:10 UTC
*** Bug 369076 has been marked as a duplicate of this bug. ***
Comment 21 Christoph Cullmann 2016-10-07 16:16:50 UTC
*** Bug 368946 has been marked as a duplicate of this bug. ***
Comment 22 Christoph Cullmann 2016-10-07 16:17:17 UTC
*** Bug 368991 has been marked as a duplicate of this bug. ***
Comment 23 Christoph Cullmann 2016-10-07 16:17:43 UTC
*** Bug 368761 has been marked as a duplicate of this bug. ***
Comment 24 Christoph Cullmann 2016-10-07 16:18:04 UTC
*** Bug 368699 has been marked as a duplicate of this bug. ***
Comment 25 Nate Graham 2017-10-27 17:00:34 UTC
*** Bug 371813 has been marked as a duplicate of this bug. ***
Comment 26 Nate Graham 2017-10-27 17:02:20 UTC
*** Bug 376403 has been marked as a duplicate of this bug. ***
Comment 27 Nate Graham 2017-10-27 17:02:27 UTC
*** Bug 380882 has been marked as a duplicate of this bug. ***
Comment 28 Nate Graham 2017-10-27 17:03:46 UTC
*** Bug 370347 has been marked as a duplicate of this bug. ***
Comment 29 Nate Graham 2017-10-27 17:04:35 UTC
*** Bug 364520 has been marked as a duplicate of this bug. ***
Comment 30 Nate Graham 2017-10-27 17:10:08 UTC
*** Bug 363491 has been marked as a duplicate of this bug. ***
Comment 31 Nate Graham 2017-10-27 17:13:25 UTC
*** Bug 354705 has been marked as a duplicate of this bug. ***
Comment 32 Nate Graham 2017-10-27 17:14:50 UTC
*** Bug 358657 has been marked as a duplicate of this bug. ***
Comment 33 Nate Graham 2017-10-27 17:21:17 UTC
*** Bug 368442 has been marked as a duplicate of this bug. ***
Comment 34 Nate Graham 2017-10-27 17:21:22 UTC
*** Bug 368519 has been marked as a duplicate of this bug. ***
Comment 35 Nate Graham 2017-10-27 17:21:26 UTC
*** Bug 364013 has been marked as a duplicate of this bug. ***
Comment 36 Nate Graham 2017-10-27 17:21:31 UTC
*** Bug 363511 has been marked as a duplicate of this bug. ***
Comment 37 Nate Graham 2017-10-27 17:21:39 UTC
*** Bug 366293 has been marked as a duplicate of this bug. ***
Comment 38 Nate Graham 2017-10-27 17:22:52 UTC
*** Bug 376525 has been marked as a duplicate of this bug. ***
Comment 39 Nate Graham 2017-10-27 18:33:46 UTC
*** Bug 384505 has been marked as a duplicate of this bug. ***
Comment 40 Nate Graham 2017-10-27 18:36:12 UTC
*** Bug 382610 has been marked as a duplicate of this bug. ***
Comment 41 Nate Graham 2017-10-27 18:37:07 UTC
*** Bug 381213 has been marked as a duplicate of this bug. ***
Comment 42 Nate Graham 2017-10-27 18:37:38 UTC
*** Bug 379788 has been marked as a duplicate of this bug. ***
Comment 43 Nate Graham 2017-10-27 18:38:46 UTC
*** Bug 370622 has been marked as a duplicate of this bug. ***
Comment 44 Nate Graham 2017-10-27 18:40:07 UTC
*** Bug 377311 has been marked as a duplicate of this bug. ***
Comment 45 Nate Graham 2017-10-27 18:53:31 UTC
*** Bug 373006 has been marked as a duplicate of this bug. ***
Comment 46 Nate Graham 2017-10-27 18:54:32 UTC
*** Bug 372549 has been marked as a duplicate of this bug. ***
Comment 47 Nate Graham 2017-10-27 19:47:49 UTC
I'm willing to bet that Christoph Cullman's work that went into 5.28 (especially https://cgit.kde.org/baloo.git/commit/?id=e34da150d82a57cf417a59b8b632b2fecb32a6f7) has fixed this. There isn't a single report of this crash happening when using a KDE Frameworks version later than 5.26.
Comment 48 Nate Graham 2017-10-27 19:49:53 UTC
*** Bug 368809 has been marked as a duplicate of this bug. ***
Comment 49 Nate Graham 2017-10-27 19:52:34 UTC
*** Bug 375293 has been marked as a duplicate of this bug. ***
Comment 50 Nate Graham 2017-10-27 20:02:35 UTC
*** Bug 364521 has been marked as a duplicate of this bug. ***
Comment 51 Nate Graham 2017-10-27 20:03:03 UTC
*** Bug 376932 has been marked as a duplicate of this bug. ***
Comment 52 Nate Graham 2017-10-27 20:04:05 UTC
*** Bug 369092 has been marked as a duplicate of this bug. ***
Comment 53 John Aylward 2017-10-27 20:10:22 UTC
How is this resolved if crashes keep getting reported as duplicates? What version of KDE was this fixed in?
Comment 54 Nate Graham 2017-10-27 20:12:25 UTC
All the crashes I'm marking as duplicates were reported against versions of KDE Frameworks 5.26 and lower. It was fixed in KDE Frameworks 5.28. I haven't run into a crash with the same characteristics on KF 5.28 or later, and if I do, it's likely a different crash.
Comment 55 John Aylward 2017-10-27 20:20:52 UTC
Created attachment 108591 [details]
New crash information added by DrKonqi

plasmashell (5.10.5) using Qt 5.7.1

- What I was doing when the application crashed:

When I was querying from the "start menu" the search results appeared and then the desktop crashed and restarted.

-- Backtrace (Reduced):
#7  0x00007fab4086a4c0 in QByteArray::QByteArray(char const*, int) () from /usr/lib64/libQt5Core.so.5
#8  0x00007faa3a54aa77 in DBPostingIterator::DBPostingIterator(void*, unsigned int) () from /usr/lib64/libKF5BalooEngine.so.5
#9  0x00007faa3a54ade4 in Baloo::PostingDB::prefixIter(QByteArray const&) () from /usr/lib64/libKF5BalooEngine.so.5
#10 0x00007faa3a55535f in Baloo::Transaction::postingIterator(Baloo::EngineQuery const&) const () from /usr/lib64/libKF5BalooEngine.so.5
#11 0x00007faa3a9ada2a in Baloo::SearchStore::constructQuery(Baloo::Transaction*, Baloo::Term const&) () from /usr/lib64/libKF5Baloo.so.5
Comment 56 Nate Graham 2017-10-27 20:23:04 UTC
That looks like a different crash; your backtrace isn't showing a crash in Query::exec(). I didn't say that *everything* had been fixed (I wish) :). Please feel free to report that as in new bug report!
Comment 57 John Aylward 2017-10-27 20:24:29 UTC
Thanks. I'll do that.
Comment 58 John Aylward 2017-10-27 20:28:58 UTC
Created attachment 108592 [details]
New crash information added by DrKonqi

krunner (5.10.5) using Qt 5.7.1

When searching from the desktop search (ALT+space), the query results are returned and then the desktop crashes and restarts.

This trace does show a call to Baloo::Query::exec(). So I decided to add it here. Let me know if I should open a seprate ticket for this as well.

-- Backtrace (Reduced):
#7  0x00007fd6a675c4c0 in QByteArray::QByteArray(char const*, int) () from /usr/lib64/libQt5Core.so.5
#8  0x00007fd66be26a77 in DBPostingIterator::DBPostingIterator(void*, unsigned int) () from /usr/lib64/libKF5BalooEngine.so.5
#9  0x00007fd66be26de4 in Baloo::PostingDB::prefixIter(QByteArray const&) () from /usr/lib64/libKF5BalooEngine.so.5
#10 0x00007fd66be3135f in Baloo::Transaction::postingIterator(Baloo::EngineQuery const&) const () from /usr/lib64/libKF5BalooEngine.so.5
#11 0x00007fd66c289a2a in Baloo::SearchStore::constructQuery(Baloo::Transaction*, Baloo::Term const&) () from /usr/lib64/libKF5Baloo.so.5
Comment 59 Nate Graham 2017-10-27 20:30:14 UTC
Can you file a whole new ticket, with a full backtrace? That's just another comment on this bug report (and all 52 people CCd on it are getting spammed).
Comment 60 Yury Gubich 2020-03-04 13:49:52 UTC
Created attachment 126594 [details]
New crash information added by DrKonqi

dolphin (19.12.2) using Qt 5.14.1

- What I was doing when the application crashed:
It always crashes when I select icons in dolphin. It doesn't matter which way it's done - it crashes with ctrl selection, with shift selection, with rectangle selection, with little plus button on top of the icon. 
It doesn't matter which kind of icon - it crashes on folders on textfiles on images.

- Custom settings of the application:
I have detailed info panel opened in Dolphin (F11 button). As soon as I close that panel i can select as many items as I wish without crashing

-- Backtrace (Reduced):
#8  0x00007f20faea6bd0 in mdb_get () from /usr/lib/liblmdb.so
#9  0x00007f20fcd24f53 in Baloo::DocumentDataDB::get(unsigned long long) () from /usr/lib/libKF5BalooEngine.so.5
#10 0x00007f20fcd45684 in Baloo::Transaction::documentData(unsigned long long) const () from /usr/lib/libKF5BalooEngine.so.5
#11 0x00007f20ff1bdc32 in Baloo::File::load() () from /usr/lib/libKF5Baloo.so.5
[...]
#13 0x00007f20fd8b826a in QObject::event(QEvent*) () from /usr/lib/libQt5Core.so.5