Bug 332253

Summary: krunner goes out-of-memory with desktop search plugin
Product: Baloo Reporter: Martin Koller <kollix>
Component: GeneralAssignee: Vishesh Handa <me>
Status: RESOLVED FIXED    
Severity: normal CC: alejandronova, auxsvr, cfeck, donolwenn, hrvoje.senjan, kde, mombelli.mauro, nepomuk-bugs, nicolamori, p.r.worrall, valdikss
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: openSUSE RPMs   
OS: Linux   
Latest Commit: Version Fixed In: 4.13.1
Attachments: result of $ valgrind -v --tool=massif --log-file=baloo baloosearch '=1.05*1.05'

Description Martin Koller 2014-03-17 14:36:34 UTC
I have KDE 4.12.90 and in krunner I only have desktop search activated.
Simply typing "=2*2" and wait a few seconds will consume as much memory as you have and the kernel OOM handler will kill the process.

(I have no longer anything Nepomuk related installed on my system, so the search plugin hopefully now already uses baloo)

Reproducible: Always
Comment 1 Nolwenn 2014-04-22 09:57:51 UTC
More or less the same on 4.13 archlinux package, after typing "=1.05*1.05" krunner consume 2 GiO of memory.
Comment 2 Vishesh Handa 2014-04-25 15:51:58 UTC
I cannot seem to reproduce this. Could one of you please try doing the same thing with the baloosearch tool?

$ baloosearch '=1.05*1.05'

and see if baloosearch spikes up in memory usage? If it does, then please run it under massif and attach the relevant report.

$ valgrind --tool=massif baloosearch '=1.05*1.05'
Comment 3 Martin Koller 2014-04-25 16:01:15 UTC
On Friday 25 April 2014 15:51:58 you wrote:

> I cannot seem to reproduce this. Could one of you please try doing the same
> thing with the baloosearch tool?

I can reproduce it with
baloosearch '=2*2'

but using
valgrind --tool=massif baloosearch '=2*2'

leads to OOM and the tool is getting killed ...
Comment 4 Vishesh Handa 2014-04-25 16:29:17 UTC
You probably have a very large number of terms in your xapian db which start with 2. Do you want to try and count?

vlap:~/.local/share/baloo/file $ delve . -a -1 | g '^2' | wc -l
Comment 5 Martin Koller 2014-04-25 16:52:10 UTC
On Friday 25 April 2014 16:29:17 you wrote:
> https://bugs.kde.org/show_bug.cgi?id=332253
> 
> --- Comment #4 from Vishesh Handa <me@vhanda.in> ---
> You probably have a very large number of terms in your xapian db which start
> with 2. Do you want to try and count?
> 
> vlap:~/.local/share/baloo/file $ delve . -a -1 | g '^2' | wc -l
> 

That is not that large:

delve . -a -1 | grep '^2' | wc -l
164678
Comment 6 Nolwenn 2014-04-25 18:15:32 UTC
Created attachment 86267 [details]
result of $ valgrind -v --tool=massif --log-file=baloo baloosearch '=1.05*1.05'
Comment 7 Christoph Feck 2014-04-29 08:39:30 UTC
*** Bug 334041 has been marked as a duplicate of this bug. ***
Comment 8 Christoph Feck 2014-05-04 10:27:10 UTC
*** Bug 334315 has been marked as a duplicate of this bug. ***
Comment 9 Christoph Feck 2014-05-04 10:28:42 UTC
Vishesh, does comment #5 / comment #6 provide the requested information? Please set the bug status.
Comment 10 Christoph Feck 2014-05-04 20:57:59 UTC
*** Bug 333947 has been marked as a duplicate of this bug. ***
Comment 11 Vishesh Handa 2014-05-05 10:09:32 UTC
> delve . -a -1 | grep '^2' | wc -l
> 164678

That is quite large! I have an idea of how to reproduce this. I'll write a test case, and try to fix it before 13.1
Comment 12 Vishesh Handa 2014-05-05 16:17:10 UTC
Git commit 5bdb32928dc3eabf482676bb039104d1137cf1fb by Vishesh Handa.
Committed on 05/05/2014 at 14:28.
Pushed by vhanda into branch 'KDE/4.13'.

XapianSearchStore: Split the string based on + * / - =

When we receive a search string, we typically split it up by spaces and
dots. Then each sub string is checked, if it is <= 3 characters, we use
our own expansion scheme (imperfect) otherwise xapians.

We need to use our own completion scheme because xapian's consumes too
much memory in the case of very few characters. It basically expands the
string to every possible completion result it has in its db, this
results in loads and loads of memory being consumed.

We now split based on some extra characters so that when searching for
'2*2=' will not consume all your RAM in the case when your DB has many
words starting with 2.
FIXED-IN: 4.13.1

M  +1    -1    src/xapian/xapiansearchstore.cpp

http://commits.kde.org/baloo/5bdb32928dc3eabf482676bb039104d1137cf1fb
Comment 13 Alejandro Nova 2014-05-06 13:22:40 UTC
Vishesh, I use KRunner to calculate simple math operations, and always put a = to tell KRunner to ignore every other runner while I'm calculating, but I'm getting bitten by this bug. I'll wait for 4.13.1, but is it possible to add an additional check to make the desktop search plugin ignore every expression with a leading "="?
Comment 14 Vishesh Handa 2014-05-08 11:07:14 UTC
@Alejandro: Yes. That is possible. Would you mind filing another bug for it? I don't want to forget about it.
Comment 15 Nicola Mori 2014-06-12 14:37:40 UTC
It seems that this bug is present again in 4.13.2 (ArchLinux 64 bit). Typing a simple calculation like:

   =sqrt(12)

my system goes immediately out-of-memory. On my machine oom-killer is able to kill krunner and recover the system, but krunner is almost unusable as a calculator.