Bug 253115 - Strigi uses an uncontrollable, monotonically growing amount of space that cannot be reclaimed
Summary: Strigi uses an uncontrollable, monotonically growing amount of space that can...
Status: RESOLVED DUPLICATE of bug 231989
Alias: None
Product: nepomuk
Classification: Miscellaneous
Component: general (show other bugs)
Version: unspecified
Platform: Ubuntu Linux
: NOR major
Target Milestone: ---
Assignee: Sebastian Trueg
URL:
Keywords:
: 252676 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-10-03 13:55 UTC by Sergio
Modified: 2011-01-06 19:21 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sergio 2010-10-03 13:55:11 UTC
Version:           unspecified (using KDE 4.5.1) 
OS:                Linux

The actual bug is "that cannot be reclaimed".

Scenario. Multiuser system. Each user has a kde desktop. Strigi is enabled by default with (IMHO) very silly defaults about what to index. Result: after a few months, each user has a > 3GB space wasted with the strigi indexing information. Either the machine comes low with disk space in home and nobody cannot work anymore, or users come low with disk space wrt their quotas and most of them cannot work anymore. Users start complaining since they cannot understand why they are low on disk, since given the actual things they had put in their home, all of them think that they should have a few GB left.

You tell them that it is the indexer. They tell you that they do not use it. You tell them that maybe they do not use it, but still it is working. They ask you why you do not disable it by default. You tell them that there is no documentation saying how to create users' skeleton environments with strigi disabled. They tell you "ok, I gonna disable it from my system settings". Finally most of them do so (either by disabling strigi completely or by telling strigi to index only much more selected stuff).

For the remaining users you need to find out on your own where the strigi configuration file is to disable it from there. Otherwise, there is no way for "root" to run "systemsetting" to configure the environment of one of his users without knowing the user password and logging in to the kde environment as the user, which seems a rather intrusive thing to do.

Ok, finally you have strigi disabled for everybody. What is the result? Absolutely nothing. When you tell strigi either not to index files at all (or not to index directory XXX), the space used for indexing (or at least for indexing directory XXX) is not reclaimed at all.

You go on google, and the hint is to kill the whole nepomuk database for each user. It is quite lucky that most users had never ever used nepomuk tags and ontologies at all. Otherwise they would get quite upset at seeing them wiped away.

Please... do not enable Nepomuk by default (or at least the nepomuk-strigi combination) until it is ready. You cannot consider nepomuk ready until there is a console allowing to control what is stored and to remove it selectively.

Note that there might also be strong privacy concerns about it until a controlling console is shipped (e.g. people erases data and is unaware about weather how and when its associated search database is removed).

Furthermore, the kde 4.0 release should have shown something about the mistake of forcing users into something that is incomplete. 

Currently, letting strigi be disabled for all users is the only possible sane default. Users who know what it is for will also know how to enable it and how to deal with its quirks.

Reproducible: Always

Steps to Reproduce:
Install kde.

Use it for a while.

See the strigi data take GBs in the nepomuk database.

Tell strigi to stop indexing (either some folders or to stop indexing alltogether).


Actual Results:  
Observe in puzzlement that none of the space taken by the strigi data is given back.

Erase the whole nepomuck database as the sole way to reclaim space.

See that with this all the rest of the data you could have in nepomuk is happily lost.

Expected Results:  
Have a way to tell strigi to selectively discard pieces of its search database.

Have a nepomuk control panel to see what is actually stored and to selectively erase data.

Bug is marked as major since:

1) It causes data loss when the only way to reclaim space from strigi is to _erase_ the whole of the nepomuk database.

2) It causes full loss of operation when the disk or the user quota fills up due to the uncontrollable operation of strigi (when you disable it, it is already to late, since you cannot reclaim the space)

3) It causes privacy concern, since you might end up erasing stuff and not knowing if it is really erased in the search database.
Comment 1 L Nix 2010-10-06 23:35:00 UTC
If you rename/delete the file /usr/share/autostart/nepomukserver.desktop, it will not start for any user.  Users may add the shortcut for the nepomukserver to their own autostart folders as desired, but global start for all users will be disabled.

For example, I renamed the file to 'nepomukserver.desktop.goaway'.  Essentially removing the .desktop suffix removes it from the autostart process, and keeps the file in case I ever (!) wish to restore autostarting.

/usr/share/autostart/nepomukserver.desktop
Comment 2 Sergio 2010-10-07 14:32:23 UTC
On 06/10/2010 23:35, L Nix wrote:
> https://bugs.kde.org/show_bug.cgi?id=253115
>
>
>
>
>
> --- Comment #1 from L Nix<lornix lornix com>   2010-10-06 23:35:00 ---
> If you rename/delete the file /usr/share/autostart/nepomukserver.desktop, it
> will not start for any user.  Users may add the shortcut for the nepomukserver
> to their own autostart folders as desired, but global start for all users will
> be disabled.
>
> For example, I renamed the file to 'nepomukserver.desktop.goaway'.  Essentially
> removing the .desktop suffix removes it from the autostart process, and keeps
> the file in case I ever (!) wish to restore autostarting.
>
> /usr/share/autostart/nepomukserver.desktop
>
>    
Thanks for posting! This is a very good advice, since it will not remove 
functionalities from those users that really want them (and are ready to 
cope with the current status of things).

Sergio
Comment 3 snvv101 2010-11-27 22:48:29 UTC
I confirm this bug in KDE 4.4.5.
This is a major bug, the other is the strigi rescanning of unchanged folder sub-trees (and a minor loosing tags when moving files).

Apart for that the project is brilliant.
Comment 4 Sebastian Trueg 2011-01-05 20:34:07 UTC
*** Bug 252676 has been marked as a duplicate of this bug. ***
Comment 5 Sebastian Trueg 2011-01-06 19:21:34 UTC

*** This bug has been marked as a duplicate of bug 231989 ***