Bug 293665 - DATA LOSS - Nepomuk eats disk space and kde freezes/crashes
Summary: DATA LOSS - Nepomuk eats disk space and kde freezes/crashes
Status: RESOLVED DUPLICATE of bug 264465
Alias: None
Product: nepomuk
Classification: Miscellaneous
Component: fileindexer (show other bugs)
Version: 4.7
Platform: Ubuntu Linux
: NOR major
Target Milestone: ---
Assignee: Sebastian Trueg
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-02-08 18:33 UTC by Matthias Gies
Modified: 2012-02-09 11:24 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Gies 2012-02-08 18:33:28 UTC
Version:           4.7 (using KDE 4.7.4) 
OS:                Linux

Working on Kubuntu 11.10.
Before update to 4.7.4 came, I already had a problem, that nepomuk totally bloated my disk space, until /home was full, resulting in a freeze and then crash of kde. It returned to KDM login and I lost some work I had open before, since my encrypted /home/user was not unmounted properly.

After upgrade to 4.7.4 I cleaned out my home folder and freshly configured KDE. Also I moved some 100GB of *.avi files to my NAS because it seemed to me the nepomuk index DB was growing massively when scanning these. Nonetheless the index has been growing from 30GB to 50GB, then 80GB, and now it occupies 149GB.

I also had strigi activated, but it seems the index keeps growing, even when I deactivate strigi.

Atm, I have deactivated nepomuk completely, otherwise it would fill my filsystem again.

Even after moving the 100G of *.avi to my NAS, the index did not shrink. Does nepomuk/strigi not cleanup index for vanished files?

At least there should be an option to reset nepomuk completely or to flush all index.

If I have 300G of data in my /home/user, what should be a regular index size with nepomuk&strigi activated?

Reproducible: Always

Steps to Reproduce:
Installed Kubuntu 11.10 from live-DVD. Then upgraded full system with regular updates to 4.7.4.
Activate nepomuk and/or strigi in systemsettings.

Actual Results:  
indexer bloating my disk space step by step. especially when leaving system unattended. index never shrinks. cannot be flushed/reset/limited by gui.

DATA LOSS when disk was running out of space for the first time and kde crashed!

Expected Results:  
should be aware of disk space! leave at least a few gig free and bring a warning. should be possible to limit/reset/flush index or complete nepomuk/strigi service config.

If any info from my running system is needed, please let me know before march. I am planning to do a completely fresh install of my system in the first week of march.
I'll try using Kubuntu 11.10 again, but with netinstall, directly installing latest kde 4.7.4.
Maybe the switch from 4.7.3 to 4.7.4 caused the chaos... I'll report then...
Comment 1 Matthias Gies 2012-02-09 08:50:55 UTC
I found some hint, maybe.

Today I went to /home/matthias/.kde/share/ to remove all nepomuk related files to reset it again.
I looked a little closer, to see where the space of 149G was wasted. Turned out the file /home/matthias/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.log consumes all of it. 

My actual index db /home/matthias/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.db only has 2.5G.

Something wrong here?

Also I found that all these files exist a second time in /home/matthias/.kde/share/apps/nepomuk/repository/main/soprano-virtuoso.* but are either empty or consume only 2-8M.

So what's the deal with the logfile consuming 149G?
Comment 2 Matthias Gies 2012-02-09 11:15:34 UTC
So, I have reset the whole thing by deleting all nepomuk folders and cfgs in my home-dir.

After logging back in, nepomuk is configured with defaults, strigi turned off.
Logfile contains nothing, only after another reboot contains following message:

		Thu Feb 09 2012
11:59:02 Unable to remove /home/matthias/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.lck (No such file or directory)

The 149G logfile I couldn't open due to it's size. I'll keep an eye on this. If nothing happens after few days of leaving the pc inactive at night, I'll turn on strigi again, to see if that's the source for infinite log-spam.

What I already noticed though: when I had strigi activated, dolphin used to return bogus search results.
Now, after resetting and having nepomuk run without strigi, dolphin returns accurate results, even about file contents.

Leads me to the question: why does the systemsettings module for nepomuk tell me, that I need strigi to search contents of files, if nepomuk itself already seems to do that?
Comment 3 Matthias Gies 2012-02-09 11:24:48 UTC

*** This bug has been marked as a duplicate of bug 264465 ***