Bug 265510 - Nepomuk backup eating too much memory when creating backup file
Summary: Nepomuk backup eating too much memory when creating backup file
Status: RESOLVED WORKSFORME
Alias: None
Product: nepomuk
Classification: Miscellaneous
Component: general (show other bugs)
Version: unspecified
Platform: openSUSE Linux
: NOR normal
Target Milestone: ---
Assignee: Vishesh Handa
URL:
Keywords:
: 265737 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-02-05 13:21 UTC by Jiří Appl
Modified: 2011-11-27 22:20 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jiří Appl 2011-02-05 13:21:27 UTC
Version:           unspecified (using KDE 4.6.0) 
OS:                Linux

This bug is against nepomukbackup.

I ran the tool today twice, it starts to create a temporary file under /tmp and nepomukservices process starts to allocate more and more memory. I have 8 GB of RAM and it manages to run out of that space too. This happens when the temporary backup file is around 1.2 GB big. At this point I had to kill the process. (I had some other memory intensive processes running, so it did not eat the entire RAM space, however it was a significant portion.)

Second maybe related issue: is the backup file supposed to be this big? (when I had to kill nepomukservices process, the backup procedure was still unfinished).

Size of my nepomuk store is: 1.7 GB

However, if I understand correctly, the backup should back up only data, which cannot be retrieved in some other way. Therefore, I thought that it should not be backing up all indexed data.

On the other hand the backup temporary file was in plain text, so I suppose that made it much bigger.

Reproducible: Always

Steps to Reproduce:
I ran the tool twice, rebooting in the meantime. Both times, same behavior.

Actual Results:  
Tool have not finished backing data up, I had to kill it because of excessive memory usage. Also, backup file was quite big (however it was a temporary uncompressed file).

Expected Results:  
Tool should finish a backup procedure and return significantly smaller backup file. It should also not require so much memory.

I am running kdepim 4.6 from http://download.opensuse.org/repositories/KDE:/Unstable:/SC:/kdepim46/KDE_Distro_Factory_openSUSE_11.3/.

The rest of my kde packages are from Factory aka. KDF (KDE SC 4.6):
Core packages: http://download.opensuse.org/repositories/KDE:/Distro:/Factory/openSUSE_11.3/
 
Extra: http://download.opensuse.org/repositories/KDE:/Extra/openSUSE_11.3_KDE_Distro_Factory/
 
Unstable:Playground: http://download.opensuse.org/repositories/KDE:/Unstable:/Playground/openSUSE_11.3_KDE_Distro_Factory/

However this probably should not have any influence.

Please let me know if you require some additional information.
Comment 1 Roman K. 2011-03-17 11:52:56 UTC
I don't know if this is related, but for me nepomuk (or virtuoso-t in the process-list) also uses too much memory, I allowed it to use 200 MB of my RAM, but currently, it uses around 760 MB RAM. 

I'm using openSUSE 11.4 with KDE SC 4.6.1 and Qt 4.7.2. I'm not know exactly, but I think this didn't happen with KDE SC 4.6.0
Comment 2 Matija Šuklje 2011-05-07 19:48:29 UTC
I can confirm identical behaviour on my Gentoo using KDE 4.6.2.
Comment 3 Nico Kruber 2011-05-07 22:23:35 UTC
is it possible, that nepomuk stays with the highest amount of RAM _ever_ set? I observed that behaviour on one of my PCs: I increased it for indexing (why not use more when I'm not there) and decreased it to a normal level afterwards.

Recently, I deleted the whole old nepomuk data repository and let it re-index everything (strigi) - et voilà: it now uses the configured amount of 256MB which I did not change during indexing...
Comment 4 Jiří Appl 2011-06-05 00:29:08 UTC
I have changed distro to ArchLinux, my current KDE SC version is 4.6.3.

When I ran the manual backup today, the memory usage was relatively ok, however the temporary backup file grew to approximately 1.3 GB. Then, the memory usage increased considerably. I had to kill it. Overall, I am still unable to perform the manual backup.
Comment 5 Dominic Lyons 2011-06-13 23:56:23 UTC
*** Bug 265737 has been marked as a duplicate of this bug. ***
Comment 6 Matija Šuklje 2011-08-03 18:04:55 UTC
In KDE 4.7.0 it seems to work OK for me now.
Comment 7 Vishesh Handa 2011-08-03 18:09:41 UTC
(In reply to comment #6)
> In KDE 4.7.0 it seems to work OK for me now.

Minor fixes - I no longer backup akonadi stuff, so there is less stuff to backup, which results in less memory consumption.

I need to do a major rewrite, but I need to run it by the other developers before that. Desktop Summit FTW! :D
Comment 8 Dominic Lyons 2011-08-07 23:26:52 UTC
After the Update to 4.7.0 the problem seemed solved.

Today there's the next scheduled backup. I'll have an eye on it...
Comment 9 Jiří Appl 2011-11-27 22:20:23 UTC
Ok, it looks like it is fixed now. Closing the bug. Please feel free to reopen if it is not really fixed, just masked.