In system-settings, I have set nepomuk to take up to 1GB for file-indexing (out of 16gb), now my system started swapping and I saw virtuoso-t process to take up ~6gb RAM. After killing it, mem got freed. $top 5305 marek 39 19 7998028 5.831g 2852 S 51.1 37.4 360:57.62 virtuoso-t 5296 marek 39 19 578812 106784 2468 S 31.9 0.7 210:27.43 nepomukstorage ... Reproducible: Always
The setting controls nepomukstorage, not virtuoso-t . Memory not used is wasted. Do you feel any slow down?
Swapping will always cause a major slow down.
what the setting in question actually does is control the NumberOfBuffers parameter passed to virtuoso (not nepomukstorage). Essentially this is the size of the cache used to store database elements (see http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtConfigScale ) The total memory usage of virtuoso is not the amount used as a cache, so its memory usage will usually be greater than the value you set by a factor of a few. Also unless you are making a large number of queries at once there will probably be little advantage to increasing the cache so far (if your main memory is large enough to store the whole db, linux's file cache will probably keep everything in memory anyway). So I strongly suggest you just leave the setting at the default.
Hey Simeon Do you think maybe we should remove this Memory option from the KCM? Virtuoso still consumes more memory, even after specifying the number of buffers. Maybe it would be better to not advertise a feature which clearly does not work from a user's point of view.
@Chao: Like Christoph said, after swapping the drop was very visible on the machine. Maybe set the priority of the consumed mem lower (if possible) so the cache gets dropped when a user run program (eg compile in my case) needs it? @Simeon: Might be the case(!), I was fiddling with the parameter, set it to 1gb, which seemed reasonable. Will try set it back to lower. @Vishesh: I wouldn't be for removing the option, as for users who don't actually use the nepomuk/virtuoso/etc functionality, limiting the mem is an effective way to keep memory for other uses. Instead, better display eg more accurate est. mem. consumption (the option value now time n-cores?) or atleast a warning it consumes actually much more.
@Marek: I'm voting for removing the setting and keep the memory to a minimum always. The user shouldn't have to care about such a low level detail, and Nepomuk should always consume the minimum amount of memory. There isn't much point in displaying a warning that it may consume more and then allowing the user to configure the memory usage. Could you elaborate on your "the option value now time n-cores"? I'm not sure I understand.
@Vishesh: I was going to ask you about this - I tend to agree we should remove the KCM option, and probably reset it to 50MB for 4.11. If people really care they can still change the config file... It seems to cause a lot of unhappiness in people when they set the memory usage to one value and virtuoso uses more. Plus I suspect that the minimum is exactly the same performance-wise as any other value, although I haven't measured it. Resources are cached anyway, and I would guess few people do the same search repeatedly. Maybe you profiled it at some point? Even if the minimum is not a good value, there is no reasonable way for the user to choose a better one.
@Simeon: Lets do it! Do you patch to it up? Or should I?
@Vishesh: Ok! Could you code it please? I'm away from my development machine for a few days.
(In reply to comment #9) > @Vishesh: Ok! Could you code it please? I'm away from my development machine > for a few days. Done. Committed.
I watched this Bug as I could not use indexing on my machine (with 8Gb ram nonetheless). But after upgrading to 4.11 virtouso-t still takes ~1GB RAM. I could live with that most of the time, but i really need every GB sometimes:) Thing is; does the memory-footprint go down significantly when nepomuk is not indexing? Is it lower when it is just updating the index later? Because i never got around to finish the initial indexing (i know, bad style for someone complaining about a bug), because is was no finished after 3 hours and I could no work smoothly with so much I/O going on, and it was just not worth it to let run overnight.