Bug 264465 - Nepomuk creates very big soprano-virtuoso.log
Summary: Nepomuk creates very big soprano-virtuoso.log
Status: CLOSED UNMAINTAINED
Alias: None
Product: nepomuk
Classification: Miscellaneous
Component: general (show other bugs)
Version: unspecified
Platform: Arch Linux Linux
: NOR major
Target Milestone: ---
Assignee: Nepomuk Bugs Coordination
URL:
Keywords:
: 280750 293665 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-01-27 13:14 UTC by Bogdan
Modified: 2015-01-23 23:22 UTC (History)
25 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bogdan 2011-01-27 13:14:42 UTC
Version:           unspecified (using KDE 4.6.0) 
OS:                Linux

I see that nepomuk create very big logfile, up to 100 GB for now. File is  soprano-virtuoso.log in .kde4/share/apps/nepomuk/repository/main/data/virtuosobackend
The main text in logfile is 
14:09:36 Reference to page with free remap dp = 1859138421, remap = 1859138421

I think that KDE developers need to add option to chose maxumum size of different logfiles, because sometimes it may "eat" all free space on the disk.
Thank you.

Reproducible: Didn't try

Steps to Reproduce:
Enable nepomuk

Actual Results:  
Very big logfile

Expected Results:  
Maximum size of logfile ~ 10mb
Comment 1 Matthias Gies 2012-02-09 11:24:48 UTC
*** Bug 293665 has been marked as a duplicate of this bug. ***
Comment 2 Matthias Gies 2012-02-09 11:39:59 UTC
Found a duplicate, but could mark it: Bug 280750 discusses the same problem.
Comment 3 Martin 2012-02-09 15:30:44 UTC
*** Bug 280750 has been marked as a duplicate of this bug. ***
Comment 4 GSC 2012-03-28 14:10:17 UTC
For me it fills my rootfs , making computer very slow. I'm on chakra x86_64.
Comment 5 Jan 2012-08-05 13:23:13 UTC
I'm affected too.

janmalte@ubuntu:~$ tail -f .kde/share/apps/nepomuk/repository_old/main/data/virtuosobackend/soprano-virtuoso.log
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
17:38:29 Reference to page with free remap dp = 1090, remap = 1090
Comment 6 Martin 2012-08-05 20:51:03 UTC
Jan, can you add version numbers, please?
Comment 7 trumpetthor+kde 2012-09-12 22:01:27 UTC
I also have this problem. Did a clean install from DVD of openSUSE 12.2 x86_64, KDE 4.8.4 "release 2" and within 2 days the log file grew to 379.2 GiB which pushed the partition to 100%. Also, the file .xsession-errors grew to  46.7 Gib. I have NOT been able to access the files, they simply fail to load. I'm deleting the soprano-virtuoso.log, the .db file, and .xsession-errors. I did copy in /home from my previous install, and understand that this may be the trigger in my case.

Please let me know if more info is needed, and good luck swiftly resolving this bug.
Comment 8 jms 2013-04-10 15:08:18 UTC
 I just got hit (again) by this bug wich is really a pain.
soprano-virtuoso.log
get filled up by the same line over and over untill home directory run out of space.
tail soprano-virtuoso.log
10:10:20 Reference to page with free remap dp = 120006, remap = 120006
10:10:20 Reference to page with free remap dp = 120006, remap = 120006
10:10:20 Reference to page with free remap dp = 120006, remap = 120006
10:10:20 Reference to page with free remap dp = 120006, remap = 120006
10:10:20 Reference to page with free remap dp = 120006, remap = 120006
10:10:20 Reference to page with free remap dp = 120006, remap = 120006
10:10:20 Reference to page with free remap dp = 120006, remap = 120006
10:10:20 Reference to page with free remap dp = 120006, remap = 120006
10:10:20 Reference to page with free remap dp = 120006, remap = 120006


line should written once and not been duplicate.
maybe should triger the database cleaner or a dialog asking u to clean db....

Note:this is anew install (using gentoo):
here is my environment emerge --info
http://bpaste.net/show/90371/
and 
I have 
 emerge -pv virtuoso-odbc virtuoso-server soprano

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] dev-db/virtuoso-odbc-6.1.6  0 kB
[ebuild   R    ] dev-db/virtuoso-server-6.1.6  USE="ldap readline -kerberos" 0 kB
[ebuild   R    ] dev-libs/soprano-2.9.0  USE="dbus raptor redland virtuoso -debug -doc {-test}" 0 kB


please have a look at this
https://bugzilla.redhat.com/show_bug.cgi?id=746499
Comment 9 Mark Brandis 2013-05-20 14:45:31 UTC
Had the same problem today on Debian. Gigabytes of soprano.log and xsession-errors.
Comment 10 Sergey Zolotarev 2013-06-01 13:21:21 UTC
Same problem on Kubuntu 13.04, soprano-virtuoso.log is 50 GiB on my system (I wouldn't notice this if it didn't eat all free space lol). Why the hell does this thing produce such a big log file?
Comment 11 rdnetto 2013-07-17 09:58:17 UTC
I'm affected as well - it reached 950 GB for me. This bug should be marked Critical, since it affects the entire OS. My .xsession-errors is a moderate 6 MB and does not appear to contain anything special.

For anyone looking for a temporary workaround, take a look at this:
http://wiki.debian.org/Limits

OS: Sabayon amd64
Package versions: 
dev-db/virtuoso-server-6.1.6
dev-libs/soprano-2.9.0

Log contents:
                Sat Jun 29 2013
00:35:49 transact while cpt atomic, trx no 38173, cpt trx no 106

                Wed Jul 10 2013
21:54:06 Seek failu21:54:06 Seek failure on database /home/reuben/.kde/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.db

                Thu Jul 11 2013
23:15:15 Reference to page with free remap dp = 16843009, remap = 16843009
[last line repeated ad infinitem]
Comment 12 wintonian 2013-08-08 22:22:09 UTC
(In reply to comment #7)
> I also have this problem. Did a clean install from DVD of openSUSE 12.2
> x86_64, KDE 4.8.4 "release 2" and within 2 days the log file grew to 379.2
> GiB which pushed the partition to 100%. Also, the file .xsession-errors grew
> to  46.7 Gib. I have NOT been able to access the files, they simply fail to
> load. I'm deleting the soprano-virtuoso.log, the .db file, and
> .xsession-errors. I did copy in /home from my previous install, and
> understand that this may be the trigger in my case.
> 
> Please let me know if more info is needed, and good luck swiftly resolving
> this bug.

I have installed Fedora 19 less than 24 hours ago, I have reinstalled most of the apps I use and also copied over most of (but not all) of the hidden files from the old Fedora 18 '/home/user' partition. 

Just to add to the stats 'soprano-virtuoso.log' has reached just over 689gb and I have run out of space on the drive - which was quite a shock.
Comment 13 Stefan Hepp 2013-11-22 01:50:18 UTC
Same problem here, on a clean install of Kubuntu 13.10. Same 'reference to page with free remap' log entry all over filling the disk to 100%. 

I am running Kubuntu inside VirtualBox under Windows. Looking at the log file, it seems that this started around the time I rebooted the VM after my Host-PC crashed, so maybe this is some issue with resuming after an unclean shutdown?
Comment 14 rdnetto 2013-11-22 15:32:56 UTC
I haven't seen soprano peg the CPU at 100% recently since I upgraded to a newer version of KDE, so this bug may be fixed in newer versions.

KDE: 4.11.2
dev-libs/soprano-2.9.4
dev-db/virtuoso-server-6.1.6
Comment 15 Nikos Alexandris 2013-12-02 09:55:34 UTC
*** This bug has been confirmed by popular vote. ***
Comment 16 Nikos Alexandris 2013-12-02 09:58:23 UTC
Working here with KDE 4.11.0 and sopranod 2.9.3, I've got a 87+GB file:

-rw------- 1 nik users  87G Dec  2 11:49 soprano-virtuoso.db

What can be done?
Comment 17 Nikos Alexandris 2013-12-02 10:00:56 UTC
Ah... wrong hit again!  The problem of my /home being filled is quite important so I am ignoring some of the readings here. Apologies. Taking votes back. Need to look for a "soprano-virtuoso.db is huge" related thread.
Comment 18 Tuomas Nurmi 2013-12-05 09:35:27 UTC
Affects me too. Filled up the drive with 48GB in about an hour.

Distribution: opensuse 13.1.
nepomuk-core 4.11.2-2.1
tail -1 of the file:

"11:17:16 Reference to page with free remap dp = 3502, remap = 3502"

Note: I didn't have this problem last week, and since then I have only been doing system suggested updates. It only started at the turn of the month.
Comment 19 Nikos Alexandris 2013-12-05 12:21:20 UTC
@Tuomas Nurmi: please vote so as to make this bug collect more than 100 votes. It is a critical one.
Comment 20 wintonian 2013-12-05 12:33:34 UTC
Just added 10 to help things along. 

This hasn't  affected me since but I agree  it is a biggie and needs fixing.
Comment 21 Nikos Alexandris 2013-12-05 12:40:58 UTC
"My" log has literally thousands of... :

--%<---
                Thu Dec 05 2013
14:39:01 GPF: bitmap.c:693 singleton bm not in use

                Thu Dec 05 2013
14:39:13 GPF: bitmap.c:693 singleton bm not in use

                Thu Dec 05 2013
14:39:29 GPF: bitmap.c:693 singleton bm not in use

                Thu Dec 05 2013
14:39:45 GPF: bitmap.c:693 singleton bm not in use

                Thu Dec 05 2013
14:40:06 GPF: bitmap.c:693 singleton bm not in use


                Thu Dec 05 2013
14:40:27 GPF: bitmap.c:693 singleton bm not in use

                Thu Dec 05 2013
14:40:41 GPF: bitmap.c:693 singleton bm not in use

                Thu Dec 05 2013
14:40:56 GPF: bitmap.c:693 singleton bm not in use

                Thu Dec 05 2013
14:41:17 GPF: bitmap.c:693 singleton bm not in use
--->%--
Comment 22 Nikos Alexandris 2013-12-15 13:50:46 UTC
Is bug https://bugs.kde.org/show_bug.cgi?id=231989 related with the one confirmed here?
Comment 23 RJVB 2014-03-29 11:58:10 UTC
Just noticed the same thing, fortunately at 14Gb only with room to spare because filling up a ZFS dataset is NOT a good idea!!

In fact, I'd suggest putting that logfile into /tmp/kde-$USER

Running nepomuk-core 4.12.3, akonadi 1.10.8 and virtuoso 6.1 (virtuoso-t is the app that writes to the logfile, so I wonder if this is a nepomuk bug ...)
Comment 24 Nikos Alexandris 2014-03-29 12:05:25 UTC
My drive got filled 100% few days ago. I just erased the `soprano-virtuoso.db` file without thinking about losing all kinds of tags and other media-related metadata. Nothing bad happened. My drive has 95GB free again. I am happy with that :-)
Comment 25 Vishesh Handa 2014-03-29 16:00:16 UTC
I'm marking this bug as WONTFIX.

With KDE SC 4.13, we have dropped Nepomuk and have moved away from virtuoso. The newer solution works much much better and does not create crazy log files.
Comment 26 RJVB 2014-03-29 16:24:11 UTC
Baloo, right?
That doesn't really resolve the issue for those of us who will have to wait until their distribution decides to move to KDE 4.13 (even Debian/Experimental is still at 4.11...)
Any chance that there'll be a backport, e.g. through a layer/interface that lets Baloo masquerade as Nepomuk so that it becomes a drop-in replacement?

Alternatively, is it possible to control the settings in the virtuoso.ini file that seems to be generated on the fly?
Comment 27 Vishesh Handa 2014-03-29 16:37:05 UTC
(In reply to comment #26)
> Baloo, right?
> That doesn't really resolve the issue for those of us who will have to wait
> until their distribution decides to move to KDE 4.13 (even
> Debian/Experimental is still at 4.11...)
> Any chance that there'll be a backport, e.g. through a layer/interface that
> lets Baloo masquerade as Nepomuk so that it becomes a drop-in replacement?
> 

Sorry no. The changes are just far too intrusive. 

> Alternatively, is it possible to control the settings in the virtuoso.ini
> file that seems to be generated on the fly?

Not without modifying the code. Even then I'm not sure what settings you'd want to change. We already try to use the minimum number of resources.
Comment 28 RJVB 2014-03-29 17:04:16 UTC
On Saturday March 29 2014 16:37:05 you wrote:
> > Alternatively, is it possible to control the settings in the virtuoso.ini
> > file that seems to be generated on the fly?
> 
> Not without modifying the code. Even then I'm not sure what settings you'd want
> to change. We already try to use the minimum number of resources.

Simply switch off the logging. Frequent logging like I saw (the same message at least 10x a second) must represent a resource draw even if the logfile is a symlink to /dev/null. What binary is responsible for creating the virtuoso.ini file, I presume it should be possible to comment out the error log string in there (put a # instead of the 1st character) ;)

R.
Comment 29 Fernando Bobbio 2015-01-23 22:45:14 UTC
I just noticed that bug too.  The solution in my case was to delete soprano-virtuoso.db and to check on the cause of it.  The cause was a malformed E-Mail in KMail.  I logged into the webmail part of my E-Mail provider and removed the E-Mail.  The issue has not reoccurred so far.