Bug 287187

Summary:	akonadi_nepomuk_feeder can make Virtuoso eat all my CPU, when it autosuspends.
Product:	[Frameworks and Libraries] Akonadi	Reporter:	Alejandro Nova <alejandronova>
Component:	Nepomuk Feeder Agents	Assignee:	Tobias Koenig <tokoe>
Status:	RESOLVED NOT A BUG
Severity:	normal	CC:	kdepim-bugs, manutortosa, vkrause
Priority:	NOR
Version:	4.8
Target Milestone:	---
Platform:	Chakra
OS:	Linux
Latest Commit:		Version Fixed In:
Sentry Crash Report:

Description Alejandro Nova 2011-11-21 20:16:52 UTC

Version:           4.8 (using Devel) 
OS:                Linux

When I have the akonadi_nepomuk_feeder in KDE 4.8 beta 1, Virtuoso eats one of my cores, and, when akonadi_nepomuk_feeder is doing more than one task at once, Virtuoso will eat all of my CPU.

Ironically, this happens when akonadi_nepomuk_feeder decides to suspend its processing, since "the system is busy". If the Nepomuk feeder restarts, nothing happens and Virtuoso CPU usage won't recede, but a new suspension has the potential to make Virtuoso eat all of my CPU.

KDE 4.8 Beta 1, packages from kde-unstable, Chakra.

Reproducible: Always

Steps to Reproduce:
1. Boot. akonadi_nepomuk_feeder starts after all agents.
2. Press a key. akonadi_nepomuk_feeder will decide that "system is busy" and it will stop indexing.
3. Virtuoso CPU usage will skyrocket.

Actual Results:  
The CPU usage for Virtuoso goes between 50% and 100%. A strace shows that Virtuoso spends all that CPU waiting for something, so, it's a waste.

Expected Results:  
If Virtuoso isn't doing anything, it should consume no CPU.

Comment 1 Alejandro Nova 2011-11-21 20:40:32 UTC

Update: a lot of messages flow when I interrupt Virtuoso. This is a snippet (something like this happens with my entire contact list, around 1,000 contacts)

"
"" Soprano: "SQLExecDirect failed on query 'sparql select distinct ?r count(?p) as ?cnt where { ?r ?p ?o. filter( ?p in (<http://www.semanticdesktop.org/ontologies/2007/08/15/nao#prefLabel>,<http://www.semanticdesktop.org/ontologies/2007/03/22/nco#fullname>) ). optional { ?r <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#prefLabel> ?o0 . } . filter(!bound(?o0) || ?o0="CONSTANZA NAVARRETE VALENZUELA").  ?r a <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#PersonContact> .  optional { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#fullname> ?o1 . } . filter(!bound(?o1) || ?o1="CONSTANZA NAVARRETE VALENZUELA"^^<http://www.w3.org/2001/XMLSchema#string>). filter(  bound(?o0) ||  bound(?o1) ) . } order by desc(?cnt)' (iODBC Error: [OpenLink][Virtuoso iODBC Driver]CL064: Lost connection to server)"
[/usr/bin/nepomukservicestub] Query failed: "sparql select distinct ?r count(?p) as ?cnt where { ?r ?p ?o. filter( ?p in (<http://www.semanticdesktop.org/ontologies/2007/08/15/nao#prefLabel>,<http://www.semanticdesktop.org/ontologies/2007/03/22/nco#fullname>) ). optional { ?r <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#prefLabel> ?o0 . } . filter(!bound(?o0) || ?o0="CONSTANZA NAVARRETE VALENZUELA").  ?r a <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#PersonContact> .  optional { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#fullname> ?o1 . } . filter(!bound(?o1) || ?o1="CONSTANZA NAVARRETE VALENZUELA"^^<http://www.w3.org/2001/XMLSchema#string>). filter(  bound(?o0) ||  bound(?o1) ) . } order by desc(?cnt)" 
"" Soprano: "SQLExecDirect failed on query 'sparql select distinct ?r count(?p) as ?cnt where { ?r ?p ?o. filter( ?p in (<http://www.semanticdesktop.org/ontologies/2007/08/15/nao#prefLabel>,<http://www.semanticdesktop.org/ontologies/2007/03/22/nco#fullname>) ). optional { ?r <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#prefLabel> ?o0 . } . filter(!bound(?o0) || ?o0="CONSTANZA NAVARRETE VALENZUELA").  ?r a <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#PersonContact> .  optional { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#fullname> ?o1 . } . filter(!bound(?o1) || ?o1="CONSTANZA NAVARRETE VALENZUELA"^^<http://www.w3.org/2001/XMLSchema#string>). filter(  bound(?o0) ||  bound(?o1) ) . } order by desc(?cnt)' (iODBC Error: [OpenLink][Virtuoso iODBC Driver]CL064: Lost connection to server)"
"" Soprano: "SQLExecDirect failed on query 'sparql select distinct ?r count(?p) as ?cnt where { ?r ?p ?o. filter( ?p in (<http://www.semanticdesktop.org/ontologies/2007/08/15/nao#prefLabel>,<http://www.semanticdesktop.org/ontologies/2007/03/22/nco#fullname>) ). optional { ?r <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#prefLabel> ?o0 . } . filter(!bound(?o0) || ?o0="CONSTANZA NAVARRETE VALENZUELA").  ?r a <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#PersonContact> .  optional { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#fullname> ?o1 . } . filter(!bound(?o1) || ?o1="CONSTANZA NAVARRETE VALENZUELA"^^<http://www.w3.org/2001/XMLSchema#string>). filter(  bound(?o0) ||  bound(?o1) ) . } order by desc(?cnt)' (iODBC Error: [OpenLink][Virtuoso iODBC Driver]CL064: Lost connection to server)"

I can reproduce them always, so, the process causing this is the Contact indexer.

Comment 2 Alejandro Nova 2011-11-23 10:06:49 UTC

Tried with a newly created database to isolate.

1. When the Akonadi activity detector decides the system is not idle (when I'm typing this, for instance), CPU usage is normal.
2. When I'm away from the computer, CPU usage is 100%.

So, this is related to the activity detection code. IMHO, that code must go.

Comment 3 Manuel Tortosa 2011-11-23 21:02:27 UTC

Could you be so kind to test the new Akonadi and VIrtuoso packages provided by Chakra?

Comment 4 Alejandro Nova 2011-11-23 21:59:00 UTC

Gracias, Manuel, por tomar mi solicitud ;)

We have another issue in Chakra: we NEED shared-desktop-ontologies 0.8.0. I installed it, and the behaviour of Nepomuk improved immediately. I'll see how my testing goes, but I think we can close this as INVALID after we have some confirmations. Please confirm.

Comment 5 Manuel Tortosa 2011-11-23 22:22:45 UTC

Ok, please next time use our bugtracker instead before reporting upstream.

Comment 6 Alejandro Nova 2011-11-23 22:46:21 UTC

Ok, but the issue remains when you try to mix results indexed with KDE 4.7 and KDE 4.8, so it's a valid bug. 

You will have either to tell users to recreate their database when they upgrade, or they will face trouble. I got only a good behaviour when I used a clean database and s-d-o 0.8.

Comment 7 Alejandro Nova 2011-11-24 14:53:14 UTC

Tested in another computer, couldn't reproduce, INVALID ;).

Seems I created some packaging issues on my own...