Bug 289475

Summary:	[Akonadi Nepomuk Feeder] If you are going to fail, please, fail. Don't waste CPU instead of displaying proper error messages.
Product:	[Frameworks and Libraries] Akonadi	Reporter:	Alejandro Nova <alejandronova>
Component:	Nepomuk Feeder Agents	Assignee:	kdepim bugs <kdepim-bugs>
Status:	RESOLVED NOT A BUG
Severity:	normal	CC:	vkrause
Priority:	NOR
Version:	4.8
Target Milestone:	---
Platform:	Chakra
OS:	Linux
Latest Commit:		Version Fixed In:
Sentry Crash Report:

Description Alejandro Nova 2011-12-21 03:21:45 UTC

Version:           4.8 (using Devel) 
OS:                Linux

Whenever I try to index mail with KDE 4.8, CPU usage rises to 100%. This is strictly related to the mail component of akonadi_nepomuk_feeder, present in KDE 4.8 Beta 2.

- The spikes are related to mail indexing. When I disable mail indexing (using the UI piece) and restart Akonadi, CPU usage returns to normal.
- If I strace akonadi_nepomuk_feeder, I notice large periods of inactivity. If I strace virtuoso_t, it is locked in a FUTEX waiting loop. The symptoms look like a  fixed Nepomuk bug that was fixed between KDE 4.6.0 and KDE 4.6.1.

Reproducible: Always

Steps to Reproduce:
Let akonadi_nepomuk_feeder index some mail.

Actual Results:  
- akonadi_nepomuk_feeder will have less than 1% of CPU, virtuoso_t will use 100% of one core.
- If you strace both programs, both are waiting I don't know what (this is waste)
- The actual indexing is superslow (~18 hours to index 40,000 mails)

Expected Results:  
- When indexing, akonadi_nepomuk_feeder will have between 10 and 30% of CPU, and virtuoso_t will use between 10% and 30% of all cores (normal behaviour).
- If you strace both programs, you'll notice them really working.
- The actual indexing is fast (~30 minutes to index 40,000 mails).

Comment 1 Alejandro Nova 2011-12-22 01:46:15 UTC

This improved A LOT after I deleted my Akonadi database and recreated it. Even though there are bugs and mail indexing remains slow, points number 1 and 2 are accomplished.

- Virtuoso gets a peak of 45% CPU usage, on both my cores (total: 90%). There is a remaining 10% to 20% used by nepomukstorage. So, that's normal. 90% of the time, however, CPU usage is less than 10%, and Akonadi eats no CPU.
- I don't need to strace; the behavior is radically different from what I saw before.
- Indexing, however, remains super slow.

Conclusions:

1. You have to forcefully delete outdated databases, as part of an upgrade.
2. Please, speed up mail indexing.

Comment 2 Christophe Marin 2011-12-22 23:33:14 UTC

> 2. Please, speed up mail indexing.

There were several issues before:
- several nepomuk feeders,
- indexing was using a lot of cpu

For 4.8, Nepomuk received lots of fixes, so did Virtuoso 6.1.4 and on the Akonadi side, there's only one feeder now.

The idle detection may be what you notice: when the machine is used, the Akonadi feeder stops indexing and resumes later.

There's an option to disable the idle detection in 4.8 (but no kcm afaict):
stop the akonadi server (akonadictl stop)
open ~/.kde/share/config/akonadi_nepomuk_feederrc

change:
[akonadi_nepomuk_feeder]
DisableIdleDetection=false

to:
[akonadi_nepomuk_feeder]
DisableIdleDetection=true

then restart Akonadi

Comment 3 Alejandro Nova 2011-12-23 01:05:05 UTC

Christophe:

1. Thanks for joining me!.
2. I've done more testing and narrowed this bug enough to be fixable.

-

Comment 4 Alejandro Nova 2011-12-23 01:09:26 UTC

- I had configuration errors.
a) The Invitations resource was misconfigured and locking up the Akonadi Nepomuk feeder.
b) I had several invalid mail folders.

- I corrected those errors, and the indexing is normal.

So, please, rephrase this bug report as follows:

- When you have errors in your Akonadi setup, instead of failing, the Akonadi Nepomuk feeder will behave strangely (eating CPU cycles like crazy, locking up Virtuoso, and other funky behaviour).

Steps to reproduce.

- Produce errors in your Akonadi setup
- Let akonadi_nepomuk_feeder start.

Actual results.

- Funky behaviour (100% cpu, dying akonadi_nepomuk_feeder)

Expected results.

- A big, bad, error message, or akonadi_nepomuk_feeder refusing to start.
- Better yet; Akonadi overwriting a broken setup with sane defaults.

Comment 5 Alejandro Nova 2011-12-23 01:15:40 UTC

BTW, I noticed that commit. Now I'm happy! I really HATED the idle detection!

Comment 6 Alejandro Nova 2011-12-23 19:28:20 UTC

Couldn't replicate anymore after I upgraded Akonadi to 1.6.90. Packaging issues.