Bug 325132 - long mail repeatedly fails to be indexed, indexation never ends, virtuoso keeps crashing , some programs are completely unresponsive
Summary: long mail repeatedly fails to be indexed, indexation never ends, virtuoso kee...
Status: RESOLVED UNMAINTAINED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: Nepomuk Feeder Agents (show other bugs)
Version: 4.11
Platform: Kubuntu Linux
: NOR major
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-09-21 00:30 UTC by dmatt
Modified: 2015-03-17 12:38 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
anonymized and modified offending email (3.70 MB, application/x-extension-eml)
2013-09-21 00:34 UTC, dmatt
Details
feeder error query for email (2.33 KB, text/plain)
2013-09-21 00:36 UTC, dmatt
Details

Note You need to log in before you can comment on or make changes to this bug.
Description dmatt 2013-09-21 00:30:20 UTC
After receiving very long mail (3.6MB of plain text - log of rsync from cron job), indexing on my PC is stuck in evil loop. After each boot normal indexing starts, than suddenly virtuoso-t process uses one core to full capacity, memory requirements of virtuoso-t grow to something like 80MB of memory and then virtuoso ends and new virtuoso process spawns. Once at this stage, this repeats cca every 15 seconds. Indexing never ends and percentage of mails indexed shown in system setting randomly jumps to anything under 100%. 
in the akonadictl output or .xsession-errors file could be seen the failing query (with whole 3.6MB of text inside).

To verify the problem I setup clean Kubuntu install. Connecting to the same IMAP account leads to same corruption.  

There are also other bugs:
1, once corrupted stays corrupted and user cannot fix it via nondestructive means:
Even after offending email is deleted, akonadi is trying to run the query and system behaves like the email is still on IMAP. Nepomuk cleaner does not help (probably even shouldn't in this case). Only option is create clean install and connect to IMAP only after the mail was deleted.
Or manually delete akonadi database maybe.

2. dolphin and some gui elements do not react to inputs for long seconds. One core is free and some other programs react adequatly, so there must be some blocking behind the scene. 

I should add, that at the beginning I thought there was problem with folders because of this error:
akonadi_nepomuk_feeder ItemQueue::fetchJobResult: Not all items were fetched: 21 2
I attempted to move few folders to newly created one. Only part of the emails were copied. The rest was deleted from source folder but not copied into target, so I lost lots of emails and had to restore backup
This might or might be not due to this email being among copied files.

Reproducible: Always

Steps to Reproduce:
1.send long email to IMAP account
2.run Kontaxt with email indexing on
3.
Actual Results:  
indexing is stuck in loop and eating system resources

Expected Results:  
email is successfully indexed

bug manifested on notebook (core2duo, 4GB RAM) with Kubuntu 12.04 and KDE 4.10.5, which was later updated to 4.11.0 and 4.11.1. Test clean install was 13.10 beta with 4.11.1.
Comment 1 dmatt 2013-09-21 00:34:10 UTC
Created attachment 82431 [details]
anonymized and modified offending email
Comment 2 dmatt 2013-09-21 00:36:52 UTC
Created attachment 82432 [details]
feeder error query for email
Comment 3 Andreas Hartmetz 2013-09-21 22:48:40 UTC
I can reproduce this, just didn't bother filing a bug so far...
What I remember from my observations is that Virtuoso crashed repeatedly and the error message, which contained the full query, was thousands of lines of e-mail contents.
Comment 4 Vishesh Handa 2015-03-17 12:38:57 UTC
The Nepomuk project is no longer maintained in KDE since 4.13. For email indexing, Baloo provided an Akonadi resource to index emails, contacts and events. Tags are now maintained by Akonadi itself.