Bug 327988

Summary: Server removes header and body data of maildir records in parttable
Product: [Frameworks and Libraries] Akonadi Reporter: Patrick <mail>
Component: serverAssignee: kdepim bugs <kdepim-bugs>
Status: RESOLVED FIXED    
Severity: normal CC: dvratil
Priority: NOR    
Version: 4.11   
Target Milestone: ---   
Platform: Arch Linux   
OS: Linux   
Latest Commit: Version Fixed In: 1.12.1
Sentry Crash Report:

Description Patrick 2013-11-23 19:10:12 UTC 
I'm heaving trouble with Akonadi Nepomuk Feeder which doesn't index existing mails, and the reason seems to be that the data in the data field of all mail records in parttable with name "PLD:HEAD" and name "PLD:RFC822" is erased. When a new mail arrives (either sent or received), it is stored correctly in the database, but after a few minutes, the data field changes. I activated logging of the mysql server, and it shows queries like these:

3 Prepare	UPDATE PartTable SET pimItemId = ?, name = ?, data = ?, datasize = ?, version = ?, external = ? WHERE ( id = ? )
3 Reset stmt	
3 Execute	UPDATE PartTable SET pimItemId = 17353, name = 'PLD:HEAD', data = NULL, datasize = 0, version = 1, external = 0 WHERE ( id = 34943 )
3 Reset stmt	
3 Execute	UPDATE PartTable SET pimItemId = 17353, name = 'PLD:RFC822', data = NULL, datasize = 0, version = 1, external = 0 WHERE ( id = 34945 )
3 Reset stmt	

I already created a new database, but that didn't change anything. Other resources seem to be fine, only the maildir resource is affected. The Akonadi Debugger, the Job Tracker, and the Notification Monitor don't show any output when the update queries are executed.

Reproducible: Always

Steps to Reproduce:
1. Open Akonadi Console.
2. Select parttable in the DB browser (or use the DB console with a meaningful query).
3. Send an email.
4. Find the email in the parttable.
5. Wait a few minutes and monitor the parttable for changes (by refreshing the table).
Actual Results:  
The data field of the records with name "PLD:HEAD" or  "PLD:RFC822" is cleared and the datasize is set to 0.

Expected Results:  
The data field should not have changed.
Comment 1 Daniel Vrátil 2013-12-06 18:21:09 UTC 
The only moment when I can imagine this is happening is when the external file with the payload cannot be found.

If you run Akonadi from console (akonadictl restart), and try to reproduce this issue, do you see any errors?
Comment 2 Patrick 2013-12-06 19:34:31 UTC 
No errors, but I do get the following message when the problem occurs:

found 1 item parts to expire in collection "sent-mail"
Comment 3 Daniel Vrátil 2014-01-13 14:26:14 UTC 
In KMail, right click the top-level "Local Folders" folder and go to "Folder Properties". In the "Retrieval" tab, check whether you have "Always retrieve full messages" or "Retrieve message bodies on demand" selected. If it's the later, check that "Keep message bodies locally for" is set to 0 ("Forever").

The default value seems to be 1 minute, so Akonadi removes the data from PartTable after one minute - which is most probably what you are seeing. The 1 minute seems to be default settings, so I'll see whether we can change it.
Comment 4 Daniel Vrátil 2014-04-05 21:45:54 UTC 
Closing as fixed. The removal is by-design, because retrieving data from local maildir is very cheap so there's no need to keep all your emails duplicated in the maildir and in the database.

The problem with Nepomuk not being able to index these items has been fixed. We didn't test the code with Nepomuk, but it works with Baloo in KDE 4.13/Akonadi 1.12.1