Bug 322083 - CollectionSync always start a database transaction, even when no update needs to be done
Summary: CollectionSync always start a database transaction, even when no update needs...
Status: RESOLVED FIXED
Alias: None
Product: Akonadi
Classification: Frameworks and Libraries
Component: libakonadi (show other bugs)
Version: unspecified
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-07-07 19:25 UTC by Alessandro Pignotti
Modified: 2013-08-22 12:19 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In: 4.12
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alessandro Pignotti 2013-07-07 19:25:24 UTC
Disclaimer: I'm not expert in akonadi codebase, so the terminology might not be 100% correct.

CollectionSync derives from TransactionSequence, which means that all the Jobs added are wrapped in a database transaction. This probably make sense to guarantee correctness in case of concurrent updates.

The problem is that a transaction is started even in the case when no updates needs to be made. The effect of this is that mysql (when it is the backend) is no longer able to optimize the load of the local collection data and needs to write book-keeping data on the log. This means that in the very frequent case of no updates needed (so, theoretically read only operation) mysqld will still be writing to disk, with an effect on system performance and battery usage.

I'm not sure what could be a good solution to this problem, my guess is that it could make sense to operate in two phases:

1) Load the local collection without starting a transaction and compare with the remote one to see if anything needs to be updated

2) If needed, start a transaction and do the complete sync. This will cause overhead when updates are needed, but hopefully this is balanced by the optimization of the read only case.

I think another possible solution would be to use explicit table locks, but akonadi currently seems to have no concept of locking.

Reproducible: Always

Steps to Reproduce:
1. Add an IMAP resource that is polled regularly
2. Watch akonadi mysqld process with strace, grep for 'pwrite'

Actual Results:  
I can see write operations happening when apparently no new mails are arriving

Expected Results:  
Read only operations should be read-only
Comment 1 Alessandro Pignotti 2013-07-07 21:01:37 UTC
For info on mysql read-only performance and transactions see http://dev.mysql.com/doc/refman/5.6/en/innodb-performance.html section 14.2.4.2.3.
Comment 2 Daniel Vrátil 2013-08-22 12:19:55 UTC
Git commit f7bf8bd51a4e03771419c91ff2cd155b56445073 by Dan Vrátil, on behalf of Alessandro Pignotti.
Committed on 22/08/2013 at 12:17.
Pushed by dvratil into branch 'master'.

Avoid starting a database transaction in CollectionSync when no updates are needed

Unconditionally starting a transaction on every collection sync
as an high overhead on disk activity since transaction bookkeping
on the database requires writing to disk. Most often collection sync
are unnecessary since collection are rarely modified, so we end up
causing a write to disk on an operation which would be read-only.

The proposed solution is two run the collection sync in 2 phases:

1) Without starting a transaction, the currently known local collections
are loaded and are compared against the received remote ones. If no
difference is found update is stopped right away

2) If there are differences a transaction is started, local collections
are loaded again to guarantee consistency and the update is applied
inside the transaction

REVIEW: 111885
FIXED-IN: 4.12

M  +1    -1    CMakeLists.txt
M  +167  -17   akonadi/collectionsync.cpp
M  +7    -1    akonadi/collectionsync_p.h

http://commits.kde.org/kdepimlibs/f7bf8bd51a4e03771419c91ff2cd155b56445073