Bug 361876 - Poor database performance after update
Summary: Poor database performance after update
Status: RESOLVED LATER
Alias: None
Product: kmymoney
Classification: Applications
Component: database (show other bugs)
Version: 4.7.2
Platform: Mint (Ubuntu based) Linux
: NOR major
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-04-16 21:54 UTC by lp.allard.1
Modified: 2016-05-05 22:53 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description lp.allard.1 2016-04-16 21:54:16 UTC
Since migration to 4.7.2 from 4.6.2, KMM is extremely slow, to the point of not being production worthy.  Every click of the mouse takes between 5 & 10 seconds no matter what I am  loading (contextual popup, or loading a ledger, or payees, etc).  This is noticeably slower when in the Accounts page but overall, its slow  to the point that it becomes frustrating to use it.  For example, opening the config popup for the reports takes up to a minute.

I eliminated database problems with a replication to another DB, then using 4.6.2 on this replicated DB, it is much much faster and snappier (when I do something something happens).

CPU and RAM are comfortably low (<10% CPU, RAM < 1.5GB) but nevertheless, I noticed CPU is always between 7-10% unless I do nothing at all (let it alone).

Is there a debug mode or something I could do to pinpoint the source of this extreme lack of performance?

bug 360874 seems to be similar or somewhat related to what I am experiencing, except that I am running on Linux Mint 17.3.  Workstation has quad core AMD Phenom II X4 965 at 3.4GHz, and 12GB DDR3-1600 RAM.

Reproducible: Always
Comment 1 Christian David 2016-04-17 06:54:17 UTC
I am surprised this happens. Unfortunately there is no debug mode. However, to solve this issue we need to know what exactly causes the slow down. KMyMoney does not start any action without prior user interaction (I think). Does it get faster again if you just wait long enough (like one or two hours)? What exactly did you do before it slows down? Does is slow down if you just open KMyMoney and do nothing for some time?

Thank you for testing both versions! This was very important and extremely helpful.

Also a last note: I recommend to use a .kmy file. It is way faster and better. I am afraid that the database backend is of low quality regarding speed.

Let us continue the discussion in this bug.
Comment 2 lp.allard.1 2016-04-17 13:43:47 UTC
Hello Christian,

it is difficult to tell.  For starters, my database has been quick for several years, until I had a recent issue with memo not recording what I manually entered in them (from the scheduled transactions).  I was suggested to try 4.7.2, this is when the DB became notoriously slower, but at that time, KMM (doing a consistency check) found several hundred errors which fixed 99% of them (a few still need manual fix).

I may revert to a local file.  I must however mention that the database back end is of pretty good performance, running on a Centos server with MariaDB (InnoDB) and 2x RAID1 SAS 15k600 Hitachi enterprise hard drives.  The InnoDB buffer has over 24GB of dedicated RAM.

Im not sure if there's a test that would simulate KMM's workload towards the database server but it would be useful.  I noticed the "Performance-Test" menu entry in KMM (Tools menu), what does it do?  I ran it but I get no message or any other output from that...

I need also to ask:  right now I am working with 4.6.4 (sorry all that time I said 4.6.2 I meant 4.6.4, I got mixed up in the version numbers).  Should I permanently move to 4.7.2 and go from there?
Comment 3 Christian David 2016-04-22 06:33:50 UTC
We can assume that the RDBMS (aka “the database”) is probably not the issue. The poor performance is caused by the way KMyMoney requests the data from the RDBMS.

To pinpoint the issue you must use 4.7.2.
Comment 4 lp.allard.1 2016-04-24 14:13:14 UTC
OK I am back with 4.7.2.

Database is still slow, especially after a few minutes of working with it.  With 4.6.4, it was faster and did not slow down over time like it does with 4.7.2

What do you suggest I try next?
Comment 5 Christian David 2016-04-24 16:13:04 UTC
Can you try this:

Does it get faster again if you just wait long enough (like one or two hours)? What exactly did you do before it slows down? Does is slow down if you just open KMyMoney and do nothing for some time?
Comment 6 lp.allard.1 2016-04-24 19:21:01 UTC
"Does it get faster again if you just wait long enough (like one or two hours)?"
No

"What exactly did you do before it slows down?"
I navigate between account ledgers and the scheduled transactions.  I enter a few transactions and process scheduled transactions.  Nothing major IMO.  I feel the application is getting slower as I go because its loading everything in the ledgers as my reconciled transactions are displayed (not hidden) so every time I enter a ledger, everything is displayed.

"Does is slow down if you just open KMyMoney and do nothing for some time?"
Yes.  I launched KMM, worked with the DB for a while (perhaps for an hour), then I let it "sit" for another hour or so, and came back to it.  I can confirm it was much slower and less responsive than before I take a break.  Even the GUI was slow to resume operation from a reduced state in the taskbar.

Other observations:
While KMM seems to hang or become unresponsive, htop reports CPU usage around 10% (on one core out of four). RAM doesnt fluctuate much, but other applications are failing to launch until KMM becomes responsive once again.  By fail to launch, I mean nothing happens until KMM becomes responsive then my application launches normally..

Also while working, if I collapse the sections in accounts and scheduled transactions, navigating is noticeably quicker.

This is a little profiling I just did:
Launching KMM and loading the database: extremely quick
Navigating: initially quick, becoming slower as I go
Opening a ledger: slow (8 seconds for a ledger with 45 transactions...)
Opening the config dialog box for a graph report: 8sec
Closing the report: 8 sec
Comment 7 Christian David 2016-05-05 08:07:15 UTC
I checked what we changed between the versions carefully. Unfortunately I could not find anything that could cause the behavior you have described. The developers have in mind that the database system has performance issues. If we have time, we will address that. However, please do not expect a change in near future. Maybe you should consider to change to a .kmy file.
Comment 8 lp.allard.1 2016-05-05 22:53:06 UTC
I performed a quick performance benchmark on the database server:

Running the test with following options:
Number of threads: 8

Doing OLTP test.
Running mixed OLTP test
Doing read-only test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Threads started!
Time limit exceeded, exiting...
(last message repeated 7 times)
Done.

OLTP test statistics:
    queries performed:
        read:                            373534
        write:                           0
        other:                           53362
        total:                           426896
    transactions:                        26681  (444.61 per sec.)
    deadlocks:                           0      (0.00 per sec.)
    read/write requests:                 373534 (6224.50 per sec.)
    other operations:                    53362  (889.21 per sec.)

Test execution summary:
    total time:                          60.0103s
    total number of events:              26681
    total time taken by event execution: 479.8500
    per-request statistics:
         min:                                  0.83ms
         avg:                                 17.98ms
         max:                                 60.14ms
         approx.  95 percentile:              29.97ms

Threads fairness:
    events (avg/stddev):           3335.1250/12.10
    execution time (avg/stddev):   59.9812/0.00


I am relying on the DB devs to tell me if this is considered acceptable for usage with KMM..