Bug 360938

Running version 4.6.4 on Linux mint 17.3 (ubuntu based)
Installed from stock repos.

I just performed another payment to the credit card account (from my chequing account) which consists in a simple transfer, and while the memo's content of the transaction is FINE in the transaction in the chequing ledger, it is NOT fine in the transaction in the credit card ledger.

This is a serious bug.

Can you try to upgrade to a more recent version (e.g. 4.7.2) and see if the problem persists?

I am not off to a good start.  First I downloaded the sources of 4.7.2 then followed the instructions of the README.cmake file which mentions that I should run "sudo apt-get build-dep kmymoney" since I am on ubuntu (mint)..  The output was

Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to find a source package for kmymoney

Then I tried to manually start the cmake compilation process and got :

CMake Error at CMakeLists.txt:72 (find_package):
  By not providing "FindQGpgme.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "QGpgme", but
  CMake did not find one.

  Could not find a package configuration file provided by "QGpgme" with any
  of the following names:

    QGpgmeConfig.cmake
    qgpgme-config.cmake

  Add the installation prefix of "QGpgme" to CMAKE_PREFIX_PATH or set
  "QGpgme_DIR" to a directory containing one of the above files.  If "QGpgme"
  provides a separate development package or SDK, be sure it has been
  installed.
-- Configuring incomplete, errors occurred!

I am not sure how to proceed since I am not using KDE as my DE and I am not developing apps for KDE..

On 26/03/16 15:55, via KDE Bugzilla wrote:
> https://bugs.kde.org/show_bug.cgi?id=360938
>
> --- Comment #4 from lp.allard.1@gmail.com ---
> I am not off to a good start.  First I downloaded the sources of 4.7.2 then
> followed the instructions of the README.cmake file which mentions that I should
> run "sudo apt-get build-dep kmymoney" since I am on ubuntu (mint)..  The output
> was
>
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> E: Unable to find a source package for kmymoney
>
> Then I tried to manually start the cmake compilation process and got :
>
> CMake Error at CMakeLists.txt:72 (find_package):
>    By not providing "FindQGpgme.cmake" in CMAKE_MODULE_PATH this project has
>    asked CMake to find a package configuration file provided by "QGpgme", but
>    CMake did not find one.
>
>    Could not find a package configuration file provided by "QGpgme" with any
>    of the following names:
>
>      QGpgmeConfig.cmake
>      qgpgme-config.cmake
>
>    Add the installation prefix of "QGpgme" to CMAKE_PREFIX_PATH or set
>    "QGpgme_DIR" to a directory containing one of the above files.  If "QGpgme"
>    provides a separate development package or SDK, be sure it has been
>    installed.
> -- Configuring incomplete, errors occurred!
>
> I am not sure how to proceed since I am not using KDE as my DE and I am not
> developing apps for KDE..
>

KMyMoney is a KDE application, and to compile from source you would need 
to have a number of dependencies, "QGpgme" included.

However, if you are not familiar with KDE application development, it 
might be better for you to locate the Claydoh PPA, and you should find a 
recent KMyMoney release there.

Allan

Well.. I added the PPA you suggested and upgraded KMM to 4.7.2.  First thing I noticed, it crashes hard when I perform a consistency check.  I am storing my data on a MySQL backend, with about 7800 transactions (been using KMM since 2008).

Can it corrupt my data (I do have backups but I dont want to jeopardize my data either)?

What I could gather from the crash (happens every time, with all accounts).  I have tried with another account which is almost empty and still it crashes.

Application: KMyMoney (kmymoney), signal: Segmentation fault
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Current thread is 1 (Thread 0x7fb6d8a8f7c0 (LWP 26966))]

Thread 2 (Thread 0x7fb6b51bf700 (LWP 26970)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
#1  0x00007fb6d3cb67f4 in QWaitCondition::wait(QMutex*, unsigned long) () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#2  0x00007fb6d3caa0fa in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#3  0x00007fb6d3cb632f in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#4  0x00007fb6ce5a4182 in start_thread (arg=0x7fb6b51bf700) at pthread_create.c:312
#5  0x00007fb6d2c9047d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Thread 1 (Thread 0x7fb6d8a8f7c0 (LWP 26966)):
[KCrash Handler]
#5  0x00007fb6d3cf99eb in QString::operator==(QString const&) const () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#6  0x00007fb6d841c243 in MyMoneyFile::consistencyCheck() () from /usr/lib/libkmm_mymoney.so.4
#7  0x0000000000465996 in _start ()

Aside from doing more backups :-) the first thing I would do is to see if it also crashes using a file instead of a database.  If it works with a file, but crashes with the database, that helps focus where to look for the problem.  You might also try testing with a newly created (and thus essentially empty) file and database.  That would show whether the crash has anything to do with your data or not.

I did a few tests:

created a new flat file (.kmy) and consistency check does not crash with 4.7.2......good
created a new database file with 4.7.2, consistency check does not crash either..... good
Consistency check does crash on my two existing pre-4.7.2 databases

I saved one of the crashing databases (the smaller one) to a flat file, then ran the consistency check.  First when saving, it said that there was some 27 errors that were fixed, then saved.

I tried to do the same thing with the larger DB, it crashes when saving to a .kmy file.  I suspect there are really lots of errors in that DB. 

Now how can I fix those without hosing the entire DB?

I'm pretty much guessing here, but if you can capture the 27 errors that were fixed in the smaller version, can you make those corrections manually on the larger version? If so, you might get farther in then saving the larger one as a file.  

Am I correct that it crashes either running the consistency check on the larger database, and also crashes trying to save that database as a file, probably because it runs the consistency check.  If so, I would try to figure out from the backtrace (perhaps running in a debugger?) and try to trap the error, so although it may not correct the error, it wont just abort.  That may also show you what it is choking on, and you might be able to fix it manually.  I don't know if the Claydoh repository includes a version with debug symbols, but if so, that would produce more useful backtraces and be easier to run in a debugger.

Hello Jack,  I was too quick on the "OK button" and did not have time to carefully read what was fixed when saving to a flat file.....

I'd like to try to manually fix whats wrong in the large database but where to start!?

It does run a consistency check automatically when saving to a file.

I setup a copy of the problematic DB in MySQL (replicated the entire DB).  Strange thing, the original and copy have different sizes! Weird.  Anyways, now I can mess with the copy as we need to find the culprit.

For the debugger, you lost me there!

I just noticed:  the tables "kmmKeyValuePairs" are 64kb in the copied DB, and 2MB in the original DB, and tables "kmmSplits" are 4.9MB in the original DB and 4MB in the copied DB...

Well, if you don't have a backup copy of the small database before fixing, then capturing the fixes is no longer an option.

Different sizes of databases containing the same content does not surprise me too much - there are lots of local configuration issues like block size, minimum table size, ... (I'm mainly guessing, but I do have some database background.)

Unfortunately, it sounds like running a debugger would be too difficult for you.  The simple explanation is that you run the debugger, and it runs KMyMoney.  If it has the right information (source code and debugging symbols compiled in) then you can watch how it runs in the source code, and can set breakpoints (places it just stops for further instructions instead of just going on.)  It can be very useful for a developer, as you can also examine the value of variables, so if you know where it blows up, you can set a breakpoint a few lines earlier, or before it calls that function, and explore the state of things right before the problem.  Unfortunately, it is not something to learn quickly.  

One thing I just thought of - can you reinstall an earlier  version, where you were not getting the crash?  If so, you might be able to save as a file, or even as an anonymized file.  Then you could try the new version, or submit the anonymized file to the developers.  (An anonymized file changes all account numbers and all actual financial values - changing them to random values.)

I suppose it would be helpful if we could replicate the consistency checks as sql queries, but I'm not sure how difficult that would be.  Maybe one of the developers who works specifically on the database back end can make a suggestion.

Jack,  I managed to copy my "production" DB to a "testing" copy and performed a check for consistency.  For whatever reason, it worked, and the output is rather scary:

  * Opening date of Account 'Opening Balances' cannot be changed to support transaction 'T000000000000000013' post date.
  * Transaction 'T000000000000000013' has a post date '01/01/09' before one of the referenced account's opening date.
    Referenced accounts: Opening Balances
    The post date was not updated to '15/02/09'.
  * Transaction 'T000000000000001390' has a post date '01/01/09' before one of the referenced account's opening date.
    Referenced accounts: Opening Balances
    The post date was not updated to '15/02/09'.
  * Transaction 'T000000000000000010' post date '2009-02-15' is older than opening date '2015-01-25' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000016' post date '2009-02-17' is older than opening date '2009-02-18' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000019' post date '2009-02-21' is older than opening date '2009-02-22' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000019' post date '2009-02-21' is older than opening date '2009-02-22' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000040' post date '2009-02-26' is older than opening date '2016-02-27' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.

 [cropped about 40 similar messages to the aboves]

    Account opening date updated.
  * Transaction 'T000000000000000210' post date '2009-05-17' is older than opening date '2010-03-05' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000232' post date '2009-05-23' is older than opening date '2014-05-11' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000245' post date '2009-05-30' is older than opening date '2009-05-31' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000281' post date '2009-06-09' is older than opening date '2009-06-10' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000291' post date '2009-06-14' is older than opening date '2015-01-25' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000401' post date '2009-07-23' is older than opening date '2015-02-01' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000422' post date '2009-07-28' is older than opening date '2015-01-18' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000424' post date '2009-07-29' is older than opening date '2015-01-25' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000449' post date '2009-08-02' is older than opening date '2015-01-25' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000481' has a post date '13/08/09' before one of the referenced account's opening date.
    Referenced accounts: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    The post date was not updated to '23/10/09'.
  * Transaction 'T000000000000000482' post date '2009-08-15' is older than opening date '2015-01-31' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000492' post date '2009-08-22' is older than opening date '2011-05-01' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000496' post date '2009-08-24' is older than opening date '2016-02-27' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000497' post date '2009-08-24' is older than opening date '2015-01-25' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000497' post date '2009-08-24' is older than opening date '2015-01-31' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000577' post date '2009-09-18' is older than opening date '2015-01-25' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Opening date of Account 'XXXXXX' cannot be changed to support transaction 'T000000000000000622' post date.
  * Transaction 'T000000000000000622' has a post date '10/10/09' before one of the referenced account's opening date.
    Referenced accounts: XXXXXXXXXXXXXXXXXx
    The post date was not updated to '23/10/09'.
  * Transaction 'T000000000000000630' post date '2009-10-13' is older than opening date '2015-01-25' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000000630' post date '2009-10-13' is older than opening date '2015-01-31' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.

 [cropped about 50 similar messages to the aboves]

  * Transaction 'T000000000000003065' post date '2012-02-17' is older than opening date '2012-04-16' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000003068' post date '2012-02-17' is older than opening date '2016-02-27' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000003068' has a post date '17/02/12' before one of the referenced account's opening date.
    Referenced accounts: XXXXXXXXXXXXXXXXXXX
    The post date was not updated to '27/02/16'.
  * Transaction 'T000000000000006609' post date '2012-02-17' is older than opening date '2016-02-27' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000003092' post date '2012-02-29' is older than opening date '2012-04-10' of account 'XXXXXXXXXXXXXXXXX'.
    Account opening date updated.
  * Transaction 'T000000000000003106' has a post date '02/03/12' before one of the referenced account's opening date.
    Referenced accounts: XXXXXXXXXXXXXXXXXXXXXx

 [cropped about 20 similar messages to the aboves]

Finished: 122 problems corrected. 111 problems still present.

Then I did another CC on the DB and this time it reported identical output but at the bottom:

Finished: 0 problems corrected. 111 problems still present.

Im not sure how to proceed from there... Why so much errors about account creation dates and such?  Previously I never had errors like that, but I must be honest, last month I did a lot of cleanup in the database (renamed accounts, changed transactions categories, created and deleted categories and reassigned transactins to the categories, etc....)

On 4/2/2016 1:14 PM, via KDE Bugzilla wrote:
> Then I did another CC on the DB and this time it reported identical output but
> at the bottom:
>
> Finished: 0 problems corrected. 111 problems still present.
That is not really as bad as it looks.  All those errors about account 
opening dates are from recent changes to the program, so you would not 
have seen them previously.  However, I don't think any of these errors 
should cause a crash.  The first step is to recover your data and let 
you keep working.  After that, if saved a copy of the bad database, then 
at a lower priority, it might be possible to find the source of the crash.

The issue is that an account should not have any transaction with a date 
earlier than the opening date of the account.  In KMM, payees and 
categories are internally treated as accounts, and when they are 
created, their opening date is the date they were created, not the date 
of the first transaction.  This is where most of these issues come from.

As you saw, KMM was able to fix most of those errors by altering the 
opening date of the account.  For the others, you can try to manually 
edit the account and change the opening day to be early enough.  In 
general, the best way I have found to do this is to open the ledger for 
the target account, then click the edit account icon (if you have it 
displayed) or else select the Account/Edit menu.  It seems the "Opening 
Balances" account is a bit difficult to find - but the first transaction 
in the ledger for most other accounts is the initial transfer of the 
opening balance for that account, so you can right click on the 
transaction, and there should be a choice "Go to 'Opening Balances"'.

OK I managed to fix most of the errors CC threw at me.  There are still some errors (6) which is much better than 120...

Another thing, theres a "Equity" account with a sub-account named "Opening balances".  This was not there with KMM 4.6.2.. What is this account?  Some (4/6) of the errors left are related to this account...  Knowing what it does and its purpose will help me fix the remaining errors.

For now, I can save the DB to a file (.kmy) and perform CC as I want, without crashes!!!

Next time I see a seg fault I will try to setup the debugger...

Thanks!!