Bug 420761

I can confirm this bug and add my observations.

Immediately before upgrading from F31 to F32 on one of my systems, I compiled and installed KMM from the master branch.

After upgrading my system to F32, I again compiled and installed KMM from the master branch. Then I saw this bug report so I opened KMM and found the cleared amount did not match the balance as expected in at least one checking account and one savings account.

I then opened the exact same KMM file on another system that was still running F31 and a recently compiled version of KMM. The cleared amount matched the balance as expected.

I went back to the system running F32 and removed the version of KMM compiled in F32 and reinstalled the version of KMM that was compiled in F31 immediately before the upgrade and the cleared amounts went back to normal.

It appears that this bug is being caused by the compile process on F32.

Fedora32 ships with gcc-10, possible that's relevant.

To confirm, you're saying both the fedora packaged and local-build behaves the same way on f32?

I can only comment on the local build. I have not tried the package provided by Fedora.

Unfortunately that didn't answer my question.  Does the local build on f32 work ok or not?

See my first comment and let me know what question you still need answered.

It wasn't clear to me, that's why I asked for clarification.  My guess is that your local build on f32 was bad too, which supports my theory that this is related to building with gcc 10 somehow.  Either a compiler bug, or a code issue exposed by gcc10

Do the unit tests show different results or do they pass on both F31 and F32?

I guess I was not clear. I will try again.

I did not test the package provided with F32.

1) F31 & KMM compiled on F31: NO BUG

2) F32 & KMM Compiled on F32: BUG

3) F32 & KMM Compiled on F31: NO BUG

I assume KMM from Fedora for F32 is compiled with F32 so it probably has the same bug as option 2 above. I will download the latest release version and test it on F32.

running tests on f31:

The following tests FAILED:
          1 - appstreamtest (Failed)
         38 - qsqlcipher-test (Failed)

1 is a false positive
      Start  1: appstreamtest
 1/42 Test  #1: appstreamtest .........................***Failed    0.06 sec
CMake Error at /usr/share/ECM/kde-modules/appstreamtest.cmake:32 (message):
  File '/usr/share/metainfo/org.kde.kmymoney.appdata.xml' does not exist.

Assumes kmymoney appstream file is already installed, instead of testing the one in the build directory

38/42 Test #38: qsqlcipher-test .......................***Failed    0.02 sec
********* Start testing of qsqlciphertest *********
Config: Using QtTest library 5.13.2, Qt 5.13.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 9.3.1 20200317 (Red Hat 9.3.1-1))
FAIL!  : qsqlciphertest::initTestCase() 'drivers.contains(sqlDriverName)' returned FALSE. ()
   Loc: [/var/tmp/kdecache-rdieter/BUILDROOT/kmymoney-5.0.8/kmymoney/plugins/sqlcipher/tests/qsqlcipher-test.cpp(42)]
PASS   : qsqlciphertest::cleanupTestCase()
Totals: 1 passed, 1 failed, 0 skipped, 0 blacklisted, 1ms
********* Finished testing of qsqlciphertest *********

Similar assumption, assuming sql driver in builddir is usable?


On f32,

The following tests FAILED:
          1 - appstreamtest (Failed)
          9 - mymoneyfile-test (Failed)
         23 - mymoneytransactionfilter-test (Failed)
         31 - reports-chart-test (Failed)
         33 - reports-pivottable-test (Failed)
         38 - qsqlcipher-test (Failed)

 9/42 Test  #9: mymoneyfile-test ......................***Failed    0.12 sec
********* Start testing of MyMoneyFileTest *********
Config: Using QtTest library 5.14.2, Qt 5.14.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 10.0.1 20200328 (Red Hat 10.0.1-0.11))
...
FAIL!  : MyMoneyFileTest::testCountTransactionsWithSpecificReconciliationState_transactionWithUnwantedReconcileState() Compared values are not the same
   Actual   (m->countTransactionsWithSpecificReconciliationState(accountId, eMyMoney::TransactionFilter::State::NotReconciled)): 1
   Expected (0)                                                                                                                : 0
   Loc: [/builddir/build/BUILD/kmymoney-5.0.8/kmymoney/mymoney/tests/mymoneyfile-test.cpp(2345)]
...
FAIL!  : MyMoneyFileTest::testClearedBalance() Compared values are not the same
   Loc: [/builddir/build/BUILD/kmymoney-5.0.8/kmymoney/mymoney/tests/mymoneyfile-test.cpp(2543)]

23/42 Test #23: mymoneytransactionfilter-test .........***Failed    0.10 sec
********* Start testing of MyMoneyTransactionFilterTest *********
Config: Using QtTest library 5.14.2, Qt 5.14.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 10.0.1 20200328 (Red Hat 10.0.1-0.11))
...
FAIL!  : MyMoneyTransactionFilterTest::testMatchTransactionState() Compared values are not the same
   Actual   (filter.matchingSplits(transaction).size()): 4
   Expected (1)                                        : 1
   Loc: [/builddir/build/BUILD/kmymoney-5.0.8/kmymoney/mymoney/tests/mymoneytransactionfilter-test.cpp(454)]
...
FAIL!  : MyMoneyTransactionFilterTest::testMatchTransactionTypeAllTypes() '!filter.match(transaction)' returned FALSE. ()
   Loc: [/builddir/build/BUILD/kmymoney-5.0.8/kmymoney/mymoney/tests/mymoneytransactionfilter-test.cpp(575)]
FAIL!  : MyMoneyTransactionFilterTest::testMatchTransactionTypeDeposits() Compared values are not the same
   Actual   (filter.matchingSplits(transaction2).size()): 1
   Expected (0)                                         : 0
   Loc: [/builddir/build/BUILD/kmymoney-5.0.8/kmymoney/mymoney/tests/mymoneytransactionfilter-test.cpp(623)]
FAIL!  : MyMoneyTransactionFilterTest::testMatchTransactionTypePayments() Compared values are not the same
   Actual   (filter.matchingSplits(transaction).size()): 2
   Expected (0)                                        : 0
   Loc: [/builddir/build/BUILD/kmymoney-5.0.8/kmymoney/mymoney/tests/mymoneytransactionfilter-test.cpp(663)]
FAIL!  : MyMoneyTransactionFilterTest::testMatchTransactionTypeTransfers() Compared values are not the same
   Actual   (filter.matchingSplits(transaction).size()): 3
   Expected (0)                                        : 0
   Loc: [/builddir/build/BUILD/kmymoney-5.0.8/kmymoney/mymoney/tests/mymoneytransactionfilter-test.cpp(724)]

31/42 Test #31: reports-chart-test ....................***Failed   10.09 sec
********* Start testing of ChartTest *********
Config: Using QtTest library 5.14.2, Qt 5.14.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 10.0.1 20200328 (Red Hat 10.0.1-0.11))
PASS   : ChartTest::initTestCase()
libEGL warning: MESA-LOADER: failed to open swrast (search paths /usr/lib64/dri)

(probably harmless, buildsystem cannot use gui)

33/42 Test #33: reports-pivottable-test ...............***Failed    0.15 sec
********* Start testing of reports::PivotTableTest *********
Config: Using QtTest library 5.14.2, Qt 5.14.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 10.0.1 20200328 (Red Hat 10.0.1-0.11))
...
FAIL!  : reports::PivotTableTest::testAdvancedFilter() 'networth_f4.m_grid["Asset"].m_total[eActual][11] == moCheckingOpen + moChild' returned FALSE. ()
   Loc: [/builddir/build/BUILD/kmymoney-5.0.8/kmymoney/plugins/views/reports/core/tests/pivottable-test.cpp(671)]

(Temporary) link to full f32 build.log,
https://kojipkgs.fedoraproject.org//work/tasks/7321/43947321/build.log

Had a quick glimpse at the log. Looks like the failing MyMoneyFile test is related to/dependent on the failing MyMoneyTransactionFilter tests.

@Rex: as a side note, bcooksley added some magic to run the graphical chart tests on the KDE CI infra. I have no idea what he did in detail, though.

I have the same issue. It looks like the first line in the register is not taken in account. When adding a line with a value of zero just before, and marking it as cleared, the result is good.

Nice catch, hopefully that will help find the cause.

I'm a little concerned that this might be affecting other calculation that are not as obvious.

Can someone provide a simply kmy file showing the problem?  Is the reconciling process necessary to show the problem, or is it just seen opening the file?

I'm on Gentoo, but have gcc-10 installed, and KMM seems to work fine for me compiled with gcc-10 from git head 5.0 branch.  Also, for me, only the appimage test fails.  However, I'm using ninja instead of make and that may well make a difference.  I've just kicked off a build using make, and will report on the results.

I also suspect it might matter whether the various tools used were themselves compiled with gcc-9 or gcc-10.

Compiling with make (4.3) and gcc (10.1.0) gives no test errors at all.

The suggested fix did not make fix to my accounts. Adding a zero transaction before the first transaction and marking it cleared or reconciled did not change the Cleared value at the bottom.

In my savings account, the Cleared value at the bottom of the page is equal to the account balance as long as everything is marked cleared or reconciled in the F31 version of KMM. In the F32 version, it's the cleared value minus the opening balance amount in the account which was in 2003. This results in a cleared amount that is a very large number. Again, adding a zero amount transaction before the opening balance transaction did not change this, which is surprising since subtracting zero from the account balance should work. I have not saved the file which may be necessary for this to reset.

My checking account is not that straight forward, I have not be able to figure out which numbers are being used to get the cleared amount and adding a zero amount transaction at he beginning changed the cleared amount but not by an amount I can make sense of. it makes no difference if the new zero amount first transaction is marked cleared, reconciled or unmarked.

Changing the cleared/reconciled status of the first couple of transactions in my checking account is causing all kinds of strange behavior. At one point the cleared amount at the bottom match the account balance as it should but ti's not consistent and I can't get back to that point. At one point I uncleared a $500 transaction that used to be the first transaction and the cleared amount changed by about $40,000. It should be around $1,400.

For what it's worth, the savings account cleared amount is a huge negative number and the checking is a huge positive number.

When I switch back to the last version of KMM that I compiled on F31 the problem goes away. To be clear, this is all happening on the same F32 machine that I upgraded last week. I am able to remove the F32 compiled version and reinstall the F31 version with my KMM build scripts.

I did make a test file and took 2 screen captures from it to show the difference between the F31 cleared amount and the F32 cleared amount. of course they don't match the theory that it's the balance minus the first transaction.

Created attachment 128315 [details]
Compiled on F31

Created attachment 128316 [details]
Compiled on F32

Created attachment 128317 [details]
F32 Test File

Brendan - I don't see any opening balance amounts in either account in the attached kmy file.  I don't even see an opening balance account, so I don't know how you think the opening balance plays any role here at all.  If the version of KMM I just compiled with gcc-10, I see what you show in the F31 picture.

I didn't mean to say opening balance, I meant to say the first transaction. In my case it was called "Opening Balance Adjustment" causing me to write the wrong thing. Sorry for the confusion.

I just did a test with F31 and F32. I changed the status of a $2.36 deposit from Reconciled to not marked. In KMM compiled on F31 the cleared amount changed by $2.36. In the F32 compiled version the cleared amount changed by $5,497.04. This happens with payments or deposits from my savings account.

When I do the same thing for a $2.50 deposit, the cleared amount changes by 14,616.78. 

When I do the same thing on an older $2.50 deposit, the cleared amount changes by 138,203.33.

Very odd.

The cleared amount is a positive number in KMM/F31. When I unclear a deposit the cleared amount drops as expected. In KMM/F32 the cleared amount is about 10 times greater in absolute value but it's negative. When I unclear a deposit, it gets smaller (bigger negative number) so at least it's going in the right direction.

I added a couple of transactions to my test file. There are 5 transactions. Changing the status of the first 3 has no affect on the cleared amount. The most recent 2 transactions work as expected. The more things I try the stranger this gets.

Note that F32 is up to date so if this is a bug in the compiler it hasn't been fixed yet.

I'm pretty much at a loss here.  I don't see any evidence of this, but I'll ask anyway for you to confirm there are no currency conversions involved anywhere here, right?  Everything in US Dollars?

Also, just to avoid confusion of changing the displayed order of transactions when changing state, try to make each transaction on a different date.  It shouldn't make any difference, but we're already taking about stuff that doesn't make any sense.

Created attachment 128584 [details]
attachment-14636-0.html

On top of what Jack suggested, may I also ask that you perform the same
tests using Skrooge? If you confirm it has the same problem it would point
to LibAlkimia, which is a library that both share.

On Mon, May 18, 2020 at 2:33 PM Jack <bugzilla_noreply@kde.org> wrote:

> https://bugs.kde.org/show_bug.cgi?id=420761
>
> --- Comment #22 from Jack <ostroffjh@users.sourceforge.net> ---
> I'm pretty much at a loss here.  I don't see any evidence of this, but
> I'll ask
> anyway for you to confirm there are no currency conversions involved
> anywhere
> here, right?  Everything in US Dollars?
>
> Also, just to avoid confusion of changing the displayed order of
> transactions
> when changing state, try to make each transaction on a different date.  It
> shouldn't make any difference, but we're already taking about stuff that
> doesn't make any sense.
>
> --
> You are receiving this mail because:
> You are the assignee for the bug.

If you decide to test Skrooge as well, please make sure it is compiled with the same versions of LibAlkimia and libMPIR/libGMP as KMyMoney.

In my test file, I made all the dates different. The first 3 were the same date and the last 2 were today. Once I mde them all different, only the first transaction ignored the cleared status.

Testing on Skrooge will have to wait. I'll have to assume the Fedora package of Skrooge uses the same versions of the libraries that I'm using when I compile KMM.

I know this is all pretty confusing so let me repeat what I said earlier.

KMM does not have a problem when I run the last version of KMM that I compiled on F31 immediately prior to upgrading to F32, even when I'm running the previously compiled version of KMM on F32. My KMM compile scripts have 2 steps, one that downloads and builds from fresh source every time and another that removes the existing version and installs any version that I still have in my build area. This makes it trivial for me to switch between any compiled version that has not be deleted from my system.

I'm not sure what this means but I thought it might be important.

Git commit a8386dc173f51fd56c5c9e42fef5fa5f2df477b7 by Thomas Baumgart.
Committed on 21/05/2020 at 11:31.
Pushed by tbaumgart into branch 'master'.

Fix transaction filter due to compiler issue with g++ 10

g++ version 10 seems to have a problem with our ancient union and struct
based bitmap handling.

Changing it to be based on QFlags solves the problem.

M  +61   -65   kmymoney/mymoney/mymoneytransactionfilter.cpp
M  +15   -16   kmymoney/mymoney/mymoneytransactionfilter.h
M  +3    -3    kmymoney/plugins/sql/mymoneystoragesql.cpp

https://invent.kde.org/office/kmymoney/commit/a8386dc173f51fd56c5c9e42fef5fa5f2df477b7

Git commit 9e1e074d0a2232d6528b64567e7424956da70aee by Thomas Baumgart.
Committed on 21/05/2020 at 11:32.
Pushed by tbaumgart into branch '5.0'.

Fix transaction filter due to compiler issue with g++ 10

g++ version 10 seems to have a problem with our ancient union and struct
based bitmap handling.

Changing it to be based on QFlags solves the problem.
FIXED-IN: 5.0.9

(cherry picked from commit a8386dc173f51fd56c5c9e42fef5fa5f2df477b7)

M  +61   -65   kmymoney/mymoney/mymoneytransactionfilter.cpp
M  +15   -16   kmymoney/mymoney/mymoneytransactionfilter.h
M  +3    -3    kmymoney/plugins/sql/mymoneystoragesql.cpp

https://invent.kde.org/office/kmymoney/commit/9e1e074d0a2232d6528b64567e7424956da70aee

Works for me. Thanks Thomas.

Thanks, can also confirm relevant autotests now pass.

Patched update submitted for Fedora 32:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-a3cb50e3dd

I wish I could confirm the solution for my bank account.  Three out of four accounts do perfectly reconcile but still not the one I reported in the beginning.

Any idea?

Just checked - my version is still 5.0.8 .  Will 5.0.9 be released to the Fedora project repository any time soon?

Downloaded latest AppImage KMyMoney-5.0.8-9e1e074-x86_64.AppImage and tested my ref. bank account - reconciles and tallies OK.

Will wait for next repository update (last one was yesterday 20200521).  Hope to get it soon.

Thanks for the fix!

Re: comment 31
"Will 5.0.9 be released to the Fedora project repository any time soon?"

When kmymoney 5.0.9 is released, yes.  At this moment, it is not (yet).

If you have any interest in compiling from source I can send you a link to my build scripts. Once you have the dependencies installed it's very easy to install from git or the released tars.

Thanks Brendan - I'll consider that.  For the moment I am fine with the app image I downloaded.