Bug 312692

Here's more detailed info about the failing test:
---
20: QDEBUG : MyMoneyDatabaseMgrTest::testAddTransactions() Error in function const QMap<QString, MyMoneyTransaction> MyMoneyStorageSql::fetchTransactions(const QString&, const QString&, bool) const : reading Splits
20: Driver = QSQLITE, Host = localhost, User = marko, Database = /var/folders/Dn/Dna0JhQ5HL4lXLJ0U0EATk+++TM/-Tmp-/kmm_test_driver
20: Driver Error: 
20: Database Error No -1: 
20: Text:  
20: Error type 0
20: Executed:  WHERE txType = 'N'  AND transactionId IN (select distinct transactionId from kmmSplits  where txType = 'N' and accountId in ( 'A000006')) ORDER BY transactionId, splitId;
20: Query error No -1: No query Unable to fetch row
20: Error type 1
20: QFATAL : MyMoneyDatabaseMgrTest::testAddTransactions() Received signal 11
20: FAIL!  : MyMoneyDatabaseMgrTest::testAddTransactions() Received a fatal error.
20:    Loc: [Unknown file(0)]

I turns out that the error gets thrown in creation of 'transactionsList' in kmymoney/mymoney/storage/mymoneydatabasemgrtest.cpp in line 707 of git version 79544b5ed8152436f407e492d38658ee36304a89:
---
    //QMap<QString, QString>::ConstIterator it_k;
    MyMoneyTransactionFilter f;
->  QList<MyMoneyTransaction> transactionList(m->transactionList(f));
    QList<MyMoneyTransaction>::ConstIterator it_t(transactionList.constBegin());
---

Line 3202 sets up the query, and for some reason it is not getting the whole query, only the where clause. Line 3195 is where we get the reference for the table so we can later call selectAllString.

That may be getting optimized away on a Mac, since it works everywhere else. One option would be to get a copy of the object instead of a reference. If that works, we can convert MyMoneyDbTable to use a d-pointer and COW to make copying cheap.

There are probable other options that are not occurring to me right now.

Hi Fernando, thanks for your reply. Great that you spotted it, since I did not even notice the missing part of the query. :-)

Which file are you referring to with your line numbers?

(In reply to comment #3)
> Line 3202 sets up the query, ...
> Line 3195 is where we get the reference for ...

Neither mymoneydatabasemgrtest.cpp nor mymoneydatabasemgr.cpp have that many lines...

I am puzzled.

mymoneystoragesql.cpp has most of it. mymoneydatabasemgr.cpp is an interface like mymoneyseqaccessmgr.cpp. The main difference is that there is just one storage class, instead of "bin", "anon", and "xml".

I changed 
  const MyMoneyDbTable& ts = m_db.m_tables["kmmSplits"];
to
  const MyMoneyDbTable ts(m_db.m_tables["kmmSplits"]);
but the problem persists.

The worst thing is that I always only get this error thrown out:
---
20: mymoneydatabasemgrtest(20687,0x7fff7126ecc0) malloc: *** error for object 0x1032aa9e0: pointer being freed was not allocated
20: *** set a breakpoint in malloc_error_break to debug
1/1 Test #20: kmymoney-mymoneydatabasemgrtest ...***Exception: Other  1.59 sec
---
and nothing else.

Only once or twice I saw the DEBUG message from my comment #1.

Running the test in gdb didn't produce a stack trace either.

(My knowledge of running applications in gdb step by step is zero, so I haven't tried that yet.)

Wondering how to proceed...

Sorry about the delayed response...

When you ran the test in gdb, after it crashed, the command "where" should have given you a stack trace.

To try to get a better trace, you could try "break filename:line_number" and then use "step" to get into a function and "next" to execute the line but not step in. I will say that a GUI debugger tends to help this process.

The other option is using qDebug() to get a ton of information. There is already a DBG macro that is commented out, but could be changed to actually provide output. From there, you could add additional debugging statements.

I have used both GUI debuggers and the print-statement method, and which is the right answer depends on the problem at hand and the mood of the developer at the time.

So, now I have information with line number, eventually.
My Xcode wouldn't want to help me with debugging in an IDE, so that I had to do all this manually in gdb:
----
.
.
.
PASS   : MyMoneyDatabaseMgrTest::testReparentAccount()
mymoneydatabasemgrtest(55218,0x7fff7126ecc0) malloc: *** error for object 0x103155820: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug

Program received signal SIGABRT, Aborted.
0x00007fff8a4720b6 in __kill ()
(gdb) where
#0  0x00007fff8a4720b6 in __kill ()
#1  0x00007fff8a5129f6 in abort ()
#2  0x00007fff8a50162d in szone_error ()
#3  0x00007fff8a42cbbe in szone_free ()
#4  0x000000010555d272 in sqlite3_free ()
#5  0x00000001055a9fd2 in sqlite3Select ()
#6  0x00000001055c9f2a in sqlite3Parser ()
#7  0x00000001055cccf7 in sqlite3RunParser ()
#8  0x00000001055cd7eb in sqlite3Prepare ()
#9  0x00000001055cd90d in sqlite3LockAndPrepare ()
#10 0x00000001055cda2d in sqlite3Prepare16 ()
#11 0x00000001037844da in QSQLiteResult::prepare ()
#12 0x00000001015f8fa3 in QSqlResult::savePrepare ()
#13 0x00000001015eac90 in QSqlQuery::prepare ()
#14 0x000000010002ffef in QFlags [inlined] () at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:4090
#15 0x000000010002ffef in QFlags [inlined] () at /opt/macports-test/Library/Frameworks/QtCore.framework/Versions/4/Headers/qglobal.h:2310
#16 0x000000010002ffef in MyMoneyStorageSql::readKeyValuePairs (this=0x10329b370, kvpType=@0x7fff5fbfcdb0, kvpIdList=@0x7fff5fbfcec0) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:4091
#17 0x0000000100074d73 in ~QString [inlined] () at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:3266
#18 ~QString [inlined] () at /opt/macports-test/Library/Frameworks/QtCore.framework/Versions/4/Headers/qstring.h:880
#19 0x0000000100074d73 in MyMoneyStorageSql::fetchTransactions (this=0x10329b370, tidList=@0x7fff5fbfd590, dateClause=@0x7fff5fbfd580) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:3266
#20 0x0000000100078f2c in ~QString [inlined] () at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:3467
#21 ~QString [inlined] () at /opt/macports-test/Library/Frameworks/QtCore.framework/Versions/4/Headers/qstring.h:880
#22 0x0000000100078f2c in MyMoneyStorageSql::fetchTransactions (this=0x10329b370, filter=@0x7fff5fbfdd50) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:3467
#23 0x0000000100092956 in MyMoneyDatabaseMgr::transactionList (this=0x103152870, list=@0x7fff5fbfe090, filter=@0x7fff5fbfdd50) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneydatabasemgr.cpp:945
#24 0x000000010008a2e3 in MyMoneyDatabaseMgr::transactionList (this=<value temporarily unavailable, due to optimizations>, filter=<value temporarily unavailable, due to optimizations>) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneydatabasemgr.cpp:934
#25 0x0000000100021fab in QListData::begin () at /opt/macports-test/Library/Frameworks/QtCore.framework/Versions/4/Headers/qlist.h:707
#26 0x0000000100021fab in QList<MyMoneyTransaction>::constBegin () at /opt/macports-test/Library/Frameworks/QtCore.framework/Versions/4/Headers/qlist.h:269
#27 0x0000000100021fab in MyMoneyDatabaseMgrTest::testAddTransactions (this=0x7fff5fbff010) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneydatabasemgrtest.cpp:708
#28 0x000000010177f32c in QMetaMethod::invoke ()
#29 0x00000001017817dc in QMetaObject::invokeMethod ()
#30 0x00000001015bb30f in QTest::qInvokeTestMethodDataEntry ()
#31 0x00000001015bc02c in QTest::qInvokeTestMethod ()
#32 0x00000001015bc7ca in QTest::qExec ()
#33 0x0000000100008c3c in main (argc=1, argv=0x7fff5fbff090) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneydatabasemgrtest.cpp:26
---

The above probably corresponds to comment #1.

After the previous gdb run failed like this I simply reran it and here's what I got then:
---
.
.
.
PASS   : MyMoneyDatabaseMgrTest::testReparentAccount()

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000003000000
0x0000000101678d20 in QHashData::free_helper ()
(gdb) where
#0  0x0000000101678d20 in QHashData::free_helper ()
#1  0x000000010008811d in MyMoneyDbTable::~MyMoneyDbTable (this=0x100111780) at qhash.h:283
#2  0x0000000100071cef in MyMoneyStorageSql::readSplit (this=0x10321cbd0, s=@0x7fff5fbfc9f0, q=@0x7fff5fbfcf50) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:3549
#3  0x0000000100074793 in MyMoneyStorageSql::fetchTransactions (this=0x10321cbd0, tidList=@0x7fff5fbfd590, dateClause=@0x7fff5fbfd580) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:3251
#4  0x0000000100078f2c in ~QString [inlined] () at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:3467
#5  ~QString [inlined] () at /opt/macports-test/Library/Frameworks/QtCore.framework/Versions/4/Headers/qstring.h:880
#6  0x0000000100078f2c in MyMoneyStorageSql::fetchTransactions (this=0x10321cbd0, filter=@0x7fff5fbfdd50) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneystoragesql.cpp:3467
#7  0x0000000100092956 in MyMoneyDatabaseMgr::transactionList (this=0x103198ce0, list=@0x7fff5fbfe090, filter=@0x7fff5fbfdd50) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneydatabasemgr.cpp:945
#8  0x000000010008a2e3 in MyMoneyDatabaseMgr::transactionList (this=<value temporarily unavailable, due to optimizations>, filter=<value temporarily unavailable, due to optimizations>) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneydatabasemgr.cpp:934
#9  0x0000000100021fab in QListData::begin () at /opt/macports-test/Library/Frameworks/QtCore.framework/Versions/4/Headers/qlist.h:707
#10 0x0000000100021fab in QList<MyMoneyTransaction>::constBegin () at /opt/macports-test/Library/Frameworks/QtCore.framework/Versions/4/Headers/qlist.h:269
#11 0x0000000100021fab in MyMoneyDatabaseMgrTest::testAddTransactions (this=0x7fff5fbff010) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneydatabasemgrtest.cpp:708
#12 0x000000010177f32c in QMetaMethod::invoke ()
#13 0x00000001017817dc in QMetaObject::invokeMethod ()
#14 0x00000001015bb30f in QTest::qInvokeTestMethodDataEntry ()
#15 0x00000001015bc02c in QTest::qInvokeTestMethod ()
#16 0x00000001015bc7ca in QTest::qExec ()
#17 0x0000000100008c3c in main (argc=1, argv=0x7fff5fbff090) at /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/kmymoney/mymoney/storage/mymoneydatabasemgrtest.cpp:26
---

It looks like this whole error is caused by my somehow messed up Qt4 configuration with regard to SQLITE!

I could prove that the error actually occurring is NOT an incomplete SQL statement [1].
It rather seems so that the rare error output at the top of comment 9 is closer to the real problem here.


[1] https://git.reviewboard.kde.org/r/108630/

I agree with your analysis that it is not related to the query. At least, not anymore. That probably makes some things easier, but I am not sure exactly what is going on. The post you made on reviewboard should make it a lot easier to debug.

Well, the problem with my reviewboard contribution is that even with all the debug messages included it doesn't seem to be so easy to really locate the reason for this strange behavior.

I have made updates to my initial post which show that the exception shows itself in different ways...

Created attachment 76799 [details]
May fix splits crashes on Mac

This may fix the issue. It turns out that readSplit() was clearing the splitId, which is only needed in one case, so swapping the order of a couple of calls may fix it. It passes all the unit tests, so it should be safe to use. The splitId fields for the tags query is being pulled from the kmmSplits query anyway.

Hmmm, unfortunately this didn't solve the issue on Mac OS X.
:-(
The same error still occurs.

I ran valgrind on this test and it failed right away. Looking into valgrind's source code reveals the following comment:
---
   /* This may fail if the client stack is busted; if that happens,
      the whole process will exit rather than simply calling the
      signal handler. */
---
and that is what actually happens, BUT unfortunately I realized after all that this is failing for ALL tests!!! :-( 
---
$ valgrind --leak-check=full  /opt/macports-test/bin/ctest --force-new-ctest-process -R databasemgrtest -V
==52857== Memcheck, a memory error detector
==52857== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==52857== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==52857== Command: /opt/macports-test/bin/ctest --force-new-ctest-process -R databasemgrtest -V
==52857== 
UpdateCTestConfiguration  from :/Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/build/DartConfiguration.tcl
Test project /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/build
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 20
    Start 20: kmymoney-mymoneydatabasemgrtest

20: Test command: /Users/marko/WC/GIT/kmymoney.mymoneydatabasemgrtest/build/kmymoney/mymoney/storage/mymoneydatabasemgrtest.app/Contents/MacOS/mymoneydatabasemgrtest
20: Test timeout computed to be: 9.99988e+06

valgrind: m_signals.c:953 (handle_SCSS_change): Assertion 'ksa_old.sa_flags == skss_old.skss_per_sig[sig].skss_flags' failed.
==52861==    at 0x1380373F7: ???
==52861==    by 0x138037634: ???
==52861==    by 0x13804B175: ???
==52861==    by 0x13804D279: ???
==52861==    by 0x1380CC03B: ???
==52861==    by 0x1380A98C9: ???
==52861==    by 0x1380A7098: ???
==52861==    by 0x1380CE05F: ???

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable
==52861==    at 0x100A65032: __sigaction (in /usr/lib/libSystem.B.dylib)
==52861==    by 0x1002F7494: kwsysProcessCreate (in /opt/macports-test/bin/ctest)
==52861==    by 0x1002F88F2: cmsysProcess_Execute (in /opt/macports-test/bin/ctest)
==52861==    by 0x100026404: cmProcess::StartProcess() (in /opt/macports-test/bin/ctest)
==52861==    by 0x10007908B: cmCTestRunTest::ForkProcess(double, bool, std::vector<std::string, std::allocator<std::string> >*) (in /opt/macports-test/bin/ctest)
==52861==    by 0x100079874: cmCTestRunTest::StartTest(unsigned long) (in /opt/macports-test/bin/ctest)
==52861==    by 0x10007198B: cmCTestMultiProcessHandler::StartTestProcess(int) (in /opt/macports-test/bin/ctest)
==52861==    by 0x100072384: cmCTestMultiProcessHandler::StartTest(int) (in /opt/macports-test/bin/ctest)
==52861==    by 0x1000726B6: cmCTestMultiProcessHandler::StartNextTests() (in /opt/macports-test/bin/ctest)
==52861==    by 0x1000727D0: cmCTestMultiProcessHandler::RunTests() (in /opt/macports-test/bin/ctest)
==52861==    by 0x1000A0837: cmCTestTestHandler::ProcessDirectory(std::vector<cmsys::String, std::allocator<cmsys::String> >&, std::vector<cmsys::String, std::allocator<cmsys::String> >&) (in /opt/macports-test/bin/ctest)
==52861==    by 0x1000A1F79: cmCTestTestHandler::ProcessHandler() (in /opt/macports-test/bin/ctest)
==52861==    by 0x100012CDE: cmCTest::ProcessTests() (in /opt/macports-test/bin/ctest)
==52861==    by 0x1000200A7: cmCTest::Run(std::vector<std::string, std::allocator<std::string> >&, std::string*) (in /opt/macports-test/bin/ctest)
==52861==    by 0x10000AB0B: main (in /opt/macports-test/bin/ctest)


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

1/1 Test #20: kmymoney-mymoneydatabasemgrtest ...***Failed    0.11 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   1.28 sec

The following tests FAILED:
	 20 - kmymoney-mymoneydatabasemgrtest (Failed)
Errors while running CTest
==52857== 
==52857== HEAP SUMMARY:
==52857==     in use at exit: 327,984 bytes in 1,770 blocks
==52857==   total heap usage: 17,182 allocs, 15,412 frees, 2,197,080 bytes allocated
==52857== 
==52857== 16 bytes in 1 blocks are definitely lost in loss record 93 of 399
==52857==    at 0x100662ABD: malloc (vg_replace_malloc.c:274)
==52857==    by 0x1009800CF: __cxa_get_globals (in /usr/lib/libstdc++.6.0.9.dylib)
==52857==    by 0x10097FEA9: std::uncaught_exception() (in /usr/lib/libstdc++.6.0.9.dylib)
==52857==    by 0x10095F096: std::ostream::sentry::~sentry() (in /usr/lib/libstdc++.6.0.9.dylib)
==52857==    by 0x10095FBF9: std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) (in /usr/lib/libstdc++.6.0.9.dylib)
==52857==    by 0x10095FC57: std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*) (in /usr/lib/libstdc++.6.0.9.dylib)
==52857==    by 0x10000B583: cmCTest::Log(int, char const*, int, char const*) (in /opt/macports-test/bin/ctest)
==52857==    by 0x1000127D8: cmCTest::ProcessTests() (in /opt/macports-test/bin/ctest)
==52857==    by 0x1000200A7: cmCTest::Run(std::vector<std::string, std::allocator<std::string> >&, std::string*) (in /opt/macports-test/bin/ctest)
==52857==    by 0x10000AB0B: main (in /opt/macports-test/bin/ctest)
==52857== 
==52857== LEAK SUMMARY:
==52857==    definitely lost: 16 bytes in 1 blocks
==52857==    indirectly lost: 0 bytes in 0 blocks
==52857==      possibly lost: 0 bytes in 0 blocks
==52857==    still reachable: 320,465 bytes in 1,694 blocks
==52857==         suppressed: 7,503 bytes in 75 blocks
==52857== Reachable blocks (those to which a pointer was found) are not shown.
==52857== To see them, rerun with: --leak-check=full --show-reachable=yes
==52857== 
==52857== For counts of detected and suppressed errors, rerun with: -v
==52857== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 3 from 3)
---

This isn't an issue anymore in version 4.12.4 of KDE software.