Bug 282763 - Kmymoney crashes while importing .csv
Summary: Kmymoney crashes while importing .csv
Status: RESOLVED FIXED
Alias: None
Product: kmymoney
Classification: Unclassified
Component: general (show other bugs)
Version: 4.6.0
Platform: openSUSE RPMs Linux
: NOR crash (vote)
Target Milestone: ---
Assignee: allan
URL:
Keywords:
: 324991 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-09-25 21:09 UTC by Matthew
Modified: 2013-09-29 10:37 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Patch file for this bug (6.10 KB, patch)
2013-09-19 12:46 UTC, allan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matthew 2011-09-25 21:09:30 UTC
Application: kmymoney (4.6.0)
KDE Platform Version: 4.7.1 (4.7.1) "release 8"
Qt Version: 4.7.4
Operating System: Linux 2.6.37.6-0.7-desktop x86_64
Distribution: "openSUSE 11.4 (x86_64)"

-- Information about the crash:
- What I was doing when the application crashed:
I had an institution and account saved. I was running the csv importer and everything went fine until the actual import. After pressing import I get a dialog stating that row 1 doesnt have enough fields and I need to cancel or accept the inconsistancy. If I select anything except cancel then Kmymoney shuts down completly.

The crash can be reproduced every time.

-- Backtrace:
Application: KMyMoney (kmymoney), signal: Segmentation fault
[Current thread is 1 (Thread 0x7f5fa0689760 (LWP 11923))]

Thread 2 (Thread 0x7f5f86f9a700 (LWP 11931)):
#0  0x00007f5f9aeba503 in poll () from /lib64/libc.so.6
#1  0x00007f5f95b3f114 in ?? () from /lib64/libglib-2.0.so.0
#2  0x00007f5f95b3f650 in g_main_context_iteration () from /lib64/libglib-2.0.so.0
#3  0x00007f5f9c97f636 in QEventDispatcherGlib::processEvents (this=0x2a9d9f0, flags=<value optimized out>) at kernel/qeventdispatcher_glib.cpp:424
#4  0x00007f5f9c953c22 in QEventLoop::processEvents (this=<value optimized out>, flags=...) at kernel/qeventloop.cpp:149
#5  0x00007f5f9c953e35 in QEventLoop::exec (this=0x7f5f86f99de0, flags=...) at kernel/qeventloop.cpp:201
#6  0x00007f5f9c868be4 in QThread::exec (this=<value optimized out>) at thread/qthread.cpp:498
#7  0x00007f5f9c935358 in QInotifyFileSystemWatcherEngine::run (this=0x2af4730) at io/qfilesystemwatcher_inotify.cpp:248
#8  0x00007f5f9c86b4d5 in QThreadPrivate::start (arg=0x2af4730) at thread/qthread_unix.cpp:331
#9  0x00007f5f9da99a3f in start_thread () from /lib64/libpthread.so.0
#10 0x00007f5f9aec367d in clone () from /lib64/libc.so.6
#11 0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7f5fa0689760 (LWP 11923)):
[KCrash Handler]
#6  ref (this=0x7fffa5a7a050, other=...) at ../../src/corelib/arch/qatomic_x86_64.h:121
#7  QString::operator= (this=0x7fffa5a7a050, other=...) at tools/qstring.cpp:1358
#8  0x00007f5f8932da36 in CsvProcessing::processQifLine(QString&) () from /usr/lib64/kde4/kmm_csvimport.so
#9  0x00007f5f89332f9c in CsvProcessing::readFile(QString const&, int) () from /usr/lib64/kde4/kmm_csvimport.so
#10 0x00007f5f893335ec in CsvProcessing::importClicked() () from /usr/lib64/kde4/kmm_csvimport.so
#11 0x00007f5f8931c73c in CsvProcessing::qt_metacall(QMetaObject::Call, int, void**) () from /usr/lib64/kde4/kmm_csvimport.so
#12 0x00007f5f9c968e8f in QMetaObject::activate (sender=0x2b5eed0, m=<value optimized out>, local_signal_index=<value optimized out>, argv=0x7fffa5a7a620) at kernel/qobject.cpp:3287
#13 0x00007f5f9c111252 in QAbstractButton::clicked (this=<value optimized out>, _t1=false) at .moc/release-shared/moc_qabstractbutton.cpp:206
#14 0x00007f5f9be5721b in QAbstractButtonPrivate::emitClicked (this=<value optimized out>) at widgets/qabstractbutton.cpp:546
#15 0x00007f5f9be587db in QAbstractButtonPrivate::click (this=0x2b601c0) at widgets/qabstractbutton.cpp:539
#16 0x00007f5f9be58a4c in QAbstractButton::mouseReleaseEvent (this=0x2b5eed0, e=0x7fffa5a7afa0) at widgets/qabstractbutton.cpp:1121
#17 0x00007f5f9baf1e1c in QWidget::event (this=0x2b5eed0, event=0x7fffa5a7afa0) at kernel/qwidget.cpp:8295
#18 0x00007f5f9baa0264 in QApplicationPrivate::notify_helper (this=0xbfc0c0, receiver=0x2b5eed0, e=0x7fffa5a7afa0) at kernel/qapplication.cpp:4481
#19 0x00007f5f9baa909c in QApplication::notify (this=<value optimized out>, receiver=0x2b5eed0, e=0x7fffa5a7afa0) at kernel/qapplication.cpp:4042
#20 0x00007f5f9ecc1b76 in KApplication::notify(QObject*, QEvent*) () from /usr/lib64/libkdeui.so.5
#21 0x00007f5f9c9547cc in QCoreApplication::notifyInternal (this=0xbf70b0, receiver=0x2b5eed0, event=0x7fffa5a7afa0) at kernel/qcoreapplication.cpp:787
#22 0x00007f5f9baa1275 in sendEvent (receiver=0x2b5eed0, event=0x7fffa5a7afa0, alienWidget=0x2b5eed0, nativeWidget=0x2972aa0, buttonDown=0x7f5f9c577198, lastMouseReceiver=..., spontaneous=true) at ../../src/corelib/kernel/qcoreapplication.h:215
#23 QApplicationPrivate::sendMouseEvent (receiver=0x2b5eed0, event=0x7fffa5a7afa0, alienWidget=0x2b5eed0, nativeWidget=0x2972aa0, buttonDown=0x7f5f9c577198, lastMouseReceiver=..., spontaneous=true) at kernel/qapplication.cpp:3141
#24 0x00007f5f9bb1fd28 in QETWidget::translateMouseEvent (this=0x2972aa0, event=<value optimized out>) at kernel/qapplication_x11.cpp:4461
#25 0x00007f5f9bb1e3e9 in QApplication::x11ProcessEvent (this=0xbf70b0, event=0x7fffa5a7b900) at kernel/qapplication_x11.cpp:3587
#26 0x00007f5f9bb46512 in x11EventSourceDispatch (s=0xc00ab0, callback=0, user_data=0x0) at kernel/qguieventdispatcher_glib.cpp:146
#27 0x00007f5f95b3ebd3 in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
#28 0x00007f5f95b3f3b0 in ?? () from /lib64/libglib-2.0.so.0
#29 0x00007f5f95b3f650 in g_main_context_iteration () from /lib64/libglib-2.0.so.0
#30 0x00007f5f9c97f5df in QEventDispatcherGlib::processEvents (this=0xbd59d0, flags=<value optimized out>) at kernel/qeventdispatcher_glib.cpp:422
#31 0x00007f5f9bb461ae in QGuiEventDispatcherGlib::processEvents (this=<value optimized out>, flags=<value optimized out>) at kernel/qguieventdispatcher_glib.cpp:204
#32 0x00007f5f9c953c22 in QEventLoop::processEvents (this=<value optimized out>, flags=...) at kernel/qeventloop.cpp:149
#33 0x00007f5f9c953e35 in QEventLoop::exec (this=0x7fffa5a7bc20, flags=...) at kernel/qeventloop.cpp:201
#34 0x00007f5f9c95827b in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1064
#35 0x000000000045a64a in runKMyMoney(KApplication*, KStartupLogo*) ()
#36 0x000000000045c516 in main ()

Reported using DrKonqi
Comment 1 allan 2011-09-25 21:33:25 UTC
(In reply to comment #0)
> Application: kmymoney (4.6.0)
> KDE Platform Version: 4.7.1 (4.7.1) "release 8"
> Qt Version: 4.7.4
> Operating System: Linux 2.6.37.6-0.7-desktop x86_64
> Distribution: "openSUSE 11.4 (x86_64)"

Would it be possible for you to send direct to me a copy of your csv
file, if necessary with private data changed, as long as the format
remains intact.

Allan
Comment 2 Matthew 2011-09-26 00:22:53 UTC
I managed to get it to work right before I left home. The csv had 3 lines at 
the top that only appeared in colum 1 in csv format, and then the header row. 
I deleted the 3 lines and the header from the file and the import went 
perfect. I did not back up the file first so I do not have a copy of the 
original formatting to send. If I remember correctly the formatting was like 
this:

name of institution
account number
some description

transaction number,date,payee,description,debit,deposit
data,data,data,data,data,data
data,data,data,data,data,data
data,data,data,data,data,data
data,data,data,data,data,data
data,data,data,data,data,data
data,data,data,data,data,data
etc...


Not backing up the original was carless of me. This is the first time I have 
submitted a bug and I diddnt think it through first. Ill be more thorough next 
time and have some better feedback to help with the troubleshooting.



On Sunday, September 25, 2011 09:33:26 PM allan wrote:
> https://bugs.kde.org/show_bug.cgi?id=282763
> 
> 
> allan <agander93@gmail.com> changed:
> 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
> CC|                            |agander93@gmail.com
> 
> 
> 
> 
> --- Comment #1 from allan <agander93 gmail com>  2011-09-25 21:33:25 ---
> (In reply to comment #0)
> 
> > Application: kmymoney (4.6.0)
> > KDE Platform Version: 4.7.1 (4.7.1) "release 8"
> > Qt Version: 4.7.4
> > Operating System: Linux 2.6.37.6-0.7-desktop x86_64
> > Distribution: "openSUSE 11.4 (x86_64)"
> 
> Would it be possible for you to send direct to me a copy of your csv
> file, if necessary with private data changed, as long as the format
> remains intact.
> 
> Allan
Comment 3 allan 2011-09-26 10:59:47 UTC
(In reply to comment #2)
> I managed to get it to work right before I left home. The csv had 3 lines at 
> the top that only appeared in colum 1 in csv format, and then the header row. 
> I deleted the 3 lines and the header from the file and the import went 
> perfect. I did not back up the file first so I do not have a copy of the 
> original formatting to send. If I remember correctly the formatting was like 
> this:
> 
> name of institution
> account number
> some description
> 
> transaction number,date,payee,description,debit,deposit
> data,data,data,data,data,data
> data,data,data,data,data,data
> data,data,data,data,data,data
> data,data,data,data,data,data
> data,data,data,data,data,data
> data,data,data,data,data,data
> etc...
> 
> 
> Not backing up the original was carless of me. This is the first time I have 
> submitted a bug and I diddnt think it through first. Ill be more thorough next 
> time and have some better feedback to help with the troubleshooting.
> 

I constructed a test file by adding your header lines to the top of a valid csv file.  The only way I could reproduce your problem was by failing to set the first line to the correct value on the Settings tab.  Leaving it set to '1' will result in that error message appearing.  Prior to that, another warning appears, an empty cell having been detected in the amount column on row 1.  Ignoring that and proceeding to import, your message appears.  Clicking 'accept'
or accept all brings up an error message that an invalid date has been detected on row 1.

As far as I can see, the logic is correct, and can only assume you failed to set the first line correctly on the Settings tab.

If you will be able to retry with another file, I'll leave this bug open, unless you con confirm that an incorrect setting was used.

In the event of a further problem, it might help if you can enable debug symbols.
> On Sunday, September 25, 2011 09:33:26 PM allan wrote:
> > https://bugs.kde.org/show_bug.cgi?id=282763
> > 
> > 
> > allan <agander93@gmail.com> changed:
> > 
> >            What    |Removed                     |Added
> > ----------------------------------------------------------------------------
> > CC|                            |agander93@gmail.com
> > 
> > 
> > 
> > 
> > --- Comment #1 from allan <agander93 gmail com>  2011-09-25 21:33:25 ---
> > (In reply to comment #0)
> > 
> > > Application: kmymoney (4.6.0)
> > > KDE Platform Version: 4.7.1 (4.7.1) "release 8"
> > > Qt Version: 4.7.4
> > > Operating System: Linux 2.6.37.6-0.7-desktop x86_64
> > > Distribution: "openSUSE 11.4 (x86_64)"
> > 
> > Would it be possible for you to send direct to me a copy of your csv
> > file, if necessary with private data changed, as long as the format
> > remains intact.
> > 
> > Allan
Comment 4 Thomas Baumgart 2011-09-27 07:16:43 UTC
Even if an invalid setting has been selected by the user, the application should not respond with a crash.
Comment 5 allan 2011-09-27 09:48:24 UTC
(In reply to comment #4)
> Even if an invalid setting has been selected by the user, the application
> should not respond with a crash.

Of course, I agree totally.  Unfortunately, the BT wasn't very helpful, just giving the routine but no address.  Also unfortunately, my attempt to reproduce the problem was unsuccessful as well, but I haven't washed my hands on it.
Comment 6 Cristian Oneț 2013-09-16 10:49:47 UTC
Is this still relevant to git master?
Comment 7 allan 2013-09-16 12:07:32 UTC
(In reply to comment #6)
> Is this still relevant to git master?

I've constructed another file based on the format 'remembered' by the user.  Attempting to import with a starting row other than the correct one always resulted in one or more error messages.  Accepting or cancelling these never resulted in a crash.  As the BT provided was unhelpful, without symbols, and it was a very old revision, I think this should be closed as 'works for me'.

While there was no crash, there was a condition, depending on responses to the errors, where it was not possible to get to the import stage without reselecting the file to import, and that needs to be looked at.  So, either close as suggested, or leave open, but for a quite different issue??
Comment 8 Cristian Oneț 2013-09-16 12:14:24 UTC
Please open an issue for that problem you observed if it's still happening on git master.
Comment 9 allan 2013-09-17 10:44:41 UTC
(In reply to comment #8)
> Please open an issue for that problem you observed if it's still happening
> on git master.

See https://bugs.kde.org/show_bug.cgi?id=324991
Comment 10 allan 2013-09-17 23:21:58 UTC
I've now found a condition which will result in a crash.  It may not be the original issue as it only appears to affect investment files.  I might as well add it here though.  Again, it involves the user ignoring error messages, in a similar way to https://bugs.kde.org/show_bug.cgi?id=324991 above. (Comment #9)

Steps to Reproduce:
1. Perform the usual steps to import an investment CSV file, until reaching the Lines page.
2. On that page, select an incorrect start (or end) line, such that a line with invalid format is
-  included in the import selection, then click Next.  
3. An error message is likely to appear indicating that an incorrect value has been detected on
-  the invalid row.
4. Instead of clicking Cancel, click an Accept button.
5. On the Finish page, click Import CSV.
6. Another error message now is likely to appear, when the import data is parsed. Generally,
-  the error first detected is likely to be an incorrect date value. Click OK.
7a. What the user should do next is to click on Back and correct the line number then click Next.
7b. Instead of correcting the line number, click to change date format.
8. Now click Next, and a crash occurs.
Comment 11 allan 2013-09-19 12:46:17 UTC
Created attachment 82405 [details]
Patch file for this bug

Both this bug and 324991 involve the same areas of code.  I started off intending to submit separate patches, but they were too interwoven.  It's a fairly small patch, and some of the inserts/deletes are really moves, to obtain a more logical sequence.

They are in the wizard version of the CSV plugin, which is not in stable, and the issues only arise following the user attempting to continue without correcting errors which had been flagged.  The crash condition probably would not arise in the tabbed wizard in stable.

Assuming no objections, I would intend to commit in a week or so.
Comment 12 allan 2013-09-19 16:09:32 UTC
(In reply to comment #11)

> and the issues only arise following the user attempting to continue without
> correcting errors which had been flagged.  The crash condition probably
> would not arise in the tabbed wizard in stable.

The crash condition does *not* arise in the tabbed wizard in stable.

> Assuming no objections, I would intend to commit in a week or so.

The intention of the patch is that, when KMM app detects the error in the import, it will pass control to the LinesDatePage wizard page, where the error needs to be corrected, instead of remaining on the CompletionPage, removing the possibility of the user re-importing a file with invalid parameters.
Comment 13 allan 2013-09-20 12:27:13 UTC
Finally, I've found the cause of the original crash.  It was fixed 28 Nov 2011 in Bug  287786 -
 If a CSV file contains lines with differing numbers of columns, and if one of the selected columns happens to be the last column in a line with fewer than the maximum number, an array goes out of bounds. It was backported into 4.6.3.

So the cause was the user  importing the header lines, after several warnings.
Comment 14 allan 2013-09-20 17:27:51 UTC
?
Comment 15 allan 2013-09-21 09:47:01 UTC
*** Bug 324991 has been marked as a duplicate of this bug. ***
Comment 16 allan 2013-09-29 10:37:36 UTC
Git commit 1a7483ef7d73cec1be91df1e2bf18fb4b91bdcf9 by Allan Anderson.
Committed on 19/09/2013 at 12:21.
Pushed by allananderson into branch 'master'.
Related: bug 324491

M  +18   -15   kmymoney/plugins/csvimport/csvdialog.cpp
M  +31   -3    kmymoney/plugins/csvimport/investprocessing.cpp

http://commits.kde.org/kmymoney/1a7483ef7d73cec1be91df1e2bf18fb4b91bdcf9