Bug 295373 - KMyMoney hangs when importing a large QIF file
Summary: KMyMoney hangs when importing a large QIF file
Status: RESOLVED WORKSFORME
Alias: None
Product: kmymoney
Classification: Applications
Component: general (show other bugs)
Version: 4.5.3
Platform: unspecified Linux
: NOR crash
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-03-05 16:42 UTC by John Veasey
Modified: 2012-03-08 13:32 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description John Veasey 2012-03-05 16:42:16 UTC
Application: KMyMoney (kmymoney), signal: Aborted
[Current thread is 1 (Thread 0x7f6f3548b780 (LWP 6958))]

Thread 3 (Thread 0x7f6f1d330700 (LWP 6964)):
#0  0x00007f6f32aba070 in pthread_getspecific () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007f6f2b5e93dd in g_thread_self () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2  0x00007f6f2b5c1610 in g_main_context_acquire () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#3  0x00007f6f2b5c2d81 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#4  0x00007f6f2b5c3792 in g_main_loop_run () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#5  0x00007f6f24c54516 in ?? () from /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0
#6  0x00007f6f2b5e82b6 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#7  0x00007f6f32ab4efc in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#8  0x00007f6f2fe5289d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#9  0x0000000000000000 in ?? ()

Thread 2 (Thread 0x7f6f1dd34700 (LWP 7002)):
#0  0xffffffffff60017b in ?? ()
#1  0x00007f6f1dd33af0 in ?? ()
#2  0x00007fffa08947e7 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0x7f6f3548b780 (LWP 6958)):
[KCrash Handler]
#6  0x00007f6f2fda73a5 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x00007f6f2fdaab0b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#8  0x00007f6f30665d7d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#9  0x00007f6f30663f26 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007f6f30663f53 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#11 0x00007f6f3066404e in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#12 0x0000000000455599 in main (argc=1, argv=<optimized out>) at /build/buildd/kmymoney-4.5.3/kmymoney/main.cpp:266
Comment 1 Thomas Baumgart 2012-03-05 19:15:42 UTC
Please add more details about what you were doing. Just dropping the backtrace is not enough. Also, please use a newer version. 4.5.3 is way too old and your problem might have been corrected in the meantime.
Comment 2 John Veasey 2012-03-05 19:27:33 UTC
On Mon, 2012-03-05 at 19:15 +0000, Thomas Baumgart wrote:

> https://bugs.kde.org/show_bug.cgi?id=295373
> 
> Thomas Baumgart <ipwizard@users.sourceforge.net> changed:
> 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|                            |ipwizard@users.sourceforge.
>                    |                            |net
> 
> --- Comment #1 from Thomas Baumgart <ipwizard@users.sourceforge.net> ---
> Please add more details about what you were doing. Just dropping the backtrace
> is not enough. Also, please use a newer version. 4.5.3 is way too old and your
> problem might have been corrected in the meantime.
> 

What I was doing?

Using KMYMONEY:  attempting to import transactions from my bank to the
ledger/program.  Not sure about the version of the Bug Report software,
I'll have to check.  Just updated system with 400 updates, I'm sorry.
John
Comment 3 John Veasey 2012-03-05 19:30:09 UTC
I was attempting to update ledger with a file imported from my bank, I think it was an OFX file suffix.
Comment 4 John Veasey 2012-03-05 19:42:50 UTC
Hello Thomas Baumgart,
Okay, my system update has the newer program for KMyMoney, v 4.5.3 now.
The actual action I was attempting:  import file in QIF format that
covers a whole year of transactions.  Not sure how long the program
would take to complete the action and may have forced an error by force
closure of the program.  Hopefully this is helpful information.  Was not
sure if you wanted email info or added comments on the Bug Report Site?
Sincerely,
John


On Mon, 2012-03-05 at 19:32 +0000, Thomas Baumgart wrote:

> https://bugs.kde.org/show_bug.cgi?id=295373
> 
> Thomas Baumgart <ipwizard@users.sourceforge.net> changed:
> 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|ipwizard@users.sourceforge. |
>                    |net                         |
>
Comment 5 Jack 2012-03-05 20:18:08 UTC
First, you said there is a new version available - but you still say 4.5.3.  The latest release is 4.6.2.  There have been many bug fixes, including in the area of file imports.

Did the program really crash by itself, or only when you tried to kill it because the import was taking too long.  There have been known problems with importing very large QIF files.  A newer version of KMM should do better, or you can export in smaller increments - just a few months at a time.
Comment 6 John Veasey 2012-03-05 22:34:23 UTC
Hi Jack,
I could not duplicate the error, using one month of transactions file.
The QIF file imported never finished over a one minute timed period.
The program window "grayed out" and some items at the top menu items
were action able.  I hope this makes sense?  Not sure if the institution
name must be real or not, also the account number and routing number
were fictional too.  I did download some transactions in OFX format
(January 1 to present) and they imported fine in my initial program
setup but not the QIF format, where I got the error the first and only
time.  Not sure if I could have imported my Quicken files, nearly 400mb
now, covering some 5 years of my checking/savings account.  
Sincerely,
John

On Mon, 2012-03-05 at 20:18 +0000, Jack wrote:

> https://bugs.kde.org/show_bug.cgi?id=295373
> 
> Jack <ostroffjh@users.sourceforge.net> changed:
> 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|                            |ostroffjh@users.sourceforge
>                    |                            |.net
> 
> --- Comment #5 from Jack <ostroffjh@users.sourceforge.net> ---
> First, you said there is a new version available - but you still say 4.5.3. 
> The latest release is 4.6.2.  There have been many bug fixes, including in the
> area of file imports.
> 
> Did the program really crash by itself, or only when you tried to kill it
> because the import was taking too long.  There have been known problems with
> importing very large QIF files.  A newer version of KMM should do better, or
> you can export in smaller increments - just a few months at a time.
>
Comment 7 Jack 2012-03-05 23:28:06 UTC
Minor point - the institution name, and account and routing numbers are not important, but if they are set for the account in KMM, and they are included in the download file, they will be used to tell which account to import into.

As I understand it, you can successfully import short QIF or OFX files, but if you try to import a very large QIF file, the program just seems to hang until you kill it.  If that is wrong, please provide more detail.  

If that is correct, I can suggest three ways to go.  The best would be to upgrade to a newer version of KMM.  Next would be if you could get your data in smaller files, instead of all in a single file.  Finally, you can try just waiting for the big file to import.  I don't know if it will work, but I would not be surprised if it takes a very long (hours) time.  Use some system monitor just to see that KMM is doing something and not actually frozen, although if there is a problem, I would expect a crash rather than a hang.
Comment 8 Cristian Oneț 2012-03-08 11:35:54 UTC
As requested by Jack please provide more detail since we can't do much with the current information.
Comment 9 John Veasey 2012-03-08 11:58:29 UTC
Okay, frustration, frustration.  Currently using v4.5.3 and have no problems if I use the import function with transactions covering 3 - 10 days.  So this bug report is closed, is my opinion.  

However this newer version in not in the repository for upgrade.  I'm not smart enough to figure the manual install so I'm asking for help.  Located in USA.  Using Ubuntu 11.10.  Using a Dell 1545 laptop w/4gb memory, 80gb hd w/ Ubuntu 11.10 and 320gb hdives dual boot has Win7 and Linux Mint 11.  I don't know where you want me to input info I should know and you don't,  but I can't tell the difference when I should.  So the install note you give about 'cmake' - where there are 4 - 6 lines you me to type in terminal mode/sudo.  Okay, I keep getting anerror message about command not found or syntax error "=" should not be there.   So I just gave up and re installed the v4.5.3.   I don't have any training or experience with terminology or terminal mode inputs.  
Sincerely,
John
PS:  I can use Skype for personal contact if you like.  Wildcrazyman  Also, I have two computers available to use one for skype and the other to get KMyMoney v4.6.3 installed.
Comment 10 Cristian Oneț 2012-03-08 12:23:36 UTC
I'm sorry that your distribution (Ubuntu/Linux mint) does not provide the latest packages we can't do nothing about that.

You say you have Win7 then you could try the Windows version which does provide a package with the latest version. See http://kmymoney2.sourceforge.net/windows-install.html for more information about installing on Windows. Current versions should be KMyMoney 4.6.2 with KDE 4.8.0.

I got it that importing small files works as expected. Does importing large file fail consistently? Or did you try to wait longer for the import process to finish? (400mb seems a lot of data). And finally, was it the QIF or the OFX import that failed?
Comment 11 John Veasey 2012-03-08 13:11:30 UTC
Initial issue appears to be my fault.  I could not duplicate.  What I now get (QIF inport):  Date Filter Error: they don't match and places all imported transactions to 'todays date'.   It is easier/works to use OFX format.  If I ever get KMyMoney v4.6.3 installed, overall my issues will go away.  

If I take your suggestion to use Window 7 & KMyMoney; I might as well keep using Quicken, where I have been trying to move away from MS OS.

Maybe it is best to Close this report...???
Thank you
Comment 12 Cristian Oneț 2012-03-08 13:19:02 UTC
> If I take your suggestion to use Window 7 & KMyMoney; I might as well keep using Quicken, where I > have been trying to move away from MS OS.

Well I haven't suggested to use it on a daily basis, just to try to reproduce the bug you've reported.

> Maybe it is best to Close this report...???

OK, if it happens again in a reproducible manner please report it. Even if it's an older version it does not matter, the two major requirements for a good bug report are:
1. A stack trace that contains useful information (debug information needs to be installed for this)
2. A way to reproduce the issue

Sometimes we can figure out what went wrong by using only 1 but if we can't reproduce the bug we can't say certainly that it's fixed if the fixed is only based on 1.
Comment 13 John Veasey 2012-03-08 13:24:20 UTC
Hi Cristian,
Okay, just imported one year of transaction in OFX format without any issues, 127kb file.  My actual Quicken file covers multiple years is why it is so large too.  Thank you again and best regards to the other people that made suggestions.


Touch someone...make them smile,
John



-----Original Message-----
From: Cristian Oneț <onet.cristian@gmail.com>
To: jvz1p1 <jvz1p1@aol.com>
Sent: Thu, Mar 8, 2012 7:19 am
Subject: [Bug 295373] KMyMoney hangs when importing a large QIF file


https://bugs.kde.org/show_bug.cgi?id=295373

Cristian Oneț <onet.cristian@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDSINFO                   |RESOLVED
         Resolution|WAITINGFORINFO              |WORKSFORME

--- Comment #12 from Cristian Oneț <onet.cristian@gmail.com> ---
> If I take your suggestion to use Window 7 & KMyMoney; I might as well keep 
using Quicken, where I > have been trying to move away from MS OS.

Well I haven't suggested to use it on a daily basis, just to try to reproduce
the bug you've reported.

> Maybe it is best to Close this report...???

OK, if it happens again in a reproducible manner please report it. Even if it's
an older version it does not matter, the two major requirements for a good bug
report are:
1. A stack trace that contains useful information (debug information needs to
be installed for this)
2. A way to reproduce the issue

Sometimes we can figure out what went wrong by using only 1 but if we can't
reproduce the bug we can't say certainly that it's fixed if the fixed is only
based on 1.
Comment 14 allan 2012-03-08 13:32:01 UTC
(In reply to comment #11)
> snip
> If I take your suggestion to use Window 7 & KMyMoney; I might as well keep
> using Quicken, where I have been trying to move away from MS OS.
> 

Alternatively, your Ubuntu 11.10 does have 4.6.1 in the Claydoh PPA - see 
https://launchpad.net/~claydoh/+archive/kmymoney2-kde4?field.series_filter=oneiric 

Just follow the instructions there to add that repo to your system.  You may need the liblakimia too.

Can't have you sticking with Quicken.

Allan