Bug 255133 - skrooge crash on ofx import
Summary: skrooge crash on ofx import
Status: RESOLVED FIXED
Alias: None
Product: skrooge
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Ubuntu Linux
: NOR crash
Target Milestone: ---
Assignee: Stephane MANKOWSKI
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-10-24 19:17 UTC by Luka Renko
Modified: 2011-04-18 17:52 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
test.ofx - sample file to reproduce the crash (1.73 KB, application/xml)
2010-10-24 19:31 UTC, Luka Renko
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Luka Renko 2010-10-24 19:17:54 UTC
Application: skrooge (0.7.4)
KDE Platform Version: 4.5.1 (KDE 4.5.1)
Qt Version: 4.7.0
Operating System: Linux 2.6.35-22-generic x86_64
Distribution: Ubuntu 10.10

-- Information about the crash:
I have clean installation of skrooge, built today from SVN in order to collect stack trace.

I tried to import OFX file from my bank and it crashed on import.

The crash can be reproduced every time.

-- Backtrace:
Application: Skrooge (skrooge), signal: Aborted
[Current thread is 1 (Thread 0x7f1b58c80760 (LWP 16482))]

Thread 2 (Thread 0x7f1b3e99d710 (LWP 16485)):
#0  0x00007f1b552dd1d3 in poll () from /lib/libc.so.6
#1  0x00007f1b4fcce009 in ?? () from /lib/libglib-2.0.so.0
#2  0x00007f1b4fcce45c in g_main_context_iteration () from /lib/libglib-2.0.so.0
#3  0x00007f1b560d71e6 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQtCore.so.4
#4  0x00007f1b560a9a02 in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQtCore.so.4
#5  0x00007f1b560a9dec in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQtCore.so.4
#6  0x00007f1b55fb42fd in QThread::exec() () from /usr/lib/libQtCore.so.4
#7  0x00007f1b560895f8 in ?? () from /usr/lib/libQtCore.so.4
#8  0x00007f1b55fb727e in ?? () from /usr/lib/libQtCore.so.4
#9  0x00007f1b55d2c971 in start_thread () from /lib/libpthread.so.0
#10 0x00007f1b552e991d in clone () from /lib/libc.so.6
#11 0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7f1b58c80760 (LWP 16482)):
[KCrash Handler]
#6  0x00007f1b55236ba5 in raise () from /lib/libc.so.6
#7  0x00007f1b5523a6b0 in abort () from /lib/libc.so.6
#8  0x00007f1b55ada6bd in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/libstdc++.so.6
#9  0x00007f1b55ad8906 in ?? () from /usr/lib/libstdc++.so.6
#10 0x00007f1b55ad8933 in std::terminate() () from /usr/lib/libstdc++.so.6
#11 0x00007f1b55ad8a86 in __cxa_rethrow () from /usr/lib/libstdc++.so.6
#12 0x00007f1b560a9f3d in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQtCore.so.4
#13 0x00007f1b560adebb in QCoreApplication::exec() () from /usr/lib/libQtCore.so.4
#14 0x000000000040185a in main (argc=1, argv=0x7fff99f2b048) at /misc/k/src/skrooge/skrooge/main.cpp:85

Possible duplicates by query: bug 255074, bug 255054, bug 254937, bug 254892, bug 254856.

Reported using DrKonqi
Comment 1 Luka Renko 2010-10-24 19:21:24 UTC
This are last messages on console:


##    >SKGDocument::beginTransaction
##    Input parameter [name]=[Import with codec UTF-8]
##    Input parameter [iSqlOrder]=[BEGIN;]
##      >SKGDocument::getTransactionToProcess
##      <SKGDocument::getTransactionToProcess
##    Input parameter [iSqlOrder]=[insert into doctransaction (d_date, t_name, i_parent) values ('2010-10-24 19:21:45','Import with codec UTF-8', 6);]
##      >SKGDocument::getTransactionToProcess
##      <SKGDocument::getTransactionToProcess
##    <SKGDocument::beginTransaction RC=[SUC-0]
##    >SKGDocument::setParameter
##    Input parameter [iName]    =[SKG_LAST_CODEC_USED_FOR_IMPORT]
##    Input parameter [iValue]   =[UTF-8]
##      >SKGObjectBase::save
##        >SKGDocument::checkExistingTransaction
##        <SKGDocument::checkExistingTransaction RC=[SUC-0]
##      Input parameter [iSqlOrder]=[INSERT INTO parameters ('t_name','t_value','t_uuid_parent') VALUES ('SKG_LAST_CODEC_USED_FOR_IMPORT','UTF-8','document')]
##        >SKGObjectBase::load
##        <SKGObjectBase::load RC=[SUC-0]
##      <SKGObjectBase::save RC=[SUC-0]
##    <SKGDocument::setParameter RC=[SUC-0]
##    >SKGImportExportManager::setCodec
##    <SKGImportExportManager::setCodec RC=[SUC-0]
##    >SKGImportExportManager::importFile
##    Input parameter [iSqlOrder]=[ANALYZE]
##      >SKGDocumentBank::importOFX
##      Input filename=/home/lukar/Documents/Fin/Izvozi/2010/test.ofx
##        >SKGDocument::beginTransaction
##        Input parameter [name]=[#INTERNAL#]
##        <SKGDocument::beginTransaction RC=[SUC-0]
Qt has caught an exception thrown from an event handler. Throwing
exceptions from an event handler is not supported in Qt. You must
reimplement QApplication::notify() and catch all exceptions there.

terminate called after throwing an instance of 'std::out_of_range'
  what():  basic_string::substr
QSocketNotifier: Invalid socket 12 and type 'Read', disabling...
KCrash: Application 'skrooge' crashing...
KCrash: Attempting to start /usr/lib/kde4/libexec/drkonqi from kdeinit
sock_file=/home/lukar/.kde/socket-lure/kdeinit4__0
Comment 2 Luka Renko 2010-10-24 19:31:13 UTC
Created attachment 52831 [details]
test.ofx - sample file to reproduce the crash

I have used binary search to limit crash to single day of transactions. The attached file is only edited to remove personal details.
On request of developers, I can provide original file from bank.
Comment 3 Luka Renko 2010-10-24 20:21:28 UTC
It looks like, this is bug in ofx library:

$ ofxdump test.ofx 

LibOFX INFO: libofx_proc_file(): File format not specified, autodecting...
(Above message occured on Line 18446744073709551615, Column 18446744073709551615)
LibOFX INFO: libofx_proc_file(): Detected file format: OFX (Open Financial eXchange (OFX or QFX))
(Above message occured on Line 18446744073709551615, Column 18446744073709551615)
LibOFX STATUS: find_dtd():DTD found: /usr/share/libofx4/0.9.0/dtd/opensp.dcl
(Above message occured on Line 18446744073709551615, Column 18446744073709551615)
LibOFX STATUS: find_dtd():DTD found: /usr/share/libofx4/0.9.0/dtd/ofx160.dtd
(Above message occured on Line 18446744073709551615, Column 18446744073709551615)
LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate SIGNONMSGSRSV1
(Above message occured on Line 2, Column 3)
LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate SONRS
(Above message occured on Line 3, Column 5)
ofx_proc_status():
    Ofx entity this status is relevent to: SONRS 
    Severity: INFO
    Code: 0, name: Success
    Description: The server successfully processed the request.

LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate BANKMSGSRSV1
(Above message occured on Line 13, Column 3)
LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate STMTTRNRS
(Above message occured on Line 14, Column 5)
ofx_proc_status():
    Ofx entity this status is relevent to: STMTTRNRS 
    Severity: INFO
    Code: 0, name: Success
    Description: The server successfully processed the request.

terminate called after throwing an instance of 'std::out_of_range'
  what():  basic_string::substr
Aborted
Comment 4 Luka Renko 2010-10-24 20:57:13 UTC
Build libofx 0.9.1 from source to collect stack trace:


#0  0x00007ffff6888ba5 in raise (sig=<value optimized out>)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007ffff688c6b0 in abort () at abort.c:92
#2  0x00007ffff712c6bd in __gnu_cxx::__verbose_terminate_handler()
    () from /usr/lib/libstdc++.so.6
#3  0x00007ffff712a906 in ?? () from /usr/lib/libstdc++.so.6
#4  0x00007ffff712a933 in std::terminate() ()
   from /usr/lib/libstdc++.so.6
#5  0x00007ffff712aa3e in __cxa_throw ()
   from /usr/lib/libstdc++.so.6
#6  0x00007ffff70d538e in std::__throw_out_of_range(char const*) ()
   from /usr/lib/libstdc++.so.6
#7  0x00007ffff7bb31dd in ofxdate_to_time_t(std::basic_string<char, std::char_traits<char>, std::allocator<char> >) ()
   from /usr/lib/libofx.so.4
#8  0x00007ffff7bbc36f in OfxStatementContainer::add_attribute(std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >)
    () from /usr/lib/libofx.so.4

#9  0x00007ffff7bbfe16 in OfxPushUpContainer::add_attribute(std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >) ()
   from /usr/lib/libofx.so.4
#10 0x00007ffff7bca799 in OFXApplication::endElement(SGMLApplication::EndElementEvent const&) () from /usr/lib/libofx.so.4
#11 0x00007ffff78a4842 in OpenSP::GenericEventHandler::endElement(OpenSP::EndElementEvent*) () from /usr/lib/libosp.so.5
#12 0x00007ffff7911e08 in OpenSP::Parser::acceptEndTag(OpenSP::EndElementEvent*) () from /usr/lib/libosp.so.5
#13 0x00007ffff7914d63 in OpenSP::Parser::doContent() ()
   from /usr/lib/libosp.so.5
#14 0x00007ffff78bacf8 in OpenSP::Parser::parseAll(OpenSP::EventHandler&, int const volatile*) () from /usr/lib/libosp.so.5
#15 0x00007ffff78bdb0e in OpenSP::ParserApp::parseAll(OpenSP::SgmlParser&, OpenSP::EventHandler&, int const volatile*) ()
   from /usr/lib/libosp.so.5
#16 0x00007ffff78bf161 in OpenSP::ParserEventGenerator::run(SGMLApplication&) () from /usr/lib/libosp.so.5

#17 0x00007ffff7bc9f92 in ofx_proc_sgml(LibofxContext*, int, char**)
    () from /usr/lib/libofx.so.4
#18 0x00007ffff7bb6e57 in ofx_proc_file () from /usr/lib/libofx.so.4
#19 0x00007ffff7bb40ab in libofx_proc_file ()
   from /usr/lib/libofx.so.4
#20 0x0000000000402a46 in main (argc=<value optimized out>, 
    argv=<value optimized out>) at ofxdump.cpp:436
Comment 5 Luka Renko 2010-10-24 21:04:38 UTC
It looks like, date in OFX files from my bank are not in correct format (w/o time), and libofx code is not written well for such case:

from lib/ofx_utilities.cpp:

    if(ofxdate.size()>8) {
    /* if exact time is specified */
exact_time_specified = true;
      time.tm_hour=atoi(ofxdate.substr(8,2).c_str());
      time.tm_min=atoi(ofxdate.substr(10,2).c_str());
      time.tm_sec=atoi(ofxdate.substr(12,2).c_str());
    }


Sample dates:
          <DTSTART>201009099.9</DTSTART>

          <DTEND>201009099.9.</DTEND>

As a workaround, will fix this dates by hand...
Comment 6 Stephane MANKOWSKI 2010-10-25 20:45:15 UTC
Hi Luka,

Thank you for your analysis.
The incident seems to be already opened in launchpad.
https://bugs.launchpad.net/ubuntu/+source/libofx/+bug/661809

I close this incident because the problem must be closed in libOfx.

Thank you for using Skrooge
Comment 7 Stephane MANKOWSKI 2010-10-25 22:20:47 UTC
I will deliver a correction to catch libofx exceptions
Comment 8 Stephane MANKOWSKI 2010-10-25 22:22:41 UTC
SVN commit 1189768 by smankowski:

BUG: 255133: Skrooge crash on ofx import (With wrong data format. Bug in libOFX)

 M  +1 -0      CHANGELOG  
 M  +2 -0      skgbankmodeler/CMakeLists.txt  
 M  +5 -0      skgbankmodeler/skgimportexportmanager.cpp  
 M  +14 -0     skgbankmodelertest/skgtestimportofx.cpp  
 A             tests/input/skgtestimportofx/exception.ofx  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1189768
Comment 9 Benoit Grégoire 2011-04-18 17:52:28 UTC
Fixed upstream, the sample file parses (but probably with the wrong date, it's unclear what to do generically with the extra 0.)