Bug 396831 - 5.0 Version - Crashes when importing OFX/QFX File Using Firefox
Summary: 5.0 Version - Crashes when importing OFX/QFX File Using Firefox
Status: RESOLVED FIXED
Alias: None
Product: kmymoney
Classification: Applications
Component: importer (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR crash
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords: triaged
: 396147 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-07-24 23:24 UTC by Brendan
Modified: 2018-11-25 16:31 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In: 5.0.3
Sentry Crash Report:


Attachments
Console logs of first and second instance (5.94 KB, text/plain)
2018-08-07 10:59 UTC, Thomas Baumgart
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Brendan 2018-07-24 23:24:32 UTC
I'm running KMM that I compiled from the 5.0 branch on Fedora 28. KMM
Version 5.0.2-7561bb34

Online banking is working for banks that I have mapped in KMM.

For a few other banks I have to download the ofx/qfx file from the
bank and import it to KMM.

There are two ways to do this. The hard way is to save the ofx/qfx
file on my hard disk and then import it into KMM. This works.

The easy way is to download it in Firefox and when Firefox asks if it
should open the file with a program or save it to disk I tell it to
open it with KMM. It crashes KMM in 5.0 if KMM is already running. If
I do this when KMM is not running, Firefox launches KMM and KMM does
not crash after importing the data. Very strange. This worked great in
KMM 4.8 either way.

I ran this from the command line to capture the differences in the
output from KMM.
1c1
< Import OFX from file on disk:
---
> From Website using Firefox:

3d2
< WebConnect: Starting server failed. Try to remove stale socket.


7c6
< Gtk-Message: 16:56:59.493: GtkDialog mapped without a transient
parent. This is discouraged.
---
> WebConnect: Processing QUrl("file:///tmp/mozilla_user/Bank2894_RecentActivity_20180724-5.QFX")


28a28
> Segmentation fault (core dumped)
Comment 1 Brendan 2018-08-03 23:08:15 UTC
I have done more testing. It turns out that if I try to open the ofx file with KMM directly from Firefox when KMM is already running, a second copy of KMM tries to start. 

I get an error "Unable to load accounts:" from the new instance of KMM. Clicking on details I get:

No storage object attached to MyMoneyFile /usr/local/src/kmm/kmymoney-MASTER-xxx/kmymoney/mymoney/mymoneyfile.cpp:212

When I click OK the detailed message pops up in it's own window. Clicking OK on it causes the new instance of KMM to crash.

I assume it can't load accounts because it's trying to open the KMM file that is already open in the first instance.

The first instance attempts to import the OFX file, let's me select the account that I want to impost the OFX file into and then gives me the summary message as if it completed successfully. KMM then crashes and nothing is saved (so I don't know for sure that it imported the data properly).

Reversing the order (import into first instance and then clear erros is second instance) does not change the outcome.

This is not how 4.8 worked. It correctly tried to import the OFX file into the open KMM instance and did not launch a second instance.
Comment 2 Thomas Baumgart 2018-08-04 04:48:19 UTC
Good hint, that it is the second instance that is crashing. Here's how you can simulate this from the command line (w/o using the browser):

- Download the OFX file to disk
- Start KMyMoney and open your file
- Start a second instance and pass it the name of the download OFX file (could be an absolute path). Make sure to use the same KMyMoney executable.

I am interested in the CLI output of both instances. Maybe, you can attach it here to the bug.

BTW: Even 4.8 started a second instance but you did not notice and in fact, it is how master works for me here.
Comment 3 Brendan 2018-08-05 16:49:58 UTC
Below is the output from both consoles when running KMM and then KMM Bank.qfx.

When I only run kmymoney Bank.qfx it imports the data and does not crash.

=============================================

Console output:

First instance:
/usr/bin/kmymoney 

WebConnect: Running in server mode
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
WebConnect: Processing QUrl("file:///Shared/Documents/KMM/Bank.qfx")
LibOFX INFO: libofx_proc_file(): File format not specified, autodetecting...
(Above message occured on Line 18446744073709551615, Column 18446744073709551615)
LibOFX INFO: libofx_proc_file(): Detected file format: OFX (Open Financial eXchange (OFX or QFX))
(Above message occured on Line 18446744073709551615, Column 18446744073709551615)
LibOFX INFO: sanitize_proprietary_tags() (end of line) removed: <INTU.BID>10898

(Above message occured on Line 18446744073709551615, Column 18446744073709551615)
LibOFX STATUS: find_dtd():DTD found: /usr/share/libofx/dtd/opensp.dcl
(Above message occured on Line 18446744073709551615, Column 18446744073709551615)
LibOFX STATUS: find_dtd():DTD found: /usr/share/libofx/dtd/ofx160.dtd
(Above message occured on Line 18446744073709551615, Column 18446744073709551615)
LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate SIGNONMSGSRSV1
(Above message occured on Line 2, Column 1)
LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate SONRS
(Above message occured on Line 3, Column 1)
LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate FI
(Above message occured on Line 10, Column 1)
LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate CREDITCARDMSGSRSV1
(Above message occured on Line 16, Column 1)
LibOFX INFO: Created OfxDummyContainer to hold unsupported aggregate CCSTMTTRNRS
(Above message occured on Line 17, Column 1)
Segmentation fault (core dumped)

=============================================

Second instance:
/usr/bin/kmymoney Bank.qfx

WebConnect: Running in client mode
WebConnect: Pass to server QUrl("file:///Shared/Documents/KMM/Chase.qfx")
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
terminate called after throwing an instance of 'MyMoneyException'
  what():  No storage object attached to MyMoneyFile /usr/local/src/kmm/kmymoney-MASTER-2018.08.03-12.23.55/kmymoney/mymoney/mymoneyfile.cpp:212
Aborted (core dumped)
Comment 4 Thomas Baumgart 2018-08-07 10:59:07 UTC
Created attachment 114352 [details]
Console logs of first and second instance

I added the logs that I get here when running the scenario using KMyMoney master. They certainly look different. Are you also running master? If so, something else interferes.
Comment 5 Brendan 2018-08-07 16:43:53 UTC
Yes I am running the master from a couple of days ago.

This is the same system that does not recognize encrypted files.
Comment 6 Thomas Baumgart 2018-08-11 19:56:01 UTC
It looks like there is something very strange going on. Please check for any parts of an older installation which might be in the way. Your startup needs to look the same as I have posted it as attachment in comment 4.
Comment 7 Brendan 2018-08-13 21:47:55 UTC
How do I find and remove left over files from 4.8?

I found a directory:

/usr/share/kde4/apps/kmymoney

Should I remove it?

What else should I be looking for.
Comment 8 Brendan 2018-08-13 22:51:07 UTC
I removed KMM 5 from my system and then I deleted every directory and file on my system with kmymoney in the name. I compiled KMM from the master, installed it and there is no change in the behavior when I try to open an ofx file from Firefox while KMM is already running.

Not sure what to do next.
Comment 9 Thomas Baumgart 2018-09-08 13:09:41 UTC
*** Bug 396147 has been marked as a duplicate of this bug. ***
Comment 10 Brendan 2018-09-21 23:12:18 UTC
I removed the version of KMM that I compiled from the master and installed KMM 5 from the Fedora 28 repositories. I still have the same problem opening and OFX file when KMM is already running.

The version from the repos also does not recognize an encrypted KMM file.

If there is something left on my system from the 4.8 version I have not been able to track it down.
Comment 11 Thomas Baumgart 2018-09-30 20:55:13 UTC
Git commit 73dc6c0e163c3f5ce34b54595bdf1e989f14ff21 by Thomas Baumgart.
Committed on 30/09/2018 at 20:55.
Pushed by tbaumgart into branch '5.0'.

Prevent crash during webconnect import using firefox

Calling the eventloop during the emission of a signal emitted by a
QTcpSocket causes problems. The fix is to call the receiver with all the
processing and dialog logic from within the eventloop itself using a
delayed call.
FIXED-IN: 5.0.2

M  +6    -1    kmymoney/kmymoney.cpp
M  +1    -1    kmymoney/kmymoney.h

https://commits.kde.org/kmymoney/73dc6c0e163c3f5ce34b54595bdf1e989f14ff21
Comment 12 Brendan 2018-09-30 22:00:27 UTC
Good news, bad news.

Good: KMM does not crash and it does import the new transactions.

Bad: KMM seems to be opening a second copy of itself and I'm getting a couple of error messages, both containing the "No storage object attached to MyMoneyFile ..." error that I mentioned earlier.

Will this be moving to the master branch? I had to switch to the 5.0 branch to get this to work.
Comment 13 Thomas Baumgart 2018-10-13 14:49:17 UTC
Can you double check again if it still fails? You could do that either on master or 5.0. TIA.
Comment 14 wgking99@yahoo.com 2018-10-13 18:18:00 UTC
I am running KMM as delivered by KUBUNTU repos:
wgking@KDEKMM:~$ cat /proc/version
Linux version 4.15.0-36-generic (buildd@lgw01-amd64-031) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018
wgking@KDEKMM:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.1 LTS
Release:        18.04
Codename:       bionic

Kmymoney looks to be v5.0.1-2 .

If I try to open a bank transaction download with KMM, it analyzes the input then crashes after the OK is clicked and the transactions are not processed.

Since I did not build this version, I am reluctant to start patching it from source. I will gladly retest when the patch hits the Kubuntu repos.
Comment 15 Jack 2018-10-13 19:31:28 UTC
UNMAINTAINED would only be appropriate if this were a problem in 4.x.  5.x is definitely still maintained.  This should be left in WAITINGFORINFO.  Note that we are not asking anyone to patch something, but to compile from source from git.  If you can't or won't compile from source, then as you say, you will just have to wait.  The issue is that we prefer to know if a patch really fixes the problem before we include it in a released version, and that is best done by the person who reported the bug in the first place, since often, the developer cannot reproduce exactly the same situation.

However, it is not even clear if you are talking about the same problem.  You say you are trying to open a transaction downloaded with KMM.  Is that the case, or is it crashing trying to do the import?  If the download completes OK, then pleas file a separate bug.  If the crash is during the download, then just follow this bug for updates,
Comment 16 wgking99@yahoo.com 2018-10-13 22:45:34 UTC
It is the same problem. Steps to reproduce are:
 1. Start KMM 
 2. visit your bank website (using Firefox in my case)
 3. Ask for a transaction download (msmoney/.ofx file format in my case)
 4. Select "open with KMM" rather than "Download file"
 5. KMM appears to scan the transactions and recognize new, dups, etc. but crashes if transactions accepted.
 
Downloading the .ofx file and using "File->Import" in KMM works flawlessly and is my workaround for now.
Comment 17 Jack 2018-10-14 00:26:17 UTC
Thanks.  That's good information that might help the developers.  Hopefully someone else (Brendon?) who experienced this crash, and can compile a version from git can check whether the new version fixes it.
Comment 18 Andrew Crouthamel 2018-10-29 03:33:56 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days, the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information.

For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please set the bug status as REPORTED so that the KDE team knows that the bug is ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 19 Bug Janitor Service 2018-11-13 14:43:40 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!
Comment 20 Brendan 2018-11-14 18:41:38 UTC
I have finally tracked down the cause of KMM crashing when importing an OFX/QFX file directly from Firefox. It only happens when I have selected "Tabbed View".

It does not happen when I use "List View" or "Tree View".

I had Deja Vu  when I figured this out after a lot of trial and error. I think this was solved in version 4 a long time ago.
Comment 21 Brendan 2018-11-14 18:52:00 UTC
See bug 257515 for info on my deja vu. It refers to bug 270493.

Tabbed View mode was causing a crash when I exited KMM at that time so it's not the same symptom. There is some mention of storage so maybe the bug fix didn't make it to KMM5.
Comment 22 Thomas Baumgart 2018-11-25 16:31:35 UTC
Git commit f580ae94efa58bfbe96cb5418cb12d717f83df2b by Thomas Baumgart.
Committed on 25/11/2018 at 16:19.
Pushed by tbaumgart into branch '5.0'.

Don't update views if no storage element is attached

In case of WebConnect and tabbed view views become active during
destruction of objects with no storage attached. This change prevents to
update the views in this case causing the crash.
FIXED-IN: 5.0.3

M  +12   -11   kmymoney/plugins/views/reports/kreportsview.cpp
M  +36   -32   kmymoney/views/kgloballedgerview.cpp
M  +12   -10   kmymoney/views/kpayeesview.cpp
M  +1    -1    kmymoney/views/kscheduledview.cpp
M  +12   -10   kmymoney/views/ktagsview.cpp
M  +5    -3    kmymoney/views/simpleledgerview.cpp

https://commits.kde.org/kmymoney/f580ae94efa58bfbe96cb5418cb12d717f83df2b