Bug 412494

Summary:	Skrooge flatpak silently fails to import anything from Amex QFX
Product:	[Applications] skrooge	Reporter:	skierpage <info>
Component:	general	Assignee:	Guillaume DE BURE <guillaume.debure>
Status:	RESOLVED FIXED
Severity:	normal	CC:	stephane
Priority:	NOR
Version First Reported In:	Trunk from git
Target Milestone:	25.1.0
Platform:	Flatpak
OS:	Linux
Latest Commit:	https://commits.kde.org/skrooge/b5fce96af90467786b862d5fd9652851cde39682	Version Fixed In:
Sentry Crash Report:
Attachments:	dummy QFX file with unexpected attribute on <OFX>

Description skierpage 2019-10-01 00:31:55 UTC

SUMMARY
Skrooge fails to import anything with no warning from Amex QFX transaction download file. The format changed in 2019, it now produces a QFX 2 file with <OFX xmlns:ns2="http://ofx.net/types/2003/04"> , and libofx 0.9.14 can't handle the XML namespacing.

STEPS TO REPRODUCE
1. Download transactions from Amex in Quicken format.
2. Open Skrooge, File > Import > Import... the QFX file
3. Run the command-line ofxdump path/to/file.qfx

OBSERVED RESULT
Skrooge footer displays "1 file successfully imported", but no accounts or operations are created.
Even SKGTRACE=15 produces no error or warning output.

ofxdump prints errors like
  LibOFX ERROR: libofx_proc_file(): Detected file format not yet supported ou couldn't detect file format; aborting.


EXPECTED RESULT
a) It would be immensely helpful if Skrooge displayed a message like "0 operations imported" that would hint that nothing useful happened.
b) Skrooge needs to use the latest libofx 0.9.15 which the very helpful cstim just released.

SOFTWARE/OS VERSIONS

Linux/KDE Plasma: 
KDE Frameworks Version: flatpak runtime org.kde.Platform  5.12

ADDITIONAL INFORMATION
The bug is the same as https://bugs.gnucash.org/show_bug.cgi?id=797081

I thought I filed this already but can't find the bug report.

Comment 1 Stephane MANKOWSKI 2019-10-01 18:30:34 UTC

Hi,

Could you provide a sample QFX file?

Comment 2 skierpage 2019-10-02 07:15:35 UTC

Created attachment 122975 [details]
dummy QFX file with unexpected attribute on <OFX>

libofx 0.9.14 and below can't handle this file

Comment 3 skierpage 2019-10-02 07:18:08 UTC

(In reply to Stephane MANKOWSKI from comment #1)
> Could you provide a sample QFX file?
The attached file errors with ofxdump 0.9.13 and prints its one transaction with ofxdump 0.9.15. The Skrooge 2.21.0BETA flatpak silently fails to import its one transaction yet displays "1 file successfully imported". If I delete the xmlns:ns2 attribute from <OFX xmlns:ns2="http://ofx.net/types/2003/04">, both ofxdump and skrooge process the file and display/import its one transaction.

Comment 4 Stephane MANKOWSKI 2019-10-02 18:26:36 UTC

Git commit b5fce96af90467786b862d5fd9652851cde39682 by Stephane MANKOWSKI.
Committed on 02/10/2019 at 18:26.
Pushed by smankowski into branch 'master'.

Skrooge flatpak silently fails to import anything from Amex QFX

M  +1    -0    CHANGELOG
M  +4    -0    plugins/import/skrooge_import_ofx/skgimportpluginofx.cpp
M  +5    -2    skgbankmodeler/skgimportexportmanager.cpp
A  +3    -0    tests/input/skgtestimportofx/412494.qfx
M  +24   -0    tests/skgbankmodelertest/skgtestimportofx.cpp

https://commits.kde.org/skrooge/b5fce96af90467786b862d5fd9652851cde39682

Comment 5 Stephane MANKOWSKI 2019-10-02 18:28:56 UTC

Hi,

libofx library doesn't allow to check if file is well detected.
I opened 2 incidents to improve libofx:
https://github.com/libofx/libofx/issues/30
https://github.com/libofx/libofx/issues/29

I did a modification in skrooge to display the number of operations imported.

Comment 6 skierpage 2019-10-03 02:07:12 UTC

Sounds great, thanks. https://phabricator.kde.org/D24327 updates the Flatpak recipe to use newer libofx revision.

Comment 7 Stephane MANKOWSKI 2019-10-04 20:06:07 UTC

Done

Comment 8 skierpage 2019-10-05 06:04:09 UTC

(In reply to skierpage from comment #6)
> Sounds great, thanks. https://phabricator.kde.org/D24327 updates the Flatpak
> recipe to use newer libofx revision.

It turns out that KDEapps uses its own org.kde.skrooge.json, in flatpak-kde-application.git, so the fix still isn't in latest flatpak (though it does say "0 operations imported", nice!). Either the same change has to be propagated to that repo; or KDE sysadmin bcooksley said in #kde-sysadmin "https://cgit.kde.org/flatpak-kde-applications.git/tree/org.kde.skrooge.json is the authoritative file
if Stephane would like to change that [then that repository needs to be updated with a .remoteapp file for Skrooge instead."  I submitted https://phabricator.kde.org/D24418 that I believe makes Skrooge's flatpak recipe authoritative, but of course it's your decision whether you want to couple them more closely.

Comment 9 Stephane MANKOWSKI 2019-10-05 09:01:00 UTC

Fixed