Bug 412494 - Skrooge flatpak silently fails to import anything from Amex QFX
Summary: Skrooge flatpak silently fails to import anything from Amex QFX
Status: RESOLVED FIXED
Alias: None
Product: skrooge
Classification: Applications
Component: general (other bugs)
Version First Reported In: Trunk from git
Platform: Flatpak Linux
: NOR normal
Target Milestone: 25.1.0
Assignee: Guillaume DE BURE
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-10-01 00:31 UTC by skierpage
Modified: 2019-10-05 09:01 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
dummy QFX file with unexpected attribute on <OFX> (1.06 KB, application/xml)
2019-10-02 07:15 UTC, skierpage
Details

Note You need to log in before you can comment on or make changes to this bug.
Description skierpage 2019-10-01 00:31:55 UTC
SUMMARY
Skrooge fails to import anything with no warning from Amex QFX transaction download file. The format changed in 2019, it now produces a QFX 2 file with <OFX xmlns:ns2="http://ofx.net/types/2003/04"> , and libofx 0.9.14 can't handle the XML namespacing.

STEPS TO REPRODUCE
1. Download transactions from Amex in Quicken format.
2. Open Skrooge, File > Import > Import... the QFX file
3. Run the command-line ofxdump path/to/file.qfx

OBSERVED RESULT
Skrooge footer displays "1 file successfully imported", but no accounts or operations are created.
Even SKGTRACE=15 produces no error or warning output.

ofxdump prints errors like
  LibOFX ERROR: libofx_proc_file(): Detected file format not yet supported ou couldn't detect file format; aborting.


EXPECTED RESULT
a) It would be immensely helpful if Skrooge displayed a message like "0 operations imported" that would hint that nothing useful happened.
b) Skrooge needs to use the latest libofx 0.9.15 which the very helpful cstim just released.

SOFTWARE/OS VERSIONS

Linux/KDE Plasma: 
KDE Frameworks Version: flatpak runtime org.kde.Platform  5.12

ADDITIONAL INFORMATION
The bug is the same as https://bugs.gnucash.org/show_bug.cgi?id=797081

I thought I filed this already but can't find the bug report.
Comment 1 Stephane MANKOWSKI 2019-10-01 18:30:34 UTC
Hi,

Could you provide a sample QFX file?
Comment 2 skierpage 2019-10-02 07:15:35 UTC
Created attachment 122975 [details]
dummy QFX file with unexpected attribute on <OFX>

libofx 0.9.14 and below can't handle this file
Comment 3 skierpage 2019-10-02 07:18:08 UTC
(In reply to Stephane MANKOWSKI from comment #1)
> Could you provide a sample QFX file?
The attached file errors with ofxdump 0.9.13 and prints its one transaction with ofxdump 0.9.15. The Skrooge 2.21.0BETA flatpak silently fails to import its one transaction yet displays "1 file successfully imported". If I delete the xmlns:ns2 attribute from <OFX xmlns:ns2="http://ofx.net/types/2003/04">, both ofxdump and skrooge process the file and display/import its one transaction.
Comment 4 Stephane MANKOWSKI 2019-10-02 18:26:36 UTC
Git commit b5fce96af90467786b862d5fd9652851cde39682 by Stephane MANKOWSKI.
Committed on 02/10/2019 at 18:26.
Pushed by smankowski into branch 'master'.

Skrooge flatpak silently fails to import anything from Amex QFX

M  +1    -0    CHANGELOG
M  +4    -0    plugins/import/skrooge_import_ofx/skgimportpluginofx.cpp
M  +5    -2    skgbankmodeler/skgimportexportmanager.cpp
A  +3    -0    tests/input/skgtestimportofx/412494.qfx
M  +24   -0    tests/skgbankmodelertest/skgtestimportofx.cpp

https://commits.kde.org/skrooge/b5fce96af90467786b862d5fd9652851cde39682
Comment 5 Stephane MANKOWSKI 2019-10-02 18:28:56 UTC
Hi,

libofx library doesn't allow to check if file is well detected.
I opened 2 incidents to improve libofx:
https://github.com/libofx/libofx/issues/30
https://github.com/libofx/libofx/issues/29

I did a modification in skrooge to display the number of operations imported.
Comment 6 skierpage 2019-10-03 02:07:12 UTC
Sounds great, thanks. https://phabricator.kde.org/D24327 updates the Flatpak recipe to use newer libofx revision.
Comment 7 Stephane MANKOWSKI 2019-10-04 20:06:07 UTC
Done
Comment 8 skierpage 2019-10-05 06:04:09 UTC
(In reply to skierpage from comment #6)
> Sounds great, thanks. https://phabricator.kde.org/D24327 updates the Flatpak
> recipe to use newer libofx revision.

It turns out that KDEapps uses its own org.kde.skrooge.json, in flatpak-kde-application.git, so the fix still isn't in latest flatpak (though it does say "0 operations imported", nice!). Either the same change has to be propagated to that repo; or KDE sysadmin bcooksley said in #kde-sysadmin "https://cgit.kde.org/flatpak-kde-applications.git/tree/org.kde.skrooge.json is the authoritative file
if Stephane would like to change that [then that repository needs to be updated with a .remoteapp file for Skrooge instead."  I submitted https://phabricator.kde.org/D24418 that I believe makes Skrooge's flatpak recipe authoritative, but of course it's your decision whether you want to couple them more closely.
Comment 9 Stephane MANKOWSKI 2019-10-05 09:01:00 UTC
Fixed