Bug 353372

Summary: OFX files are double encoded which messes up the £ sign
Product: [Applications] kmymoney Reporter: Barry Scott <barry>
Component: generalAssignee: KMyMoney Devel Mailing List <kmymoney-devel>
Status: RESOLVED FIXED    
Severity: normal    
Priority: NOR    
Version: 4.7.1   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In: 4.8.0
Attachments: Fix double encoding of OFX files with £ signs

Description Barry Scott 2015-09-30 13:58:58 UTC
OFX import over HTTP/HTTPS downloads data in CP1252 typically.
This is assumed, in error, to be unicode by the OFX import code and saved into a tempfile
encoded as UTF-8.
The UTF-8 encoded tempfile is passed into libofx to parse.
libofx assumes that the encoding is exactly as describer by the OFX headers.
It then uses iconv to convert from the declated encoding of CP1252 into utf-8
ready to pass to OFX imported code.

You can see this problem with an OFX file that contains a £ sign.
After import it shows as two characters.

The attached patch fixes two places I found that this mistake.


Reproducible: Always

Steps to Reproduce:
1. Import an OFX file encoded in CP1252 that containing:
          <STMTTRN>
            <TRNTYPE>OTHER
            <DTPOSTED>20150803000000[-5:EST]
            <TRNAMT>0.04
            <FITID>+201508030000001            
            <NAME>INT EARNED £0.05LESS TAX £0.01IN
          </STMTTRN>
2. See that the Ledger entry is: INT EARNED £0.05LESS TAX £0.01IN


Actual Results:  
£ is shown as £

Expected Results:  
Show £ as £
Comment 1 Barry Scott 2015-09-30 14:01:01 UTC
Created attachment 94785 [details]
Fix double encoding of OFX files with £ signs

This patch has been tested by rpmbuild of fedora kmymoney 4.7.1 + this patch.
Comment 2 Cristian Oneț 2015-12-01 10:46:36 UTC
Git commit 2d4f13c067c44365f535906aabc180d9d8e57a9f by Cristian Oneț.
Committed on 01/12/2015 at 10:43.
Pushed by conet into branch 'master'.

Fix OFX direct connect data encoding.

Write the byte array directly into the file without passing it trough
QTextStream which uses QTextCodec::codecForLocale() to interpret the
data. The encoding is handled by libofx.
FIXED-IN: 4.8.0

M  +1    -2    kmymoney/plugins/ofximport/dialogs/kofxdirectconnectdlg.cpp
M  +2    -3    kmymoney/plugins/ofximport/ofxpartner.cpp

http://commits.kde.org/kmymoney/2d4f13c067c44365f535906aabc180d9d8e57a9f
Comment 3 Christian David 2016-02-24 13:41:02 UTC
Git commit bde946b1e800e40414219397032b8ff484e17733 by Christian Dávid, on behalf of Cristian Oneț.
Committed on 24/02/2016 at 13:30.
Pushed by christiand into branch 'frameworks'.

Fix OFX direct connect data encoding.

Write the byte array directly into the file without passing it trough
QTextStream which uses QTextCodec::codecForLocale() to interpret the
data. The encoding is handled by libofx.
FIXED-IN: 4.8.0

M  +1    -2    kmymoney/plugins/ofximport/dialogs/kofxdirectconnectdlg.cpp
M  +2    -3    kmymoney/plugins/ofximport/ofxpartner.cpp

http://commits.kde.org/kmymoney/bde946b1e800e40414219397032b8ff484e17733