Bug 276201 - OFX direct connect fails to match downloaded transactions with cleared transactions
Summary: OFX direct connect fails to match downloaded transactions with cleared transa...
Status: RESOLVED FIXED
Alias: None
Product: kmymoney
Classification: Applications
Component: general (show other bugs)
Version: SVN
Platform: Debian testing Linux
: NOR normal
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-21 16:06 UTC by Mike Wolfe
Modified: 2011-06-22 13:32 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mike Wolfe 2011-06-21 16:06:49 UTC
Version:           SVN trunk
OS:                Linux

This problem was introduced within the last few weeks.  I have downgraded to revision 1236403, which does not have the problem.  

When performing an OFX direct download, the transactions are all marked as new, regardless of the fact that they have already been matched and cleared previously.  This leads to having ~30+ transactions that are exact duplicates of the already cleared transactions in accounts.

Reproducible: Always

Steps to Reproduce:
Open an account, select either "update all accounts" or "update".

Actual Results:  
All downloaded transactions are treated as new and unique.

Expected Results:  
Only new transactions should be treated as new and unique.
Comment 1 Thomas Baumgart 2011-06-21 20:12:43 UTC
I don't see a difference between said revision and current HEAD which should influence this. Please do the following:

a) create a directory called '/home/thb' and make it writable to your user
b) run an update of a single account
c) There should be a file called /home/thb/kmm-statement-0.txt.
d) keep a copy of it
e) revert to current HEAD, recompile and install
f) repeat step b)
g) There should be a file called /home/thb/kmm-statement-1.txt
h) run a diff c) and g), make sure to replace any information you don't want to disclose with # for each digit and X for each letter.
i) send the result to me via private e-mail
j) once done, remove the directory created in a) and all files in it
Comment 2 Brendan 2011-06-21 21:00:16 UTC
I'm having the same problem. This started after I upgraded from Fedora 14 to
Fedora 15 but I don't think it's related.

A few days before I upgraded I could no longer compile the latest source
from SNV. I was getting version errors for libofx, aqbanking and gwenhywfar.
F15 had all of the necessary versions so I upgraded.

I'm currently running:

KMM 4.5.96-svn1237895

libofx-0.9.4

gwenhywfar-4.0.5

aqbanking-5.0.2


When I download my transactions for 30 days I get duplicates.


I uninstalled KMM from source and installed the package that comes with F15
(kmymoney-4.5.3). The duplicate problem goes away. I appears something has
changed recently that has caused this problem. The last day I was able to
compile from SVN on my old system (F14) was June 10 so I suspect the
duplicate problem was introduced after June 10.


For now I am shortening my days back for OFX imports to avoid this problem
as much as possible. Let me know if you need someone to test a fix and I
will gladly try to download duplicates again.


I'm pretty sure this has happened before. I will search the archives to see
if I can figure out how it was solved the last time it happened.


----
Brendan





On Tue, Jun 21, 2011 at 7:06 AM, Mike Wolfe <wolfemi1@gmail.com> wrote:

> https://bugs.kde.org/show_bug.cgi?id=276201
>
>           Summary: OFX direct connect fails to match downloaded
>                    transactions with cleared transactions
>           Product: kmymoney4
>           Version: SVN trunk
>          Platform: Debian testing
>        OS/Version: Linux
>            Status: UNCONFIRMED
>          Severity: normal
>          Priority: NOR
>         Component: general
>        AssignedTo: kmymoney-devel@kde.org
>        ReportedBy: wolfemi1@gmail.com
>
>
> Version:           SVN trunk
> OS:                Linux
>
> This problem was introduced within the last few weeks.  I have downgraded
> to
> revision 1236403, which does not have the problem.
>
> When performing an OFX direct download, the transactions are all marked as
> new,
> regardless of the fact that they have already been matched and cleared
> previously.  This leads to having ~30+ transactions that are exact
> duplicates
> of the already cleared transactions in accounts.
>
> Reproducible: Always
>
> Steps to Reproduce:
> Open an account, select either "update all accounts" or "update".
>
> Actual Results:
> All downloaded transactions are treated as new and unique.
>
> Expected Results:
> Only new transactions should be treated as new and unique.
>
> --
> Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You are the assignee for the bug.
> _______________________________________________
> KMyMoney-devel mailing list
> KMyMoney-devel@kde.org
> https://mail.kde.org/mailman/listinfo/kmymoney-devel
>
Comment 3 Mike Wolfe 2011-06-22 02:21:25 UTC
Thomas,

I followed your instructions.  However, for step f), I had to rename the 
kmm-statement-0.  I think I may have odd permissions issues.

Anyway, here's the (hopefully) sufficiently sanitized output files.  To 
my untrained eye, it looks like the part after the bankid field is 
simply not written, and the closing quotation mark is missing there.  I 
also checked the actual bankid fields (which I sanitized), and the 
entire bankid number is present in both the broken and working versions.

-Mike


BROKEN:

<!DOCTYPE KMYMONEY-STATEMENT>
<KMYMONEY-STATEMENT>
<STATEMENT version="1.1" currency="USD" accountid="A000138" 
skipCategoryMatching="0" accountnumber="# " accountname="Credit card #" 
type="creditcard" begindate="2011-05-21" enddate="2011-06-21" 
routingnumber="" closingbalance="-15467/25">
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-05-20" 
reconcile="0" amount="-153/20" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="PHO SQUARE RESTAURANT" dateposted="2011-05-30" 
reconcile="0" amount="-823/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="PHO SQUARE RESTAURANT"/>
<TRANSACTION payee="CITY OF ROCKFORD ONLINE" dateposted="2011-06-09" 
reconcile="0" amount="-907/20" action="none" bankid="ID # 
                                                                                                                                                                                                                                " number="" memo="CITY OF ROCKFORD ONLINE"/>
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-05-26" 
reconcile="0" amount="-273/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="BIEN ETRE KIM" dateposted="2011-05-22" reconcile="0" 
amount="-30/1" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="BIEN ETRE KIM"/>
<TRANSACTION payee="Amazon.com" dateposted="2011-06-07" reconcile="0" 
amount="-462/25" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="Amazon.com"/>
<TRANSACTION payee="STATE FARM INSURANCE" dateposted="2011-05-20" 
reconcile="0" amount="-9751/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="STATE FARM INSURANCE"/>
<TRANSACTION payee="PHO SQUARE RESTAURANT" dateposted="2011-06-14" 
reconcile="0" amount="-823/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="PHO SQUARE RESTAURANT"/>
<TRANSACTION payee="TCGPLAYER.COM" dateposted="2011-05-31" reconcile="0" 
amount="-629/20" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="TCGPLAYER.COM"/>
<TRANSACTION payee="HAPPY WOK RESTAURANT I" dateposted="2011-05-23" 
reconcile="0" amount="-78/5" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="HAPPY WOK RESTAURANT I"/>
<TRANSACTION payee="HAPPY WOK" dateposted="2011-06-15" reconcile="0" 
amount="-153/25" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="HAPPY WOK"/>
<TRANSACTION payee="TENSUKE MARKET" dateposted="2011-05-29" 
reconcile="0" amount="-3579/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="TENSUKE MARKET"/>
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-05-19" 
reconcile="0" amount="-273/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="GRAY'S FOODS" dateposted="2011-05-25" reconcile="0" 
amount="-733/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="GRAY'S FOODS"/>
<TRANSACTION payee="FIVE BELOW #708" dateposted="2011-06-13" 
reconcile="0" amount="-379/25" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="FIVE BELOW #708"/>
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-06-12" 
reconcile="0" amount="-287/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="HAPPY WOK RESTAURANT I" dateposted="2011-06-06" 
reconcile="0" amount="-217/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="HAPPY WOK RESTAURANT I"/>
<TRANSACTION payee="ROCK VALLEY GREENHOUSE" dateposted="2011-05-22" 
reconcile="0" amount="-3643/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="ROCK VALLEY GREENHOUSE"/>
<TRANSACTION payee="PHO SQUARE RESTAURANT" dateposted="2011-06-03" 
reconcile="0" amount="-823/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="PHO SQUARE RESTAURANT"/>
<TRANSACTION payee="MENARDS 3042" dateposted="2011-05-20" reconcile="0" 
amount="-4063/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="MENARDS 3042"/>
<TRANSACTION payee="Payment - Thank You" dateposted="2011-06-16" 
reconcile="0" amount="400/1" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="Payment - Thank You"/>
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-05-22" 
reconcile="0" amount="-273/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="Amazon.com" dateposted="2011-06-17" reconcile="0" 
amount="462/25" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="Amazon.com"/>
<TRANSACTION payee="STATE FARM INSURANCE" dateposted="2011-06-20" 
reconcile="0" amount="-9751/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="STATE FARM INSURANCE"/>
<TRANSACTION payee="GRAY'S FOODS" dateposted="2011-05-29" reconcile="0" 
amount="-1167/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="GRAY'S FOODS"/>
<TRANSACTION payee="TENSUKE MARKET" dateposted="2011-05-29" 
reconcile="0" amount="-217/10" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="TENSUKE MARKET"/>
<TRANSACTION payee="AMAZON MKTPLACE PMTS" dateposted="2011-06-08" 
reconcile="0" amount="-471/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="AMAZON MKTPLACE PMTS"/>
<TRANSACTION payee="SYNDICATE SALES KITCHEN" dateposted="2011-06-05" 
reconcile="0" amount="-406/5" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SYNDICATE SALES KITCHEN"/>
</STATEMENT>
</KMYMONEY-STATEMENT>


UNBROKEN:

<!DOCTYPE KMYMONEY-STATEMENT>
<KMYMONEY-STATEMENT>
<STATEMENT version="1.1" currency="USD" accountid="A000138" 
skipCategoryMatching="0" accountnumber="# " accountname="Credit card #" 
type="creditcard" begindate="2011-05-21" enddate="2011-06-21" 
routingnumber="" closingbalance="-15467/25">
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-05-20" 
reconcile="0" amount="-153/20" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="PHO SQUARE RESTAURANT" dateposted="2011-05-30" 
reconcile="0" amount="-823/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="PHO SQUARE RESTAURANT"/>
<TRANSACTION payee="CITY OF ROCKFORD ONLINE" dateposted="2011-06-09" 
reconcile="0" amount="-907/20" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="CITY OF ROCKFORD ONLINE"/>
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-05-26" 
reconcile="0" amount="-273/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="BIEN ETRE KIM" dateposted="2011-05-22" reconcile="0" 
amount="-30/1" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="BIEN ETRE KIM"/>
<TRANSACTION payee="Amazon.com" dateposted="2011-06-07" reconcile="0" 
amount="-462/25" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="Amazon.com"/>
<TRANSACTION payee="STATE FARM INSURANCE" dateposted="2011-05-20" 
reconcile="0" amount="-9751/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="STATE FARM INSURANCE"/>
<TRANSACTION payee="PHO SQUARE RESTAURANT" dateposted="2011-06-14" 
reconcile="0" amount="-823/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="PHO SQUARE RESTAURANT"/>
<TRANSACTION payee="TCGPLAYER.COM" dateposted="2011-05-31" reconcile="0" 
amount="-629/20" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="TCGPLAYER.COM"/>
<TRANSACTION payee="HAPPY WOK RESTAURANT I" dateposted="2011-05-23" 
reconcile="0" amount="-78/5" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="HAPPY WOK RESTAURANT I"/>
<TRANSACTION payee="HAPPY WOK" dateposted="2011-06-15" reconcile="0" 
amount="-153/25" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="HAPPY WOK"/>
<TRANSACTION payee="TENSUKE MARKET" dateposted="2011-05-29" 
reconcile="0" amount="-3579/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="TENSUKE MARKET"/>
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-05-19" 
reconcile="0" amount="-273/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="GRAY'S FOODS" dateposted="2011-05-25" reconcile="0" 
amount="-733/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="GRAY'S FOODS"/>
<TRANSACTION payee="FIVE BELOW #708" dateposted="2011-06-13" 
reconcile="0" amount="-379/25" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="FIVE BELOW #708"/>
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-06-12" 
reconcile="0" amount="-287/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="HAPPY WOK RESTAURANT I" dateposted="2011-06-06" 
reconcile="0" amount="-217/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="HAPPY WOK RESTAURANT I"/>
<TRANSACTION payee="ROCK VALLEY GREENHOUSE" dateposted="2011-05-22" 
reconcile="0" amount="-3643/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="ROCK VALLEY GREENHOUSE"/>
<TRANSACTION payee="PHO SQUARE RESTAURANT" dateposted="2011-06-03" 
reconcile="0" amount="-823/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="PHO SQUARE RESTAURANT"/>
<TRANSACTION payee="MENARDS 3042" dateposted="2011-05-20" reconcile="0" 
amount="-4063/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="MENARDS 3042"/>
<TRANSACTION payee="Payment - Thank You" dateposted="2011-06-16" 
reconcile="0" amount="400/1" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="Payment - Thank You"/>
<TRANSACTION payee="SUBWAY        00028902" dateposted="2011-05-22" 
reconcile="0" amount="-273/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SUBWAY        00028902"/>
<TRANSACTION payee="Amazon.com" dateposted="2011-06-17" reconcile="0" 
amount="462/25" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="Amazon.com"/>
<TRANSACTION payee="STATE FARM INSURANCE" dateposted="2011-06-20" 
reconcile="0" amount="-9751/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="STATE FARM INSURANCE"/>
<TRANSACTION payee="GRAY'S FOODS" dateposted="2011-05-29" reconcile="0" 
amount="-1167/100" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="GRAY'S FOODS"/>
<TRANSACTION payee="TENSUKE MARKET" dateposted="2011-05-29" 
reconcile="0" amount="-217/10" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="TENSUKE MARKET"/>
<TRANSACTION payee="AMAZON MKTPLACE PMTS" dateposted="2011-06-08" 
reconcile="0" amount="-471/50" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="AMAZON MKTPLACE PMTS"/>
<TRANSACTION payee="SYNDICATE SALES KITCHEN" dateposted="2011-06-05" 
reconcile="0" amount="-406/5" action="none" bankid="ID 
#                                                                                                                                                                                                                                
" number="" memo="SYNDICATE SALES KITCHEN"/>
</STATEMENT>
</KMYMONEY-STATEMENT>





















On 6/21/2011 1:12 PM, Thomas Baumgart wrote:
> https://bugs.kde.org/show_bug.cgi?id=276201
>
>
>
>
>
> --- Comment #1 from Thomas Baumgart<ipwizard users sourceforge net>   2011-06-21 20:12:43 ---
> I don't see a difference between said revision and current HEAD which should
> influence this. Please do the following:
>
> a) create a directory called '/home/thb' and make it writable to your user
> b) run an update of a single account
> c) There should be a file called /home/thb/kmm-statement-0.txt.
> d) keep a copy of it
> e) revert to current HEAD, recompile and install
> f) repeat step b)
> g) There should be a file called /home/thb/kmm-statement-1.txt
> h) run a diff c) and g), make sure to replace any information you don't want to
> disclose with # for each digit and X for each letter.
> i) send the result to me via private e-mail
> j) once done, remove the directory created in a) and all files in it
>
Comment 4 Thomas Baumgart 2011-06-22 13:32:08 UTC
SVN commit 1238045 by tbaumgart:

Don't add binary 0's to the IDs.

BUG: 276201

 M  +2 -2      ofximporterplugin.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1238045