Bug 359874

Most is not a strict term and in my case most imported transactions automatically match.
Please attach .kmy and .csv file you've got problems with.

My .kmy file contains several years of my financial history.  I'm not going to upload it to a public Bugzilla.  I'd send you an "anonymous file" version of it, but since that would change all of the payee names, it would render it essentially useless for diagnosing this issue.  I realize that makes it much more difficult for you to help me, but I'm not comfortable making that much personal data public.

If it helps to quantify the term "most," I just imported 25 transactions today, and 2 of them matched.  Only 6 new payees were created.  I then had to manually match 19 transactions which would have been automatically matched before this bug was introduced.  This is about par for the course.  Each pair of transactions I manually matched were within 30 days of each other.  I don't have time right now to check every one, but at least two or three of the payees involved had payee matching disabled.

How hard would it be to add a global setting to turn off payee matching?  CSV imports worked great until that was introduced.

As of this writing, I am still running KMyMoney 4.7.2.

Today I imported a statement with 45 transactions.  None were automatically matched, and I had to manually match 33 transactions that should have been matched automatically.

Today I imported 22 transactions.  None of them matched, but 19 should have.

(In reply to lwilson1337 from comment #2)
> My .kmy file contains several years of my financial history.  I'm not going
> to upload it to a public Bugzilla.  I'd send you an "anonymous file" version
> of it, but since that would change all of the payee names, it would render
> it essentially useless for diagnosing this issue.  I realize that makes it
> much more difficult for you to help me, but I'm not comfortable making that
> much personal data public.

Couldn't you just invent some names and figures that should cause the same problem? 
If yes then prepare new purely fictional .kmy and .csv and submit it here.

Created attachment 103932 [details]
Fictional CSV

Created attachment 103933 [details]
Fictional KMY

You're right, I could do that quite easily.  When importing the attached .csv into the attached .kmy, KMyMoney fails to match any of the transactions, though they should match 6 of the 7 transactions in the .kmy file.  The payee names differ, but the amounts are the same, and all payees in the .kmy are set to "No matching."

I am still running KMyMoney 4.7.2 and tested with that version.

(In reply to lwilson1337 from comment #8)
> You're right, I could do that quite easily.  When importing the attached
> .csv into the attached .kmy, KMyMoney fails to match any of the
> transactions, though they should match 6 of the 7 transactions in the .kmy
> file.  The payee names differ, but the amounts are the same, and all payees
> in the .kmy are set to "No matching."
> 
> I am still running KMyMoney 4.7.2 and tested with that version.

I used git master version with your .kmy and .csv file and took just single transaction under consideration
in kmy: 9 Feb 2017, Bob's House of cards, payment 15.45
in csv: 11 Feb 2017, BOBSHOUSE EFGHI000001, deposit 15.45

1) Why did you disable payee matching? Do you want "BOBSHOUSE EFGHI000001" payee to be created near "Bob's House of cards" after csv import?

2) Values you try to match are opposite i.e. one is payment and one is deposit. Why should they match?

Anyways, if I change
02/11/2017,15.45,,BOBSHOUSE EFGHI000001
to
02/11/2017,-15.45,,BOBSHOUSE EFGHI000001

and set payee matching to "match on list below" and on the list there is only "^BOB*" then this single transaction gets matched.

Of course you chose the one transaction that I entered into the CSV file incorrectly.  Yes, that should have been a negative value (like all of the others that were in the Payment column in the KMY file).

Could you please explain or point me to good documentation on the matching string format?  What you entered below looks like a regular expression because it uses ^, but it isn't because it uses * instead of .* to match on any character.  Also, the GENERIC UTILITIES DISTRICT entry in my file matched on "UTIL" but not "*UTIL*".  Payee matching didn't work for me at all when I tried it before, and I think this is why.

(Yes, I did RTFM.  It wasn't helpful: http://kmymoney2.sourceforge.net/online-manual/details.payees.personalinformation.html)

As to 1), no, of course I don't want all the bogus names assigned by my bank imported as payees, but I wasn't aware that payee matching could stop that behavior.  I turned it off because I wanted my transactions to match like they did before.  I would still like to be able to turn this off completely and go back to the way things were before.

Thank you for reporting this issue in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the issue with a recent software version?

If you can reproduce the issue, please change the status to "REPORTED" when replying. Thank you!

I don't recall when this was originally filed, but it seems to me KMM is acting as designed.  If I have an imported transactions with the same amount, close date, but different name/payee, I personally would NOT want it to match an existing transaction.  If I did want it to match, I would use the matching tab in the Payees View.  My remaining question is how the matching was done prior to the change introduced in 4.7.1.  However, it's now so long ago, reverting is not really an option, so at best, this might be converted to a wishlist requesting the ability to configure things so the matching behaves as it did prior to 4.7.1, with a clear description of the new criteria for matching in such cases.

If this is "working as designed", could you please provide some documentation on how it's designed?  I haven't tried in a while, but every time I try to set up payee matching, it goes horribly wrong - possibly because I have no idea how the "regular expression" syntax is supposed to work.

I've been matching every transaction manually for so long that I'm almost used to it, but it still bothers me because I know it could be so much easier.  Please don't use the argument that "it's been so long, it's not worth fixing it now," because I've been asking for this since the first broken version came out.

I can reproduce the behavior in the current version of kmymoney, but please be patient with me; I'm very busy and can't devote a lot of time to this (especially since the bug is costing me time every month).  If I had more time, I'd figure out how to write a fix myself.

Created attachment 153624 [details]
Fixed Fictional KMY file

I attached a 'fixed' version of your fictional file. I added the necessary regex to the payees so that matching can happen. Except two, they all match on the full name. Exceptions are "Bob's House of cards" and "Joe's" which only requires the lead-in of the imported payee name to match. I have checked this against https://binary-factory.kde.org/view/AppImage/job/KMyMoney_Release_appimage-centos7/lastStableBuild/ and my development build (based on master). It matches 5 of your 6 transactions in the CSV. The one that is not matched is due to the different sign of the amount. Once you change the sign it matches it as well. When importing the file a second time all transactions are detected as duplicates.

Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!

Thank you, your changes to the sample file gave me a few more clues as to how the payee matching syntax works.  It still didn't answer all my questions, though, and I'd still like to see some documentation.  For example, do I have to escape out asterisks?

I applied what I learned to my personal kmymoney file, and it didn't work reliably.  One or two patterns that only had letters and a single space worked, while many others fitting that same description didn't.  I don't have access to my file at the moment, but when I do I'll try to find some examples that I'm comfortable sharing.

I assume in your testing with that sample file, your reproduced the original issue on the current version of kmymoney, so I've marked the bug REPORTED again.