Bug 272712 - Option to automatically place transactions in categories by fields other than Payee
Summary: Option to automatically place transactions in categories by fields other than...
Status: REPORTED
Alias: None
Product: kmymoney
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Unlisted Binaries Linux
: NOR wishlist
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-07 16:33 UTC by David Alston
Modified: 2022-01-31 20:15 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Regex filter memo field (1.87 KB, application/gzip)
2022-01-31 20:15 UTC, TobIs
Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Alston 2011-05-07 16:33:38 UTC
Version:           unspecified (using KDE 4.6.2) 
OS:                Linux

For some reason my Bank has decided to use what Kmymoney's OFX import sees as the Payee field for strings that are (for the most part) useless for auto-assigning categories (eg. "CHK CRD 100").  The Memo field usually has a better description of who was being paid.

I'd like to reduce the amount of data-entry that I have to do when I import statements as much as possible.  It would be nice if I could auto-assign categories based on arbitrary fields so that it can handle cases like...

* If the Memo field has the word "Kroger" in it assign the category "Grocery"
* If the Payment field has the amount $54.66 and Payee matches "CHECK" assign to category "Internet"

Reproducible: Didn't try
Comment 1 David Alston 2011-05-14 22:39:09 UTC
Honestly, the only part I really need is to be able to match strings in the Memo field to categories.  I even downloaded the source code to see if I could figure out how to do it myself, but I'm not a coder and can't even get it to compile without any changes yet.
Comment 2 David Alston 2011-06-13 22:37:28 UTC
Am I the only one who seems to be running into this problem?
Comment 3 Thomas Baumgart 2011-06-14 09:21:14 UTC
SVN commit 1236585 by tbaumgart:

Allow to base the payee name from either the PAYEEID, NAME or MEMO field in an OFX transaction

BUG: 272712

 M  +2 -4      dialogs/konlinebankingstatus.cpp  
 M  +23 -46    dialogs/konlinebankingstatusdecl.ui  
 M  +51 -17    ofximporterplugin.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1236585
Comment 4 Jack 2011-08-31 16:44:21 UTC
It looks like the original request here was about assigning category, not payee.  Should this be reopened as a wishlist, or should I file a new one?
Comment 5 Thomas Baumgart 2011-08-31 17:43:56 UTC
Yes, I must have mixed things up here. So we simply reopen it.
Comment 6 David Alston 2011-09-03 22:02:55 UTC
@Thomas: thanks for catching this and re-opening it.  It would be extremely helpful to my personal finance workflow.
Comment 7 David Alston 2012-01-04 04:49:49 UTC
poke.  I still can't use kmymoney because this feature doesn't exist.  If no one else is interested we might as well close the bug report.  Even using the CSV importer and making the "Memo" field from my bank (the usually more useful field) the Payee field doesn't work because there are too many transactions where the Memo field in the CSV are either blank, or the "Description" field is better suited to be the payee.

I'll throw some votes on this bug and wait awhile, but if there isn't a big enough need for this feature in the kmymoney community I'll go head and close this bug report.
Comment 8 allan 2012-01-04 10:48:47 UTC
(In reply to comment #7)
> poke.  I still can't use kmymoney because this feature doesn't exist.  If no
> one else is interested we might as well close the bug report.  Even using the
> CSV importer and making the "Memo" field from my bank (the usually more useful
> field) the Payee field doesn't work because there are too many transactions
> where the Memo field in the CSV are either blank, or the "Description" field is
> better suited to be the payee.
> 
> I'll throw some votes on this bug and wait awhile, but if there isn't a big
> enough need for this feature in the kmymoney community I'll go head and close
> this bug report.

In your (comment #1) you said - 
"Honestly, the only part I really need is to be able to match strings in the
Memo field to categories." 

and I'm trying to reconcile this with -
"....doesn't work because there are too many transactions where the Memo field in the CSV are either blank, or the "Description" field is better suited to be the payee."

Are you able to give a better statement of your requirement?
Comment 9 David Alston 2012-01-04 16:25:32 UTC
I can see why that might seem confusing.

Restated: The problem is that sometimes I want to automatically categorize a transaction by the text in the Memo field and sometimes I want to automatically categorize based on the text in the Payee field.  For some reason my banks CSV exports are inconsistent about which field they put information like store names.

Let me know if you need any more information.  Like I said, though, if this feature would only be useful for me we can close this bug report.
Comment 10 Jack 2012-01-04 17:12:37 UTC
David - you are not the only one who would find this useful.  The confusion is whether we are talking about assigning Payee or assigning Category.  Assigning Payee has already been dealt with Thomas' commit in Comment 3.  Assigning Categories has not been done, and should remain a wishlist.  Although you say (in Comment 9) you want to "categorize" a transaction, you also mention "store names" which is more likely to affect Payee then Category.

Note that assigning Category is more difficult than assigning Payee, since even if there is a store name in either of the two fields, a single Payee might have transactions assigned to more than one category, which might or might not be obvious from text in any field in an import file.

However, if you agree we are talking about Category here, I'll add my vote.
Comment 11 David Alston 2012-01-05 16:47:50 UTC
I'm more interested in automatically assigning the Category based on arbitrary fields than setting which field is the Payee.  The latter is a very welcome feature and I'm glad that Thomas has written it into kmymoney, but the assignment of Categories is what takes most of my time when I'm importing bank statements.

I had an idea that Categories could be treated like mail filter rules in an email application (eg. Setup a "Grocery" rule that applies the "Grocery" category to anything that matches "Kroger" or "Aldi" and "Debit > $10", etc..) but that might require much more effort to add.

As long as I have a way to automatically set a Category by matching against at least one arbitrary field in a transaction (not just the Payee field) then I'll be able to import my data into kmymoney with much less effort.
Comment 12 Jack 2012-01-05 17:20:56 UTC
Thanks for the clarification.  

Are you aware you can set a default category for each Payee?

I think the biggest problem here is for Payees that have transactions in more than one category.  I  have two Payees that are generally "groceries," but are "gasoline" if "FUEL" appears in the memo.

Wondering out loud - could this be done as a modification of the default category for a payee by adding additional criteria such as text appearing in a particular or any of a set of import fields?
Comment 13 David Alston 2012-01-05 18:00:43 UTC
I have been using the "set default category per payee" feature.  Unfortunately, the export my bank provides me doesn't consistently provide useful strings to match on in the same field.  Sometimes the more useful string for matching (eg. the grocery store name) is listed in the Memo field while other times it is listed in the Payee field.

As far as transactions that might appear in more than one category are concerned I suspect that that will need to be managed manually by whoever is importing the data and setting up the auto-categorizing.  If I get both gas and groceries at Kroger then I should probably not setup a rule to automatically categorize anything with Kroger in it.. or setup the rule for Groceries and remember to manually change the Category to "Fuel" when I review the transactions.

Some additional thoughts.. I suppose what my feature request is ultimately suggesting is a shift from a Payee-based workflow to a Category-based workflow.  I don't know how it is on other people's statements, but the Payee category seems to be very flexible in the eyes of my bank (sometimes obscure strings, sometimes strings that vary only mildly for transactions with franchises) while my categories remain much more consistent month-to-month.  The more flexible the rules for auto-categorizing the more easily I can manage those rules to reflect my actual cashflow usage.  Right now adjusting the Payee to match on arbitrary strings provides quite a bit of that functionality already; unfortunately I can't fully leverage that feature because my bank's particular export formats does not consistently provide the relevant information in consistent fields.
Comment 14 Jack 2012-01-05 18:35:30 UTC
I think we are going in circles.  I do understand about the bank not being consistent.  However, given that you can look in any of several fields, can you consistently and correctly get transactions assigned to the right Payee?  If so, then using the "set default category" will always set transactions matched to that payee to the same category.  The only issue is that if an import fails to match because the bank has changed again, redo the import telling KMM to use a different field to match the Payee.

One possible enhancement request here is a setting for KMM to look in more than one field in an import (payee, memo, description) to match the payee, and only create a new payee if none of them match.  If this sounds useful, I'll be glad to create a new wishlist entry for it.

Regarding Payee vs Category based workflow, I think that's all in your perception.  For example, I have a Payee of "Misc Grocery" that has entries to match any of the grocery stores that don't have their own Payee.  The "Misc Grocery" payee always uses the "groceries" category.  Even if sometimes I have to change which field KMM matches on, depending on the bank changing its format, it gets the right payee and category.

What you propose - having KMM match the Category before it matches the Payee, still has exactly the same problem.  Having a default Payee for a given Category is essentially the same as having a "Misc X" payee which always uses Category "X" since the effort for KMM to make the match is exactly the same, since you can't predict which import field will have the useful string to match.
Comment 15 David Alston 2012-01-05 21:37:17 UTC
> The only issue is that if an import fails
> to match because the bank has changed again, redo the import telling KMM to use
> a different field to match the Payee.

This would work if I was only importing one transaction.  In the same monthly statement some stores have strings useful for matching Payee's in the Payee field while other stores have their useful string in the "Memo" field.


> One possible enhancement request here is a setting for KMM to look in more than
> one field in an import (payee, memo, description) to match the payee, and only
> create a new payee if none of them match.  If this sounds useful, I'll be glad
> to create a new wishlist entry for it.

We do seem to be typing in circles :^)  I don't really mind what the Payee field is as long as the transaction is categorized correctly.  If we need to, as you suggest, adjust what kmymoney considers the Payee field to be on a per-transaction basis then the "matching strings" functionality of the Payees section would be able to apply the categories to my liking.

Another way to approach it would be to tie the category to the transaction (or it's splits) instead.  This would give the auto-categorizor much more flexability since it wouldn't necessarily be tied to the "Payee" at all.  The auto-categorizor would watch during the imports for anything that matches its rules (eg. "Payee == Aldi" or "Memo == Kroger" or "Memo == Kroger && Debit > 10.00") and apply the appropriate category.  Using this approach we could leave the data as close to the format of the original import as possible since we wouldn't have to figure out rules for "The payee is now in the memo field" or "the Payee is now in the Payee field."

I'm happy with whichever approach fulfills the communities general consensus.  I'm still very impressed with the rich feature set that kmymoney already has.  It really is an exceptional product.


If this wishlist request is too confusing to be useful then it probably does need a new bug number and a new description.
Comment 16 TobIs 2020-08-24 19:21:02 UTC
Pretty old discussion, but I would also be very happy with such a feature.
For e.g. direct debits everything works as expected, payee is detected properly.
But for payments with debit card, all useful information is in the memo field and could be extracted with regex.
So currently I have to manually edit all card payments.
Comment 17 TobIs 2022-01-31 20:15:20 UTC
Created attachment 146116 [details]
Regex filter memo field

Please find attached a quick and dirty approach to implement a more fine granular filter than with a per account setting to extract information from the memo field to identify the payee. Filter settings can be added to a simple JSON file.