Bug 405293 - CSV importer does not handle transfers
Summary: CSV importer does not handle transfers
Status: REPORTED
Alias: None
Product: kmymoney
Classification: Applications
Component: importer (show other bugs)
Version: git (master)
Platform: unspecified All
: NOR normal
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL: https://forum.kde.org/viewtopic.php?f...
Keywords:
Depends on:
Blocks:
 
Reported: 2019-03-10 08:40 UTC by Thomas Baumgart
Modified: 2023-02-22 15:28 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Sample Bank CSV Export #1 (995 bytes, text/plain)
2019-03-14 03:49 UTC, Jim Davis
Details
Sample Bank CSV Export #2 of 3 (7.91 KB, text/plain)
2019-03-14 03:50 UTC, Jim Davis
Details
Sample Bank CSV Export #3 of 3 (6.28 KB, text/plain)
2019-03-14 03:51 UTC, Jim Davis
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Baumgart 2019-03-10 08:40:08 UTC
SUMMARY
In case the CSV input data does contain a category column, and the column is filled with the name of another (asset or liability) account then the CSV importer will create an income/expense account with that name.

STEPS TO REPRODUCE
1. Create a kmy file with two asset accounts A1 and A2
2. Create a csv file with a category column
3. Add a transaction to that file with A2 in the category column  
4. Import the file

OBSERVED RESULT
Depending on the sign of the amount, an income or expense category with name A2 is created

EXPECTED RESULT
Account A2 will be used

ADDITIONAL INFORMATION
I think that it is very unlikely that a bank will deliver categorized entries. So this problem more or less appears when you try to move historical data from another "application" into KMyMoney.

It is therefor valid from my POV that in case of an asset/liability account a special mark (e.g. enclose name in brackets ) can be used to support the importer in making a decision. Another way would be to allow an optional path element as first part of the name that provides information about the account type (e.g Asset, Liability, Expense, Income) or just the first letter. For this, we have to consider translations as well.
Comment 1 Jim Davis 2019-03-10 17:05:20 UTC
As I was the one who had the problem which Thomas describes in opening this bug (and heled me fix- Thanks Thomas!), I want to clarify that here in the U.S. it appears to be common for banks to go to CSV file formats as the ONLY transaction download option.

In the case of my two banks ("Credit Union" Banks) both offer "Category" AND "Tag" columns, and both include an "Account" column. Both banks allow a user of their web site to custom categorize transactions, one bank putting these in a "tags" column and the other in 2 cols of "Parent Category" and "Category". 

Understanding how KMyMoney processes (Allocates Payee/Description vs. Category) these columns was my dilemma, and I think the documentation should detail the process, unless Thomas's suggested fix is considered a better option.
Comment 2 Jim Davis 2019-03-14 03:49:11 UTC
Created attachment 118793 [details]
Sample Bank CSV Export #1

Attached are three csv files labelled BankA, BankB, and BankA_all_accts. These are a sample of 1 month of historical transactions, with redacted (replaced) account numbers and in some cases a note in the format (jd note: xxx). Bank "B" is my primary checking & savings, Bank "A" is a secondary checking but has better organizational and external account linking abilities.

In both cases, csv output can be combined (all internal and external linked accounts) or separated by account.
Currently only Bank "A" is linked to Bank "B" but both have an option to link to other (external) bank transactions.

Bank "A" allows for both a custom description as well as 2 levels of categories with splits, plus tags.
Bank "B" allows for a custom "nickname (description) and has only 1 category with splits and call them "tags" in the csv.


Bank "A" web software:		MX	http://www.mx.com/company/
			also ref:   https://webcache.googleusercontent.com/search?q=cache:uZjwPbc_rVoJ:https://www.crunchbase.com/organization/moneydesktop+&cd=4&hl=en&ct=clnk&gl=us

Bank "B" web software:		Fiserv	https://www.fiserv.com/customer-channel-management/internet-banking.aspx
Comment 3 Jim Davis 2019-03-14 03:50:28 UTC
Created attachment 118794 [details]
Sample Bank CSV Export #2 of 3

Sample Bank CSV Export #2 of 3
Comment 4 Jim Davis 2019-03-14 03:51:29 UTC
Created attachment 118795 [details]
Sample Bank CSV Export #3 of 3

Sample Bank CSV Export #3 of 3
Comment 5 Jim Davis 2019-03-14 03:52:22 UTC
Attached are three csv files labelled BankA, BankB, and BankA_all_accts. These are a sample of 1 month of historical transactions, with redacted (replaced) account numbers and in some cases a note in the format (jd note: xxx). Bank "B" is my primary checking & savings, Bank "A" is a secondary checking but has better organizational and external account linking abilities.

In both cases, csv output can be combined (all internal and external linked accounts) or separated by account.
Currently only Bank "A" is linked to Bank "B" but both have an option to link to other (external) bank transactions.

Bank "A" allows for both a custom description as well as 2 levels of categories with splits, plus tags.
Bank "B" allows for a custom "nickname (description) and has only 1 category with splits and call them "tags" in the csv.


Bank "A" web software:		MX	http://www.mx.com/company/
			also ref:   https://webcache.googleusercontent.com/search?q=cache:uZjwPbc_rVoJ:https://www.crunchbase.com/organization/moneydesktop+&cd=4&hl=en&ct=clnk&gl=us

Bank "B" web software:		Fiserv	https://www.fiserv.com/customer-channel-management/internet-banking.aspx
Comment 6 Don S. 2023-02-21 19:10:39 UTC
This bug bit me too while trying to import a csv file that contains a lot of tranfers. I tried putting [brackets] around the account name in the csv file but that didn't work either.

 I hoped to be able to pre-process csv files and import into KMyMoney to save some time and errors but that doesn't seem possible since this bug has existed for more than 3 years.
Comment 7 Don S. 2023-02-21 19:12:32 UTC
This bug bit me too while trying to import a csv file that contains a lot of tranfers. I tried putting [brackets] around the account name in the csv file but that didn't work either.

 I hoped to be able to pre-process csv files and import into KMyMoney to save some time and errors but that doesn't seem possible since this bug has existed for more than 3 years.
Comment 8 Jack 2023-02-21 19:29:17 UTC
Yes, the "bug" still exists, but how does that make it impossible for you to import your data?  The only issue is that you will need to manually edit each transfer transaction and specify the other account.  In case a new category was created with the name of the other account, you can delete it.  Finding unused Categories is not as easy as finding unused Payees, but if you make sure not to hide unused Categories, you can sort the Categories view by Total Value, putting the unused ones at the top of the expense and income lists.
Comment 9 Don S. 2023-02-21 19:45:18 UTC
Jack, I should have said 'impractical' rather than 'impossible'. It make my play to pre-process the csv files to save time every month impractical.

But,, I just thought of a ?possible? workaround. I can put something in the csv Payee field and use KMyMoney's Payees Match and Default category. That's assuming that KMyMoney's csv import processes those rules when importing.
Comment 10 Jack 2023-02-21 19:49:47 UTC
As far as I know, it does, so give that a try and let us know how it works.  As a separate possibility, what about just using a single dummy Category with no Payee?  You could even put the other account in the memo if it's not going to be obvious to you what it should be.  That way, you would just have to change the dummy Category to the other account, and not need to remove the (dummy?) Payee.
Comment 11 Don S. 2023-02-21 20:11:03 UTC
Good idea. Also, after years of using KMyMoney, I just realized that I can select multiple transactions and it can change their categories in one step. Using the filter to find them will make it even easier.

But I'll try the other idea with the Payee processing tomorrow and let you know how it goes. I have to go somewhere, otherwise I'd try it now.
Comment 12 Don S. 2023-02-22 14:06:03 UTC
I just tried using Payee Matching to import a test file and that didn't work to create a transfer transaction. It didn't work. It created a new category like it did before.

Jack: Your idea of entering a unique category in the csv and changing the transactions manually seems to be the quickest way to handle it. Thanks!
Comment 13 Jack 2023-02-22 15:28:24 UTC
Note I did not expect Payee matching to create a transfer.  That make sense to me, as transfers very rarely need a Payee, although there are use cases, which have been discussed on the mailing list in the past.  I actually wonder what happens if there is already a transfer transaction for that Payee  - might the imported transaction match, and thus be imported as a transfer.  Remember importing involves both Payee matching and transaction matching, which are not the same thing.  Transaction matching fills in some of the fields missing from the import if it finds a an existing transaction which matches closely enough.  More stuff to test.