Bug 338076 - Download OXF of investment transactions all set to 'category' "investment transaction".
Summary: Download OXF of investment transactions all set to 'category' "investment tra...
Status: RESOLVED WORKSFORME
Alias: None
Product: kmymoney
Classification: Applications
Component: general (show other bugs)
Version: git (master)
Platform: Fedora RPMs Linux
: NOR normal
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords: triaged
Depends on:
Blocks:
 
Reported: 2014-08-06 18:34 UTC by george
Modified: 2018-10-27 03:42 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description george 2014-08-06 18:34:17 UTC
When downloading transactions for an investment account all the new entries in the 'Brokerage' account get a default "category' of "investment transaction".  It would be far better if this actually reflected the nature of the transaction, i.e.:

interest expense
interest income
Dividend income
investment expense  (for general fees and adjustments)

investment transaction (for buy and sell transactions)

I would like to suggest an extension to the account info data where the user could specify what categories s/he wants to use for each of these.

Attempting to 'edit' these transactions is another horror story.
First the security field is cleared.  

The 'category' field becomes 'split' and can not be changed except by clearing the splits and all the 'yes I really do want to do this, please may I'

Sure could be a bit more friendly here, or am I missing something?
Comment 1 george 2014-08-06 18:42:25 UTC
(In reply to george from comment #0)
> When downloading transactions for an investment account all the new entries
> in the 'Brokerage' account get a default "category' of "investment
> transaction".  It would be far better if this actually reflected the nature
> of the transaction, i.e.:
> 
> interest expense
> interest income
> Dividend income
> investment expense  (for general fees and adjustments)
> 
> investment transaction (for buy and sell transactions)
> 
> I would like to suggest an extension to the account info data where the user
> could specify what categories s/he wants to use for each of these.
> 
> Attempting to 'edit' these transactions is another horror story.
> First the security field is cleared.  
> 
> The 'category' field becomes 'split' and can not be changed except by
> clearing the splits and all the 'yes I really do want to do this, please may
> I'
> 
> Sure could be a bit more friendly here, or am I missing something?

And now I find that I can not restore the security name AND the transaction is no longer in the investment account either.
Comment 2 allan 2014-08-06 19:43:57 UTC
(In reply to george from comment #0)
> When downloading transactions for an investment account all the new entries
> in the 'Brokerage' account get a default "category' of "investment
> transaction".  It would be far better if this actually reflected the nature
> of the transaction, i.e.:
> 
> interest expense
> interest income
> Dividend income
> investment expense  (for general fees and adjustments)
> 
> investment transaction (for buy and sell transactions)
> 
> I would like to suggest an extension to the account info data where the user
> could specify what categories s/he wants to use for each of these.
> 
> Attempting to 'edit' these transactions is another horror story.
> First the security field is cleared.  
> 
> The 'category' field becomes 'split' and can not be changed except by
> clearing the splits and all the 'yes I really do want to do this, please may
> I'
> 
> Sure could be a bit more friendly here, or am I missing something?

See my last comment in your other bug,  Bug 337251.  You should be getting the correct category information now.  However, the heading in this bug refers to OFX, where we have been dealing with QIF, I hope.  Or, are you saying that OFX is exhibiting the same issue as was present pre your git download?
Comment 3 george 2014-08-06 22:04:26 UTC
(In reply to allan from comment #2)
> (In reply to george from comment #0)
> > When downloading transactions for an investment account all the new entries
> > in the 'Brokerage' account get a default "category' of "investment
> > transaction".  It would be far better if this actually reflected the nature
> > of the transaction, i.e.:
> > 
> > interest expense
> > interest income
> > Dividend income
> > investment expense  (for general fees and adjustments)
> > 
> > investment transaction (for buy and sell transactions)
> > 
> > I would like to suggest an extension to the account info data where the user
> > could specify what categories s/he wants to use for each of these.
> > 
> > Attempting to 'edit' these transactions is another horror story.
> > First the security field is cleared.  
> > 
> > The 'category' field becomes 'split' and can not be changed except by
> > clearing the splits and all the 'yes I really do want to do this, please may
> > I'
> > 
> > Sure could be a bit more friendly here, or am I missing something?
> 
> See my last comment in your other bug,  Bug 337251.  You should be getting
> the correct category information now.  However, the heading in this bug
> refers to OFX, where we have been dealing with QIF, I hope.  Or, are you
> saying that OFX is exhibiting the same issue as was present pre your git
> download?

One of the reasons for a new bug report is that, yes, this refers to an OFX download. I suspect the bit of code I am talking about is shared by both OFX
QIF.

Let me see if I understand.  I do see the info in the split.  Are you saying that the information in the ledger is from the split lines?
How would I go about getting a report on Div income? Him, it looks like if I 
tell the "Transactions by account" report generator to look for "Div" I get what I want.

Not sure if this is the place, but I have done a few translations in my QIF "filter" that I think ought to be in KMM.  How should I present these to you?
Comment 4 allan 2014-08-06 22:51:52 UTC
> > See my last comment in your other bug,  Bug 337251.  You should be getting
> > the correct category information now.  However, the heading in this bug
> > refers to OFX, where we have been dealing with QIF, I hope.  Or, are you
> > saying that OFX is exhibiting the same issue as was present pre your git
> > download?
> 
> One of the reasons for a new bug report is that, yes, this refers to an OFX
> download. I suspect the bit of code I am talking about is shared by both OFX
> QIF.

No, that's not the case.  They are handled by quite different classes, /kmymoney/kmymoney/converter/mymoneyqifreader.cpp and /kmymoney/kmymoney/plugins/ofximport/ofximporterplugin.cpp.  The code I fixed was in the former.

I don't have any OFX accounts, but if you are able to provide an OFX file that demonstrates this behaviour, I will try to look at it.

> 
> Let me see if I understand.  I do see the info in the split.  Are you saying
> that the information in the ledger is from the split lines?

Yes, more or less.  The transactions, including the splits, are created and then read into the Ledger.

If you look in the Category view, you should see the different categories and their contents. 
> How would I go about getting a report on Div income? Him, it looks like if I 
> tell the "Transactions by account" report generator to look for "Div" I get
> what I want.

> Not sure if this is the place, but I have done a few translations in my QIF
> "filter" that I think ought to be in KMM.  How should I present these to you?
Comment 5 allan 2014-08-06 23:04:34 UTC
(In reply to george from comment #0)
> When downloading transactions for an investment account all the new entries
> in the 'Brokerage' account get a default "category' of "investment
> transaction".  It would be far better if this actually reflected the nature
> of the transaction, i.e.:
> 
> interest expense
> interest income
> Dividend income
> investment expense  (for general fees and adjustments)
> 
> investment transaction (for buy and sell transactions)

I don't know how much of the above is still relevant now that the imports are producing the relevant categories?

 I would probaly expect to see the 'interest expense' as a fee against the relevant transaction, or as a 'Miscellaneous expense'.  There is also an 'Interest' activity type, similar to 'Dividend', which could be used.
Comment 6 george 2014-08-06 23:16:03 UTC
(In reply to allan from comment #5)
> (In reply to george from comment #0)
> > When downloading transactions for an investment account all the new entries
> > in the 'Brokerage' account get a default "category' of "investment
> > transaction".  It would be far better if this actually reflected the nature
> > of the transaction, i.e.:
> > 
> > interest expense
> > interest income
> > Dividend income
> > investment expense  (for general fees and adjustments)
> > 
> > investment transaction (for buy and sell transactions)
> 
> I don't know how much of the above is still relevant now that the imports
> are producing the relevant categories?
> 
>  I would probaly expect to see the 'interest expense' as a fee against the
> relevant transaction, or as a 'Miscellaneous expense'.  There is also an
> 'Interest' activity type, similar to 'Dividend', which could be used.

Well, actually what I see is a category for _Fees and "<account name>:<security>" for the buy/sell.  

For a Div I see the same account/security entry with amount 0 and "<account> (Brokerage)" with the amount of the dividend. 
Only the Memo tells what the amount is for.
Comment 7 george 2014-08-06 23:31:52 UTC
(In reply to allan from comment #4)
> > > See my last comment in your other bug,  Bug 337251.  You should be getting
> > > the correct category information now.  However, the heading in this bug
> > > refers to OFX, where we have been dealing with QIF, I hope.  Or, are you
> > > saying that OFX is exhibiting the same issue as was present pre your git
> > > download?
> > 
> > One of the reasons for a new bug report is that, yes, this refers to an OFX
> > download. I suspect the bit of code I am talking about is shared by both OFX
> > QIF.
> 
> No, that's not the case.  They are handled by quite different classes,
> /kmymoney/kmymoney/converter/mymoneyqifreader.cpp and
> /kmymoney/kmymoney/plugins/ofximport/ofximporterplugin.cpp.  The code I
> fixed was in the former.

But don't these bits of code just reformat the transaction and send it up the line to the code that actually puts it in the account?

In any case, I seem to be getting the same result from my QIF import as the OFX import is generating. Here is a scrap of QIF that shows the same thing:

!Account
NFidelity 58
D(800) 544-6666
TInvst
^
!Type:Invst
D8/31' 6
NDiv
YFDRXX FIDELITY CASH RESERVES
CR
U248.19
T248.19
MDIVIDEND RECEIVED
^

> 
> I don't have any OFX accounts, but if you are able to provide an OFX file
> that demonstrates this behavior, I will try to look at it.
> 
> > 
> > Let me see if I understand.  I do see the info in the split.  Are you saying
> > that the information in the ledger is from the split lines?
> 
> Yes, more or less.  The transactions, including the splits, are created and
> then read into the Ledger.
> 
> If you look in the Category view, you should see the different categories
> and their contents. 
> > How would I go about getting a report on Div income? Him, it looks like if I 
> > tell the "Transactions by account" report generator to look for "Div" I get
> > what I want.
> 
> > Not sure if this is the place, but I have done a few translations in my QIF
> > "filter" that I think ought to be in KMM.  How should I present these to you?
Comment 8 allan 2014-08-07 11:33:57 UTC
> > > ...I suspect the bit of code I am talking about is shared by both OFX
> > > QIF.
> > 
> > No, that's not the case.  They are handled by quite different classes,
> > /kmymoney/kmymoney/converter/mymoneyqifreader.cpp and
> > /kmymoney/kmymoney/plugins/ofximport/ofximporterplugin.cpp.  The code I
> > fixed was in the former.
> 
> But don't these bits of code just reformat the transaction and send it up
> the line to the code that actually puts it in the account?
>
They handle things differently.  For instance, the code of ofximporterplugin.cpp does not contain any mention of "brokerage".  When I last looked at this, I came to the conclusion that in OFX parlance, brokerage refers to the broker supplying the file, and it has no knowledge of KMM innards, such as default brokerage accounts.

> In any case, I seem to be getting the same result from my QIF import as the
> OFX import is generating. Here is a scrap of QIF that shows the same thing:
> 
> !Account
> NFidelity 58
> D(800) 544-6666
> TInvst
> ^
> !Type:Invst
> D8/31' 6
> NDiv
> YFDRXX FIDELITY CASH RESERVES
> CR
> U248.19
> T248.19
> MDIVIDEND RECEIVED
> ^
> 
When I first looked at this, it seemed more or less identical to your previous QIF snippet, so I was quite surprised when I imported it and it gave different results.  That is, the imported transaction was missing the brokerage account.  I didn't think it relevant, but the "!Account" entry was the cause.  When I removed it, I was prompted to supply the account for the import.  I then realised that previously your import had worked because I had selected an account tha had an existing brokerage account.  The "!Account" entry was creating a new account, which would not have a brokerage account. So, I had misled myself, and you, into thinking your problem had been fixed.  It has not.

The reason is that the existing code will only create a new brokerage account if the transaction either has an "X" suffix to the action code, or a transfer account name in [] .

This area of code is long eastablished, and it seems, if I understand you correctly, the problem is the result of recent Quicken versions moving the goalposts away from their long-established "standard".

@Thomas. Thomas, what is your view on how this issue should be resolved?  Do we commit to following Quicken, where-ever it chooses to go?
> > 
> > I don't have any OFX accounts, but if you are able to provide an OFX file
> > that demonstrates this behavior, I will try to look at it.
> > 

To be able to investigate the OFX issue, I will need a file that demonstrates the problem.  Comparing with QIF is useless, I'm afraid.  However, from what I said above, OFX has no awareness of KMM's file structure.
Comment 9 Jack 2014-11-11 23:27:42 UTC
I've just re-read through this bug, and I now wonder if the issue isn't just a misunderstanding of the relation between the investment account and the associated brokerage account.  When you enter (or import) an investment transaction, you do so in the investment account.  That transaction includes one or more categories for dividend, interest, fees, ... (the categories you seem to be referring to.)  If the transaction includes a net income or expense, the transaction will also show up in the brokerage account.  When you look at the brokerage account ledger, you will only see "investment transaction" and the amount of income or expense.  In order to deal with any of the details, don't try to edit the transaction in the brokerage account, edit it in the investment account.  All the details related to categories are aspects of the investment activity.  The brokerage account simply provides or receives the necessary funds for the transaction.  

As far as I can tell, the origin of the distinction between investment and brokerage accounts is lost in the mists of history, and although it may or may not really be necessary from an accounting or programming perspective, once you understand how KMM works, it shouldn't cause any problems.
Comment 10 Cristian Oneț 2014-11-12 07:41:44 UTC
Please read comment #9 and tell us if the explanation make things clearer or you would still suggest a change in the behavior.
Comment 11 george 2014-11-14 21:30:31 UTC
(In reply to Jack from comment #9)
> I've just re-read through this bug, and I now wonder if the issue isn't just
> a misunderstanding of the relation between the investment account and the
> associated brokerage account.  When you enter (or import) an investment
> transaction, you do so in the investment account.  That transaction includes
> one or more categories for dividend, interest, fees, ... (the categories you
> seem to be referring to.)  If the transaction includes a net income or
> expense, the transaction will also show up in the brokerage account.  When
> you look at the brokerage account ledger, you will only see "investment
> transaction" and the amount of income or expense.  In order to deal with any
> of the details, don't try to edit the transaction in the brokerage account,
> edit it in the investment account.  All the details related to categories
> are aspects of the investment activity.  The brokerage account simply
> provides or receives the necessary funds for the transaction.  

Yes, I under stand this.  The only issue that "may" be remaining is how KMM handles cash transactions related to a security.  These seem to come in tagged as miscellaneous transactions. They seem to, mostly, be corrections to prior mis stated transaction.  They are new transactions and are not DIV BUY or SELL, and, it seems should only affect the cash amount related to the given security (although I might want DIV corrections to be correctly tagged for tax reasons).
> 
> As far as I can tell, the origin of the distinction between investment and
> brokerage accounts is lost in the mists of history, and although it may or
> may not really be necessary from an accounting or programming perspective,
> once you understand how KMM works, it shouldn't cause any problems.
Comment 12 Jack 2014-11-15 00:33:49 UTC
(In reply to george from comment #11)
Ah yes, miscellaneous transactions.  I think we are now in a much fuzzier area, similar to issues I have with Merrill Lynch reporting a dividend reinvestment as three transactions:  a dividend, a transfer in (of securities, but different from a buy), and a transfer (of case, but not clear to where).  As I understand it, KMM does not interpret the OFX itself, but leaves that task to libofx.  KMM acts on what transactions libofx says are in the file.  You might try running ofxdump (which should get installed  with the rest of libofx) on the OFX file to see what it says.  I've only recently done that on my ofx files, and realized that ofx is not quite as standard a standard as might be desired.
Comment 13 Andrew Crouthamel 2018-09-25 03:33:38 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days, the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information.

For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please set the bug status as REPORTED so that the KDE team knows that the bug is ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 14 Andrew Crouthamel 2018-10-27 03:42:32 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least 30 days. The bug is now closed as RESOLVED > WORKSFORME due to lack of needed information.

For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!