Bug 305630 - QIF Import Does Not Create All Security Accounts When Associated SECURITY fields are created
Summary: QIF Import Does Not Create All Security Accounts When Associated SECURITY fie...
Status: RESOLVED NOT A BUG
Alias: None
Product: kmymoney
Classification: Applications
Component: general (show other bugs)
Version: 4.6.2
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-08-22 20:29 UTC by MikeB
Modified: 2017-11-15 05:19 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description MikeB 2012-08-22 20:29:35 UTC
The QIF file was generated from Quicken 2002. But, interestingly, it will fail the import if I use the Quicken 2002 import method. Specifically, it does nothing except display a message that states "The statements have been processed with the following results" of which there are none. 

So, I used the default import method which seems to work. In fact it worked with several other accounts.The one that failed is an Investment account. The QIF contained Price changes as well as Transactions.

Before importing, I manually created an Investment account and asked KMyMoney to create an associated Brokerage account.

Many transactions were not imported. As well, all the SECURITY entries were created in the XML file, but the associated ACCOUNT entries were not created. As a result, in KMyMoney I am unable to delete the Securities. As well, these Securities do not show up as selectable items when trying to add a new Investment transaction. I was forced to edit the XML file to get rid of the entries (and/or manually add the associated ACCOUNT entries). 

Reproducible: Always

Steps to Reproduce:
1.  Create Investment account with associated Brokerage account
2.  Import QIF (which includes transactions + price changes)
3.  Go to securities list and attempt deletion or try and add a new Investment transaction and then attempt to select one of the imported Securities.
Actual Results:  
Delete button is greyed-out. And the Securities do not show up in the selection list.

Expected Results:  
Should have been able to delete the Security if I wished.
All the Securities should have shown up so I could select one of them when adding a new transaction.
Comment 1 MikeB 2012-08-22 21:19:12 UTC
The reason the QIF File import failed to import all the transactions and to create all the associated ACCOUNT entries for each SECURITY was that two of the Security entries  (in the QIF) did not have symbols.

So, the problems that need to get fixed in the code are:

1.   Make certain that an ACCOUNT entry is created in the XML file if an associated SECURITY is imported, regardless of any other conditions.

2.  In the case of a Security which has no Symbol, either refuse to import any associated transactions, but still continue on to import all other transactions and  inform the user that this has happened OR import the Securities even though they don't have associated symbols. The user could then add symbols after the fact.
Comment 2 MikeB 2012-08-22 21:26:05 UTC
Another thing to keep in mind is that a Quciken 2002 QIF contains all Securities whether they are used transactions in this Investment account or not. This is another reason why the associated ACCOUNT entry MUST be created for each Security that is imported. Otherwise, you end up with the two problems I cited earlier.
Comment 3 allan 2012-08-22 21:30:10 UTC
(In reply to comment #0)
> The QIF file was generated from Quicken 2002. But, interestingly, it will
> fail the import if I use the Quicken 2002 import method. Specifically, it
> does nothing except display a message that states "The statements have been
> processed with the following results" of which there are none. 
> 
> So, I used the default import method which seems to work. In fact it worked
> with several other accounts.The one that failed is an Investment account.
> The QIF contained Price changes as well as Transactions.
> 
> Before importing, I manually created an Investment account and asked
> KMyMoney to create an associated Brokerage account.
> 
> Many transactions were not imported. As well, all the SECURITY entries were
> created in the XML file, but the associated ACCOUNT entries were not
> created. As a result, in KMyMoney I am unable to delete the Securities. As
> well, these Securities do not show up as selectable items when trying to add
> a new Investment transaction. I was forced to edit the XML file to get rid
> of the entries (and/or manually add the associated ACCOUNT entries). 
> 
> Reproducible: Always
> 
> Steps to Reproduce:
> 1.  Create Investment account with associated Brokerage account
> 2.  Import QIF (which includes transactions + price changes)
> 3.  Go to securities list and attempt deletion or try and add a new
> Investment transaction and then attempt to select one of the imported
> Securities.
> Actual Results:  
> Delete button is greyed-out. And the Securities do not show up in the
> selection list.
> 
> Expected Results:  
> Should have been able to delete the Security if I wished.
> All the Securities should have shown up so I could select one of them when
> adding a new transaction.

Importing QIF files can be a bit of a lottery because the specification is so loose, and it is not uncommon to have to edit the files to get them to import successfully.

I've imported numerous qif files in the past, but don't recognise your mention of 'the Quicken 2002 import method.'  Can you explain, please.

> Before importing, I manually created an Investment account and asked
> KMyMoney to create an associated Brokerage account.

Did you also create a security account with ticker symbol?

Are you able to provide a copy of your file, ideally the whole file, or at least extract from your file all the investment type transactions, with private details replaced, confirm that importing it, too, still fails then provide a copy.  Multiple similar (price) entries can be dropped, as long as the format is retained.

Send it to me off-list, if you prefer not to attach to the bug report.

(When submitting this reply, your second post clashed, but I'll send it anyway.)
(And again, with your added comment!)
Comment 4 MikeB 2012-08-22 21:39:45 UTC
That's my point: If you import a QIF file that contains a Security entry with no Ticker Symbol it results in many transactions not being imported as well as a bunch of SECURITY entries in the XML file with no associated ACCOUNT entries.

In answer to your query regarding the "Quicken 2002 import mechanism:  You can find this by changing the QIF Profile from "default".

In addition, but this maybe should be added to a separate bug report, any transactions of the type "MiscExp", "XIn", or "IntInc" are disregarded by the import mechanism.
Comment 5 allan 2012-08-22 22:44:41 UTC
(In reply to comment #4)
> That's my point: If you import a QIF file that contains a Security entry
> with no Ticker Symbol it results in many transactions not being imported as
> well as a bunch of SECURITY entries in the XML file with no associated
> ACCOUNT entries.
> 

I'll see if I can have a look at detecting tickerless entries.  Symbols are fundamental to the way KMM handles securities, so I think the only way round this would be th flag them on import.

> In answer to your query regarding the "Quicken 2002 import mechanism:  You
> can find this by changing the QIF Profile from "default".
> 

Ah, that must have been added for the Windows version.  I do not see that.

> In addition, but this maybe should be added to a separate bug report, any
> transactions of the type "MiscExp", "XIn", or "IntInc" are disregarded by
> the import mechanism.

No, that's not the case.  They are catered for in the code.  That's not to say that they are handled the way you expect.  Can you provide an example file?
Comment 6 MikeB 2012-08-22 23:14:20 UTC
How do I get the file to you?

The "MiscExp", "XIn", "XOut" & "IntInc" definitely did not get imported. The import procedure showed an error message. I think it said something like accounts don't exist for some transactions and that it was ignoring these transactions.
Comment 7 allan 2012-08-22 23:33:46 UTC
(In reply to comment #6)
> How do I get the file to you?
>
See private email. 
> The "MiscExp", "XIn", "XOut" & "IntInc" definitely did not get imported. The
> import procedure showed an error message. I think it said something like
> accounts don't exist for some transactions and that it was ignoring these
> transactions.

I just happen to have one file with an intinc item, and it imported correctly, as I see it, so we'll see what your file does.
Comment 8 MikeB 2012-08-22 23:59:23 UTC
My mistake "XIn" and "XOut"'s do work, but "IntInc" and MiscExp" don't.
Comment 9 allan 2012-08-23 11:26:29 UTC
(In reply to comment #8)
> My mistake "XIn" and "XOut"'s do work, but "IntInc" and MiscExp" don't.

I've started looking at the MiscExp transactions.  When I launch KMM from the command line, by typing 'kmymoney', I get a log, starting :-

"Read 41992 bytes
Selected date format: '%m %d %y'
Statement with 116 transactions, 1137 prices and 31 securities added to the statement list
3000 lines processed"

Would you check that you get something similar.

Then, in your RRSP.QIF file, I see six MiscExp entries, the first three of which appear, by date and amount,  to relate to the preceeding sell transaction.  All three show an amount of 5.35.  After import, if I click Edit>Find transaction and enter 5.35  in the 'Amount' tab, then search, I get listed the same six transactions.  So, for me, those, at least, are getting imported.  Then, looking at the apparently related sell transactions, they have been flagged as missing a category assignment, because there is no account shown to receive the funds, which possibly is supposed to cover the expense, although I can't see in the file that the pairs are supposed to be related.

How do you expect to see these items recorded in KMM?
Comment 10 allan 2012-08-23 15:41:49 UTC
(In reply to comment #0)

> 3.  Go to securities list and attempt deletion or try and add a new
> Investment transaction and then attempt to select one of the imported
> Securities.
> Actual Results:  
> Delete button is greyed-out. And the Securities do not show up in the
> selection list.
> 
> Expected Results:  
> Should have been able to delete the Security if I wished.
> All the Securities should have shown up so I could select one of them when
> adding a new transaction.

If a security in the list cannot be deleted, it is probable that the reason is that that security is still active.  It may have price entries associated with it.  Go to Tools>Prices, enter the symbol in the search box then click Show all stored prices.  All prices will have to be deleted to allow the security to be deleted.  Multiple entries may be selected by the usual method - shift or Ctrl and click.
Comment 11 allan 2012-08-23 15:49:47 UTC
(In reply to comment #9)
> (In reply to comment #8)
> > My mistake "XIn" and "XOut"'s do work, but "IntInc" and MiscExp" don't.
> 

Also in this file, I find six intinc entries.  One has the value 0.17, so following the same procedure as above and searching for 0.17, the transaction shows up, and double-clicking on it shows the other entries, too.  So, again, the intinc entries have been imported.
Comment 12 MikeB 2012-08-23 18:19:29 UTC
On 23/08/2012 8:41 AM, allan wrote:
> https://bugs.kde.org/show_bug.cgi?id=305630
>
> --- Comment #10 from allan <agander93@gmail.com> ---
> (In reply to comment #0)
>
>> 3.  Go to securities list and attempt deletion or try and add a new
>> Investment transaction and then attempt to select one of the imported
>> Securities.
>> Actual Results:
>> Delete button is greyed-out. And the Securities do not show up in the
>> selection list.
>>
>> Expected Results:
>> Should have been able to delete the Security if I wished.
>> All the Securities should have shown up so I could select one of them when
>> adding a new transaction.
> If a security in the list cannot be deleted, it is probable that the reason is
> that that security is still active.  It may have price entries associated with
> it.  Go to Tools>Prices, enter the symbol in the search box then click Show all
> stored prices.  All prices will have to be deleted to allow the security to be
> deleted.  Multiple entries may be selected by the usual method - shift or Ctrl
> and click.

 From a user perspective would it not be better for KMyMoney to allow 
the delete regardless whether there are attached entries or not. Then 
what you do is tell the user that there are associated entries and that 
they will be lost if the user continues with the deletion.

The reason this is better is because the way it is now, a user has no 
idea how to fix the problem because it doesn't tell the user what the 
associations are and hence they don't know where to look to try and 
resolve the problem. This results in questions being asked on the forum 
eating up precious time.
Comment 13 MikeB 2012-08-23 18:55:46 UTC
On 23/08/2012 8:49 AM, allan wrote:
> https://bugs.kde.org/show_bug.cgi?id=305630
>
> --- Comment #11 from allan <agander93@gmail.com> ---
> (In reply to comment #9)
>> (In reply to comment #8)
>>> My mistake "XIn" and "XOut"'s do work, but "IntInc" and MiscExp" don't.
> Also in this file, I find six intinc entries.  One has the value 0.17, so
> following the same procedure as above and searching for 0.17, the transaction
> shows up, and double-clicking on it shows the other entries, too.  So, again,
> the intinc entries have been imported.

OK, I can get the IntInc and MiscExp transactions to import by choosing 
BMO RRSP (Brokerage) as the import destination; however, when I do this 
none of the stock/fund purchase transactions show up in the BMO RRSP 
Investment account. In fact the Investment account contains no 
transactions. The Brokerage account shows transactions for each of the 
fund purchases, but all there is is a date, a  0 dollar debit and the 
balance carried forward from the previous transaction.

If I choose the Investment account as the import destination then I get 
all the stock/fund transactions imported but no "IntInc" or "MiscExp" 
transactions..

This testing was done on a brand new XML database file which contained 
nothing but a Savings account, a RRSP Brokerage account and a RRSP 
Investment account with no transactions in any of them.
Comment 14 Cristian Oneț 2014-07-29 10:36:52 UTC
Is this still relevant with using the latest version?
Comment 15 Ralf Habacker 2017-11-15 05:19:59 UTC
It is not possible to retest this issue with newer version without having a QIF and kmy test file. If this issue is still present, feel free to reopen with the mentioned test file appended.