Bug 277861 - Improvements to CSV Plugin
Summary: Improvements to CSV Plugin
Status: RESOLVED FIXED
Alias: None
Product: kmymoney
Classification: Applications
Component: general (show other bugs)
Version: SVN
Platform: openSUSE Linux
: NOR wishlist
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-07-15 20:53 UTC by allan
Modified: 2011-12-13 11:11 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Snapshot of proposed CSV Importer Wizard (30.83 KB, image/png)
2011-07-26 00:14 UTC, allan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description allan 2011-07-15 20:53:02 UTC
Version:           SVN trunk (using KDE 4.6.0) 
OS:                Linux

If either a banking or an investment file has been opened, and then the user selects the opposite tab, a message box appears, warning that the current settings will be cleared and asking whether or not to proceed.  If the user elects to proceed, the table widget is cleared and it is necessary for the user to reopen the file selector box. and reload the required file.

During discussions regarding the user handbook, there was a query as to why that was necessary, and in fact it is not, as long as an appropriate file has already been opened during the current session.  This is a wishlist item for an improvement to the operation.

A second query was raised about why there was a spinbox for the start line number, but an editbox for the  last line number.  Early on. I had decided that a spinbox for the last line would be a waste of the user's time as there could be a large number of lines to spin through.  Later, however, the plugin was modified to preset the last line number to the last line in the file.  Now, therefore, it makes sense to adopt a spinbox for this setting, too.

Reproducible: Didn't try

Steps to Reproduce:
Does not apply.

Actual Results:  
Does not apply.

Expected Results:  
Does not apply.
Comment 1 allan 2011-07-20 10:08:23 UTC
Firstly, a correction to the spinbox paragraph.  I was writing from memory and misrememebered.  The lastline widget is in fact already a spinbox, but with the up/down arrow buttons not enabled.  I have now enabled them here.

Another query arose, which boils down to the suitability of a tab for the settings.  This actually is something that I have been unhappy about since changing to using tabs.  The problem comes from the need to switch tabs several times during the process of preparing to import.  Firstly, the correct tab has to be active, either Banking or Investment.  Then the file has to be selected and loaded.  Once loaded, it may be that the wrong field selector is active, causing the fields to display incorrectly.  So, the Settings tab has then to be selected to correct this.  Now, the fields are displaying correctly, so switch back to Banking or Investment and select the appropriate columns.  Then, back to Settings to ensure the date format is correct, to select the start and end lines and finally to select the decimal symbol.  That's the operational side of the issue.  The other aspect of it is that the Banking and Investment tabs are associated with file or operation type, whereas the Settings are not, but associate with the data format.

As a way to resolve both aspects, what I have tried is to remove the Settings tab and instead transfer the widgets into a new small window, which opens when a file is selected, and which positions clear of the main window (on a large enough screen.)  So, the settings are visible all the time and the data column selections are too, and bouncing tabs back and forth isn't required.  While I say 'the settings are visible all the time and ....", this isn't strictly true as it's possible for another window to overlay.  However, the widget can be made visible again as it shows on the task bar.

Do any team members have any objections to or comments  on this approach, please?  Obviously this is for post 4.6 release.
Comment 2 Thomas Baumgart 2011-07-20 11:32:55 UTC
Ever thought about using a multi page wizard for this instead of a single page dialog?
Comment 3 allan 2011-07-20 12:33:10 UTC
(In reply to comment #2)
> Ever thought about using a multi page wizard for this instead of a single page
> dialog?

It did enter my mind a while back, but I think I was too deeply entrenched then to give enough thought.

So, it would start off with a page with just a couple of buttons, to select either Banking or Investment, and that would lead on to the relevant column headings, but with the addition of decimal selection on the same page, then the remaining settings following on?
Comment 4 allan 2011-07-20 12:42:46 UTC
That would be a couple of radio buttons, plus the first page could also have the file open button.  The next would be like the left side of the current UI, column settings above and table widget below. Hmmm....
Comment 5 Jack 2011-07-20 13:32:54 UTC
I also like this.  There could be fewer control widgets on each page, but the user could easily back up if something needed to be changed.
Comment 6 allan 2011-07-26 00:12:14 UTC
Before I get too deeply into this, if you can, would you have a quick
look at the attached snapshot.

I have a bare-bones wizard working just in Qt at the moment.  The
wizard is just the lower part of the window, the intention being for
the file to be visible at all times, once loaded.  I thought it best for
the 'next' etc buttons to be at the bottom, not somewhere in the
middle, which, I think, means having the wizard at the bottom too.

Do you have any problems with this approach?

I'm not sure yet about having a button for file load, then another for
'next'.  At the moment, I'm thinking just import the file as the first
step, so the user knows he's selected the right file.  He'll
immediately see any separator problems, which may be corrected in the next
stage, etc.  The alternative would be to have the choice of separators
on this page, too, cutting down on the number of steps, which might be
better, but makes the UI more complicated.  (Tried this now and don't
really like the look of it.)

I've not used the KMM wizards, as I wanted to start from scratch so I
understood more fully what did what, but not to look too 'foreign'.
Comment 7 allan 2011-07-26 00:14:29 UTC
Created attachment 62190 [details]
Snapshot of proposed CSV Importer Wizard
Comment 8 allan 2011-09-06 23:42:02 UTC
(In reply to comment #7)

I have the wizard version working pretty well now, with just a few loose ends to tidy up.  whilst I'm biased, I must say it's a lot better than the previous versions.

In parallel, I working on improving the handling of broker csv files.  In parallel because changes to the ui are needed.  I've come across a couple of issues on which I need advice.

They both concern categories.  Up to now, as I started off from bank statements, I did not need splits, but, because in broker files there is useful category information, it would be a shame not to use it.  With a bit of a struggle, I have this sort of working.  One issue is that some entries have no transaction type, or rather have the information in a different column.  Another is that a broker statement may include information on more than one security, plus broker fees and interest, and transfers of money.  Now the queries.

1) If the needed category already exists, all is fine-ish.  I've borrowed some code from mymoneyqifreader - the checkCategory() function, which largely works for me, to the point where it needs to create a new category.  This involves the KMyMoneyApp, to which the plugin has no access, I think?  Can anything be done about this, or do I accept a limitation, requiring manual creation of categories?  Or, would it be legit. to instantiate a new KMyMoneyApp in order to create a new category?  Or, maybe there's another way.......

2) I'm confused about transaction matching.  I have two transactions with the same payee, one being a dividend and the other investment/broker interest.  The first one imports OK, but the second gets its category changed to dividend, apparently as a result of transaction matching.  I wouldn't have thought it correct to change a category without a query being flagged.  Am I wrong?  Am I missing something?  There's a subsidiary one about why a _Dividend category gets created sometimes when there is already a dividend category, but I'll have a look at that one.
Comment 9 allan 2011-09-08 12:07:54 UTC
(In reply to comment #8)
> (In reply to comment #7)
> 
> I have the wizard version working pretty well now, with just a few loose ends
> to tidy up.  whilst I'm biased, I must say it's a lot better than the previous
> versions.
> 
> In parallel, I working on improving the handling of broker csv files.  In
> parallel because changes to the ui are needed.  I've come across a couple of
> issues on which I need advice.
> 
> They both concern categories.  Up to now, as I started off from bank
> statements, I did not need splits, but, because in broker files there is useful
> category information, it would be a shame not to use it.  With a bit of a
> struggle, I have this sort of working.  One issue is that some entries have no
> transaction type, or rather have the information in a different column. 
> Another is that a broker statement may include information on more than one
> security, plus broker fees and interest, and transfers of money.  Now the
> queries.
> 
> 1) If the needed category already exists, all is fine-ish.  I've borrowed some
> code from mymoneyqifreader - the checkCategory() function, which largely works
> for me, to the point where it needs to create a new category.  This involves
> the KMyMoneyApp, to which the plugin has no access, I think?  Can anything be
> done about this, or do I accept a limitation, requiring manual creation of
> categories?  Or, would it be legit. to instantiate a new KMyMoneyApp in order
> to create a new category?  Or, maybe there's another way.......
> 
> 2) I'm confused about transaction matching.  I have two transactions with the
> same payee, one being a dividend and the other investment/broker interest.  The
> first one imports OK, but the second gets its category changed to dividend,
> apparently as a result of transaction matching.  I wouldn't have thought it
> correct to change a category without a query being flagged.  Am I wrong?  Am I
> missing something?  There's a subsidiary one about why a _Dividend category
> gets created sometimes when there is already a dividend category, but I'll have
> a look at that one.

I think my 'problems' mentioned above may be safely ignored now.  Looking more healthy now.
Comment 10 Cristian Oneț 2011-12-13 11:01:31 UTC
Does this still have any relevance since the UI of the plugin has been rewritten?
Comment 11 allan 2011-12-13 11:11:36 UTC
(In reply to comment #10)
> Does this still have any relevance since the UI of the plugin has been
> rewritten?

No longer.  From here, activity switched to the reviewboard, and this bug is a left-over.