Bug 396886

Summary: Online Banking Behavior Change Since 4.8
Product: [Applications] kmymoney Reporter: Brendan <KDE>
Component: onlinebankingAssignee: KMyMoney Devel Mailing List <kmymoney-devel>
Status: RESOLVED FIXED    
Severity: normal CC: ralf.habacker
Priority: NOR    
Version: git (master)   
Target Milestone: ---   
Platform: Other   
OS: Linux   
Latest Commit: Version Fixed In: 5.0.2
Sentry Crash Report:
Attachments: Summary Window
Statement stats

Description Brendan 2018-07-26 17:05:00 UTC
I'm running KMM compiled from the git master branch on July 20, 2018 on Fedora 28.

When running the "Update all accounts" command, KMM KMM displays the "Statement stats" page for every mapped account. It requires user interaction to move on the the next account. This behavior was fixed in 2009 so that KMM updated all accounts and then gave a single "Statement stats" page for the accounts that had one of more new transactions. If there were no new transactions in any account you did not get a "Statement stats" window. If you had new transactions in 3 of your accounts, KMM displayed a single "Statement stats" window at the end with the OXF information from the three accounts that had one of more new transactions.

I also compiled KMM from the 5.0 branch on July 20, 2018. The behavior is different in Version 5.0.2-7561bb34. In this version KMM only displays the "Statement stats" window for online accounts that have new transactions. It stops updating online accounts until the users takes action to close the "Statement stats" window.

Assuming I have 13 mapped accounts and 6 of them have new transactions.

KMM 4.8: I get one "Statement stats" window at the end with the summary of the 6 accounts.

KMM 5.0.2: I get 6 "Statement stats" windows which all have to be clicked on to continue the update process.

KMM - MASTER: I get 13 "Statement stats" windows, which all have to be clicked on to continue the update process.

Assuming I have 13 mapped accounts and none of them have new transactions.

KMM 4.8: No windows pop up.

KMM 5.0.2: No windows pop up.

KMM - MASTER: I get 13 "Statement stats" windows which all have to be clicked on to finish the update process.

For some reason this behavior was changed when moving from 4.8 to 5 and it's different in the 5.0 branch and the master branch. From usability perspective it should work like 4.8.

Thomas Baumgart fixed this behavior in 2009.

I found the emails from my original "feature request" on this issue.
The first batch were in January 2009 (Subject: "Feature Request") with
a followup in April 2009 (Subject: Feature Request - Update). I think
it was finally working in June/July 2009 (Subject: Feature Request -
Update All Accounts  Behavior). The text from the last email on the
subject is copied below.

============================================================

07/02/2009 08:58 PM

Works great with OFX. I got only the results from the one bank that
had new data. I ran it again and there was no window. Perfect. Thanks,
you've saved me lots of scrolling.
--
Brendan


On Thu, Jul 2, 2009 at 1:46 AM, Thomas Baumgart <thb@net-bembel.de> wrote:
>
> On Sunday 21 June 2009 19:21:53 Brendan Coupe wrote:
> > I have 10 accounts that I update online. Most have very little
> > activity but it's easier to use that Update All Accounts option every
> > day or two than it is to download the 3 or 4 accounts that change more
> > often individually.
> >
> > In the past few months Thomas has greatly improved the behavior of
> > this process by eliminating the selection screen for each account
> > eliminating all user interaction except at the end of the process when
> > the summary screen appears. Please note that you need to get the very
> > latest CVS source and unmap and remap any online accounts that are
> > still asking you to confirm the account name before importing the
> > data. I was still getting the selection window on my credit card
> > accounts until it was fixed earlier today. The fix will not work
> > unless you remap the account.
> >
> > Now for my request. I would like the option to have Update All
> > Accounts be done in silent mode. Right now 20 different windows popup
> > during this process. It takes 45 to 60 seconds to download all 10
> > accounts and it's annoying to have them popping up while I'm trying to
> > do other things. Plus I see no useful information in them. Maybe the
> > account name that is being downloaded and a counter (1 of 10) can be
> > shown at the bottom of the KMM screen.
> >
> > Part 2 of my request is to give me the option to see a summary of only
> > the accounts that have added or matched transactions. I have to scroll
> > through 10 accounts to get to the one account that actually has
> > transactions fairly often since it's at the bottom of the list. I have
> > two suggestions here. Either keep it the way it is and only show the
> > accounts with added or matched > 0 or ideally I would like to see a
> > table with Account name in the first column and Processed, Added,
> > Matched and Duplicates plus new Payees in the rest of the columns. It
> > would be nice if the user could decide if they want to see this table
> > for all accounts or just accounts with new data (Added > 0). I think
> > there should also be an option to completely turn off the download
> > summary if you do not want to see it.
> >
> > I understand that other people may want to keep the download process
> > exactly as it is and it may have some usefulness, especially when
> > troubleshooting download problems. I would like to see my suggestions
> > added as options so that each user see what they want to see. I just
> > prefer to have a simple table that requires no scrolling at the end of
> > the download process and see nothing else.
>
> I just added the following to CVS HEAD:
>
> Skip statement import statistics if no transactions have been imported for
> the 'Update all accounts' function
>
> I tested it with the AqBanking plugin and HBCI and that works as expected.
> Brendan, please test it with your set of OFX accounts. The suppression only
> works when you select the 'Update all accounts' feature. One account will
> still give you the results.
>
> > These requests is only possible because of the great work done to make
> > the online banking features work so well in the past few years. I used
> > to have to download each OFX file manually and the ability to do it
> > with one click of the mouse for most of my accounts saves a lot of
> > time. You've spoiled me and now I want more:) Next I'll want a cronjob
> > which will be lots of fun since my KMM file is encrypted.
> >
> > Thanks for all the hard work.
>
> Thanks for all your testing.
>
>
>
> --
>
> Regards
>
> Thomas Baumgart
>
> GPG-FP: E55E D592 F45F 116B 8429   4F99 9C59 DB40 B75D D3BA
> -------------------------------------------------------------
> If a cluttered desk is characteristic of a cluttered mind,
> what does an empty desk mean ?
> -------------------------------------------------------------
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> KMyMoney2-developer mailing list
> KMyMoney2-developer@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/kmymoney2-developer
Comment 1 Thomas Baumgart 2018-07-28 12:29:23 UTC
I added https://phabricator.kde.org/D14439 to resolve the problem. Anyone is welcome to test it and leave comments there.
Comment 2 Brendan 2018-07-28 21:27:47 UTC
Created attachment 114183 [details]
Summary Window
Comment 3 Brendan 2018-07-28 21:32:40 UTC
Comment on attachment 114183 [details]
Summary Window

I compiled the master branch with the patch. It did add the summary window at the end of the process.

It did NOT eliminate the 13 popups that I had prior to the summary at the end so this added one more window that I had to close:-(

So I had no new transactions and had to close 14 popup windows instead of 13.
Comment 4 Thomas Baumgart 2018-07-29 04:52:18 UTC
Git commit 66523b3c99812e30816668351504063d6620a9c6 by Thomas Baumgart.
Committed on 29/07/2018 at 04:52.
Pushed by tbaumgart into branch 'master'.

Only show a single summary of statement import

Summary:
Depending on the number of statements imported via online connects to
banks, the sheer number of dialogs popping up is annoying and time
consuming. This change reduces this to a single dialog which is also
shown if no transaction has been imported at all to notify the
termination of the process to the user.

Test Plan: I ran updates for single accounts as well as multiple accounts

Reviewers: wojnilowicz

Reviewed By: wojnilowicz

Differential Revision: https://phabricator.kde.org/D14439

M  +16   -3    kmymoney/converter/mymoneystatementreader.cpp
M  +2    -0    kmymoney/converter/mymoneystatementreader.h
M  +7    -30   kmymoney/kmymoney.cpp
M  +41   -0    kmymoney/kmymoneyutils.cpp
M  +4    -0    kmymoney/kmymoneyutils.h
M  +4    -40   kmymoney/views/kaccountsview.cpp
M  +44   -0    kmymoney/views/kaccountsview_p.h

https://commits.kde.org/kmymoney/66523b3c99812e30816668351504063d6620a9c6
Comment 5 Thomas Baumgart 2018-07-29 05:06:36 UTC
(In reply to Brendan from comment #3)
> Comment on attachment 114183 [details]
> Summary Window
> 
> I compiled the master branch with the patch. It did add the summary window
> at the end of the process.
> 
> It did NOT eliminate the 13 popups that I had prior to the summary at the
> end so this added one more window that I had to close:-(

Very strange, as the code for the message per account is not present anymore in the source. Can you attach a screenshot of one of those 13 popups? Is this OFX related, as I don't see any popups per account anymore, just the final summary. One difference I see is, that I am using KBanking as the online backend.
Comment 6 Brendan 2018-07-29 22:24:51 UTC
Created attachment 114203 [details]
Statement stats

I've attached the popup window that I see after each account is downloaded. I removed the account name and balance.

I am using OFX.

Come one Thomas, you fixed this 9 years ago. Don't you remember how you did that:-)
Comment 7 Brendan 2018-07-29 23:33:27 UTC
I think it's important to keep in mind that this works just like 4.8 when you compile KMM from the 5.0 branch. If none of my accounts have any new transactions I don't get the summary popup at the end and I don't get the 13 popups along the way.

Łukasz Wojniłowicz worked on this and asked me to test it back on May 15, 2018. It took me a while to get to it but it seems to work fine in 5.0.2.

Not sure why the changes are not in the master branch but they appear to be in the 5.0 branch.

Here is the email response that I received from Łukasz via the kmymoney-devel@kde.org. The subject of this email thread is "Problems Compiling KMM KF5":

===========

Łukasz Wojniłowicz <lukasz.wojnilowicz@gmail.com>
May 15

to For
I tried to bring mentioned feature back. Could you try again and see if it has 
bettered for you?

Cheers
Łukasz

Dnia piątek, 30 marca 2018 17:50:02 CEST Brendan Coupe pisze:
Comment 8 Brendan 2018-07-29 23:52:25 UTC
Please disregard most of mt last comment. The 5.0 branch is better than the master branch but it's not the same as 4.8. I switched back to 5.0.2 and did more testing to clarify the differences.

In the 5.0 branch I no longer get popup windows for every account. I only get popup windows for the accounts that have new transactions. I do NOT get a summary of the downloaded transactions for the accounts that have new transactions at the end. This means if 3 out of 13 accounts have new transactions I get 3 popup windows and the download stops until I clear each window. If none of the 13 accounts has a new transaction then I don't get any popup windows.

In the master branch I get popups for every account and with the patch I also get a summary window at the end. For 13 accounts I get 14 popups regardless of whether any of the accounts have new transactions.

The combination of eliminating the remaining popups in the 5.0 branch and adding the summary at the end when there is at least one new transaction like how it works with the patch is what is needed to get back to how 4.8 worked.

I'm running that 5.0 branch for now but can easily test changes / patches for either branch.

Sorry for the confusion.
Comment 9 Ralf Habacker 2018-07-30 06:27:34 UTC
(In reply to Brendan from comment #7)
> Not sure why the changes are not in the master branch but they appear to be
> in the 5.0 branch.

Probably because branch 5.0 was note merged into master branch.
Comment 10 Thomas Baumgart 2018-07-30 16:07:56 UTC
No, because it is OFX specific and needs to be fixed in master. I'll take care of it. BTW, the QIF importer seems to have the same problem.
Comment 11 Thomas Baumgart 2018-07-30 16:59:11 UTC
Git commit 4b9cc6a8016ccacf501cf41d8d9978436926e6fa by Thomas Baumgart.
Committed on 30/07/2018 at 16:57.
Pushed by tbaumgart into branch 'master'.

Remove per account statement summary

The OFX importer displayed a summary per account which conflicts with
the global summary after all accounts have been updated and is very
annoying in case you have many accounts.

M  +2    -20   kmymoney/plugins/ofx/import/ofximporter.cpp

https://commits.kde.org/kmymoney/4b9cc6a8016ccacf501cf41d8d9978436926e6fa
Comment 12 Brendan 2018-07-30 17:41:13 UTC
Way to go Thomas, I'm down to one popup with or without new transactions. Works almost the same as 4.8. In 4.8 there was no summary popup when there were no new transactions. In the master there is a summary screen telling you there are no new transactions. I think I might like it better this way since I wasn't always sure the update was done in 4.8.
Comment 13 Thomas Baumgart 2018-07-31 08:51:54 UTC
... and that is the reason, why I added it saying that nothing has been changed.