SUMMARY STEPS TO REPRODUCE 1. Have a Payee with entries using multiple spellings (Petsmart, PetsMart, PetSmart) 2. Merge all payees into a single spelling (PetSmart is the correct spelling) 3. OBSERVED RESULT An error popup occurs (unable to remove payee(s). Details. Cannot remove payee that is still referenced to a transaction /build/kmymoney-2XFvAS/kmymoney-5.0.8/kmymoney/mymoney/storage/mymoneystoragemgr.cpp:290 Click on OK Click on another payee in the list. Click on Petsmart, PetsMart or PetSmart and all transactions are gone. Check accounts that used one of these as a payee and all entries for that payee are gone. EXPECTED RESULT Post the error with some explanation of resolving it and do not delete all entries under those Payees. SOFTWARE/OS VERSIONS Ubuntu Linux: ADDITIONAL INFORMATION Because I had automatic backups, I was able to recover. However there is no way of knowing you will hit this error if you attempt a merge, so it can happen at any time.
It is not clear exactly what you tried to do. It is clear your goal is to merge several payees into a single one, but the only relevant action in KMM I can think of it to delete a payee and reassign it's transactions to the desired payee - I am not aware of any "merge" function in KMM. Please provide a more detailed explanation of exactly what steps to took. Also, if you get an error, before trying again or doing something else, check at that point whether anything is wrong with your data.
Apologies - I see the merge function, and it works just fine for me. When you clicked on OK after the first error, what was still shown? Did you still see the list of payees to merge, or did you still see the dropdown to choose the target payee? When you say to click on any of the payees you merged, where is that? If the merge finished, then all of them except one should no longer be present on the list. Did the merge ultimately complete without further error, or was there another error, or did you cancel the merge? It's not clear.
Jack, Here are the steps I take: 1. From the main screen click on Payees 2. Scroll down to the Payees I want to merge. (In my case Petsmart, PetsMart and PetSmart) 3. Select all three (at this point there are transactions listed under all three) 4. Click on Merge 5. Popup appears asking if I really want to merge the payees. (I click on Yes) 6. Popup appears asking me to select a payee to merge everything into. (I select PetSmart and click OK) 7. The error popup occurs that I detailed in the Bug Report. (I click on OK) 8. All three Payees are still there. When I click on them, all transactions are gone. I should note that I have successfully done merges before without any problems. The problem only occurs when the error I described in the bug report happens. If I were to hazard a guess as to what is happening, I would say it is something like: When a merge is requested, all transactions are gathered and put in a temporary spot and removed from the original Payees. When successful, the Payee selected for the merge gets all of the transactions. (It may be created as a new Payee). When a failure occurs, the temporary storage gets deleted and since the transactions were already deleted from the original Payees, they are gone and there is no way to restore them. On 10/10/20 11:56 AM, Jack wrote: > https://bugs.kde.org/show_bug.cgi?id=427519 > > --- Comment #2 from Jack <ostroffjh@users.sourceforge.net> --- > Apologies - I see the merge function, and it works just fine for me. > When you clicked on OK after the first error, what was still shown? Did you > still see the list of payees to merge, or did you still see the dropdown to > choose the target payee? When you say to click on any of the payees you > merged, where is that? If the merge finished, then all of them except one > should no longer be present on the list. Did the merge ultimately complete > without further error, or was there another error, or did you cancel the merge? > It's not clear. >
Since you have done successful merges before, my guess is there is some problem with a transaction for one of the Payees to be deleted, which interrupts the reassignment of transactions to the chosen Payee. Again, only a guess, but I wonder if something other than the Payee-id is changed in a transaction, which is then left in some invalid state, no longer associated with any payee. In any case, a few more questions to help troubleshoot. If you do the merge, and then save the file (or maybe just run a consistency check) does KMM complain about anything related to the merged payees or the missing transactions? Can you find any of the missing transactions by looking in the ledger for the relevant account and date? Here I'm wondering if the transactions are actually still present, but there is only some problem in the display in the Payees View. Have you ever saved the file (to a new name so you don't overwrite your good backup) and then opened it again? Can you save an anonymized version of the file? If so, can you identify the three payees to be merged, and which one you want to keep? In the anonymized files, Payees are just named by their internal ID, so for example, you can do 'grep "PAYEE.*PetSmart" file.xml' or 'zgrep "PAYEE.*PetSmart" file.kmy' to find the id for that Payee. If you can replicate the problem with the anonymized file, there is a better chance of myself or one of the developers identifying the underlying problem.
Jack, In answer to your questions. does KMM complain about anything related to the merged payees or the missing transactions? No, there is not any indication of a problem. Can you find any of the missing transactions by looking in the ledger for the relevant account and date? No, I have looked in the ledgers that should contain the entries and they are all gone. Have you ever saved the file (to a new name so you don't overwrite your good backup) and then opened it again? Yes, and the entries are still gone. Can you save an anonymized version of the file? I don't see a way to do that, so you would need to give me instructions. On 10/16/20 6:54 PM, Jack wrote: > https://bugs.kde.org/show_bug.cgi?id=427519 > > --- Comment #4 from Jack <ostroffjh@users.sourceforge.net> --- > Since you have done successful merges before, my guess is there is some problem > with a transaction for one of the Payees to be deleted, which interrupts the > reassignment of transactions to the chosen Payee. Again, only a guess, but I > wonder if something other than the Payee-id is changed in a transaction, which > is then left in some invalid state, no longer associated with any payee. In > any case, a few more questions to help troubleshoot. > > If you do the merge, and then save the file (or maybe just run a consistency > check) does KMM complain about anything related to the merged payees or the > missing transactions? Can you find any of the missing transactions by looking > in the ledger for the relevant account and date? Here I'm wondering if the > transactions are actually still present, but there is only some problem in the > display in the Payees View. Have you ever saved the file (to a new name so you > don't overwrite your good backup) and then opened it again? > > Can you save an anonymized version of the file? If so, can you identify the > three payees to be merged, and which one you want to keep? In the anonymized > files, Payees are just named by their internal ID, so for example, you can do > 'grep "PAYEE.*PetSmart" file.xml' or 'zgrep "PAYEE.*PetSmart" file.kmy' to find > the id for that Payee. If you can replicate the problem with the anonymized > file, there is a better chance of myself or one of the developers identifying > the underlying problem. >
Hi, > Can you save an anonymized version of the file? > > I don't see a way to do that, so you would need to give me instructions. The instructions are described here https://docs.kde.org/stable5/en/extragear-office/kmymoney/details.formats.anonymous.html
Created attachment 132665 [details] PopupMenu.png It isn't working for me. I must be missing a step. Here are the steps I did: 1. Click on File (upper right hand corner of KMyMoney window) 2. On Drop down menu, I selected Save As.. 3. I got the attached popup menu (PopupMenu.png) 4. Storage type to use for data had two options (SQL, XML). I chose XML. Clicked on OK. 5. I had created a separate directory called Anonymous 6. The attached SaveScreen.png popped up. I went to the Anonymous Directory 7. I did not see any Filter I could pick. The dropdown in the lower right of the popup allowed me to pick Anonymous files which I did. I named the file AnonData and clicked on Save. 8. When the save was complete I closed KMyMoney and checked the Anonymous directory. 9. A file had been created, but it was named AnonData.kmy not AnonData.anon.kmy as the instructions indicated it would. 10. When I opened that file with KMyMoney, it had not made anything anonymous. 11. I thought maybe I just needed manually change the name and it would open it as anonymous, so I changed the name to AnonData.anon.kmy and reopened it, but as I suspected that did not work. 12. I looked in settings and under general there is a Filter tab, but it does not give an option to save a file anonymously. Am I missing something? On 10/23/20 10:52 AM, antoine wrote: > https://bugs.kde.org/show_bug.cgi?id=427519 > > antoine <antoine@copernic.xyz> changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > CC| |antoine@copernic.xyz > > --- Comment #6 from antoine <antoine@copernic.xyz> --- > Hi, > >> Can you save an anonymized version of the file? >> >> I don't see a way to do that, so you would need to give me instructions. > The instructions are described here > https://docs.kde.org/stable5/en/extragear-office/kmymoney/details.formats.anonymous.html >
Created attachment 132666 [details] SaveScreen.png
> It isn't working for me. I must be missing a step. this was a known issue in version 5.0.1 (https://bugs.kde.org/show_bug.cgi?id=419120), I thought it was fixed in 5.0.8 >7. I did not see any Filter I could pick. The dropdown in the lower >right of the popup allowed me to pick Anonymous files which I did. I >named the file AnonData and clicked on Save. You did the right thing, this time, name your file AnonData.anon.xml it should work
Created attachment 132688 [details] AnonData.anon.xml.gz Here is the anonymous file. Wow, it anonymizes transaction amounts as well! I was able to repeat the problem with this file. The three payees are: P005411 P003550 P003804 The account that has the transactions involving these payees is A000229 Here is what I did: Checked and noted the Balance of A000229. Clicked on Payees and selected P005411, P003550 and P003804 Clicked on Merge and selected P005411 as the Payee to merge everything into. When I indicated to merge, I got the same error as in my Bug Report. Clicked on OK. Checked the three accounts and all entries were gone. Checked the account balance and it was the same. Closed and reopened KMyMoney, the Payee entries were still all gone and the account balance had changed. On 10/23/20 1:09 PM, antoine wrote: > https://bugs.kde.org/show_bug.cgi?id=427519 > > --- Comment #9 from antoine <antoine@copernic.xyz> --- > >> It isn't working for me. I must be missing a step. > this was a known issue in version 5.0.1 > (https://bugs.kde.org/show_bug.cgi?id=419120), I thought it was fixed in 5.0.8 > >> 7. I did not see any Filter I could pick. The dropdown in the lower >> right of the popup allowed me to pick Anonymous files which I did. I >> named the file AnonData and clicked on Save. > You did the right thing, > this time, name your file AnonData.anon.xml it should work >
I finally had time to download the file and I can confirm the error. You also get the same error trying to delete P003804 and assign the transactions to P005411. I get no error doing the same with P003550. That at least give a hint the problem is related to P003804. However, in process of looking for anything strange. I noticed a console error: Problem determining the category for transaction ' "T000000000000015349" '. Reason: Split not found for account !A000069 /build/kmymoney/src/kmymoney-5.1.0/kmymoney/mymoney/mymoneytransaction.cpp:278 Oddly, if I search for that transaction, it seems to be a transfer from and to account A000069, for a different Payee. There ARE two splits for the transaction, but both for A000069, and there is no assigned Category for the transaction. Given that action="Transfer", I'm not sure if a Category should be necessary. I'm guessing that A000069 is a Brokerage account, due to the presence of Investment transactions in it, but again, I don't know if it matters. I'll keep digging to see if I can find any other problems with the file, but I do wonder whether the consistency check should find that type of problem.
Correction - A000069 is a checking account. Discovery - the transaction triggering the error message is T000000000000040026, which referencees P03550 (not sure why I seemed to get the problem Payee wrong above) transferring between A000620 and A000572, both being expense Categories. I may be too tired, but I can't think of how both splits of a transaction can be for categories. Daryl - perhaps you can check on these accounts, transaction, and categories in your original file to see if there are any hints about what might be odd.
Created attachment 133153 [details] AnonDataNov8.anon.xml.gz Jack, I found the transaction you referenced. It was from 12/31/92 and indeed did a transfer from an account back to itself. Since it was a zero sum change, I just deleted the transaction. I did a consistency check and no problems were found. I tried doing the merge again and ran into the same problem. I have attached an updated anonymous file with the 12/31/92 transaction removed. Is there anything I can be looking for that might be causing this problem? On 11/7/20 4:05 PM, Jack wrote: > https://bugs.kde.org/show_bug.cgi?id=427519 > > Jack <ostroffjh@users.sourceforge.net> changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > Resolution|WAITINGFORINFO |--- > Ever confirmed|0 |1 > Status|NEEDSINFO |CONFIRMED > > --- Comment #11 from Jack <ostroffjh@users.sourceforge.net> --- > I finally had time to download the file and I can confirm the error. You also > get the same error trying to delete P003804 and assign the transactions to > P005411. I get no error doing the same with P003550. That at least give a > hint the problem is related to P003804. However, in process of looking for > anything strange. I noticed a console error: > > Problem determining the category for transaction ' "T000000000000015349" '. > Reason: Split not found for account !A000069 > /build/kmymoney/src/kmymoney-5.1.0/kmymoney/mymoney/mymoneytransaction.cpp:278 > > Oddly, if I search for that transaction, it seems to be a transfer from and to > account A000069, for a different Payee. There ARE two splits for the > transaction, but both for A000069, and there is no assigned Category for the > transaction. Given that action="Transfer", I'm not sure if a Category should > be necessary. I'm guessing that A000069 is a Brokerage account, due to the > presence of Investment transactions in it, but again, I don't know if it > matters. > > I'll keep digging to see if I can find any other problems with the file, but I > do wonder whether the consistency check should find that type of problem. >
I think the transaction I mentioned in Comment #11 was a red herring. The one which triggered the merge time error is listed in Comment #12 - T000000000000040026.
Sorry - I missed that you had uploaded a new file. I'll look at it as soon as I can. In the meantime, can you check for the other transaction I mentioned? That one is NOT caught by the consistency check, and was from 2006-08-13.
Jack, I found the transaction from 2006-08-13. It appears to be transferring money from one expense category to another which evidently is not allowed in KMyMoney. I don't see any other problems with the transaction. One of the categories involved is called Cash Account which I am guessing I set up to track cash expenditures. Why that is a category and not an Account, I can't explain. A little history that may be helpful. Years and years ago I did my accounting by hand with a ledger. At one point I transferred everything to a book keeping program (I don't remember what it was). From there I transferred it to Quicken. Eventually I transferred it to GNUCash and then earlier this year to KMyMoney. Whenever I did the transfer I used the tools provided by the new program. One of these programs must have allowed a transfer between expense categories, or something happened in one of the moves to a new program. As I look at through the Cash Account category, there appear to be a lot more of the same type of entry which I suppose has the potential to cause the same problem if I try merging payees. Is it possible to convert a category to an account? Would that resolve this problem? On 11/17/20 12:03 PM, Jack wrote: > https://bugs.kde.org/show_bug.cgi?id=427519 > > --- Comment #15 from Jack <ostroffjh@users.sourceforge.net> --- > Sorry - I missed that you had uploaded a new file. I'll look at it as soon as > I can. In the meantime, can you check for the other transaction I mentioned? > That one is NOT caught by the consistency check, and was from 2006-08-13. >
The problem seems to be that when you delete a payee, the program reassigns the transactions for that payes to the new payee you choose. Unfortunately, however it chooses the list of transactions to process, it misses that one, so at the end, when it checks for any transactions still assigned to the payee to be deleted, it finds it and raises the error. Unfortunately, as you discovered, it seems to do so in a way that all those other transactions that had the payee changed are somehow deleted. This behavior is clearly a bug. I'm not sure what to say about allowing this sort of transaction to have been imported. I can dig a bit more through the code, but we may have to wait for an opinion from one of the developers. Internally to KMM, a Category IS an Account (of type income or expense, rather than one of the more commonly used account types (checking, savings, investment, ...). I don't know of any way to change an account type between a category type and another type. I'll have to test, but I doubt it would be safe to just edit the xml file and manually change the type.) In terms of history, it sounds like "Cash Account" should be type Cash in KMM rather than one of the Category types. Note that if you do a transaction search for T000000000000040026 it shows you two results. If you click on one, it shows the ledger for the Category used for that split. If you then right click on the transaction, it displays the Payee (same for both splits) but you will note that it does NOT display that transaction. I suspect whatever filter is being used to show the transactions for a Payee is the same filter causing the transaction to not have it's Payee changed and thus causing the crash. Another bit of information - if you to a transaction search on P003550 (as text, not as Payee) that transaction is not included in the results. I don't understand why not, but I think all these oddities are somehow related. More research is needed.
Jack, I don't see what you are describing for transaction T000000000000040026. I can find the transaction and if I double click on it, it takes me to the ledger. If I right click on it, I get a menu that has an option to take me to the ledger for the payee. If I take that option, the transaction is not in the ledger. I get the same thing for both results of the transaction search. On 11/17/20 5:14 PM, Jack wrote: > https://bugs.kde.org/show_bug.cgi?id=427519 > > --- Comment #17 from Jack <ostroffjh@users.sourceforge.net> --- > The problem seems to be that when you delete a payee, the program reassigns the > transactions for that payes to the new payee you choose. Unfortunately, > however it chooses the list of transactions to process, it misses that one, so > at the end, when it checks for any transactions still assigned to the payee to > be deleted, it finds it and raises the error. Unfortunately, as you > discovered, it seems to do so in a way that all those other transactions that > had the payee changed are somehow deleted. This behavior is clearly a bug. > I'm not sure what to say about allowing this sort of transaction to have been > imported. I can dig a bit more through the code, but we may have to wait for > an opinion from one of the developers. > > Internally to KMM, a Category IS an Account (of type income or expense, rather > than one of the more commonly used account types (checking, savings, > investment, ...). I don't know of any way to change an account type between a > category type and another type. I'll have to test, but I doubt it would be > safe to just edit the xml file and manually change the type.) In terms of > history, it sounds like "Cash Account" should be type Cash in KMM rather than > one of the Category types. > > Note that if you do a transaction search for T000000000000040026 it shows you > two results. If you click on one, it shows the ledger for the Category used > for that split. If you then right click on the transaction, it displays the > Payee (same for both splits) but you will note that it does NOT display that > transaction. I suspect whatever filter is being used to show the transactions > for a Payee is the same filter causing the transaction to not have it's Payee > changed and thus causing the crash. > > Another bit of information - if you to a transaction search on P003550 (as > text, not as Payee) that transaction is not included in the results. I don't > understand why not, but I think all these oddities are somehow related. More > research is needed. >
Which file were you using? That was from the first one you posted. In the second one, you had deleted one transaction, so the numbers are different. I believe the same transaction in the newer file is just one number lower, so T000000000000040025 instead of T000000000000040026.
Git commit f0651a6ac44d197d6b70cc273d0c72c1d3a374d7 by Thomas Baumgart. Committed on 21/11/2020 at 08:24. Pushed by tbaumgart into branch '5.1'. Include all transactions in reassignment when merging/deleting payees The transaction filter skips a few tests when dealing with splits referencing categories. This causes transactions that only reference categories but no accounts not to be reported when searching e.g. for all transactions referencing a specific payee. Setting a special flag forces to include these tests even on splits referencing categories. In the given case, this will cause all transactions to be reported that reference a given set of payees. FIXED-IN: 5.1.1 M +2 -0 kmymoney/views/kpayeesview_p.h https://invent.kde.org/office/kmymoney/commit/f0651a6ac44d197d6b70cc273d0c72c1d3a374d7
Git commit 4b00598568941996cf68c8ee2744be53f1944c49 by Thomas Baumgart. Committed on 21/11/2020 at 08:50. Pushed by tbaumgart into branch 'master'. Include all transactions in reassignment when merging/deleting payees The transaction filter skips a few tests when dealing with splits referencing categories. This causes transactions that only reference categories but no accounts not to be reported when searching e.g. for all transactions referencing a specific payee. Setting a special flag forces to include these tests even on splits referencing categories. In the given case, this will cause all transactions to be reported that reference a given set of payees. Cherry-picked from f0651a6ac44d197d6b70cc273d0c72c1d3a374d7 and adjusted to master M +2 -1 kmymoney/views/kpayeesview_p.h https://invent.kde.org/office/kmymoney/commit/4b00598568941996cf68c8ee2744be53f1944c49
Jack, That makes sense, it is what I am seeing. I have been notified that the bug has been fixed and is available in version 5.1.1. Thanks for all of your help! On 11/20/20 9:41 AM, Jack wrote: > https://bugs.kde.org/show_bug.cgi?id=427519 > > --- Comment #19 from Jack <ostroffjh@users.sourceforge.net> --- > Which file were you using? That was from the first one you posted. In the > second one, you had deleted one transaction, so the numbers are different. I > believe the same transaction in the newer file is just one number lower, so > T000000000000040025 instead of T000000000000040026. >