SUMMARY *** Using kmymoney-5.1-1802-macos-clang-x86_64.dmg - Can open .kmy files created on Linux (kmymoney v5.0.8 ) with 21720 transactions. Problem with unusable Home screen. Googled solution suggested that nightly version solved the Home problem. Installed kmymoney-master-2001-macos-clang-x86_64.dmg. Home screen works but Data newer than 2016 was missing. File Information shows only 14811 transactions NOTE: If you are reporting a crash, please try to attach a backtrace with debug symbols. See https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports *** STEPS TO REPRODUCE 1. 2. 3. OBSERVED RESULT EXPECTED RESULT SOFTWARE/OS VERSIONS Windows: macOS: Linux/KDE Plasma: (available in About System) KDE Plasma Version: KDE Frameworks Version: Qt Version: ADDITIONAL INFORMATION
Could be related to the content of your data. What does KMyMoney print to the console when started from the command line? Anything suspicious? What is the output of the following command run on the command line (replace "your-data-file-name.kmy" with your actual filename: zcat your-data-file-name.kmy | grep 'TRANSACTION>' | wc -l It should be the number of transactions plus the number of schedules shown in the File information dialog.
(In reply to Thomas Baumgart from comment #1) > Could be related to the content of your data. What does KMyMoney print to > the console when started from the command line? Anything suspicious? What is > the output of the following command run on the command line (replace > "your-data-file-name.kmy" with your actual filename: > > zcat your-data-file-name.kmy | grep 'TRANSACTION>' | wc -l > > It should be the number of transactions plus the number of schedules shown > in the File information dialog. Reply from Neil Ralph: 1) Not sure if I should be using 'reply' or 'Add Comment' 2) could not get zcat to complete: neilralph@Neils-MBP ~ % zcat 9gwengood4.kmy | grep 'TRANSACTION>' | wc -l zcat: can't stat: 9gwengood4.kmy (9gwengood4.kmy.Z): No such file or directory 0 neilralph@Neils-MBP ~ % pwd /Users/neilralph neilralph@Neils-MBP ~ % ls 9gwengood4.kmy Library Applications Movies Applications (Parallels) Music BBDownloads Nextcloud Desktop Parallels Documents Pictures Downloads Public
Either way of replying to a bug works, but if you reply by email, please remove all content, as your entire email becomes the next comment, which gets very cluttered if you include the previous message. Please post the result of "ls -l 9gwengood4*". I can't tell if there might be something strange with your filename or if zcat is doing something strange.
(In reply to Jack from comment #3) > Either way of replying to a bug works, but if you reply by email, please > remove all content, as your entire email becomes the next comment, which > gets very cluttered if you include the previous message. > > Please post the result of "ls -l 9gwengood4*". I can't tell if there > might be something strange with your filename or if zcat is doing > something strange. neilralph@Neils-MBP ~ % ls -l 9gwengood4* -rw-r--r--@ 1 neilralph staff 967340 Aug 1 09:53 9gwengood4.kmy I confirmed that I can open 9gwengood4.kmy with kmymoney.app (5.1.3) I noticed that v 5.1.80 doesn't appear in the dropdown choices for the bug report I should also mention that, when opening with v5.1.80, kmymoney opens with a dialog box asking for the currency type. If I close the dialog without adding my currency (US Dollar) the program crashes. I never had issues with currency definition on previous upgrades (i have been happily using kmymoney since 2008)
Neil, please look at the bug at bugs.kde.org comments 2 and 4, and you can see how not trimming the previous content clutters the bug. Don't worry about the version, but the .80 in the daily build implies it was compiled from the head of the git branch, not from a released version. Mentioning it in the report as you have done is sufficient. Did you try what Thomas suggested - launching the program from command line and seeing if there is any relevant error message in the console output? The error that zcat gives on your file suggests some problem with the file itself. To check that, launch KMM from command line with "kmymoney -n" which tells it not to open any data file. If that crashes, the the bug is in the program. If that opens ok, and then crashes when you try to open your data file, there is some problem with the data. Is there anything in that dialog box for currency that might indicate what it needs a currency for? i would guess it would only need that if it didn't completely and successfully read the file. Do you have any other version of KMM which opens that file ok? Do you have a backup of that file to try?
Based on Neil's answer in comment #2 it could be, that zcat on MacOS requires the file to have a specific filetype. Can you try the following: cp 9gwengood4.kmy 9gwengood4.kmy.Z zcat 9gwengood4.kmy | grep 'TRANSACTION>' | wc -l rm 9gwengood4.kmy.Z As Jack pointed out, v 5.1.80 refers to "git (master)" here. I adjusted the version. The dialog box asking for the currency type is a sign that KMyMoney for whatever reason cannot read your data (completely). For further analysis it would be helpful, if you can paste the output that KMyMoney prints when running from the command line.
(In reply to Thomas Baumgart from comment #6) neilralph@Neils-MBP ~ % cp 9gwengood4.kmy 9gwengood4.kmy.Z neilralph@Neils-MBP ~ % zcat 9gwengood4.kmy | grep 'TRANSACTION>' | wc -l 21698 neilralph@Neils-MBP ~ % rm 9gwengood4.kmy.Z neilralph@Neils-MBP ~ % (In reply to Jack from comment #5) I am unable to run kmm from command line - quessing a syntax or PATH issue: neilralph@Neils-MBP ~ % kmymoney -n zsh: command not found: kmymoney (In General - I note that /Library/Preferences/ does not contain a file nor directory for kmymoney)
On MacOS usually the .app contents aren't added to your PATH variable. So to run you'll have to do something like /Applications/KMyMoney.app/Contents/MacOS/kmymoney <add arguments here> I haven't tried kmymoney on mac lately, not sure what the path is but adjust accordingly.
Still trying to start from terminal - but in mean time - closing the currency dialog (which opens when starting kmymoney from Finder) causes a crash with the following uncaught error: Security 'USD' not found. /Users/packaging/Craft/BinaryFactory/downloads/git/extragear/kmymoney/kmymoney/mymoney/mymoneyfile.cpp:3175
Possible option: rename your data file so the program can't find it. That will at least test if the program starts. Then you can open the renamed data file and see if it crashes then, indicating a problem with the file. It is still worth finding where it is installed, to see the console output, which might give more information on the actual problem.
Okay - Starting from command line opens kmm - neilralph@Neils-MBP /Applications % /Applications/kmymoney.app/Contents/MacOS/kmymoney -n Terminal prints many items ending with: Plugins: xmlstorage loaded Template successfully opened from "/Applications/kmymoney.app/Contents/Resources/checkprinting/check_template.html" Everything is OK .. Then I open an existing file and the Currency Dialog opens - 8 new lines show in Terminal: 2023-08-11 18:22:10.789 kmymoney[6110:382017] +[CATransaction synchronize] called within transaction Open file QUrl("file:///Users/neilralph/Nextcloud/Statements/9gwengood6.kmy") Model for "I" loaded with 8 items in 0 ms Model for "P" loaded with 1508 items in 8 ms Start verifying account hierarchy End verifying account hierarchy Model for accounts loaded with 358 items in 3 ms Model for "T" loaded with 33151 items in 84 ms Upon adding US Dollar and closing the Dialog – Terminal displays many items not found, ie: "Account" "A000351" "BPFCU-CD-2747 6 month 12/9/23 " Security 'E000072' not found. /Users/packaging/Craft/BinaryFactory/downloads/git/extragear/kmymoney/kmymoney/mymoney/mymoneyfile.cpp:3175 "Account" "A000352" "BPFCU-CD-6635 12 month 12/30/23" Security 'E000073' not found. /Users/packaging/Craft/BinaryFactory/downloads/git/extragear/kmymoney/kmymoney/mymoney/mymoneyfile.cpp:3175 After listing the many "not found"s Terminal displays: Start calculating balances: 33151 splits End calculating balances Processed home view section 8 in 72 ms Processed home view section 1 in 5 ms Processed home view section 2 in 4 ms Processed home view section 3 in 3 ms Processed home view section 4 in 0 ms Processed home view section 5 in 7 ms Processed home view section 6 in 170 ms Processed home view section 7 in 0 ms Processed home view section 10 in 5 ms
It sounds like your data file is actually incomplete. Can you think of any way it might have gotten truncated? You started by saying the file could be opened on Linux. Is that still true? Can you try moving the file again from Linux to Mac and compare the file sizes in both locations? I also note you said the original has 21720 but the zcat | grep | wc only showed 21698. Can you try a variant on what Thomas asked earlier: cp 9gwengood4.kmy 9gwengood4.kmy.Z zcat 9gwengood4.kmy | tail -n 10 rm 9gwengood4.kmy.Z The last few lines should probably be </REPORTS> <BUDGETS/> <ONLINEJOBS/> </KMYMONEY-FILE> If the last line is not </KMYMONEY-FILE> then the file has definitely been truncated. Do you have any backups?
Reply to Comment 12 neilralph@Neils-MBP ~ % cp 9gwengood4.kmy 9gwengood4.kmy.Z neilralph@Neils-MBP ~ % zcat 9gwengood4.kmy | tail -n 10 <ACCOUNTGROUP group="expense"/> <ACCOUNTGROUP group="income"/> <ACCOUNTGROUP group="asset"/> <ACCOUNTGROUP group="liability"/> <DATES from="2022-01-01" to="2023-01-25"/> </REPORT> </REPORTS> <BUDGETS count="0"/> <ONLINEJOBS count="0"/> </KMYMONEY-FILE> neilralph@Neils-MBP ~ % rm 9gwengood4.kmy.Z neilralph@Neils-MBP ~ %
Comment14 (reply to Comment 12) before installing v 5.1.8 I renamed kmymoney.app to kmymoney1dot3.app (in macos Applications) I can run that from command line: neilralph@Neils-MBP ~ % /Applications/kmymoney1dot3.app/Contents/MacOS/kmymoney -n KMM opens with no problem I can then open my latest Linux save (9gwengood6) using <File><Open>within KMM, which loads with no loss of data. The File Info indicates 21741 transactions I wonder if upgrading from 5.1.3 to 5.1.8 is too great a leap – Should I try 5.1.4?
Well, count me confused. Comment 13 shows your file is not truncated, and if 5.1.3 opens it without problem, then the file should be OK. If by 5.1.8 you mean 5.1.80, it should be able to read the file with no problem. There is no 5.1.4 - that will be the next release from the 5.1 branch of git, although that is not currently scheduled. The 80 is not a real version number, it just indicates that it has been compiled from a specific git commit, not from a released version. If you run the 5.1.80 version and do Help About KMyMoney, it will give a version where there is something after the 5.1.80. That will be indicate the git commit when that version was compiled, and we can check if there are any problems with that specific version, although I don't recall hearing about any. You can also try downloading a newer nightly version.
kmymoney-master-2001-macos-clang-x86_64.dmg is pretty recent. The changes that were made in between are not relevant for this problem. The file structure is OK, otherwise 5.1.3 would complain. Also, looking at the tail of the files shows that it is not truncated. Leaves the point, that the number of transactions shall be 21720 but shows 21698. Well, 22 difference. Not sure where that may come from. A gut feeling tells me, that the XML reader stops processing data too early. That may be caused by content in the data file that is not stored according to the strict XML specifications. The old (<= 5.1.3) XML reader simply reads the data but the new one (>= 5.1.80) may have stumbled upon it and simply stopped. Another explanation would be, that accounts exist that reference an unknown security. As accounts are loaded before the transactions and securities after, they are not loaded properly (better at all) and cannot be found in the process of finishing the load operation (consistency check). That is when messages like "Account" "A000351" "BPFCU-CD-2747 6 month 12/9/23 " Security 'E000072' not found. /Users/packaging/Craft/BinaryFactory/downloads/git/extragear/kmymoney/kmymoney/mymoney/mymoneyfile.cpp:3175 are generated. That also explains, why there are no further messages when loading the file after loading the model T (transactions). It usually looks like this: : : Model for "G" loaded with 5 items in 0 ms Start verifying account hierarchy End verifying account hierarchy Model for accounts loaded with 214 items in 4 ms Model for "T" loaded with 24195 items in 137 ms Model for parameters loaded with 6 items Model for schedules loaded with 82 items in 0 ms Model for "E" loaded with 4 items in 0 ms Model for currencies loaded with 184 items Model for prices loaded with 242 items in 0 ms Model for "R" loaded with 53 items in 0 ms Model for "B" loaded with 6 items in 0 ms : : The model load operations are all triggered by the XML reader. BTW, the number shown when loading the journal model (T) is the number of splits not transactions. We need to figure out at which transaction it fails (sounds easy, but may be not). Once more: make sure to have a good backup copy of your data before you start changing things!!! Or simply work on a temp copy. You know what I mean First let's convert the .kmy to .xml file so that we can look at it. This is the sequence to do it: cp 9gwengood4.kmy 9gwengood4.kmy.Z zcat 9gwengood4.kmy > 9gwengood4.xml rm 9gwengood4.kmy.Z On Linux, the tool xmllint can be used to analyze the content of the XML file and check for errors. I don't know, if that is available for MacOS or if there are other tools to verify an XML file. Does that show any errors/problems?
Thanks to all of you for your efforts - I began looking at the xml code, which led me to think that problem might be my sloppy handling of securities as there were 74 instances of currency="E0000**". At which point I thought to look how the version upgrade handled a much smaller .kmy file (216kB) with no securities. This Linux .kmy has 4519 transactions with 9174 splits. Upon opening it from 5.1.80 the currency window came up. Upon choosing USD and closing the window the Home Screen displayed the status of the accounts at the end of 2017. File Information shows 173 transactions and 345 splits. Upon saving the file size became 25kB. Transactions later than 2017 were lost. The console printout did not indicate errors. Printed Log Upon opening: 2023-08-12 14:41:42.746 kmymoney[8482:577400] +[CATransaction synchronize] called within transaction Open file QUrl("file:///Users/neilralph/Downloads/kmymoney/RalphCenterportGood.kmy") Model for "I" loaded with 7 items in 0 ms Model for "P" loaded with 1030 items in 5 ms Start verifying account hierarchy End verifying account hierarchy Model for accounts loaded with 179 items in 1 ms Model for "T" loaded with 345 items in 0 ms Printed Log Upon closing Currency Dialog: Start calculating balances: 345 splits End calculating balances Processed home view section 8 in 1 ms Processed home view section 1 in 0 ms Processed home view section 2 in 0 ms Processed home view section 3 in 1 ms Processed home view section 4 in 0 ms Processed home view section 5 in 2 ms Processed home view section 6 in 58 ms Processed home view section 7 in 0 ms Processed home view section 10 in 0 ms Plugins: budgetview unloaded Plugins: checkprinting unloaded Plugins: csvexporter unloaded Plugins: csvimporter unloaded Plugins: forecastview unloaded Plugins: gncimporter unloaded Plugins: icalendarexporter unloaded Plugins: kbanking unplugged Plugins: kbanking unloaded Plugins: ofximporter unloaded Plugins: onlinejoboutboxview unloaded Plugins: qifexporter unloaded Plugins: qifimporter unloaded Plugins: reconciliation report unloaded Plugins: reportsview unloaded Plugins: sqlstorage unloaded Plugins: xmlstorage unloaded
What might an example of "sloppy handling of securities?" Assuming you have not edited the kmy file manuallly (after uncompressing into xml) then the program should only ever delete a security if there is no remaining reference to it left in the file. Also, if the new version also truncated a file with no securities, then there is (also) something else going on. It would be nice if we could figure out why it is opening the currency dialog when opening the file. Have you been able to try xmllint (or equivalent) against any of these xml files? Can you grep for CURRENCY in the files? Also, if all these cases are with data files you have transferred from Linux to MacOS, can you try transferring one of them back to Linux and compare it to the original?
Your "Users/neilralph/Downloads/kmymoney/RalphCenterportGood.kmy" file is a good example. You mention it shows 4519 on your Linux machine but loads only the first 345 on MacOS. So it seems that either transaction 345 or 346 in the list may contain the culprit. I shoot for the latter. Here's how we can easily find out on which line that is in the XML file (assuming you have converted that file to XML format already): grep -n '<TRANSACTION' RalphCenterportGood.xml | head -n 346 | tail -n 1 should show the line number and initial data of transaction 346. How does that whole TRANSACTION element (including all the splits) looks like? Can you post that here? Maybe, also include transaction 345 just in case. BTW, the last line of transaction 346 can be figured out using grep -n '/TRANSACTION' RalphCenterportGood.xml | head -n 346 | tail -n 1 I doubt that the transport from Linux to MacOS has an influence here, because the file is compressed and would not be read by KMyMoney if something would have failed during the transport. Also, prior versions of MacOS (5.1 branch) can open the file(s). The entries of form currency="E0000**" instances are another weirdness. They may have been introduced by an older version of the application and may not cause problems, but that is another topic and should not be handled/discussed here.
(In reply to Thomas Baumgart from comment #19) neilralph@Neils-MBP kmymoney % zcat RCGcopy.kmy > RCGcopy.xml neilralph@Neils-MBP kmymoney % grep -n '<TRANSACTION' RCGcopy.xml | head -n 346 | tail -n 1 5903: <TRANSACTION postdate="2016-06-30" commodity="USD" memo="" entrydate="2017-01-01" id="T000000000000000751"> neilralph@Neils-MBP kmymoney % grep -n '<TRANSACTION' RCGcopy.xml | head -n 345 | tail -n 1 5897: <TRANSACTION postdate="2016-06-30" commodity="USD" memo="" entrydate="2017-01-01" id="T000000000000000750"> neilralph@Neils-MBP kmymoney % Using Xcode on Mac: Line 5902 </TRANSACTION> <TRANSACTION postdate="2016-06-30" commodity="USD" memo="" entrydate="2017-01-01" id="T000000000000000751"> <SPLITS> <SPLIT shares="-30/1" reconcileflag="2" memo="" action="" value="-30/1" price="1/1" payee="P000425" account="A000032" bankid="" number="" id="S0001" reconciledate="2016-07-13"/> <SPLIT shares="30/1" reconcileflag="0" memo="" action="" value="30/1" price="1/1" payee="P000425" account="A000030" bankid="" number="" id="S0002" reconciledate=""/> </SPLITS> </TRANSACTION> Line 5896 </TRANSACTION> <TRANSACTION postdate="2016-06-30" commodity="USD" memo="" entrydate="2017-01-01" id="T000000000000000750"> <SPLITS> <SPLIT shares="-10/1" reconcileflag="2" memo="" action="" value="-10/1" price="1/1" payee="P000530" account="A000032" bankid="" number="" id="S0001" reconciledate="2016-07-13"/> <SPLIT shares="10/1" reconcileflag="0" memo="" action="" value="10/1" price="1/1" payee="P000530" account="A000069" bankid="" number="" id="S0002" reconciledate=""/> </SPLITS> </TRANSACTION>
While looking through the xml I wanted to test the assumption that the transactions import sequentially in line# order – Transactions for payee P000021 stopped after processing line 4870: (4 lost before line 5902) SSA TREAS is P000021 finding “TREAS” gave 178 hits, 2/transaction Line 3902: 12/31/15 919/1 caught Line 3908: 12/31/15 383/1 caught Line 4307: 2/3/2016 919/1 caught Line 4313 2/3/2016 383/1 caught Line 4550 3/3/2016 919/1 caught Line 4556 3/3/2016 383/1 caught Line 4864 4/1/2016 919/1 caught Line 4870 4/1/2016 383/1 caught Line 5230 5/3/2016 919/1 missing Line 5236 5/3/2016 383/1 missing Line 5632 6/3/2016 919/1 missing Line 5638 6/3/2016 383/1 missing Transactions for payee P000463 stopped after processing line 4858, with two lost before line 5902 Steve is P000463 3884 12/28/15 2000/1 caught 3968 1/1/16 2001/1 caught 4238 2/4/16 2001/1 caught 4520 ` 2/29/16 2000/1 caught 4664 3/9/16 2000/1 caught 4858 3/31/16 2000/1 caught 5266 5/4/16 2000/1 missing 5758 6/10/16 2000/1 missing
The two transactions Thomas asked to you to look at start at line 5897, but you think you start losing transactions somewhere between lines 4870 and 5266. Just like you did in Comment 20, you can look at more transactions, decreasing the number, until you get to those lines, seeing if anything looks amiss. You can also look at the file in a text editor, starting around line 4870.
I am new at working with xml, and it looks like there are numerous issues with my 2 KMM apps. I think I am going to stick with using 5.3.1 for now. I don't have the time and skill set necessary for porting to nightly now. Do you imagine that the upgrade issues will be tackled before 5.3.1 goes out of support? Might there be an upgrade/conversion utility in the works? The Nightly 5.1.80 version looks good but I noticed a few little problems and issues with it. Should I use a bug report to communicate them or mention the in a email - if an email - to whom? Thank you so much for your generosity and attention in helping me !- Neil
If you want to use an e-mail you can send it to kmymoney-devel@kde.org (if you are not subscribed, the message will be moderated and approved). You can also use the forum at https://discuss.kde.org. In this case, make sure to tag your post with the kmymoney tag. Switching to a new version does not require any conversion tool. As a last step here I would recommend to run xmllint on that file. No need to do it on the MacOS side. You can use your Linux system to do it. Maybe, you need to install xmllint from your distro packages first.
(In reply to Thomas Baumgart from comment #24) I used xmllint and gedit to clean up the 9gwengood6a.xml file. There were numerous instances "/00" in TRANSACTION memo fields. How do I convert the .xml file to .kmy?
A .kmy file is just a gzipped .xml file. You can either gzip and rename it yourself, or just open the xml file in kmymoney and save as to kmy. I'm curious if you know how NULLs got into the memo fields. We can probably look for them when saving a data file, but we should compare that to being able to find them when they are first introduced.
(In reply to Jack from comment #26) The strings "/00" in the memo fields were found in credit card transactions imported from .qfx files. When looked at in KMM these transactions all had exactly 3 lines (not wrapped) in their memos .. I guess codes for new lines in the qfx files may contribute
Thomas made a commit to master branch to remove those \0 characters on import. That should already be available in any version compiled from master (5.1.80) but will not show up in the 5.1 branch. You should be able to easily switch between the 5.1 and master branch as there are no differences in the data file. However, always keep backups, just in case.