Bug 384903 - Lost data
Summary: Lost data
Status: RESOLVED WORKSFORME
Alias: None
Product: kmymoney
Classification: Applications
Component: database (show other bugs)
Version: 4.8.0
Platform: Other Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: KMyMoney Devel Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-09-20 21:14 UTC by aperali
Modified: 2021-03-19 04:33 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
attachment-11492-0.html (1.38 KB, text/html)
2017-09-24 22:30 UTC, aperali
Details
attachment-15418-0.html (1.67 KB, text/html)
2017-09-25 18:25 UTC, aperali
Details
attachment-15903-0.html (2.65 KB, text/html)
2017-09-25 18:41 UTC, aperali
Details
attachment-16351-0.html (3.79 KB, text/html)
2017-09-25 18:51 UTC, aperali
Details
attachment-21734-0.html (1.07 KB, text/html)
2017-09-25 21:13 UTC, aperali
Details
attachment-21763-0.html (134 bytes, text/html)
2017-09-25 21:13 UTC, aperali
Details
attachment-21814-0.html (2.52 KB, text/html)
2017-10-03 14:19 UTC, Brendan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description aperali 2017-09-20 21:14:52 UTC
I have been using KMM for about 3 months and included all my transactions of all my accounts since jan17 and about 3 years of my investment. 
Yesterday I was doing some entries (Im not sure in what panel, maybe in schedule or ledger) and I get an error message something like "out of memory" but KMM did not close after closing this message. I continue to use KMM normally and saved my progress before close KMM. After a while I open KMM again but my previous job was not saved! It never happened before! 
I retyped everything again, saved and quit from KMM. Reopen and it was nice. Continue to enter some others transactions, made some reconciliations, saved and close KMM. Reopen and the reconciliations was not there. It is happening today again. I tried to save as another file but nothing changed.
Someone could help me to solve it. KMM is the only manager that fulfilled my needs?
Im using windows 4.8.0 version with GPG encryption.
Is there anyway or tool to fix the KMM or make a clean copy of my database?
Unfortunately I don have any tracking bug file and dont know how to create it.
Thank you,
Comment 1 Jack 2017-09-24 17:10:52 UTC
First, always make frequent backups.  Then, if you run into this error, you will only have lost a small amount of work.  

Second, when you get any errors, it often helps to copy down the exact wording or even take a screen shot of the error.  We need to know if the error comes from KMyMoney itself, or from one of the KDE libraries it uses, or from Windows.   Also, which version of Windows are you using?

For your last questions, we cannot fix KMM if we don't know exactly what is wrong.  Are you storing your data in a file or in a database.  If you are using GPG, then it is probably a file.  You can simply copy the file, but that won't fix any bad data in it.  

The underlying data file is a plain text file in XML format, and sometimes looking at it in an editor can provide some information, but I don't think that would be the first thing I would check here.  If you want to do that, you need yo make a copy of the file, then unencrypt it with GPG, then gunzip it.  I'm not sure whether you have all the tools to do that easily available.

I think we need more information before anyone can make a particular suggestion about what to do next.
Comment 2 Thomas Baumgart 2017-09-24 17:25:44 UTC
Just a little correction: if you gpg decrypt a file you don't need to gunzip it. It is already decompressed.
Comment 3 aperali 2017-09-24 22:30:49 UTC
Created attachment 108000 [details]
attachment-11492-0.html

Thanks both you,
I'm sorry to not have any clear desorption of the error and haven't take a
screen shot. The memory error appeared just once. After that I did not got
it again but I still miss some data. What I did to minimize the problem was
to set up kmm to save each 0sec and it seems to solve the problem for now,
although each time I make a change it saves the file and take a few ms.
The next time I got the error I'll take a screen shot. I'm using Windows 10
pro regular updated.
Best regards,

Em 24 de set de 2017 2:25 PM, "Thomas Baumgart" <bugzilla_noreply@kde.org>
escreveu:

> https://bugs.kde.org/show_bug.cgi?id=384903
>
> --- Comment #2 from Thomas Baumgart <tbaumgart@kde.org> ---
> Just a little correction: if you gpg decrypt a file you don't need to
> gunzip
> it. It is already decompressed.
>
> --
> You are receiving this mail because:
> You reported the bug.
Comment 4 Thomas Baumgart 2017-09-25 06:05:11 UTC
Please make sure, that the number of automatic backups is set to 10 or above so you can always go back a step in case something goes really crazy.
Comment 5 aperali 2017-09-25 18:25:28 UTC
Created attachment 108016 [details]
attachment-15418-0.html

Today I was entering new transactions and it seems ok. Then, I try to
import an ofx file and it imported fine but when I was editing the imported
transactions it stopped to work while the saving bar below of the screen
was on 100% and KMM freeze stopping to work. I force close kmm and started
again but all my previous saving (0sec automatic saving) was not there!
I noticed that KMM created a file with the "ne.kmyBO8040.new" extension.
This is the unique clue I have.


2017-09-25 3:05 GMT-03:00 Thomas Baumgart <bugzilla_noreply@kde.org>:

> https://bugs.kde.org/show_bug.cgi?id=384903
>
> --- Comment #4 from Thomas Baumgart <tbaumgart@kde.org> ---
> Please make sure, that the number of automatic backups is set to 10 or
> above so
> you can always go back a step in case something goes really crazy.
>
> --
> You are receiving this mail because:
> You reported the bug.
>
Comment 6 aperali 2017-09-25 18:41:43 UTC
Created attachment 108018 [details]
attachment-15903-0.html

Forgot to mention that another occurrency was after an update stock and
currency prices kmm frozen again and i forced close it loosing previous
data even with 0 auto save.


2017-09-25 15:25 GMT-03:00 Alexandre Perali <aperali@gmail.com>:

> Today I was entering new transactions and it seems ok. Then, I try to
> import an ofx file and it imported fine but when I was editing the imported
> transactions it stopped to work while the saving bar below of the screen
> was on 100% and KMM freeze stopping to work. I force close kmm and started
> again but all my previous saving (0sec automatic saving) was not there!
> I noticed that KMM created a file with the "ne.kmyBO8040.new" extension.
> This is the unique clue I have.
>
>
> 2017-09-25 3:05 GMT-03:00 Thomas Baumgart <bugzilla_noreply@kde.org>:
>
>> https://bugs.kde.org/show_bug.cgi?id=384903
>>
>> --- Comment #4 from Thomas Baumgart <tbaumgart@kde.org> ---
>> Please make sure, that the number of automatic backups is set to 10 or
>> above so
>> you can always go back a step in case something goes really crazy.
>>
>> --
>> You are receiving this mail because:
>> You reported the bug.
>>
>
>
>
> --
> Atc,
> Perali
>
Comment 7 aperali 2017-09-25 18:51:29 UTC
Created attachment 108019 [details]
attachment-16351-0.html

I have about 38 accounts including savings, investment, checking,
retirement,assets, credit card, etc
Could it be the problem?
Ive just reopen the file I really sure that was saved including reconciled
accounts and even reconcliliations are not there!!! It is very frustationg!

2017-09-25 15:41 GMT-03:00 Alexandre Perali <aperali@gmail.com>:

> Forgot to mention that another occurrency was after an update stock and
> currency prices kmm frozen again and i forced close it loosing previous
> data even with 0 auto save.
>
>
> 2017-09-25 15:25 GMT-03:00 Alexandre Perali <aperali@gmail.com>:
>
>> Today I was entering new transactions and it seems ok. Then, I try to
>> import an ofx file and it imported fine but when I was editing the imported
>> transactions it stopped to work while the saving bar below of the screen
>> was on 100% and KMM freeze stopping to work. I force close kmm and started
>> again but all my previous saving (0sec automatic saving) was not there!
>> I noticed that KMM created a file with the "ne.kmyBO8040.new" extension.
>> This is the unique clue I have.
>>
>>
>> 2017-09-25 3:05 GMT-03:00 Thomas Baumgart <bugzilla_noreply@kde.org>:
>>
>>> https://bugs.kde.org/show_bug.cgi?id=384903
>>>
>>> --- Comment #4 from Thomas Baumgart <tbaumgart@kde.org> ---
>>> Please make sure, that the number of automatic backups is set to 10 or
>>> above so
>>> you can always go back a step in case something goes really crazy.
>>>
>>> --
>>> You are receiving this mail because:
>>> You reported the bug.
>>>
>>
>>
>>
>> --
>> Atc,
>> Perali
>>
>
>
>
> --
> Atc,
> Perali
>
Comment 8 Jack 2017-09-25 19:48:08 UTC
[If you are going to reply to a bug comment by email, please consider trimming unnecessary lines, as the entire message ends up as the new comment.]
I'm not sure what it means to set the automatic backup time to 0 minutes (not seconds).  I would try it at 1 or more.  Have you confirmed that the automatic backups are actually being made?  

If KMM froze when saving the file, that is very different from freezing doing the import or while editing a transaction.  Your first posting said you had received an error about "out of memory" but I wonder if this is a translation issue, and you actually mean "out of space"?  Have you checked if the disk where you are saving the files has sufficient free space?

You can also consider manually saving the file more often, such as immediately after doing an ofx import.  That file extension you mention might be the temporary file name KMM uses to save the new file - only renaming it to the real file name once it knows the new file has successfully been saved.  Have you tried renaming and opening that file?  Is it your data?  With or without the new transactions/edits?
Comment 9 aperali 2017-09-25 21:13:42 UTC
Created attachment 108022 [details]
attachment-21734-0.html

Thank you Jack!  Sorry for the email history.
I'm sure there is plenty of memory on the drive I'm saving kmm data.
I mean auto save time was set to 0.
I cannot find auto backup files except 3 user-Kmm files ended by -1, -2 and
-3 (the number of backups I set up on kmm settings) but none of them ends
with .kmm.
The 1st time, I realized that error message but it's not being showed
again. Now I realized twice that the problem (freezing) appear after saving
the file than, after I forced close kmm, some (10 min I guess) data was
lost despite I'm sure they were manually saved by me.
Kmm is freezing after saving the file (when the left bottom blue bar shows
100% complete}.
I renamed that file but it also contains the same data of the original one.
I'm quite sure the problem is related to when i shutdown kmm while it
freezes.
Comment 10 aperali 2017-09-25 21:13:44 UTC
Created attachment 108023 [details]
attachment-21763-0.html

I'll try to reproduce the error after ofx importing and editing some
transactions to see if it happens again
Comment 11 Jack 2017-09-25 21:48:13 UTC
First, try setting your backup interval to something other than 0.
Second, does KMM freeze if you save after just editing one transaction, or only if you have done something more, such as reconciling and account or importing an ofx file?
Third, if you are posting your comments by replying in email, please configure your email software to only send plain text.  It seems each of your replies has created an attachment to the bug - but the all seem to be just HTML versions of the comment itself.
Comment 12 aperali 2017-09-26 12:48:22 UTC
Autosave is set to every 3 minutes now
Numbers of backup to keep is set to 3 (all of backups have the extension: name.kmy.1~ 2~ or 3~). It curious that they havent any difference in modified time. Shouldnt it be different from each 3 min?
The freezing seems to be randomly sometimes after a simple edit of a transaction and saving sometime after another couple of other activities.
I noticed that a specific transaction with a split that I tryied to change its values was insisting to back to the old values when the freezing happened. I deleted and recreated these transactions, lets see what happens now.
Is there any way I can delete those attachment of the bug report?
Comment 13 Jack 2017-09-26 16:17:44 UTC
It is possible that when KMM alters the file name (from 1 to 2 and 2 to 3 - or perhaps the reverse) before creating the new one, Windows might consider that the last modified time.  

I see no new attachment this time, so whatever you did differently worked.
Comment 14 aperali 2017-10-02 23:03:35 UTC
Today after a lot of entries it happened again. Im almost giving up kmm. It is really very frustrating. I did many charges on credit card, some transfers and manage many investment, saved the file, quit from kmm and launch kmm again when many of data was not there! Why could it be happening?
Comment 15 aperali 2017-10-02 23:08:26 UTC
Strange that I deleted 2 accounts and now they are there again!
Comment 16 aperali 2017-10-02 23:17:19 UTC
(In reply to aperali from comment #15)
> Strange that I deleted 2 accounts and now they are there again!

It is definitely not saving sometimes. I did just one modification now, saved the file, quit and open it again and the modification was not there.
Comment 17 aperali 2017-10-02 23:27:21 UTC
(In reply to aperali from comment #16)
> (In reply to aperali from comment #15)
> > Strange that I deleted 2 accounts and now they are there again!
> 
> It is definitely not saving sometimes. I did just one modification now,
> saved the file, quit and open it again and the modification was not there.

For some reason KMM is not saving into gdrive. I just save as <name> in my document folder and it saved ok. Try to save as <same name> in gdrive it warns me about overwriting I choose yes but the file was not saved!!!

I´ll try to keep the file in my document folder for a couple of days to see if the problem is something with gdrive.
Comment 18 Jack 2017-10-02 23:44:21 UTC
That is something you should have stated at the beginning.  If you don't say otherwise, we all assume you are just saving to a local hard drive.  What is gdrive, and how is it set up/configured?  Have you had any other problems with other programs saving to gdrive?  Have you googled about other people having problems saving to gdrive?

I am assuming you are talking about the Google storage service.  Is it set up as a drive letter?  Have you tried setting it up for offline use, having KMyMoney read and save the local copy, and then let gdrive sync the local copy with the gdrive copy?  That might work better and avoid whatever underlying problem you are having.

After saying all of that, however, it will probably be good for KMM to consider some method of confirming that a file has actually been saved correctly before the save function returns without any message.
Comment 19 aperali 2017-10-03 00:49:35 UTC
(In reply to Jack from comment #18)
> That is something you should have stated at the beginning.  If you don't say
> otherwise, we all assume you are just saving to a local hard drive.  What is
> gdrive, and how is it set up/configured?  Have you had any other problems
> with other programs saving to gdrive?  Have you googled about other people
> having problems saving to gdrive?
> 
> I am assuming you are talking about the Google storage service.  Is it set
> up as a drive letter?  Have you tried setting it up for offline use, having
> KMyMoney read and save the local copy, and then let gdrive sync the local
> copy with the gdrive copy?  That might work better and avoid whatever
> underlying problem you are having.
> 
> After saying all of that, however, it will probably be good for KMM to
> consider some method of confirming that a file has actually been saved
> correctly before the save function returns without any message.

yes, gdrive is the google drive storage service. I configured  it as local folder shared with google drive server; the usual way to do it i think (as you mentioned). It is not a drive letter and it is set up to save offline. Im using this set up since I started to use KMM about 3 months ago without a issue.
Now I moved the file to \user\username\my document

One thing I noticed also was when I save as the file it change the file name including this sufix "name-09-20ne.kmy"
Comment 20 Jack 2017-10-03 13:39:43 UTC
I think it is clearly a problem with gdrive, probably some interaction with how KMM actually saves a new version of the file.  The developers may correct me, but I believe it actually saves the new file with a temporary name, and then, once it is fully written and closed, it renames it.  I think with normal files (at least on Windows and Linux) that rename operation replaces the original file with the same name - perhaps gdrive doesn't work like a local hard drive - either because of something Google does intentionally, or perhaps it's a subtle timing issue related to Google syncing the local file with the online version.

Perhaps you can try "Save as" instead of save, but using the same name - just to see if it behaves any differently.  Another test - save to a new name on gdrive.  If that always works, then there is some issue with the specific details of how KMM saves the file.
Comment 21 Jack 2017-10-03 13:42:54 UTC
Also, now that I read your comment more carefully, that odd file name may indicate KMM's temporary name as part of saving.  Another thing to think about is whether Google changed anything at about the time this started failing.  Unfortunately, I doubt there is any way we can find out.
Comment 22 aperali 2017-10-03 13:55:37 UTC
As you previous suggestion, it would be good if the developers do some KMM check for the saved file (perhaps comparing the date/time of the file with the system time), because I realized that the time of file saved was much before the time that I really saved it.

Thank you Jack for all your replies and support
Comment 23 Brendan 2017-10-03 14:19:58 UTC
Created attachment 108141 [details]
attachment-21814-0.html

Google is on the process of deprecating the Google Drive app for Windows
and they are replacing it with Backup and Sync. This is all fairly recent
so you should make sure you specify which app you are currently using.
There is no reason for the developers to waste time trying to fix this
problem if it's caused by the app that won't work soon.

I used to consult for a company that used the old Google Drive app for
Windows extensively. The problem you are having was not uncommon. There
seemed to be a conflict with the software we were using to backup the
drives which would result it Google Drive wiping out newer files. I think
we were able to recover them from either the recycle bin on the local
computer or in the Google Drive trash on the web interface. Have you
checked either of those locations?



*----Brendan Coupe*

On Tue, Oct 3, 2017 at 7:55 AM, <bugzilla_noreply@kde.org> wrote:

> https://bugs.kde.org/show_bug.cgi?id=384903
>
> --- Comment #22 from aperali@gmail.com ---
> As you previous suggestion, it would be good if the developers do some KMM
> check for the saved file (perhaps comparing the date/time of the file with
> the
> system time), because I realized that the time of file saved was much
> before
> the time that I really saved it.
>
> Thank you Jack for all your replies and support
>
> --
> You are receiving this mail because:
> You are the assignee for the bug.
>
Comment 24 aperali 2017-10-03 14:51:00 UTC
(In reply to Brendan from comment #23)

The service Im using is one that can be reached at https://www.google.com/drive/
It also creates a shared folder that is used to sync with google drive.

The suggestion is only if a new version had planned to be launch, it´s just a small routine I think. I do all my job synced with google drive, using excell, primavera, etc without an issue.

I checked both recycle bin but nothing related to kmm was there either locally or in gdrive.
Comment 25 Jack 2021-02-17 00:35:21 UTC
Is this problem still happening?  Does gdrive still exist in the form you were using?  If not, does everything work OK with however you now have Google storage set up?  I don't know that KMyMoney has changed anything that would specifically effect saving to Google, but the latest version is 5.1.1, so upgrading is advised anyway.
Comment 26 Bug Janitor Service 2021-03-04 04:33:20 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 27 Bug Janitor Service 2021-03-19 04:33:52 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!