Bug 331592

Summary: Painting with brushes slowly fills up memory
Product: [Applications] krita Reporter: vanyossi <ghevan>
Component: GeneralAssignee: Krita Bugs <krita-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: dimula73
Priority: NOR    
Version: 2.8 Beta   
Target Milestone: ---   
Platform: unspecified   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: Krita using more ram than expected
memcheck log
Screens showing different krita ram usage
cmake config log

Description vanyossi 2014-02-28 01:19:00 UTC
While painting the RAM memory used by krita starts to grow more and more. I remember setting undo steps to 5 to keep ram usage low, but now it starts to build up slowly as I paint.

Swap memory also starts to fill over time.

Reproducible: Always

Steps to Reproduce:
This is what I'm doing that causes the build up
1. Open a serialized ora file (name_002.ora) the file contains two layers, one transparent and on with a paper color
2. I sketch on the transparent layer (this is the action that causes the build up)
3. Save incremental version (to have now a name_003.ora)
4. clear the sketched out layer, keep painting.

After a couple of rounds ram gets filled.


Expected Results:  
Ram should be relative to the complexity of the image and not the amount of strokes made. (unless undos are infinite :p)

Ram seems to build up independently of the image size.
After testing with a new image, .kra I'm certain the painting action alone causes the issue

XFCE desktop
Kdelibs 4.11.2
qt 4.8.5
krita Version 2.8 Beta 3 (git e9de85d)
Comment 1 Dmitry Kazakov 2014-02-28 09:49:47 UTC
By default Krita is using up to 50% of the memory you have. That is intentional and can be configured if you need it.

Could you tell more exactly:

1) How much memory you have in your system?
2) What is the size and bit-depth of the image you use?
3) What is the undo limit of Krita (also note, that undo limit setting is applied only after restart of Krita)?

When measuring, also take into account that Linux, by default, keeps all the files Krita accesses cached in memory. That means when Krita iterates through its swap file, Linux may cache portions of it in memory, which you may qualify as "Krita eats more than 50% of memory". But in real life, that memory will be reclaimed by the OS as soon as it needs it.
Comment 2 vanyossi 2014-02-28 17:39:01 UTC
Created attachment 85364 [details]
Krita using more ram than expected

The system has 2GB.
Image bit depth is 8bit on the testings.
Krita is set to undo limits 5 since a long time, so i guess that's not it.

I don't have the real measures of the "free" command but as far as I remember it where like this.
 free  in:
-/+ buffers/cache:  86M
Swap:  350M

amount in buffers stays between 70 ~ 86M free memory, but swap keeps becoming less and less.
It takes about 3hours painting time to fill memory from 600MB used. a second attempt made the ram usage go to 1.6g on the RES column of top, but fewer programs where opened.

Attaching screenshot with top running.
Comment 3 Halla Rempt 2014-03-02 11:10:52 UTC
Created attachment 85381 [details]
memcheck log

I've tried a memcheck run and while I see lots of memory leaks, nothing that reall explains the issue reported. It's mostly stuff that doesn't get deleted at exit.
Comment 4 vanyossi 2014-03-05 00:59:30 UTC
I'll keep monitoring Krita usage, It is not a massive build up, it takes time and on normal usage I believe it's a non issue. In any case closing and reopening after 6 hours of painting frees all the memory and makes system go back to normal.
Comment 5 vanyossi 2014-03-15 06:46:36 UTC
Created attachment 85585 [details]
Screens showing different krita ram usage

Ok so It does what I described, it fills all swap memory also. This makes changing tools to lag a lot, specially when making transforms or warps. On a side note Krita crashed while saving a huge file when memory was short. So first test was partially lost.

Memory fills faster the bigger the image, which is expected, and makes me think that maybe the bug is not so much of a leak but a missing line in the limit of undo steps code. I have set mine at 5 and it does limit to that amount, but memory keeps filling on each action. I'm sure it worked fine before as I changed the limit to cope with memory issues and until this recent 2.8 build it worked great. Even at 5000x3000 memory never filled up.

I hope this helps. I'll try another test using CPU canvas. I just remembered that when everything worked ok I was not on OpenGL. (Maybe it has nothing to do, but it's worth a try as one session it began  building memory when panning)
Comment 6 vanyossi 2014-03-18 05:24:20 UTC
Created attachment 85623 [details]
cmake config log

I think I found the source of my problem. It's not a leak, just a nasty bug, possibly related to build options.

I can't turn off Actions/Macros recording. On a fresh start pressing "save all actions" in the menu "tools > recording" I get a 19 bytes file. But after 10 strokes the file grows to 1.2megs. I get to this conclusion after working half an hour on a 4000x1800 document. On a fresh start on that file Krita uses 780MB memory, but after working for about 35 minutes the same file (2 layers, one bg and one empty) uses 1.3GB. Pressing "Save all actions" gets me, to my surprise, a 634MB file which is exactly the memory surplus.

I tried to turn off recording pressing "stop recording actions" before beginning to work, but even like this pressing "save all actions" gets me a file with contents. (4.2Megs for a few strokes)

So it's not a leak as the program is not loosing memory into oblivion, but I don't need to be recording at all times. This could be a problem with my build because recorded actions does nothing if I select "Macross > Open and play" How can I prevent this recording to even happen? it eats memory and after long periods it makes screencasts impossible.

I attach my cmake configure options and log.
Comment 7 Dmitry Kazakov 2014-03-18 06:15:01 UTC
Git commit 9e88ae2828b730d74bf08367c3418290940014d0 by Dmitry Kazakov.
Committed on 18/03/2014 at 05:49.
Pushed by dkazakov into branch 'master'.

Remove the ability to record all the actions ever done by the user

This results in a drastic memory leak which grows with the time
(it seems that replaying the brush actions doesn't work, both with and
without the patch)

Please test it is fast as possible then I could backport it to 2.8.

M  +2    -2    krita/image/recorder/kis_action_recorder.cc
M  +5    -6    krita/image/recorder/kis_action_recorder.h
M  +1    -10   krita/plugins/extensions/bigbrother/bigbrother.cc
M  +0    -1    krita/plugins/extensions/bigbrother/bigbrother.h
M  +0    -2    krita/plugins/extensions/bigbrother/bigbrother.rc

http://commits.kde.org/calligra/9e88ae2828b730d74bf08367c3418290940014d0
Comment 8 vanyossi 2014-03-19 04:18:51 UTC
Hi again.
I tested in master using b20fe23.
compiled and ran without previously running 2.8 branch. Enrico splash appear and the "save all actions" menu action was gone also.

After much testing I have to say that it seems it behaves exactly as before =(. However suspecting of the actions recording I also made a few attempts to start and stop recording. That still functions but there is something weird. By default both menus options can be activated (start and stop) Pressing start will grey out the start button making the stop option the only available for pressing. After stop, the option "stop recording" becomes greyed out, that is until I make any action (For example make a stroke on canvas). Both start and stop become highlighted again.

I made tests with openGL canvas and without it, Using master and just to be sure after that I switched to 2.8 and performed the same tests. Both versions seem to behave the same. For the record I have both installs in different directories and I just switch between them using a symlink as the install directory. I wonder if there is a cache file I need to remove...

Anyways after one sketch, same size and roughly the same amount of work.
top (RES column). Fresh / Sketch -- Difference / saved krarec ( ls -h )
v2.8
424468 / 484008  -- dif 59540 / 52MB

master
406968 / 492460 -- dif 85492 / NA

Not sure if the patch is no good or if this is the normal amount of memory expected to be used. But as far as I can tell it uses the same as 2.8 that is recording actions. What bugs me is that even in master it keeps growing until swap is filled making all Krita GUI super slow.
Comment 9 vanyossi 2014-03-20 01:42:41 UTC
After testing dimitryK changes and not find a clue I set up to find the break point.

Using this forum post as reference http://forum.kde.org/viewtopic.php?f=281&t=120046 and boud git expertise I set a bisect to find the bugger. The test process was as follows

Start krita, annotate fresh RAM usage
Sketch a reference image, annotate RAM
Sketch a second image, annotate RAM again

On builds without the bug RAM upper limit was consistently around 630MB.
On builds with the bug RAM upper limit after 1st sketch hit 680MB~750MB and second sketch made ram up to 920MB~960MB usage. In this scenarios I kept random brushing with a big brush until 1GB was reached (about 15 strokes was enough for this). In this case I confirmed the build as a bad one.

The result of the bisect. (2.8 branch)

0202186ac4b8ae917d7c735adfcc1b41bb626bcb is the first bad commit
commit 0202186ac4b8ae917d7c735adfcc1b41bb626bcb
Author: Boudewijn Rempt <boud@valdyas.org>
Date:   Mon Feb 17 15:10:03 2014 +0100

    BUG:331181 Don't activate all sensors when loading old format presets

:040000 040000 93a726d4b7a981cf58c6568775b93f5b987534a7 306a69b0c6326c18ccb199d48068d25b67e937d6 M	krita


Previous commit cd49a328880 was the lowest scorer of all. 59800 was the upper limit no matter how much or how big I painted.
I hope this helps resolve the bug :D
Comment 10 Halla Rempt 2014-03-31 20:34:03 UTC
Git commit a9271d26288b7b4dc1d3e76a2dbd99e18d0622d4 by Boudewijn Rempt.
Committed on 31/03/2014 at 20:22.
Pushed by rempt into branch 'master'.

Use KisShared to handle memory management of the sensor objects that
get created when making a stroke.

M  +21   -26   krita/plugins/paintops/libpaintop/kis_curve_option.cpp
M  +5    -5    krita/plugins/paintops/libpaintop/kis_curve_option.h
M  +4    -4    krita/plugins/paintops/libpaintop/kis_curve_option_widget.cpp
M  +4    -3    krita/plugins/paintops/libpaintop/kis_curve_option_widget.h
M  +4    -6    krita/plugins/paintops/libpaintop/kis_dynamic_sensor.cc
M  +14   -5    krita/plugins/paintops/libpaintop/kis_dynamic_sensor.h
M  +4    -4    krita/plugins/paintops/libpaintop/kis_multi_sensors_model_p.cpp
M  +5    -5    krita/plugins/paintops/libpaintop/kis_multi_sensors_model_p.h
M  +6    -6    krita/plugins/paintops/libpaintop/kis_multi_sensors_selector.cpp
M  +6    -5    krita/plugins/paintops/libpaintop/kis_multi_sensors_selector.h
M  +1    -1    krita/plugins/paintops/libpaintop/kis_pressure_opacity_option.cpp
M  +3    -6    krita/plugins/paintops/libpaintop/kis_pressure_rotation_option.cpp
M  +2    -2    krita/plugins/paintops/libpaintop/tests/kis_sensors_test.cpp
M  +2    -3    krita/plugins/paintops/libpaintop/tests/kis_sensors_test.h

http://commits.kde.org/calligra/a9271d26288b7b4dc1d3e76a2dbd99e18d0622d4
Comment 11 Halla Rempt 2014-04-01 09:57:08 UTC
Git commit ab836d4737fc96019682b1e005a26250cd2eac5c by Boudewijn Rempt.
Committed on 31/03/2014 at 20:22.
Pushed by rempt into branch 'calligra/2.8'.

Use KisShared to handle memory management of the sensor objects that
get created when making a stroke.

M  +21   -26   krita/plugins/paintops/libpaintop/kis_curve_option.cpp
M  +5    -5    krita/plugins/paintops/libpaintop/kis_curve_option.h
M  +4    -4    krita/plugins/paintops/libpaintop/kis_curve_option_widget.cpp
M  +4    -3    krita/plugins/paintops/libpaintop/kis_curve_option_widget.h
M  +4    -6    krita/plugins/paintops/libpaintop/kis_dynamic_sensor.cc
M  +14   -5    krita/plugins/paintops/libpaintop/kis_dynamic_sensor.h
M  +4    -4    krita/plugins/paintops/libpaintop/kis_multi_sensors_model_p.cpp
M  +5    -5    krita/plugins/paintops/libpaintop/kis_multi_sensors_model_p.h
M  +6    -6    krita/plugins/paintops/libpaintop/kis_multi_sensors_selector.cpp
M  +6    -5    krita/plugins/paintops/libpaintop/kis_multi_sensors_selector.h
M  +1    -1    krita/plugins/paintops/libpaintop/kis_pressure_opacity_option.cpp
M  +3    -6    krita/plugins/paintops/libpaintop/kis_pressure_rotation_option.cpp
M  +2    -2    krita/plugins/paintops/libpaintop/tests/kis_sensors_test.cpp
M  +2    -3    krita/plugins/paintops/libpaintop/tests/kis_sensors_test.h

http://commits.kde.org/calligra/ab836d4737fc96019682b1e005a26250cd2eac5c
Comment 12 Halla Rempt 2014-05-30 15:44:42 UTC
Git commit 5522e745bcd1707814637241275a823f1a6bdf9d by Boudewijn Rempt.
Committed on 31/03/2014 at 20:22.
Pushed by dickson into tag 'steam-247500_247566'.

Use KisShared to handle memory management of the sensor objects that
get created when making a stroke.

M  +0    -2    krita/plugins/paintops/libpaintop/kis_dynamic_sensor.cc

http://commits.kde.org/calligra/5522e745bcd1707814637241275a823f1a6bdf9d