Bug 363320 - Very bad performance on some large files
Summary: Very bad performance on some large files
Status: RESOLVED FIXED
Alias: None
Product: krita
Classification: Applications
Component: Brush engines (show other bugs)
Version: git master (please specify the git hash!)
Platform: Other Linux
: NOR critical
Target Milestone: ---
Assignee: Krita Bugs
URL:
Keywords: regression, release_blocker
Depends on:
Blocks:
 
Reported: 2016-05-20 13:27 UTC by Camille Bissuel
Modified: 2016-05-24 12:32 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
screenshot of default performance tab (115.41 KB, image/png)
2016-05-20 14:47 UTC, Camille Bissuel
Details
screenshot of upgraded performance tab (115.96 KB, image/png)
2016-05-20 14:48 UTC, Camille Bissuel
Details
Callgrind trace (837.45 KB, application/octet-stream)
2016-05-21 08:29 UTC, Halla Rempt
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Camille Bissuel 2016-05-20 13:27:24 UTC
I work on a large file (8192x8192 pixel), as I use to when I want to do some illustration for an A2 poster. it work well in Krita 2.9 if I took care of not filling my RAM with too much layers.

It worked starting from scratch in Krita 3 (recompiled this morning)… but I stopped my work, I closed Krita, and now I want to work again on it … But now brush strokes display and computation is so slow I can't even paint.
Switching on and off layer visibility seem to allow computation and display to solve. But I can't do that between each brush stroke !

See this video : https://drive.google.com/open?id=0B-5J--XsvWh-YXBMZ1RXbVVQc0U
You can download the kra file here : https://drive.google.com/open?id=0B-5J--XsvWh-azJ0V2NOai1xU00

Might be related to Bug 363225 : https://bugs.kde.org/show_bug.cgi?id=363225
(same config :  I use Krita 3 Beta on Archlinux (Antergos distrib) build from AUR on a new shiny (since a few months) Wacom Cintiq 27 Touch tablet, with Gnome 3.20, libwacom 0.18 and xf86-input-wacom 0.32 on an AMD gpu and OpenGL enabled on a Radeon R9 290…)

Disabling OpenGl doesn't change anything, no more than disabling Instant preview.

If it's not my config, you should be able to reproduce by simply opening the file and try to draw.


Reproducible: Always
Comment 1 Halla Rempt 2016-05-20 13:39:18 UTC
Sounds like a duplicate of https://bugs.kde.org/show_bug.cgi?id=363087 -- are you sure you're not running out of memory? I cannot test right now, I'm work though.
Comment 2 Camille Bissuel 2016-05-20 13:52:43 UTC
No it's not a duplicate : you can see a small monitor (gnome system-monitor extension) in the top right in the video : green is for RAM, it's not even full, and SWAP appear in violet right to it if used.

Take your time to test, I'm back to Krita 2.9 to finish this work. But IMHO, not being a part of the Krita team, I think we should investigate this before Krita 3 release.

Please give me guidance on how I can give you more infos.
Comment 3 Halla Rempt 2016-05-20 14:19:22 UTC
The weird thing is that we've at the same time people telling us that 3.0 has amazing performance for them, with big images and everything...
Comment 4 Camille Bissuel 2016-05-20 14:24:14 UTC
Overall it's good, but I hit very bad bugs…
Maybe it's just with my config… but I can't tell how it's specific… maybe an update in a library on Arch which is not already released on other distribs ?  Or because I'm under Gnome 3 ?
I don't know where to look at to be honest.
Comment 5 Halla Rempt 2016-05-20 14:33:48 UTC
The system swap isn't what Krita uses to swap. How much memory does your system have in total? And can you add a screenshot of your performance settings tab?
Comment 6 Camille Bissuel 2016-05-20 14:38:29 UTC
I have 16 Gig of RAM… I tried to use default in the performance tab, and I try increased them almost to a maximum, and it doesn't change anything.
I need to reinstall Krita 3 to provide a screenshot, I'll try with the RC1 appimage in case it change something.
Comment 7 Camille Bissuel 2016-05-20 14:47:47 UTC
Created attachment 99097 [details]
screenshot of default performance tab

Not more luck with the RC1 appimage
Comment 8 Camille Bissuel 2016-05-20 14:48:23 UTC
Created attachment 99098 [details]
screenshot of upgraded performance tab
Comment 9 Camille Bissuel 2016-05-20 14:51:27 UTC
By the way, disable progress reporting doesn't affect this (nor the other logging or disable vector optimizations)
Comment 10 Halla Rempt 2016-05-20 14:52:19 UTC
That looks fine, and your image only takes about 2gb anyway, so that's not the problem. Does it make a difference if you disable progress reporting?
Comment 11 Camille Bissuel 2016-05-20 14:53:41 UTC
Cross posts ;)
Sorry to bring bad news in such a positive moment for Krita ;(
Comment 12 Halla Rempt 2016-05-20 16:24:59 UTC
Hm, on my mac, I can confirm that everything is really slow, and the file crashes Krita in my Linux vm. So my first guess was that there were lots of pixels outside the image, but that's not true either. On the other hand, Krita 2.8 in another VM also couldn't load the file at all... I guess I need to investigate on my desktop, back at home.
Comment 13 Camille Bissuel 2016-05-20 17:33:20 UTC
Your last comment made me do one last test : Open other big files in Krita 3 RC1 (based on the same 8K template).
Interestingly, only the file created today with Krita 3 triggered this bug.
Older files created with Krita 2.9 or Krita 2.8 are not.
But the bigger one I found ("radical-dreaming.kra" made with Krita 2.9) crashed Krita 3 when my pen was close enough to be detected. Reopening once again it worked.

I'm uploading big files here (upload will take two or three hours), so you can test :
https://drive.google.com/open?id=0B-5J--XsvWh-Z3lWVksyVlJwMzA
"D12.kra" and "radical-dreaming.kra" are 2.9 files
"illustration-mlle-solangel.kra" and "illustration-mlle-solange-final.kra" are new files created a few days ago and finished today with Krita 3.

Are there any change in the file format between 2.9 and 3 ?
Comment 14 Halla Rempt 2016-05-20 17:36:27 UTC
> Are there any change in the file format between 2.9 and 3 ?

Only for the animation support.
Comment 15 Camille Bissuel 2016-05-20 19:17:08 UTC
Another idea : the original very rought sketch in this file was done  with mypaint in the .ora format.
I add it to the Drive (named "croquis-mlle-solange-4.ora"). So one layer in the kra file is coming from ora. 
I tried to remove it with Krita 2.9 and open it again in Krita 3, but it doesn't fix the bug.
Comment 16 Halla Rempt 2016-05-21 07:34:09 UTC
The radical_dreaming.kra is much, much bigger than the mll-solange image, and that one performs fine indeed. And I can confirm on my desktop that it's impossible to work with the mlle-solange image. I'm going to take a look...

I'm wondering if the reason is that the slow file has a layer style. I'm also wondering where the icon that indicate that a layer has a style has gone.
Comment 17 Halla Rempt 2016-05-21 08:29:33 UTC
Created attachment 99109 [details]
Callgrind trace

I've run a callgrind run. Started after loading the image, ran while trying to draw a line. There is some confusion because apparently an autosave kicked in, but it looks like there is some serious time spent calculating layer bounds for calculating changerects.
Comment 18 Halla Rempt 2016-05-21 08:32:17 UTC
https://phabricator.kde.org/T2565
Comment 19 Halla Rempt 2016-05-21 08:40:04 UTC
After adding a debug line to KisLayer::changeRect, painting a single to prints this:

isLayer::changeRect "couleurs" QRect(3733,2557 31x31) 64
KisLayer::changeRect "couleurs" QRect(3733,2557 31x31) 64
KisLayer::changeRect "couleurs" QRect(3733,2557 31x31) 64
KisLayer::changeRect "shade" QRect(3733,2557 31x31) 8
KisLayer::changeRect "shade" QRect(3733,2557 31x31) 8
KisLayer::changeRect "line art" QRect(3733,2557 31x31) 8
KisLayer::changeRect "line art" QRect(3733,2557 31x31) 8
KisLayer::changeRect "highlights" QRect(3733,2557 31x31) 8
KisLayer::changeRect "highlights" QRect(3733,2557 31x31) 8
KisLayer::changeRect "group" QRect(3733,2557 31x31) 64
KisLayer::changeRect "group" QRect(3733,2557 31x31) 64
KisLayer::changeRect "root" QRect(3733,2557 31x31) 64
KisLayer::changeRect "root" QRect(3733,2557 31x31) 64
KisLayer::changeRect "couleurs" QRect(3733,2557 31x31) 64

And takes about 20 seconds.
Comment 20 Halla Rempt 2016-05-21 08:44:07 UTC
Hm, and something is wrong with the layer styles in any case: opening the layer styles document without enabling any layerstyle still means that layer styles are saved with the document and enabled for the current layer. Tested by creating an empty document, opening and closing the layer styles dialog.
Comment 21 Camille Bissuel 2016-05-23 07:38:20 UTC
It probably make no difference, but I don't remember I have used any layer style in this file… and I don't find any style by opening the document.
I may have opened the layer style panel… but by accident, I didn't meant to use it ;p
Comment 22 Halla Rempt 2016-05-23 11:57:27 UTC
*** Bug 363316 has been marked as a duplicate of this bug. ***
Comment 23 Camille Bissuel 2016-05-23 17:04:50 UTC
Investigating further, I tried to compare the file with others… 
and it is set to an unusual resolution of 75 ppi : I suppose this is coming from MyPaint because I didn't set it up.

If I change the resolution to anything else (I tried with 72, 90 and 300 ppi), with "Scale Image To New Size", and checking "Adjust print size separately", or if I downscale the definition a little, performance are way better : It's not perfectly smooth as usual, but I can draw easily. 

So maybe scaling is involved, or scaling unlock something in the file properties or in the file structure ? or there is some miscalculation happening depending on the resolution ?

Any way, that's a first workaround…
Comment 24 Halla Rempt 2016-05-23 17:39:19 UTC
Oh, that's a very interesting finding!
Comment 25 Dmitry Kazakov 2016-05-24 10:19:33 UTC
Git commit bd46e0aa30c5fded9da44566fcd35f1335a550d1 by Dmitry Kazakov.
Committed on 24/05/2016 at 10:19.
Pushed by dkazakov into branch 'master'.

Remove amortizing from the amortizeExactBounds()

Its worst-case time may be higher than 300ms, which is unacceptable
for the painter. So the proper solution would be to implement a special
stroke that walks through all the layers and

1) Purges all the default pixel tiles using KisPaintDevice::purgeDefaultPixels()
2) Calls exactBounds() for every device to update the caches

Fixes T2565
Fixes T2543
Related: bug 363316

M  +7    -11   libs/image/kis_paint_device_cache.h

http://commits.kde.org/krita/bd46e0aa30c5fded9da44566fcd35f1335a550d1
Comment 26 Camille Bissuel 2016-05-24 12:32:19 UTC
I just compiled the latest Krita, and I confirm it's fixed !
Many thanks to both of you Boud and Dmitry, even I don't have any clue how you actually fixed that ;)