Bug 413504 - Brush outline (and also stroke) lags significantly on 4k displays
Summary: Brush outline (and also stroke) lags significantly on 4k displays
Status: RESOLVED FIXED
Alias: None
Product: krita
Classification: Applications
Component: General (show other bugs)
Version: git master (please specify the git hash!)
Platform: Arch Linux Linux
: NOR normal
Target Milestone: ---
Assignee: Dmitry Kazakov
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-10-27 06:16 UTC by Raghavendra kamath
Modified: 2020-09-18 11:19 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
video demonstrating the delay in painting (588.92 KB, video/mp4)
2019-10-27 06:16 UTC, Raghavendra kamath
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Raghavendra kamath 2019-10-27 06:16:11 UTC
Created attachment 123510 [details]
video demonstrating the delay in painting

SUMMARY
When drawing long and big strokes the brush lags behind the cursor position for a bit, if you draw fast enough the brush takes a tiny second to catchup to the cursor positon. This can be evidently seen if you use the brush cursor shape as arrow and draw a long stroke with a "C) pencil-2" brush. Also this lag somewhat makes the drawing performance feel sluggish.  

I am not using any kind of stabilizer, this is with none and basic settings.

I am attaching a video to show the trailing brush stroke behind the cursor. I hope you can see it. If not please let me know how i can provide more information for this. 

This delay adds up and drawing in Krita feels a bit slow compared to Mypaint which gives better drawing feedback.

Operating System: Arch Linux 
KDE Plasma Version: 5.17.1
KDE Frameworks Version: 5.63.0
Qt Version: 5.13.1
Kernel Version: 5.3.7-arch1-1-ARCH
OS Type: 64-bit
Processors: 8 × Intel® Core™ i7-4790K CPU @ 4.00GHz
Memory: 15.5 GiB of RAM
Graphic card - Intel Integrated  Intel® HD Graphics 4600
Comment 1 Halla Rempt 2019-10-27 08:49:10 UTC
Do you also get this with the 4.2.7 appimage?
Comment 2 Raghavendra kamath 2019-10-27 09:07:47 UTC
Yes I can reproduce this in the latest appimage downloaded from the website (krita-4.2.7.1b-x86_64) For convenience of viewing I slowed the initial video you can see that the arrow shape is away than the stroke and cursor outline. This feels like a stabiliser is enabled even when there is none.
Comment 3 Raghavendra kamath 2019-10-27 09:11:34 UTC
Here is the link to the video, uploading it as there is a limit for file size here.

https://box.raghukamath.com/cloud/index.php/s/zri4ZAKK8gC6xmR

you can notice that in addition to the arrow being ahead sometimes the actual drawing of the stroke also lags behind the brush outline.
Comment 4 Halla Rempt 2019-10-27 09:20:24 UTC
Could you check some older appimages to see whether this has always been the case, as well?
Comment 5 Raghavendra kamath 2019-10-27 09:44:13 UTC
Yep, this is has bugged me for a while but I couldn't understand why I feel that way and it is only recently my attention went to it when I used arrow as cursor shape, earlier I was using no cursor shape. I tested this under 4.1.0 appimage I could reproduce it. I'll test this on different desktop with nvidia card and also on a laptop with amd card in a while and report back.
Comment 6 wolthera 2020-04-25 09:51:59 UTC
Yes, I have noticed this as well. Sometimes dmitry's spontanuous speedup proects make it less worse than it used to be, but it's still not to the level of mypaint or painttool sai.
Comment 7 andrei 2020-06-05 04:02:31 UTC
I can confirm this performance issue.

My system: windows 10, NVidia 1050Ti + IntelHD 630 (some core i7)
Krita version: beta from Steam. (can confirm for stable version too)

Krita settings (display/performance) do not influence much for me.
Brush smoothing is disabled.
It does not depend on whether I use mouse or tablet.

Interesting note: it depends on canvas size.
The bigger canvas window, the larger lag you can observe. For example, on 4k monitor it's really slow.

Interesting note 2: moving cursor even without any drawing loads both GPU heavily (up to 80% on Intel and up to 25% on NVidia).

Interesting note 3: Full Screen mode reduces this load a bit, but still it is pretty noticeable.


I checked if it's the same for other drawing applications on my PC. For some yes, for some no. One of the working ones is GIMP - almost no lag between cursor and brush, and almost no load on GPU.


Please, fix this or point me how to do it. : )
Comment 8 andrei 2020-06-05 14:59:17 UTC
Tested on other PCs today.
* windows laptop, without Nvidia GPU - same lag.
* ubuntu pc with Nvidia - no lag.
* ubuntu laptop without Nvidia - no lag.
* ubuntu in VirtualBox on windows - black canvas and freeze : )

So, apparently it works only on native ubuntu system, not related to discrete GPU presence.
Comment 9 andrei 2020-06-14 14:06:34 UTC
I really doubt somebody is reading this thread, but anyway I'd like to keep my research results somewhere...

At first, some more information about my setup. ANGLE is used, Nvidia vertical sync disabled - no difference. Also, I'm testing on 4K monitor. With smaller monitor the lag is less noticeable, but still present. Image size is not so relevant, you can take 256x256 with the same effect. The main factor is canvas widget size.

To be sure it's software problem, I run application on the same PC under Ubuntu - no lag.

So, I tried to build from source and check what's taking so much GPU resource when mouse is just moving.

It happened to be KisToolPaint::requestUpdateOutline with funny commentaries like:
// DIRTY HACK ALERT: we should fetch the assistant's dirty rect when requesting the update, instead of just dumbly update the entire canvas!

So, I commented out all that block and tried to run again. This time there was no GPU load during mouse move, but I started to notice the same effect when drawing with freehand brush. Apparently it does full canvas refresh too.

I think, even the canvas rendering is not 100% optimized, this problem is mainly due to Nvidia Optimus brokenness and it is not possible to disable integrated GPU for me now. On ubuntu, iGPU is not used, and everything is fine (some other stuff is not fine, but it's not related to krita).

Maybe I'll try to fix this in future, but it seems much easier way is to buy normal PC with GPU directly connected to monitor.
Comment 10 Dmitry Kazakov 2020-08-30 12:24:57 UTC
Well, there are different kinds of processes involved into the lag. If we try to limit the lag definition for this bug to "the lag of the outline that happens when hovering the stylus over the canvas surface and that is proportional to the display resolution", then this kind of lag is fixable.

The solution to this lag is to finally implement partial updates for the openGL canvas. That is now possible thanks to Qt's internal widget buffer that caches the canvas.
Comment 11 andrei 2020-08-31 00:49:22 UTC
That would be interesting to try. I guess not only outline, but also brush drawing can be updated only in the area of brush.

For now I just switched to Ubuntu, which does not have Nvidia Optimus. I read on some laptops it can be disabled in bios, but not on my MSI.

I wonder how games for example work on Windows. If rendering is done only with opengl then intel gpu can be disabled...? hm...

Maybe it's worth mentioning in some FAQ, that Laptop with Nvidia + 4K + Windows give noticeable lag.
Comment 12 Dmitry Kazakov 2020-08-31 09:11:11 UTC
Ok, Raghavendra confirmed that he has Quad HD display and the bugs happens while hovering the brush over canvas without painting. So the bug is definitely in openGL updates. We should try to make them partial.
Comment 13 Dmitry Kazakov 2020-09-04 06:54:23 UTC
Hi, Andrei!

Could you please test this proof-of-concept package? Does it remove the lag, or at least reduce it?

Windows: https://yadi.sk/d/vCbSms7IPlaaBQ
AppImage: https://yadi.sk/d/n59GFoYDw5IdxQ

The rendering in this package is very buggy, because its main purpose is to test brush outline rendering and painting speed. To overcome bugs, follow these steps:

1) Create an image
2) Press '2' key to "Zoom Fit" the image in Krita window (the background is not updated in the build so you will have hard time seeing the borders)
3) Hover the stylus over the image without painting and check the speed
3) Paint with various brushes and check the speed
Comment 14 andrei 2020-09-04 09:51:43 UTC
Ok, thank you. I’ll try to test on this weekend.
Comment 15 Dmitry Kazakov 2020-09-04 22:43:13 UTC
Hi, Andrei!

Could you also check this package (DK14)? It is basically the same optimization present in the previous package(DK11), but it has a few bugs fixed. Though this bugfixes could theoretically make it slower, so it would be nice to test both:

DK14: https://yadi.sk/d/0CpMkqRjTDL86w

PS:
This new package should have zooming, background and checkers fixed, though canvas decorations are still disabled.
Comment 16 andrei 2020-09-05 07:24:37 UTC
Dear Dmitry, thanks for your work~

I tried to do some tests, and here's the result:

For all tests I used "Basic 1" brush, size 100, without smoothing.
Picture size 4096x4096, 8 bit.
Laptop with Nvidia GPU as before. External monitor 4K, 60 FPS. 
For checking the lag I used video on my phone, also 60 FPS.

At first I tried to check with screen recorder, but realized that it affects Krita performance. That's why phone camera.

1. Windows, Krita 4.3, Intel GPU
Outline lag: ~4 frames (65ms), brush lag: ~5 frames (83ms).

2. Windows, Krita 4.3, Nvidia Optimus
Outline lag: ~8 frames (130ms), brush lag: ~11 frames (180ms).

3. Windows, Krita DK11, Intel GPU
Outline lag: ~3 frames (50ms), brush lag: ~3 frames (50ms).

4. Windows, Krita DK14, Intel GPU
Outline lag: ~3 frames (50ms), brush lag: ~3 frames (50ms).

5. Windows, Krita DK14, Nvidia Optimus
Outline lag: ~4 frames (65ms), brush lag: ~5 frames (83ms).

6. Ubuntu, Krita 4.2.9, Only Nivida
Outline lag: ~1 frame (17ms), brush lag: ~1 frame (17ms). Maybe even less.

7. Ubuntu, Krita 4.2.9, Intel GPU
Outline lag: ~4 frames (65ms), brush lag: ~5 frames (83ms).

So few notes on this.
* Your fix is working. It improves the performance 25~50% depending on GPU used.
* Only Intel GPU is better than Nvidia + Intel GPU.
* Performance on pure Nvidia is still ~3 times higher. (case 6)
Comment 17 Dmitry Kazakov 2020-09-05 07:49:40 UTC
Hi, Andrei!

Do I understand it right that you you used "OS-provided hardware-optimized cursor" as a baseline for you tests? You don't have screen-tablet like "Wacom Cintiq" or something?

I'm asking because I also tested the patch on a screen-tablet device. And for such devices there are two different lags: 1) between hardware stylus you hold in the hand and OS-provided cursor; 2) between OS-provided cursor and the outline. We cannot do anything about the first lag (it is in hardware and drivers) but we can (and the patch tries to) improve the second lag.
Comment 18 andrei 2020-09-05 08:09:42 UTC
Nonono. I use usual tablet (XP-Pen Deco Pro)
Comment 19 Dmitry Kazakov 2020-09-05 08:16:25 UTC
Hi, Andrei!

Just a wild guess, did you try playing with "VSync" options of your GPU driver? Perhaps disabling VSync or enabling something like "Triple buffering" addect the lag?
Comment 20 Dmitry Kazakov 2020-09-05 08:20:03 UTC
Can you also try setting FPS limit in Krita to 60fps (Preferences->Performance->Advanced->FPS LImit + restart Krita), does it make any change?
Comment 21 Halla Rempt 2020-09-05 11:25:12 UTC
Gimp cannot put any load on the GPU since it doesn't use the GPU at all.
Comment 22 Dmitry Kazakov 2020-09-08 22:26:31 UTC
Here is a merge request with testing packages: 
https://invent.kde.org/graphics/krita/-/merge_requests/488
Comment 23 Dmitry Kazakov 2020-09-17 15:26:30 UTC
The original problem has been resolved by disabling VSync in KWin compositor settings. Disabling VSync in NVidia drivers didn't do any change. It seems like KWin's take precedence over the driver's defaults.

The partial updates patch has also been merged and it reduces the delay a bit in both modes: VSync on and off.
Comment 24 andrei 2020-09-18 11:19:20 UTC
Sorry, Dmitry, I haven't noticed your questions in time.

VSync/Tripple buffering give no noticeable effect.

Reducing Krita FPS to 60 also does not change anything much,
but when I set it to 30, the lag is almost completely gone (regardless of Nvidia/Intel gpu is used).
Unfortunately, with 30 FPS artifacts start to appear.

Probably I can live with those artifacts, since they don't affect resulting image...

PS: tested on Krita 4.3.0.