Bug 456280 - WindowHeap window draw calls depend on draw calls of current desktop (freezes windows from other desktops until current desktop updates)
Summary: WindowHeap window draw calls depend on draw calls of current desktop (freezes...
Status: RESOLVED FIXED
Alias: None
Product: kwin
Classification: Plasma
Component: general (show other bugs)
Version: 6.0.0
Platform: Ubuntu Linux
: NOR minor
Target Milestone: ---
Assignee: KWin default assignee
URL:
Keywords: regression
: 455416 456279 468411 (view as bug list)
Depends on:
Blocks:
 
Reported: 2022-07-03 13:34 UTC by postix
Modified: 2024-04-10 15:27 UTC (History)
14 users (show)

See Also:
Latest Commit:
Version Fixed In: 6.0.4


Attachments
desktop grid doesn't update the inactive desktops (Wayland and X11) (3.87 MB, video/mp4)
2022-09-19 20:36 UTC, bastimeyer123
Details
firefox information (6.15 KB, image/png)
2022-11-05 20:14 UTC, spittingverse@gmail.com
Details
Screenrecording: 5.26.2 + Firefox 106 (2.61 MB, video/x-matroska)
2022-11-05 20:27 UTC, postix
Details
kwin-6.0-dev-desktopgrid-x11 (3.87 MB, video/mp4)
2023-11-20 14:00 UTC, bastimeyer123
Details
kwin-6.0-dev-desktopgrid-wayland (3.81 MB, video/mp4)
2023-11-20 14:01 UTC, bastimeyer123
Details

Note You need to log in before you can comment on or make changes to this bug.
Description postix 2022-07-03 13:34:48 UTC
STEPS TO REPRODUCE
1.  Open a window, whose content changes over time, eg play a video
2.  Move the window to another VD

OBSERVED RESULT
The video stops playing respectively the window becomes static, showing the last frame when it was dragged from its original VD.

Moving it back to its original VD, makes it animated again. If a 2nd screen is attached, moving it to the corresponding VD on the 2nd screen, has the same effect.


SOFTWARE/OS VERSIONS
Operating System: openSUSE Tumbleweed 20220702
KDE Plasma Version: 5.25.2
KDE Frameworks Version: 5.95.0
Qt Version: 5.15.5
Kernel Version: 5.18.6-1-default (64-bit)
Graphics Platform: Wayland
Graphics Processor: AMD Radeon RX 580 Series
Comment 1 postix 2022-07-03 13:40:24 UTC
> Moving it back to its original VD, makes it animated again. If a 2nd screen is attached, moving it to the corresponding VD on the 2nd screen, has the same effect.

My guess would be that this is the reason why this issue is not observable in the "present windows" and "overview" effect.
Comment 2 David Edmundson 2022-07-03 13:54:39 UTC
*** Bug 455416 has been marked as a duplicate of this bug. ***
Comment 3 postix 2022-07-03 14:18:31 UTC
I can reproduce this issue as well w/o moving a window between two VD by using again a window, which plays a video, on some VD "A" but invoking the grid effect on different VD "B": The window's content won't be updated anymore.
Comment 4 postix 2022-07-03 14:20:34 UTC
To complete comment 3: The window's content becomes animated again, when moving it to the VD "B". 

In this way it's the same bug but kind of the inverse in terms of the "steps to reproduce".
Comment 5 Nate Graham 2022-07-05 17:41:41 UTC
*** Bug 456279 has been marked as a duplicate of this bug. ***
Comment 6 ratijas 2022-08-01 08:35:42 UTC
Can confirm, Wayland-only. Works fine on X11. Maybe it's because on Wayland compositor (KWin) suspends apps from drawing their surfaces when they are not on an active desktop?
Comment 7 Maximilian Böhm 2022-08-01 16:40:18 UTC
It’s not Wayland-only. On X.org, the framerate of the window just drops dramatically low instead of a full freeze.
Comment 8 Nate Graham 2022-09-08 19:52:27 UTC
Cannot reproduce on git master Wayland with any of the WindowHeap-based effects anymore. Looks like this was fixed recently.
Comment 9 bastimeyer123 2022-09-16 21:18:50 UTC
(In reply to Nate Graham from comment #8)
> Cannot reproduce on git master Wayland with any of the WindowHeap-based
> effects anymore. Looks like this was fixed recently.

This is not fixed on X11. Window contents do not get updated on non-active desktops when the grid is active.
Just built KWin from the master branch (3aa1411d948f2ee43f99323007e7e776ce9e99f5).
Comment 10 Nate Graham 2022-09-19 19:01:13 UTC
This bug report was filed for Wayland; can you please open a new one for the X11 issue? Thanks!
Comment 11 bastimeyer123 2022-09-19 20:36:22 UTC
Created attachment 152236 [details]
desktop grid doesn't update the inactive desktops (Wayland and X11)

> Cannot reproduce on git master Wayland with any of the WindowHeap-based effects anymore.
> can you please open a new one for the X11 issue?

I was just about to open a new bug report for X11, but decided to take a look at Wayland first.
KWin built from the current HEAD of the master branch, d6234d0411401a55ca700c75f6425e6f3fbfc4d6.
The issue is **not** fixed, on both X11 and Wayland.

On Wayland, there's even a glitch sometimes where the window content switches between black and regular content each frame (as you can see in the attached video).
Comment 12 postix 2022-09-19 20:39:53 UTC
(In reply to bastimeyer123 from comment #11)
> On Wayland, there's even a glitch sometimes where the window content
> switches between black and regular content each frame 

This issue I had already observed and described in the dup bug #455416
Comment 13 Marco Martin 2022-09-28 15:02:47 UTC
could it be driver dependent? can't reproduce here
Comment 14 postix 2022-10-30 18:37:11 UTC
(In reply to Marco Martin from comment #13)
> could it be driver dependent? can't reproduce here

Which application(s) did you test? 
For me all tested apps do not update -- except for the Firefox 106 (Wayland native), which played a video from YouTube via invidio.us.
Comment 15 postix 2022-10-30 18:40:30 UTC
It seems that Firefox 106 (Wayland native) is animated fine in any cases.
Comment 16 bastimeyer123 2022-11-05 09:52:00 UTC
(In reply to Marco Martin from comment #13)
> could it be driver dependent? can't reproduce here

Which issue are you talking about? The black window contents for every other frame on Wayland, or the main issue that the window contents don't get updated anymore?

See the attachment of my post #11. Nothing has changed compared to building/running KWin from the current HEAD of the master branch from a few minutes ago. The black frames do only happen on my second screen though, the one on the right.
https://bugs.kde.org/show_bug.cgi?id=456280#c11

Using an up2date Arch system with its default repos, except for custom kwin and kdecoration packages which are built from their master branches (with kwin patches for changing some hardcoded constants like grid-spacing and spring-motion stuff which I don't agree with). Using AMDGPU with default Mesa on a Radeon RX 5700 XT. System config is two 1920x1080@60 screens, with the one on the right-hand side as primary screen. Both X11 and Wayland are broken, as shown in my post #11.

Btw, I really have to agree with the author of the following post:
https://bugs.kde.org/show_bug.cgi?id=457817#c1
I've been using a downgraded version of kwin ever since the QML rewrites were merged, so I don't have to deal with all these countless issues that make kwin unusable on a production system. Only testing kwin built from master once in a while to provide feedback.
Comment 17 spittingverse@gmail.com 2022-11-05 20:12:26 UTC
(In reply to postix from comment #15)
> It seems that Firefox 106 (Wayland native) is animated fine in any cases.

Are you sure? I am experiencing the issue with this system:
Operating System: Arch Linux
KDE Plasma Version: 5.26.2
KDE Frameworks Version: 5.99.0
Qt Version: 5.15.7
Kernel Version: 6.0.6-arch1-1 (64-bit)
Graphics Platform: Wayland
Processors: 8 × 11th Gen Intel® Core™ i7-1165G7 @ 2.80GHz
Memory: 15.4 GiB of RAM
Graphics Processor: Mesa Intel® Xe Graphics
Manufacturer: HP
Product Name: HP Pavilion Laptop 15-eg0xxx
System Version: Type1ProductConfigId
With Firefox 106.0.4 (64-bit)

To reproduce you open Firefox, play any YouTube video, and open the desktop grid. You can drag the window to any other desktop and within the grid the Firefox window will stop updating. This issue also occurs if you don't drag the window and switch to another VD and open the desktop grid from there. The issue is that no window contents update in virtual desktops that are not the one you opened the desktop grid from.
Comment 18 spittingverse@gmail.com 2022-11-05 20:14:34 UTC
Created attachment 153501 [details]
firefox information

Also just to confirm I am using the native wayland version of firefox
Comment 19 postix 2022-11-05 20:27:06 UTC
Created attachment 153502 [details]
Screenrecording: 5.26.2 + Firefox 106

> Are you sure? 

Well, here's the proof! The whole Firefox UI is not affected for me on my system. :)
Comment 20 postix 2022-11-07 15:40:59 UTC
I noticed that when recording with OBS 28.1.1 a window, in my case Chromium 107, Wayland native, the recorded window won't update in OBS, when the window is moved to a different virtual desktop. 

I guess this is somehow related and which makes the prio of the issue a little bit higher than just "minor" IMHO.
Comment 21 postix 2022-11-07 16:52:11 UTC
> I guess this is somehow related 

Checked with Firefox (which is always animated) and with mpv (showing the stuttering):

This is definitely the same issue. I.e. recording a window, if OBS is on a different VD than the window, is impossible, unless you want to have a still image.
Comment 22 postix 2022-11-07 16:53:31 UTC
Therefore moving it to "wayland-generic", since this is from my POV not specific to the grid effect.
Comment 23 bastimeyer123 2022-11-07 17:23:05 UTC
No, this is not a wayland specific issue. This affects X11 as well, if you take a look at #11
Comment 24 postix 2022-11-07 17:45:42 UTC
(In reply to bastimeyer123 from comment #23)
> No, this is not a wayland specific issue. This affects X11 as well, if you
> take a look at #11

You are right!
Comment 25 bastimeyer123 2022-11-09 16:12:49 UTC
> Severity changed to "minor"

Seriously? This issue is one of the main reasons why the grid and thus all recent versions of kwin since the QML rewrite are completely unusable. Not being able to see the actual/current contents of windows on other desktops makes the grid 100% useless because you can't effectly do any multitasking jobs on different desktops, and you need to switch to each individual desktop first before you know which content you'll see.
Comment 26 ratijas 2022-11-18 16:34:38 UTC
> Seriously? This issue is one of the main reasons why the grid and thus all recent versions of kwin since the QML rewrite are completely unusable. Not being able to see the actual/current contents of windows on other desktops makes the grid 100% useless because you can't effectly do any multitasking jobs on different desktops, and you need to switch to each individual desktop first before you know which content you'll see.

Well, in Android and iOS multitasking you only get to see apps' last frame — always — not just sometimes because of a bug. It doesn't make those systems 100% useless, heh)

I wonder, if the root cause of this bug are some optimizations in protocols, and if so, whether they can be temporarily turned off while effect is active
Comment 27 RogueRen 2023-01-08 17:55:19 UTC
I have a somewhat similar issue when capturing XWayland windows in OBS, where the capture in OBS will seemingly randomly drop framerates or freeze completely, then unfreeze if OBS is put back in focus as the active window. Only affects XWayland as far as I can tell, and after talking with the OBS support Discord channel they seems convinced this is related to KDE Wayland as OBS is rendering frames without issue, KDE just isn't sending them. 

This is only on the capture, the Xwayland windows itself runs without issue so it might have to do with the capture portal?

OS: Pop!_OS 22.04 LTS
Kernel: 6.0.12-76060006-generic 
KDE Plasma: 5.24.7
KDE Framework: 5.92.0
Qt: 5.15.3
CPU: AMD Ryzen 5 3600
GPU: AMD RX 6650 XT
Comment 28 bastimeyer123 2023-01-28 13:15:21 UTC
I'd like to very kindly bring some attention to this issue in 2023, especially with the next Plasma release in mind. Thanks!
Comment 29 bastimeyer123 2023-02-27 02:13:37 UTC
> Well, in Android and iOS multitasking you only get to see apps' last frame — always — not just sometimes because of a bug. It doesn't make those systems 100% useless, heh)

This is my desktop system and not a mobile phone with a battery. I need to see what's going on on the other desktops. Right now with the current buggy state, all I can see is which windows are on which desktops, but that information is rather useless, because I'm already aware of where the windows are because I moved them there. I need to see their contents, be it a video player, web browser, konsole output or what not. That's really frustrating and a major issue for my workflow.

> I wonder, if the root cause of this bug are some optimizations in protocols, and if so, whether they can be temporarily turned off while effect is active

That sounds very likely. Has anybody taken a look at this so far? I would really appreciate it if this could be fixed some time soon. Thank you very much.
I can't help as an end-user unfortunately, because I don't know any kwin internals, cpp or Qt. :/
Comment 30 bastimeyer123 2023-02-28 20:25:57 UTC
Btw, when the current desktop's contents get drawn/rendered, then the contents of all other desktops will be drawn/rendered too.

For example, open a video player on desktop 1 and 2, go to desktop 1 and open the grid. Then the content of the video player on desktop 2 gets drawn on each frame when player 1 gets drawn (assuming the same fps). And if your current desktop's only window is a text editor with a blinking cursor for example, then the contents of other desktops will only be drawn when the text cursor blinks, namely once each second. Also, moving the mouse cursor over the grid and hovering windows so that their border color or drop shadow gets redrawn also causes their content to be redrawn.

This means that the draw calls of all the other desktops on the grid and their windows are tied to the current desktop, which is incorrect.
Comment 31 bastimeyer123 2023-03-04 16:06:59 UTC
> This means that the draw calls of all the other desktops on the grid and their windows are tied to the current desktop, which is incorrect.

Is this a limitation of all the QML implementations, because content does only get updated on the current effect's `prePaintScreen()` / `paintScreen()` calls which seems to depend on the updates of the current desktop? (as said, no clue about any kwin internals)
https://invent.kde.org/plasma/kwin/-/blob/v5.27.2/src/libkwineffects/kwinquickeffect.cpp#L309-340
Comment 32 bastimeyer123 2023-05-20 08:48:01 UTC
Is there a chance for anyone of the kwin devs to have a look at this, so this can be fixed in the upcoming Plasma 6 release?

This issue isn't that problematic for the overview effect, but it pretty much is for the desktop grid where the desktop contents are shown in a much larger area on the screen. The whole purpose of the desktop grid is to show the contents of the other desktops, and if the desktop contents are not up2date because the grid effect only gets redrawn when the current desktop updates, then this makes the whole effect useless. It's also a big usability issue and it is annoying workflow-wise for lots of people.

Would it make sense as a temporary (inefficient and wasteful) workaround to draw the contents of each desktop of the current activity on each output frame while the respective effects (overview / grid) are shown? Otherwise, from what I understand, each desktop of the current activity would have to be checked on each output frame when something was updated, and then the effect would need to be redrawn only in these cases.

Please have a look at this issue... :/
Comment 33 postix 2023-06-30 12:42:28 UTC
(In reply to postix from comment #20)
> I noticed that when recording with OBS 28.1.1 a window, in my case Chromium
> 107, Wayland native, the recorded window won't update in OBS, when the
> window is moved to a different virtual desktop. 

At least with Plasma 5.27.6 Wayland + OBS 29.1.2 + Intel HD GPU + Firefox 114 or Chromium 114 I can no longer reproduce this issue with OBS Studio or the heap-based effects.

With all other applications (VLC, Konsole, ...) I can still reproduce it.
Comment 34 postix 2023-06-30 12:55:39 UTC
^ never mind about Chromium: The issue(s) is/are only reproducible with when it runs as an native Wayland app, but is gone when it runs under XWayland. For Firefox the Wayland version does not show the bug though.
Comment 35 postix 2023-06-30 13:31:41 UTC
Never mind my two comments above, sorry for the noise. I checked again and found the following with [1]:

1) When moving any app to another virtual desktop, their content is no longer regularly updated in the heap effects, just like Basti described.
2) When adding a window (Chromium, Firefox, Konsole or likely any other) as a source to OBS Studio, its content will always be updated in the heap effects, no matter on which virtual desktop it's moved.
3) When maximizing a floating window, which was added as source to OBS Studio, bug #469854 is triggered and also the content freezes in the heap effects, when moving the windows to a different virtual desktop.

Looking very much forward to hearing feedback from the kwin devs. :-)

[1] Plasma 5.27.6 Wayland + OBS 29.1.2 + Intel HD GPU + Firefox 114 Wayland or Chromium 114 Wayland or X11 + Konsole 23.04.2
Comment 36 bastimeyer123 2023-06-30 13:39:04 UTC
If you have OBS opened on the current desktop, then it'll trigger draw calls, so every other desktop gets updated as well, hence why you're not seeing any issues. Minimize OBS, so it doesn't trigger draw calls, and the other desktops will only get updated when a draw call on the current desktop occurs, e.g. when window contents get updated, when you hover/unhover a window on the window-heap, or when an animation is playing. The applications themselves and X11/Wayland are 100% irrelevant to this issue.

Please just read my previous comments. I have described the exact issue a couple of times already. Don't get confused when your current desktop triggers draw calls. Hint: disable all desktop elements and hide all windows, then check the other desktop contents and switch between desktops while the effects are opened/running.
Comment 37 postix 2023-06-30 14:19:53 UTC
>  Minimize OBS, so it doesn't trigger draw calls, and the other desktops will only get updated when a draw call on the current desktop
> occurs, e.g. when window contents get updated, when you hover/unhover a window on the window-heap,
> or when an animation is playing. 
If OBS is minimized or not makes no difference for what I have described in Comment 35. 

>  The applications themselves and X11/Wayland are 100% irrelevant to this issue.
I had mentioned it because upon testing when starting e.g. Chromium for (X)Wayland, the symptoms changed. Maybe this was just coincidence. 

> Please just read my previous comments. I have described the exact issue a couple of times already
Unfortunately I cannot reproduce what you stated in comment 30. After a year of guessing I'd really like to hear some feedback finally from the devs. ;)
Comment 38 bastimeyer123 2023-09-07 14:22:56 UTC
Could this bug please finally get some attention, so it can get fixed in the Plasma 6 release scheduled for February 2024?
https://pointieststick.com/2023/09/06/september-plasma-6-update/

The bug status is also still set to "REPORTED" while I have several times given clear instructions on how to reproduce it, when it occurs and why.

Please, dear KWin devs, have a look at the issue (as well as the other bugs affecting the desktop grid and related effects). Thanks.
Comment 39 Varaskkar 2023-09-07 14:57:53 UTC
I have enabled a desktop effect called "Miniatures to the side" (in english maybe the name is different).

See image: https://i.imgur.com/Q7WBT8s.jpg

With that desktop effect, I can watch a video (located in virtual desktop 1) while programming code (located virtual desktop 2) ..., and do both things from virtual desktop 2.

The problem? It's that this bug also happens with this desktop effect.
Comment 40 Nate Graham 2023-09-07 15:48:47 UTC
Can reproduce. The preview of the video window *in the virtual desktop bar at the top of the screen* does indeed stop animating when the video window is dragged there.
Comment 41 Nate Graham 2023-09-07 15:51:00 UTC
Clearer steps to reproduce:

1. Have 2 virtual desktops
2. Open a video in an app (I used DragonPlayer)
3. Open Overview
4. Drag the DragonPlayer window onto the thumbnail of another virtual desktop

In the thumbnail for that virtual desktop, the video window doesn't update in real-time.
Comment 42 bastimeyer123 2023-11-20 14:00:35 UTC
Created attachment 163315 [details]
kwin-6.0-dev-desktopgrid-x11
Comment 43 bastimeyer123 2023-11-20 14:01:57 UTC
Created attachment 163316 [details]
kwin-6.0-dev-desktopgrid-wayland
Comment 44 bastimeyer123 2023-11-20 14:04:16 UTC
The issue still remains in the latest version packaged by Arch Linux's kde-unstable package repo from today. Just checked in a VM.
See the two attachments I've just uploaded: kwin-6.0-dev-desktopgrid-{x11,wayland}.mp4
Comment 45 postix 2024-03-05 16:09:18 UTC
*** Bug 468411 has been marked as a duplicate of this bug. ***
Comment 46 bastimeyer123 2024-03-07 12:43:51 UTC
Please remember that lots of people will use wayland (or try it out again) on Plasma 6.

As explained, when using the wayland session, all window contents on other desktops that won't be updated due to this issue will flicker between a black frame and the last known content, and when closing and opening the grid again, it's just a black frame that remains. This is incredibly bad for the grid when non-static content is shown, like video players for example.

Since wayland is now the default session for new users, this makes this issue even more imporant, because the flickering is really annoying.

Please have a look at this issue.
Comment 47 postix 2024-03-07 12:57:30 UTC
Without generating too much noise, considering 3 DUPs and > 10 people subscribed and the fact that it's about not working useful functionality, I'd also kindly ask to raise the issue from `minor` back to `normal`. :)
Comment 48 Marco Martin 2024-03-07 16:08:27 UTC
it seems i can reproduce that only for xwayland clients, if they are on another desktop, and have an animation like a video, it won't play in the thumbnails if on another desktop
Comment 49 Marco Martin 2024-03-07 16:11:15 UTC
scratch that, even firefox wayland native has frozen video in the thumbnail in ie desktop grid when is on a desktop different from the current
Comment 50 postix 2024-03-14 14:03:07 UTC
Fun fact: When you record a window with OBS Studio 29.1.3 (for Wayland), the recorded window gets constantly updated in the grid no matter on which VD it is located.

Operating System: openSUSE Tumbleweed 20240311
KDE Plasma Version: 6.0.1
KDE Frameworks Version: 6.0.0
Qt Version: 6.6.2
Kernel Version: 6.7.7-1-default (64-bit)
Graphics Platform: Wayland
Comment 51 Bug Janitor Service 2024-04-10 13:38:09 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/kwin/-/merge_requests/5592
Comment 52 David Redondo 2024-04-10 14:27:09 UTC
Git commit 1573d04b5ac7f54649fd8225a42c7a9a09266eb5 by David Redondo.
Committed on 10/04/2024 at 13:50.
Pushed by davidre into branch 'master'.

Mark windows for offsceen rendering in WindowThumbnailSource

This way if an effect wants to show such  window  that is for example
on a different virtual desktop it is updated live.
FIXED-IN:6.0.4

M  +7    -0    src/scripting/windowthumbnailitem.cpp

https://invent.kde.org/plasma/kwin/-/commit/1573d04b5ac7f54649fd8225a42c7a9a09266eb5
Comment 53 David Redondo 2024-04-10 14:36:23 UTC
Git commit a516e9e6cdefd2057441d969dcfb5480c295111b by David Redondo.
Committed on 10/04/2024 at 14:27.
Pushed by davidre into branch 'Plasma/6.0'.

Mark windows for offsceen rendering in WindowThumbnailSource

This way if an effect wants to show such  window  that is for example
on a different virtual desktop it is updated live.
FIXED-IN:6.0.4


(cherry picked from commit 1573d04b5ac7f54649fd8225a42c7a9a09266eb5)

M  +7    -0    src/scripting/windowthumbnailitem.cpp

https://invent.kde.org/plasma/kwin/-/commit/a516e9e6cdefd2057441d969dcfb5480c295111b
Comment 54 bastimeyer123 2024-04-10 15:27:18 UTC
Thank you very much for the bugfix, David! <3
I applied a516e9e6cdefd2057441d969dcfb5480c295111b on top of v6.0.3.1 and I can confirm that it's working (using wayland).