Bug 493745

Summary:	kwin-wayland struggles with external GPUs
Product:	[Plasma] kwin	Reporter:	Khanich <khanich.opensource>
Component:	performance	Assignee:	KWin default assignee <kwin-bugs-null>
Status:	CONFIRMED ---
Severity:	normal	CC:	nate, xaver.hugl
Priority:	NOR
Version First Reported In:	6.1.5
Target Milestone:	---
Platform:	Other
OS:	Linux
Latest Commit:		Version Fixed/Implemented In:
Sentry Crash Report:

Description Khanich 2024-09-27 17:10:00 UTC

Sorry for the title, I am not sure what to name this.

SUMMARY
I have a laptop with an Intel iGPU (marked as default) and an AMD 6750XT as an external GPU connected via Thunderbolt. The eGPU also has a display connected via Display Port. Normally the Laptop lid is closed while using the external monitor.

While trying to run a game fullscreen the eGPU gets used as expected (at least according to radeontop), but I don't see a picture, just a black screen. When opening the overview, I get a picture in that small window, but not an updating one, more like a screenshot. The showed picture when leaving the overview again is also that picture.

When I go and open the Laptop lid, the game gets automatically moved to the internal monitor where it also runs as expected (but not as great as it could be because of the limited bandwidth).

If I then go and open the overview again, drag the game to the external monitor and then click on it, it's moved back to the internal monitor like KWin doesn't want me to let the game run on that monitor.

If I start a game in windowed mode (I tested this with War Thunder with maximum settings in the garage) I can move the window around freely. But the FPS counter of the game shows something interesting: If the window is on the external monitor, the FPS is at about 19 while on the internal monitor it's at about 29.

None of this was a problem on Plasma version 6.0 (but I am not sure when it changed because I didn't play anything for a few months).

My theory is that KWin for some reason really wants to move the window buffer to the iGPU (or system RAM) and then struggles while moving it back because of bandwidth limitation (after all, Thunderbolt 3 and 4 are practically PCIe 3.0 4x in bandwidth). usbtop gives about 200 kb/s "To Device" and about 28kb/s "From Device" and watching a video on YouTube (at fullscreen Firefox) is giving the same amount (although as I noticed YouTube does report about 1 dropped frame per second).

SOFTWARE/OS VERSIONS
Linux-Kernel: 6.10.11-200.fc40.x86_64 (64-bit)
KDE Plasma Version: 6.1.5
KDE Frameworks Version: 6.6.0
Qt Version: 6.7.2
Linux distribution: Fedora Kinoite

ADDITIONAL INFORMATION
It's independent of the game and of if it's run via Wine/Proton or a native game.
No, I didn't test in an Xorg session.

Comment 1 Zamundaaa 2024-09-30 16:55:09 UTC

Gaming on external GPUs is indeed very challenging. By default, what happens is that
1. the app renders on the external GPU. Because of limited bandwidth of the USB C connection and https://gitlab.freedesktop.org/mesa/mesa/-/issues/7340, performance is worse compared to having it in a 16x PCIe slot
2. the driver copies the rendered results to a buffer in system memory
3. KWin gets that buffer in system memory, composites a results with the integrated GPU
4. KWin either presents the composited result directly, or copies the composited result back to the eGPU and presents it on a display connected to that

There are efforts to fix that (https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/268), but none of this has changed in any way in the last few years.

> While trying to run a game fullscreen the eGPU gets used as expected (at least according to radeontop), but I don't see a picture, just a black screen
That sounds like a problem we can investigate though. If you put KWIN_DRM_NO_DIRECT_SCANOUT=1 into /etc/environment and reboot, does anything change in regard to that?

Comment 2 Khanich 2024-10-01 16:08:17 UTC

(In reply to Zamundaaa from comment #1)
> Gaming on external GPUs is indeed very challenging. By default, what happens
> is that
> 1. the app renders on the external GPU. Because of limited bandwidth of the
> USB C connection and https://gitlab.freedesktop.org/mesa/mesa/-/issues/7340,
> performance is worse compared to having it in a 16x PCIe slot
> 2. the driver copies the rendered results to a buffer in system memory
> 3. KWin gets that buffer in system memory, composites a results with the
> integrated GPU
> 4. KWin either presents the composited result directly, or copies the
> composited result back to the eGPU and presents it on a display connected to
> that
> 
> There are efforts to fix that
> (https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/
> 268), but none of this has changed in any way in the last few years.
> 

Well, hopefully there won't take as long as many other protocols did.

> > While trying to run a game fullscreen the eGPU gets used as expected (at least according to radeontop), but I don't see a picture, just a black screen
> That sounds like a problem we can investigate though. If you put
> KWIN_DRM_NO_DIRECT_SCANOUT=1 into /etc/environment and reboot, does anything
> change in regard to that?

I just tested it with this variable in /etc/environment with both a Proton game (Farming Simulator 22 with Proton version 9.0-3) and a native game fullscreen (War Thunder again) and it works. So, the black screen apparently is a bug in KWin's direct scanout implementation I guess?

I also decided to do the same test in War Thunder in windowed mode again with this variable and without (because a Patch on the side of WT could have changed this after all and I think I saw a MESA update in the last few days, although I am not sure; besides, it makes thinks more comparable):
With variable on internal monitor: ~60
With variable on external monitor connected to the eGPU where the game runs: ~55
Without variable on internal monitor: ~45
Without variable on external monitor connected to the eGPU where the game runs: ~40
I am not sure, but should this be the case in windowed mode?

Comment 3 Zamundaaa 2024-10-04 14:20:32 UTC

> I am not sure, but should this be the case in windowed mode?
No, the environment variable doesn't change anything for windowed mode (yet)

Comment 4 Khanich 2024-10-04 19:07:29 UTC

(In reply to Zamundaaa from comment #3)
> > I am not sure, but should this be the case in windowed mode?
> No, the environment variable doesn't change anything for windowed mode (yet)

Well, in this case there's another bug I guess