Bug 513538

Summary: kwin-wayland session is completely broken on RV380 while kwin-x11 works fine
Product: [Plasma] kwin Reporter: Jan Binder <wheel>
Component: wayland-genericAssignee: KWin default assignee <kwin-bugs-null>
Status: RESOLVED UPSTREAM    
Severity: major CC: xaver.hugl
Priority: NOR    
Version First Reported In: 6.3.6   
Target Milestone: ---   
Platform: Debian stable   
OS: Linux   
Latest Commit: Version Fixed/Implemented In:
Sentry Crash Report:

Description Jan Binder 2025-12-18 13:08:57 UTC 
SUMMARY
I recently upgraded the debian installationon a National Instruments 8130 PXIe embedded PC to the latest stable release and the default graphical session was unusable.

STEPS TO REPRODUCE
1. Install KDE Plasma (here 6.3.6) on the National Instruments 8130 PXIe embedded PC.
2. Use the kwin-wayland session which is selected by default.
3. After login, graphical session becomes unusable, windows do not draw contents, Plasma desktop does not react to clicks, mouse pointer flickers.
4. Kill graphical session, change session type to kwin-x11.
5. After login, session works fine, Firefox and other applications are usable.

OBSERVED RESULT
After logging into a kwin-wayland session, the mouse cursor flickers and sometimes draws a square with random content.
Windows restored from the last session are black or contain content previously drawn in their place on the screen.
Closing windows is possible but takes tens of seconds.
Plasma desktop toolbar and widgets do not react to clicks, K menu does not open.
Global shortcuts do not seem to work.
System log is continuosly filled with messages like:
kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
kernel: radeon 0000:01:00.0: 000000001d8f69be pin failed

EXPECTED RESULT
Desktop environment should work at least as well as in the X11-bases session and not fill the system log with error messages.

SOFTWARE/OS VERSIONS
Linux Kernel: 6.12.x
Linux/KDE Plasma: 6.3.6
KDE Plasma Version: 6.3.6
KDE Frameworks Version: 6.13.0
Qt Version: 6.8.2+dfsg-9+deb13u1

ADDITIONAL INFORMATION

dmesg output of graphics initialization:

Dez 18 11:29:18 ni8130 kernel: [drm] radeon kernel modesetting enabled.
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: vgaarb: deactivate vga console
Dez 18 11:29:18 ni8130 kernel: Console: switching to colour dummy device 80x25
Dez 18 11:29:18 ni8130 kernel: [drm] initializing kernel modesetting (RV380 0x1002:0x5460 0x1002:0x5460 0x00).
Dez 18 11:29:18 ni8130 kernel: resource: resource sanity check: requesting [mem 0x00000000000c0000-0x00000000000dffff], which spans more than PCI Bus 0000:00 [mem 0x000d0000-0x000dffff window]
Dez 18 11:29:18 ni8130 kernel: caller pci_map_rom+0x69/0x1a0 mapping multiple BARs
Dez 18 11:29:18 ni8130 kernel: [drm] Generation 2 PCI interface, using max accessible memory
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: VRAM: 128M 0x00000000D8000000 - 0x00000000DFFFFFFF (32M used)
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: GTT: 512M 0x00000000B8000000 - 0x00000000D7FFFFFF
Dez 18 11:29:18 ni8130 kernel: [drm] Detected VRAM RAM=128M, BAR=128M
Dez 18 11:29:18 ni8130 kernel: [drm] RAM width 64bits DDR
Dez 18 11:29:18 ni8130 kernel: [drm] radeon: 32M of VRAM memory ready
Dez 18 11:29:18 ni8130 kernel: [drm] radeon: 512M of GTT memory ready.
Dez 18 11:29:18 ni8130 kernel: [drm] GART: num cpu pages 131072, num gpu pages 131072
Dez 18 11:29:18 ni8130 kernel: [drm] radeon: 1 quad pipes, 1 Z pipes initialized
Dez 18 11:29:18 ni8130 kernel: [drm] PCIE GART of 512M enabled (table at 0x00000000D8040000).
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: WB enabled
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: fence driver on ring 0 use gpu addr 0x00000000b8000000
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: radeon: MSI limited to 32-bit
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: radeon: using MSI.
Dez 18 11:29:18 ni8130 kernel: [drm] radeon: irq initialized.
Dez 18 11:29:18 ni8130 kernel: [drm] Loading R300 Microcode
Dez 18 11:29:18 ni8130 kernel: [drm] radeon: ring at 0x00000000B8001000
Dez 18 11:29:18 ni8130 kernel: [drm] ring test succeeded in 1 usecs
Dez 18 11:29:18 ni8130 kernel: [drm] ib test succeeded in 0 usecs
Dez 18 11:29:18 ni8130 kernel: [drm] Panel ID String: 1600x1200               
Dez 18 11:29:18 ni8130 kernel: [drm] Panel Size 1600x1200
Dez 18 11:29:18 ni8130 kernel: [drm] Radeon Display Connectors
Dez 18 11:29:18 ni8130 kernel: [drm] Connector 0:
Dez 18 11:29:18 ni8130 kernel: [drm]   DVI-I-1
Dez 18 11:29:18 ni8130 kernel: [drm]   HPD1
Dez 18 11:29:18 ni8130 kernel: [drm]   DDC: 0x64 0x64 0x64 0x64 0x64 0x64 0x64 0x64
Dez 18 11:29:18 ni8130 kernel: [drm]   Encoders:
Dez 18 11:29:18 ni8130 kernel: [drm]     CRT1: INTERNAL_DAC1
Dez 18 11:29:18 ni8130 kernel: [drm]     DFP1: INTERNAL_TMDS1
Dez 18 11:29:18 ni8130 kernel: [drm] Connector 1:
Dez 18 11:29:18 ni8130 kernel: [drm]   DVI-I-2
Dez 18 11:29:18 ni8130 kernel: [drm]   HPD1
Dez 18 11:29:18 ni8130 kernel: [drm]   DDC: 0x68 0x68 0x68 0x68 0x68 0x68 0x68 0x68
Dez 18 11:29:18 ni8130 kernel: [drm]   Encoders:
Dez 18 11:29:18 ni8130 kernel: [drm]     DFP1: INTERNAL_TMDS1
Dez 18 11:29:18 ni8130 kernel: [drm]     CRT2: INTERNAL_DAC2
Dez 18 11:29:18 ni8130 kernel: [drm] Connector 2:
Dez 18 11:29:18 ni8130 kernel: [drm]   LVDS-1
Dez 18 11:29:18 ni8130 kernel: [drm]   Encoders:
Dez 18 11:29:18 ni8130 kernel: [drm]     LCD1: INTERNAL_LVDS
Dez 18 11:29:18 ni8130 kernel: [drm] Initialized radeon 2.50.0 for 0000:01:00.0 on minor 0
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: [drm] format RG16 little-endian (0x36314752) not supported
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: [drm] format RG16 little-endian (0x36314752) not supported
Dez 18 11:29:18 ni8130 kernel: radeon 0000:01:00.0: [drm] No compatible format found
Dez 18 11:29:18 ni8130 kernel: [drm] fb mappable at 0xD80C0000
Dez 18 11:29:18 ni8130 kernel: [drm] vram apper at 0xD8000000
Dez 18 11:29:18 ni8130 kernel: [drm] size 7680000
Dez 18 11:29:18 ni8130 kernel: [drm] fb depth is 24
Dez 18 11:29:18 ni8130 kernel: [drm]    pitch is 6400
Dez 18 11:29:18 ni8130 kernel: fbcon: radeondrmfb (fb0) is primary device

Log messages printed during kwin session startup:

Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: No backend specified, automatically choosing drm
Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: kwin_wayland_drm: drmSetClientCap for Atomic Mode Setting failed. Using legacy mode on GPU "/dev/dri/card0"
Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: r300 FP: Compiler Error:
Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: Too many ALU instructions used: 219, max: 64.
Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: Using a dummy shader instead.
Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: r300 FP: Compiler Error:
Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: Too many ALU instructions used: 219, max: 64.
Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: Using a dummy shader instead.
Dez 18 11:30:59 ni8130 kernel: radeon 0000:01:00.0: [drm] *ERROR* crtc 43: Can't calculate constants, dotclock = 0!
Dez 18 11:30:59 ni8130 kernel: radeon 0000:01:00.0: 000000001d8f69be pin failed
Dez 18 11:30:59 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 18 11:30:59 ni8130 kernel: radeon 0000:01:00.0: 00000000d49c4d8b pin failed
Dez 18 11:30:59 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 18 11:30:59 ni8130 kernel: radeon 0000:01:00.0: 000000001d8f69be pin failed
Dez 18 11:30:59 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 18 11:30:59 ni8130 kernel: radeon 0000:01:00.0: 00000000d49c4d8b pin failed
Dez 18 11:30:59 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 18 11:30:59 ni8130 kernel: radeon 0000:01:00.0: 000000001d8f69be pin failed

... ad infinitum
Comment 1 Bug Janitor Service 2025-12-18 13:33:40 UTC 
Thank you for the bug report!

Debian advises users to not submit bugs upstream (https://www.debian.org/Bugs/Reporting), and Plasma 6.3.6 is no longer eligible for support or maintenance from KDE. It's possible that the issue exists only in Debian at this point.

Could you report the bug to Debian using the report bug utility (https://packages.debian.org/stable/utils/reportbug)? If necessary, the maintainer of the package will forward the bug upstream.
Thanks for understanding!

Thanks again!
Comment 2 Zamundaaa 2025-12-18 13:47:13 UTC 
> Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: r300 FP: Compiler Error:
> Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: Too many ALU instructions used: 219, max: 64.
> Dez 18 11:30:58 ni8130 kwin_wayland_wrapper[1191]: Using a dummy shader instead.
I think in this case we can ignore the bot.
For situations like that, where the hardware isn't good enough to run our full shaders, we have an automatic fallback to llvmpipe. That looks like rather than failing our shader compilation, RadeonSi is replacing our actual shader with nonsense though... please report this at https://gitlab.freedesktop.org/mesa/mesa/-/issues
Comment 3 Jan Binder 2025-12-19 11:15:45 UTC 
I upgraded this system further to Debian forky:

Mesa: 25.2.8
Plasma: 6.5.4
Frameworks: 6.20.0
Qt: 6.9.2+dfsg-3

The amount of breakage is reduced considerably, probably because the llvmpipe fallback works and the session now runs with correct drawing at 170% CPU load and produces 1 frame every 3 seconds.
There is of course still a lot of logspam concerning buffer pinning and in general it is not that much more usable than before and absolutely no comparison to the X11 session.

Startup output now is:

Dez 19 10:55:23 ni8130 kwin_wayland_wrapper[1397]: No backend specified, automatically choosing drm
Dez 19 10:55:23 ni8130 kwin_wayland_wrapper[1397]: kwin_wayland_drm: drmSetClientCap for Atomic Mode Setting failed. Using legacy mode on GPU "/dev/dri/card0"
Dez 19 10:55:23 ni8130 kwin_wayland_wrapper[1397]: kwin_wayland_drm: Could not find edid for connector DrmConnector(id=53, gpu="/dev/dri/card0", name="LVDS-1", connection="Connected", countMode=14)
Dez 19 10:55:24 ni8130 kwin_wayland_wrapper[1397]: r300 FP: Compiler error: Too many ALU instructions used: 236, max: 64
Dez 19 10:55:24 ni8130 kwin_wayland_wrapper[1397]: r300 FP: Use RADEON_DEBUG=dummysh to force dummy shader instead.
Dez 19 10:55:24 ni8130 kwin_wayland_wrapper[1397]: kwin_scene_opengl: Failed to link shader:
Dez 19 10:55:24 ni8130 kwin_wayland_wrapper[1397]:  "error: Too many ALU instructions used: 236, max: 64\x00"
Dez 19 10:55:24 ni8130 kwin_wayland_wrapper[1397]: kwin_scene_opengl: Creating the OpenGL rendering failed:  "Could not initialize rendering context"
Dez 19 10:55:24 ni8130 kernel: radeon 0000:01:00.0: [drm] *ERROR* crtc 44: Can't calculate constants, dotclock = 0!
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: Failed to find a working output layer configuration! Enabled layers:
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: src QRectF(0,0 1600x1200) -> dst QRect(0,0 1600x1200)
Dez 19 10:55:25 ni8130 kernel: radeon 0000:01:00.0: 000000009c0e0206 pin failed
Dez 19 10:55:25 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_wayland_drm: Could not find edid for connector DrmConnector(id=53, gpu="/dev/dri/card0", name="LVDS-1", connection="Connected", countMode=1)
Dez 19 10:55:25 ni8130 kernel: radeon 0000:01:00.0: 000000001bf1c23a pin failed
Dez 19 10:55:25 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: Failed to find a working output layer configuration! Enabled layers:
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: src QRectF(0,0 1600x1200) -> dst QRect(0,0 1600x1200)
Dez 19 10:55:25 ni8130 kernel: radeon 0000:01:00.0: 00000000b0f092c6 pin failed
Dez 19 10:55:25 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_wayland_drm: Could not find edid for connector DrmConnector(id=53, gpu="/dev/dri/card0", name="LVDS-1", connection="Connected", countMode=1)
Dez 19 10:55:25 ni8130 kernel: radeon 0000:01:00.0: 000000001bf1c23a pin failed
Dez 19 10:55:25 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: Failed to find a working output layer configuration! Enabled layers:
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: src QRectF(0,0 1600x1200) -> dst QRect(0,0 1600x1200)
Dez 19 10:55:25 ni8130 kernel: radeon 0000:01:00.0: 00000000b0f092c6 pin failed
Dez 19 10:55:25 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_wayland_drm: Could not find edid for connector DrmConnector(id=53, gpu="/dev/dri/card0", name="LVDS-1", connection="Connected", countMode=1)
Dez 19 10:55:25 ni8130 kernel: radeon 0000:01:00.0: 000000001bf1c23a pin failed
Dez 19 10:55:25 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: Failed to find a working output layer configuration! Enabled layers:
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: src QRectF(0,0 1600x1200) -> dst QRect(0,0 1600x1200)
Dez 19 10:55:25 ni8130 kernel: radeon 0000:01:00.0: 00000000b0f092c6 pin failed
Dez 19 10:55:25 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_wayland_drm: Could not find edid for connector DrmConnector(id=53, gpu="/dev/dri/card0", name="LVDS-1", connection="Connected", countMode=1)
Dez 19 10:55:25 ni8130 kernel: radeon 0000:01:00.0: 000000001bf1c23a pin failed
Dez 19 10:55:25 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: Failed to find a working output layer configuration! Enabled layers:
Dez 19 10:55:25 ni8130 kwin_wayland_wrapper[1397]: kwin_core: src QRectF(0,0 1600x1200) -> dst QRect(0,0 1600x1200)


With RADEON_DEBUG=dummysh the session is even more broken than the old version and only a flickering mouse pointer can be seen.
Assorted debug output for that session:

Dez 19 11:25:46 ni8130 kwin_wayland_wrapper[2785]: No backend specified, automatically choosing drm
Dez 19 11:25:46 ni8130 kwin_wayland_wrapper[2785]: kwin_wayland_drm: drmSetClientCap for Atomic Mode Setting failed. Using legacy mode on GPU "/dev/dri/card0"
Dez 19 11:25:46 ni8130 kwin_wayland_wrapper[2785]: kwin_wayland_drm: Could not find edid for connector DrmConnector(id=53, gpu="/dev/dri/card0", name="LVDS-1", connection="Connected", countMode=1)
Dez 19 11:25:46 ni8130 kwin_wayland_wrapper[2785]: r300 FP: Compiler error: Too many ALU instructions used: 236, max: 64
Dez 19 11:25:46 ni8130 kwin_wayland_wrapper[2785]: r300 FP: Using a dummy shader instead.
Dez 19 11:25:46 ni8130 kwin_wayland_wrapper[2785]: r300 FP: Compiler error: Too many ALU instructions used: 236, max: 64
Dez 19 11:25:46 ni8130 kwin_wayland_wrapper[2785]: r300 FP: Using a dummy shader instead.
Dez 19 11:25:47 ni8130 kernel: radeon 0000:01:00.0: [drm] *ERROR* crtc 44: Can't calculate constants, dotclock = 0!
Dez 19 11:25:47 ni8130 kernel: radeon 0000:01:00.0: 0000000043943254 pin failed
Dez 19 11:25:47 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 11:25:47 ni8130 kwin_wayland_wrapper[2785]: kwin_core: Failed to find a working output layer configuration! Enabled layers:
Dez 19 11:25:47 ni8130 kwin_wayland_wrapper[2785]: kwin_core: src QRectF(0,0 1600x1200) -> dst QRect(0,0 1600x1200)
Dez 19 11:25:47 ni8130 kernel: radeon 0000:01:00.0: 00000000b5a7db9e pin failed
Dez 19 11:25:47 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 11:25:47 ni8130 kwin_wayland_wrapper[2785]: kwin_core: Failed to find a working output layer configuration! Enabled layers:
Dez 19 11:25:47 ni8130 kwin_wayland_wrapper[2785]: kwin_core: src QRectF(0,0 1600x1200) -> dst QRect(0,0 1600x1200)
Dez 19 11:25:47 ni8130 kernel: radeon 0000:01:00.0: 00000000dd3538cf pin failed
Dez 19 11:25:47 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 11:25:47 ni8130 kwin_wayland_wrapper[2785]: kwin_wayland_drm: Could not find edid for connector DrmConnector(id=53, gpu="/dev/dri/card0", name="LVDS-1", connection="Connected", countMode=1)
Dez 19 11:25:47 ni8130 kwin_wayland_wrapper[2785]: kwin_wayland_drm: Could not find edid for connector DrmConnector(id=53, gpu="/dev/dri/card0", name="LVDS-1", connection="Connected", countMode=1)
Dez 19 11:25:47 ni8130 kernel: radeon 0000:01:00.0: 0000000043943254 pin failed
Dez 19 11:25:47 ni8130 kernel: [drm:radeon_crtc_page_flip_target [radeon]] *ERROR* failed to pin new rbo buffer before flip
Dez 19 11:25:47 ni8130 kwin_wayland_wrapper[2785]: kwin_core: Failed to find a working output layer configuration! Enabled layers:
Dez 19 11:25:47 ni8130 kwin_wayland_wrapper[2785]: kwin_core: src QRectF(0,0 1600x1200) -> dst QRect(0,0 1600x1200)

Dez 19 11:25:51 ni8130 kwin_wayland_wrapper[2830]: r300 FP: Compiler error: Too many ALU instructions used: 67, max: 64
Dez 19 11:25:51 ni8130 kwin_wayland_wrapper[2830]: r300 FP: Using a dummy shader instead.
Dez 19 11:25:51 ni8130 kwin_wayland_wrapper[2830]: r300 FP: Compiler error: If/then statements not supported by R300/R400 shaders, should have been flattened by peephole_select.
Dez 19 11:25:51 ni8130 kwin_wayland_wrapper[2830]: Failed to link: error: If/then statements not supported by R300/R400 shaders, should have been flattened by peephole_select.

Dez 19 11:25:55 ni8130 ksplashqml[2788]: r300: CS space validation failed. (not enough memory?) Skipping rendering.
Dez 19 11:25:55 ni8130 ksplashqml[2788]: r300: CS space validation failed. (not enough memory?) Skipping rendering.


Considering that the dummy shader is still broken but the default behavior is now as expected regarding the error handling/rendering result, excluding usability, what is the sensible thing for the dummy shader to do?
Comment 4 Zamundaaa 2025-12-19 14:45:08 UTC 
There dummy shader can't help. If it doesn't work on the hardware, it simply has to fail and fall back to software rendering.
For the moment, you *could* attempt to patch KWin to reduce the amount of instructions in the colormanagement.glsl file (maybe delete the tonemapping code), but it's not a long term solution.

Software rendering should absolutely not be that slow though, the buffer pinning is probably the bigger problem. Unfortunately at this point, it seems rather unlikely for the radeon driver to see fixes for it, but you can try creating a report for it at https://gitlab.freedesktop.org/drm/amd/-/issues