Bug 512511 - Screen freeze when screen dims or entering/exiting full screen ("Atomic modeset test failed! Permission denied")
Summary: Screen freeze when screen dims or entering/exiting full screen ("Atomic modes...
Status: RESOLVED FIXED
Alias: None
Product: kwin
Classification: Plasma
Component: general (other bugs)
Version First Reported In: 6.5.80
Platform: Other Linux
: VHI major
Target Milestone: ---
Assignee: KWin default assignee
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2025-11-23 15:24 UTC by Nate Graham
Modified: 2025-12-10 16:25 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed/Implemented In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nate Graham 2025-11-23 15:24:28 UTC
Twice in the last day, I've had my screen freeze visually while watching (well, listening to) YouTube videos in the background.

Once it happened when the laptop was unplugged and the screen dimmed, and the second time happened while it was plugged in and I was entering full-screen mode.

I wasn't able to catch logs from the first time it happened, but here are the logs from the second time:

[massive flood of millions of these lines]
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to create framebuffer: Invalid argument
Nov 22 23:23:32 engine kwin_wayland[1978]: Rendering a layer failed!
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to find a working output layer configuration! Enabled layers:
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 2880x1800) -> dst QRect(0,0 2880x1800)
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 256x256) -> dst QRect(1046,1104 256x256)
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to create framebuffer: Invalid argument
Nov 22 23:23:32 engine kwin_wayland[1978]: Rendering a layer failed!
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to find a working output layer configuration! Enabled layers:
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 2880x1800) -> dst QRect(0,0 2880x1800)
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 256x256) -> dst QRect(1046,1104 256x256)
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to create framebuffer: Invalid argument
Nov 22 23:23:32 engine kwin_wayland[1978]: Rendering a layer failed!
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to find a working output layer configuration! Enabled layers:
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 2880x1800) -> dst QRect(0,0 2880x1800)
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 256x256) -> dst QRect(1046,1104 256x256)
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to create framebuffer: Invalid argument
Nov 22 23:23:32 engine kwin_wayland[1978]: Rendering a layer failed!
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to find a working output layer configuration! Enabled layers:
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 2880x1800) -> dst QRect(0,0 2880x1800)
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 256x256) -> dst QRect(1046,1104 256x256)
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to find a working output layer configuration! Enabled layers:
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 2880x1800) -> dst QRect(0,0 2880x1800)
Nov 22 23:23:32 engine kwin_wayland[1978]: Atomic modeset test failed! Permission denied
Nov 22 23:23:32 engine kwin_wayland[1978]: Applying output configuration failed!

2880x1800 is the laptop's native resolution. I have it at 175% scale.

When this happens, the system remains *functionally* responsive; it's a purely visual freeze. I can change the volume and hear the feedback sound played, and play/pause playing music or videos.

In both cases, I was able to recover by switching to another VT and manually restarting kwin_wayland.

This has only started happening since yesterday.


Operating System: KDE Linux 2025-11-22
KDE Plasma Version: 6.5.80
KDE Frameworks Version: 6.21.0
Qt Version: 6.10.0
Kernel Version: 6.17.8-arch1-1 (64-bit)
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 7 7840U w/ Radeon™ 780M Graphics
Memory: 17 GB of RAM (16.0 GB usable)
Graphics Processor: AMD Radeon 780M Graphics
Comment 1 Nate Graham 2025-11-23 22:19:48 UTC
It happened again when the screen dimmed. The log output looks largely the same:

Nov 22 23:23:32 engine kwin_wayland[1978]: Applying output configuration failed!
Nov 22 23:23:32 engine kwin_wayland[1978]: Atomic modeset test failed! Permission denied
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 2880x1800) -> dst QRect(0,0 2880x1800)
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to find a working output layer configuration! Enabled layers:
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 256x256) -> dst QRect(1046,1104 256x256)
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 2880x1800) -> dst QRect(0,0 2880x1800)
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to find a working output layer configuration! Enabled layers:
Nov 22 23:23:32 engine kwin_wayland[1978]: Rendering a layer failed!
Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to create framebuffer: Invalid argument
Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 256x256) -> dst QRect(1046,1104 256x256)
Nov 22 23 23:23:54 engine kwin_wayland[39355]: atomic commit failed: Device or resource busy
Nov 22 23:23:46 engine kwin_wayland[39355]: atomic commit failed: Device or resource busy
Nov 22 23:23:46 engine kwin_wayland[39355]: atomic commit failed: Device or resource busy
Comment 2 Nate Graham 2025-11-24 19:41:56 UTC
This appears to be 100% reproducible now when the screen dims or blanks due to inactivity.
Comment 3 Zamundaaa 2025-11-24 20:00:15 UTC
The "permission denied" warnings almost certainly come from the VT switch.

This bit is more interesting though:
> Nov 22 23:23:32 engine kwin_wayland[1978]: Failed to create framebuffer: Invalid argument
> Nov 22 23:23:32 engine kwin_wayland[1978]: src QRectF(0,0 256x256) -> dst QRect(1046,1104 256x256)
> Nov 22 23 23:23:54 engine kwin_wayland[39355]: atomic commit failed: Device or resource busy
> Nov 22 23:23:46 engine kwin_wayland[39355]: atomic commit failed: Device or resource busy
> Nov 22 23:23:46 engine kwin_wayland[39355]: atomic commit failed: Device or resource busy

and matches something I saw yesterday as well. I can't yet explain though why the kernel would disallow creating the framebuffer object... Or what changes in KWin could be related, drm_buffer.cpp had its last change 6 months ago.
I'll try bisecting.
Comment 4 Nikola Novoselec 2025-11-26 09:38:30 UTC
I'm experiencing the exact same issue on a different GPU vendor (NVIDIA), so this isn't AMD-specific.

My setup:
- RTX 3090 with proprietary nvidia-drm 580.105.08
- Samsung ultra-wide 7680x2160 @ 120Hz via DisplayPort
- KDE Plasma 6.5.3, KWin 6.5.3 on Wayland
- CachyOS (Arch-based), kernel 6.17.9-2
- What happens: Screen locks, display goes to sleep (DPMS). When I try to wake it up (move mouse, press key), the monitor powers on but the screen stays completely black. Can't see anything. But the system is still running - I can hear audio, keyboard shortcuts work, I can even type my password blind and it unlocks (just can't see it).

The only fix is Ctrl+Alt+F2 then Ctrl+Alt+F1 to switch VT and back.

Journal output shows:
Nov 26 09:57:24 kwin_wayland[4613]: Atomic modeset test failed! Permission denied
Nov 26 09:57:24 kwin_wayland[4613]: Applying output configuration failed!

This happens every single time. 100% reproducible - just lock screen, wait for DPMS timeout (I have it set to 20 seconds when locked), try to wake.

What I already tried:
- Switched from nvidia-open to nvidia proprietary driver - no change
- Disabled VRR/Adaptive Sync completely - no change
- Verified DRM/i2c ACL permissions - they're correct (user has rw access to /dev/dri/card1 and /dev/i2c-*)

My NVIDIA modprobe config:
options nvidia NVreg_PreserveVideoMemoryAllocations=1
options nvidia NVreg_TemporaryFilePath=/var/tmp
options nvidia NVreg_DynamicPowerManagement=0
options nvidia NVreg_EnableGpuFirmware=0
options nvidia_drm modeset=1 fbdev=1

Kernel params include nvidia_drm.modeset=1 nvidia_drm.fbdev=1

Possibly related:
- I noticed powerdevil logs show "Watching for DPMS state changes unimplemented" - not sure if that's connected.

Also interesting - when DPMS kicks in, I see DDCA_EVENT_DISPLAY_DISCONNECTED in dbus, and on wake attempt DDCA_EVENT_DISPLAY_CONNECTED. So DisplayPort is being treated as a hotplug event rather than just a power state change?

Looking at drm_gpu.cpp, I noticed that setActive() only forces a modeset for legacy mode, not for atomic mode setting. The comment even says "force a modeset with legacy, we can't reliably know if one is needed". Maybe atomic mode needs the same treatment on session reactivation?

Happy to test patches or debug builds if that would help. This happens every time so it's very easy to reproduce.
Comment 5 Zamundaaa 2025-11-27 16:31:41 UTC
That's almost certainly a different problem, please create a new bug report for it and attach the output of drm_info there for when the screen works normally, and for when it doesn't wake up. The "permission denied" bit is from the VT switch and not related to the actual problem.

(In reply to Nate Graham from comment #2)
> This appears to be 100% reproducible now when the screen dims or blanks due
> to inactivity.
It's not completely 100% reproducible for me anymore, but I could trigger it with a few attempts, and could not trigger it anymore with https://invent.kde.org/plasma/kwin/-/merge_requests/8282. Can you test the MR as well?
Comment 6 Nikola Novoselec 2025-11-29 22:41:27 UTC
I spent some time digging into this and I think I've found what's going on.

TL;DR: DSC modes can't wake from DPMS because KWin's atomic path doesn't request a full modeset, but DSC needs link retraining after the display powers down. The driver doesn't help either - it reports everything as "Good" even when the link is dead.

My Setup
- KWin 6.5.3, CachyOS (Arch-based), kernel 6.17.9-2-cachyos
- RTX 3090 with nvidia-open 580.105.08
- Samsung Odyssey G95NC at 7680x2160@120Hz over DisplayPort
- Wayland session, VRR off, HDR off, Night Color off
- Kernel params: nvidia_drm.modeset=1 nvidia_drm.fbdev=1

What Happens
Expected: 
- Screen wakes up when I move the mouse or press a key.

Actual:
- Lock screen, let DPMS kick in
- Try to wake up (mouse/keyboard)
- Screen stays black. Audio keeps playing, SSH works fine
- Ctrl+Alt+F2 then Ctrl+Alt+F1 brings everything back instantly
- Every. Single. Time.

What I Found
I started testing different modes to figure out what's actually broken:

Resolution	                DSC Required?	DPMS Wake
7680x2160 @ 120Hz	Yes (~48 Gbps)	Black screen
3840x2160 @ 120Hz	Borderline (~24 Gbps)	Corruption
3840x2160 @ 60Hz	No (~12 Gbps)	Works fine

So it's clearly tied to DSC. The pattern is consistent - if the mode needs DSC, wake fails.

Looking at journalctl during wake attempts:

kwin_wayland[5884]: atomic commit failed: Permission denied
kwin_wayland[5884]: Atomic modeset test failed! Permission denied
kwin_wayland[5884]: Applying output configuration failed!
Why I Think This Happens
DSC requires full link training - DPCD negotiation, encoder/decoder sync, the works. When the monitor goes to sleep, it tears down the DisplayPort link completely (at least this Samsung does).

When KWin tries to wake it up, it appears to just flip CRTC.ACTIVE from 0 to 1 via DrmPipeline::commitPipelines() and submits an atomic commit without DRM_MODE_ATOMIC_ALLOW_MODESET. The driver sees a dead link and can't satisfy the request without retraining.

Why -EPERM specifically
The -EPERM from nvidia-drm is their way of saying "I can't do this without a modeset". Standard DRM usually returns -EINVAL for bad configs, but NVIDIA uses -EPERM when resource allocation fails or the hardware state requires a full modeset to proceed.

Basically KWin is telling the driver "turn this on, but don't touch link training or bandwidth allocation" and the driver responds "permission denied - I literally cannot do that with a dead DSC link."

Why VT switch fixes it
VT switch forces drmDropMaster()/drmSetMaster(). KWin re-acquires master, treats the GPU as "new", and does full initialization including link training. This proves the hardware and driver can wake correctly - it's just the atomic fast-path that's broken for DSC.

I also looked at MR !8282 but that's addressing hotplug state consistency during display connect/disconnect, not this DPMS wake path.

The Kicker: Driver Doesn't Report the Failure
Here's where it gets interesting. I captured modetest via SSH while the screen was black (before VT switch recovery):

Connector 139 (DP-3): connected 1400x400mm
  props:
    DPMS: On (value: 0)
    link-status: Good (value: 0)
    CRTC_ID: 138
Even during the failure state, the driver reports:

DPMS: On - driver thinks display is on
link-status: Good - driver reports no link problems
The nvidia-drm driver doesn't surface the DSC link failure to userspace. It silently fails the atomic commit with -EPERM while reporting everything as healthy. This means KWin can't use link-status: Bad as a trigger for modeset - the driver never sets it.

This further supports unconditionally requesting modeset on DPMS wake, since there's no reliable signal from the driver that re-training is needed.

Proposed Fix
The atomic wake path needs to request a full modeset when leaving DPMS Off. Should be a minimal change in DrmOutput::setDpmsMode():

bool DrmOutput::setDpmsMode(DpmsMode requested)
{
    if (requested == DpmsMode::On && m_dpmsMode != requested) {
        // Link may have been torn down, allow driver to retrain
        pipeline()->setModesetRequested(true);
    }
    return commitDpms(requested);
}

This adds DRM_MODE_ATOMIC_ALLOW_MODESET to the commit, letting the kernel do its thing with link training and DSC negotiation.
Performance impact: Adds ~20-40ms to the wake path on high-refresh panels. Honestly that's nothing compared to a black screen.

Future optimization: Could theoretically be made conditional on:
- The upcoming kernel flag DRM_MODE_FLAG_DSC
- link-status property being Bad

However, as shown in the logs above, nvidia-drm doesn't set link-status: Bad when DSC link training fails - it just silently rejects the atomic commit. Until drivers reliably surface link failures, forcing modeset unconditionally on wake is the only robust fix.

Other Compositors
I'm not the only one hitting this - wlroots, Hyprland and Sway are all adding the same workaround:
swaywm/wlroots#2373
hyprwm/Hyprland#2696

There's also kernel-side discussion about making link-status flip to Bad automatically when DSC is lost, so userspace can react without guessing. Until that lands, forcing modeset on wake is the pragmatic fix.

Logs
KWin Journal (failure sequence)
Nov 29 23:24:24 kwin_wayland[6070]: Failed to delay sleep: Sender is not authorized to send message
Nov 29 23:24:57 kwin_wayland[6070]: Atomic modeset test failed! Permission denied
Nov 29 23:24:57 kwin_wayland[6070]: Applying output configuration failed!
Recovery via VT switch succeeded immediately after.

drm_info (working state)
Node: /dev/dri/card1
├───Driver: nvidia-drm (NVIDIA DRM driver) version 0.0.0
│   ├───DRM_CLIENT_CAP_ATOMIC supported
│   └───DRM_CAP_ATOMIC_ASYNC_PAGE_FLIP = 1
├───Device: PCI 10de:2204 NVIDIA Corporation GA102 [GeForce RTX 3090]
└───Connector 3 (DisplayPort)
    ├───Status: connected
    ├───Physical size: 1400×400 mm
    ├───Mode: 7680×2160@120.00 phsync nvsync
    ├───EDID: Samsung Electric Company Odyssey G95NC
    ├───Properties
    │   ├───"DPMS": enum {On, Standby, Suspend, Off} = On
    │   ├───"link-status": enum {Good, Bad} = Good
    │   ├───"CRTC_ID" (atomic): object CRTC = 62
    │   └───"vrr_capable" (immutable): range [0, 1] = 1
    └───CRTC 0
        ├───Object ID: 62
        ├───ACTIVE: 1
        ├───MODE_ID: blob 152 (7680×2160@120.00)
        └───VRR_ENABLED: 0

modetest connector (before failure)
Connector 3 (139): DisplayPort
  status: connected
  modes:
    7680x2160 120.00 7680 7728 7760 7800 2160 2163 2168 2222 207640 phsync nvsync
  props:
    DPMS: On
    link-status: Good
    CRTC_ID: 62
modetest connector (DURING failure - black screen)

Captured via SSH while screen was black, before VT switch recovery:
Connector 139 (DP-3): connected 1400x400mm
  props:
    DPMS: On (value: 0)
    link-status: Good (value: 0)
    CRTC_ID: 138
This is significant: Even during the failure state, the driver reports everything as healthy. It silently fails the atomic commit with -EPERM while claiming the link is fine.

Also tested with the proprietary NVIDIA driver (565.77) - same behavior.
Comment 7 Zamundaaa 2025-11-30 01:19:28 UTC
(In reply to Nikola Novoselec from comment #6)
> When KWin tries to wake it up, it appears to just flip CRTC.ACTIVE from 0 to
> 1 via DrmPipeline::commitPipelines() and submits an atomic commit without
> DRM_MODE_ATOMIC_ALLOW_MODESET.
KWin actually always severs connection between connector and crtc completely, and always allows a full modeset on dpms changes. It just has the optimization that, if the driver doesn't actually require a modeset, we skip it.

I'm surprised the Nvidia driver would ever allow such configuration changes without a modeset, but if you want to investigate that further, as I wrote before, make a new bug report. This one is not about a Nvidia driver bug.

> The -EPERM from nvidia-drm is their way of saying "I can't do this without a
> modeset". Standard DRM usually returns -EINVAL for bad configs, but NVIDIA
> uses -EPERM when resource allocation fails or the hardware state requires a
> full modeset to proceed.
As I mentioned before, "permission denied" errors are from the VT switch, there's nothing more to it.

> VT switch forces drmDropMaster()/drmSetMaster(). KWin re-acquires master, treats the GPU as "new", and does full initialization including link training. This proves the hardware and driver can wake correctly - it's just the atomic fast-path that's broken for DSC.
KWin doesn't really do anything special on VT switches with atomic modesetting (legacy does force a modeset, because of bugs in old and not super well maintained drivers).
If I had to guess, I'd say the VT switch changes some driver-internal state, and maybe forces a modeset on the driver side.

> This is significant: Even during the failure state, the driver reports
> everything as healthy. It silently fails the atomic commit with -EPERM while
> claiming the link is fine.
If drm_info shows the screen is on while it's actually not receiving any signal, then that is definitely a driver bug. You can report it at https://github.com/NVIDIA/open-gpu-kernel-modules/issues.

> Other Compositors
> I'm not the only one hitting this - wlroots, Hyprland and Sway are all adding the same workaround:
> swaywm/wlroots#2373
> hyprwm/Hyprland#2696
Neither of theses issues are related to what you describe, and don't hint at wlroots or Hyprland doing any such workaround either...

> Future optimization: Could theoretically be made conditional on:
> - The upcoming kernel flag DRM_MODE_FLAG_DSC
There's no such flag to be found on dri-devel or Google. Did you use an LLM to hallucinate this research?
Comment 8 Nikola Novoselec 2025-11-30 10:21:19 UTC
You're right, and I apologize for the noise.

I let an LLM have a go at analyzing the logs and the conclusions I'd drawn, and clearly didn't verify the output thoroughly enough. The `DRM_MODE_FLAG_DSC` reference is definitely a hallucination - that flag doesn't exist in the kernel and there's no proposal for it either. I should have caught that before posting.

I've since traced the actual KWin code path and confirmed you're correct: `DrmOutput::setDrmDpmsMode()` calls `commitPipelines()` with `CommitMode::TestAllowModeset`, which does set `DRM_MODE_ATOMIC_ALLOW_MODESET`. The premise of my analysis was wrong.

The DSC correlation in my testing still holds (7680x2160@120Hz fails, lower bandwidth modes work), but as you pointed out, if KWin is doing the right thing, the issue is in the NVIDIA driver. I'll open a bug with them instead.

Thanks for taking the time to respond and for catching the hallucinated content. Lesson learned on verification.
Comment 9 Zamundaaa 2025-12-01 17:19:55 UTC
Git commit f0f00551a40ec071095c7f3076221d898ffb95af by Xaver Hugl.
Committed on 01/12/2025 at 16:14.
Pushed by zamundaaa into branch 'master'.

workspace: route dpms handling through workspace and OutputConfiguration

This has a few advantages:
- dpms is now global, which means it's more robust against hotplug events
- some code from the backends is moved to Workspace and simplified
- displays are now turned off atomically (per GPU), rather than potentially
  doing multiple modesets that could cause flicker
Related: bug 493879

M  +0    -24   src/backends/drm/drm_backend.cpp
M  +0    -1    src/backends/drm/drm_backend.h
M  +1    -17   src/backends/drm/drm_gpu.cpp
M  +13   -78   src/backends/drm/drm_output.cpp
M  +0    -5    src/backends/drm/drm_output.h
M  +5    -0    src/backends/drm/drm_pipeline.cpp
M  +0    -7    src/backends/drm/drm_virtual_output.cpp
M  +0    -1    src/backends/drm/drm_virtual_output.h
M  +8    -29   src/backends/wayland/wayland_output.cpp
M  +0    -3    src/backends/wayland/wayland_output.h
M  +0    -10   src/core/backendoutput.cpp
M  +0    -25   src/core/backendoutput.h
M  +0    -2    src/core/output.cpp
M  +0    -12   src/core/output.h
M  +1    -0    src/core/outputconfiguration.h
M  +1    -4    src/dpmsinputeventfilter.cpp
M  +26   -17   src/plugins/kscreen/kscreen.cpp
M  +1    -1    src/plugins/kscreen/kscreen.h
M  +9    -32   src/wayland/dpms.cpp
M  +69   -26   src/workspace.cpp
M  +15   -2    src/workspace.h

https://invent.kde.org/plasma/kwin/-/commit/f0f00551a40ec071095c7f3076221d898ffb95af
Comment 10 Nate Graham 2025-12-10 16:25:57 UTC
*** Bug 513151 has been marked as a duplicate of this bug. ***