Bug 521333

Summary:	KWin does not properly fallback to drmModeAddFB when drmModeAddFB2 fails
Product:	[Plasma] kwin	Reporter:	Pavel Ondračka <pavel.ondracka>
Component:	platform-drm	Assignee:	KWin default assignee <kwin-bugs-null>
Status:	RESOLVED FIXED
Severity:	normal	CC:	nate, xaver.hugl
Priority:	NOR
Version First Reported In:	unspecified
Target Milestone:	---
Platform:	Other
OS:	Linux
Latest Commit:	https://invent.kde.org/plasma/kwin/-/commit/ee01169f593151ae7743143227260973c3b1df8b	Version Fixed/Implemented In:	6.6.6
Sentry Crash Report:

Description Pavel Ondračka 2026-06-11 12:14:09 UTC

DESCRIPTION

When drmModeAddFB2 fails, KWin has a fallback path to drmModeAddFB, but that one is guarded by ret == EOPNOTSUPP https://github.com/KDE/kwin/blob/master/src/backends/drm/drm_gpu.cpp#L996, while the kernel returns -EONOTSUPP.

This is problematic for some drm drivers on big endian (like for radeon driver) since kernel returns -EOPNOTSUPP for everything unless quirk_addfb_prefer_host_byte_order is set (and radeon does not).
  #ifdef __BIG_ENDIAN
  if (!dev->mode_config.quirk_addfb_prefer_host_byte_order)
      return -EOPNOTSUPP;
  #endif

So the check should probably be more robust, possibly something like:
if (ret < 0 && (ret == -EOPNOTSUPP || errno == EOPNOTSUPP))

STEPS TO REPRODUCE
1. Start kwin on a big endian machine with some driver without modifiers that does not have modifiers and quirk_addfb_prefer_host_byte_order. Or hack the kernel to always return -EOPNOTSUPP from drmModeAddFB2

OBSERVED RESULT
kwin fails to start since it cannot ge t a framebuffer.


EXPECTED RESULT
kwin starts


SOFTWARE/OS VERSIONS
Operating System: linux
kwin-6.6.5

Comment 1 Bug Janitor Service 2026-06-11 15:55:07 UTC

A possibly relevant merge request was started @ https://invent.kde.org/plasma/kwin/-/merge_requests/9373

Comment 2 Zamundaaa 2026-06-16 11:39:02 UTC

Git commit 85fa4ee22b57e834505864a20ac13de0a97a1161 by Xaver Hugl.
Committed on 16/06/2026 at 11:18.
Pushed by zamundaaa into branch 'master'.

backends/drm: fix addFB2 fallback

Returned error codes are negative, not the actual value of the define.

M  +1    -1    src/backends/drm/drm_gpu.cpp

https://invent.kde.org/plasma/kwin/-/commit/85fa4ee22b57e834505864a20ac13de0a97a1161

Comment 3 Zamundaaa 2026-06-16 12:32:23 UTC

Git commit 9f27237b14432624ee2798dc6866748e96a705dd by Xaver Hugl.
Committed on 16/06/2026 at 11:49.
Pushed by zamundaaa into branch 'Plasma/6.7'.

backends/drm: fix addFB2 fallback

Returned error codes are negative, not the actual value of the define.


(cherry picked from commit 85fa4ee22b57e834505864a20ac13de0a97a1161)

Co-authored-by: Xaver Hugl <xaver.hugl@kde.org>

M  +1    -1    src/backends/drm/drm_gpu.cpp

https://invent.kde.org/plasma/kwin/-/commit/9f27237b14432624ee2798dc6866748e96a705dd

Comment 4 Vlad Zahorodnii 2026-06-17 07:13:36 UTC

Git commit ee01169f593151ae7743143227260973c3b1df8b by Vlad Zahorodnii, on behalf of Xaver Hugl.
Committed on 17/06/2026 at 06:37.
Pushed by vladz into branch 'Plasma/6.6'.

backends/drm: fix addFB2 fallback

Returned error codes are negative, not the actual value of the define.


(cherry picked from commit 85fa4ee22b57e834505864a20ac13de0a97a1161)

Co-authored-by: Xaver Hugl <xaver.hugl@kde.org>

M  +1    -1    src/backends/drm/drm_gpu.cpp

https://invent.kde.org/plasma/kwin/-/commit/ee01169f593151ae7743143227260973c3b1df8b