Created attachment 156646 [details]
The full trace of all threads of the nested kwin_wayland crash.

SUMMARY

I booted the Fedora 38 KDE Plasma live image Fedora-KDE-Live-x86_64-38-20230221.n.1.iso on bare metal by selecting Troubleshooting > Start Fedora-KDE-Live 38 in basic graphics mode which puts nomodeset on the kernel command line and uses the llvmpipe driver from mesa-dri-drivers-23.0.0~rc4-3.fc38.x86_64 and the simpledrm driver from the 6.2.0 kernel. Plasma 5.27.0 on Wayland started. I started Konsole. I tried to run a nested kwin_wayland session using the instructions at https://community.kde.org/KWin/Wayland
export $(dbus-launch)
kwin_wayland --xwayland 

The nested kwin_wayland window didn't appear except for a Wayland icon briefly shown in the task manager. The following output was in Konsole which showed some errors and a segmentation fault of kwin_wayland.

kwin_wayland --xwayland
No backend specified, automatically choosing Wayland because WAYLAND_DISPLAY is set
unable to lock lockfile /run/user/1000/wayland-0.lock, maybe another compositor is running
kwin_wayland_backend: Failed to open drm render node /dev/dri/renderD128
kf.globalaccel.kglobalacceld: Failed to register service org.kde.kglobalaccel
OpenGL vendor string:                   Mesa
OpenGL renderer string:                 llvmpipe (LLVM 15.0.7, 256 bits)
OpenGL version string:                  4.5 (Core Profile) Mesa 23.0.0-rc4
OpenGL shading language version string: 4.50
Driver:                                 LLVMpipe
GPU class:                              Unknown
OpenGL version:                         4.5
GLSL version:                           4.50
Mesa version:                           23.0
Linux kernel version:                   6.2
Requires strict binding:                no
GLSL shaders:                           yes
Texture NPOT support:                   yes
Virtual Machine:                        no
kwin_core: Parse error in tiles configuration for monitor "7fb8c463-c102-5440-8fb7-5253b26b5d9c" : "illegal value" Creating default setup
kwin_xkbcommon: XKB: inet:323:58: unrecognized keysym "XF86EmojiPicker"
kwin_xkbcommon: XKB: inet:324:58: unrecognized keysym "XF86Dictate"
(WW) Option "-listen" for file descriptors is deprecated
Please use "-listenfd" instead.
(WW) Option "-listen" for file descriptors is deprecated
Please use "-listenfd" instead.
(EE) could not connect to wayland server
Segmentation fault (core dumped)

Nested kwin_wayland crashed when starting in gbm_bo_create_with_modifiers. gbm=0x0 so gbm->v0.backend_version might've been a null pointer dereference at at ../src/gbm/main/gbm.c:518 in mesa-libgbm.

Core was generated by `kwin_wayland --xwayland'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fa870378365 in gbm_bo_create_with_modifiers (gbm=0x0, width=64, height=64, format=875713089, modifiers=0x55aeb46ccb58, count=1) at ../src/gbm/main/gbm.c:518
518        if (gbm->v0.backend_version >= 1) {
[Current thread is 1 (Thread 0x7fa8726b2e40 (LWP 3725))]

(gdb) bt
#0  0x00007fa870378365 in gbm_bo_create_with_modifiers (gbm=0x0, width=64, height=64, format=875713089,
    modifiers=0x55aeb46ccb58, count=1) at ../src/gbm/main/gbm.c:518
#1  0x00007fa8737d49e6 in KWin::Wayland::WaylandEglLayerBuffer::WaylandEglLayerBuffer (modifiers=...,
    backend=0x55aeb4759160, format=<optimized out>, size=..., this=0x55aeb4dbf930)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/backends/wayland/wayland_egl_backend.cpp:53
#2  std::_Construct<KWin::Wayland::WaylandEglLayerBuffer, QSize const&, unsigned int&, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend*&> (__p=0x55aeb4dbf930) at /usr/include/c++/13/bits/stl_construct.h:119
#3  std::allocator_traits<std::allocator<void> >::construct<KWin::Wayland::WaylandEglLayerBuffer, QSize const&, unsigned int&, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend*&> (__p=0x55aeb4dbf930)
    at /usr/include/c++/13/bits/alloc_traits.h:660
#4  std::_Sp_counted_ptr_inplace<KWin::Wayland::WaylandEglLayerBuffer, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<QSize const&, unsigned int&, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend*&>
    (__a=..., this=0x55aeb4dbf920) at /usr/include/c++/13/bits/shared_ptr_base.h:604
#5  std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<KWin::Wayland::WaylandEglLayerBuffer, std::allocator<void>, QSize const&, unsigned int&, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend*&> (__a=...,
    __p=<optimized out>, this=<optimized out>) at /usr/include/c++/13/bits/shared_ptr_base.h:971
#6  std::__shared_ptr<KWin::Wayland::WaylandEglLayerBuffer, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, QSize const&, unsigned int&, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend*&> (__tag=...,
    this=<optimized out>) at /usr/include/c++/13/bits/shared_ptr_base.h:1712
#7  std::shared_ptr<KWin::Wayland::WaylandEglLayerBuffer>::shared_ptr<std::allocator<void>, QSize const&, unsigned int&, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend*&> (__tag=..., this=<optimized out>)
    at /usr/include/c++/13/bits/shared_ptr.h:464
#8  std::make_shared<KWin::Wayland::WaylandEglLayerBuffer, QSize const&, unsigned int&, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend*&> () at /usr/include/c++/13/bits/shared_ptr.h:1010
#9  KWin::Wayland::WaylandEglLayerSwapchain::WaylandEglLayerSwapchain (this=<optimized out>, size=...,
--Type <RET> for more, q to quit, c to continue without paging--
    format=<optimized out>, modifiers=..., backend=<optimized out>, this=<optimized out>, size=...,
    format=<optimized out>, modifiers=..., backend=<optimized out>)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/backends/wayland/wayland_egl_backend.cpp:127
#10 0x00007fa8737d76eb in std::make_unique<KWin::Wayland::WaylandEglLayerSwapchain, QSize const&, unsigned int const&, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend* const&> () at /usr/include/c++/13/bits/unique_ptr.h:1070
#11 KWin::Wayland::WaylandEglCursorLayer::beginFrame (this=0x55aeb4cda450)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/backends/wayland/wayland_egl_backend.cpp:284
#12 0x00007fa8737de2e2 in KWin::Wayland::WaylandOutput::renderCursorOpengl (this=this@entry=0x55aeb46c5e40,
    backend=<optimized out>, source=source@entry=0x55aeb4d9ac40)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/backends/wayland/wayland_output.cpp:217
#13 0x00007fa8737de805 in KWin::Wayland::WaylandOutput::setCursor (source=0x55aeb4d9ac40, this=0x55aeb46c5e40)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/backends/wayland/wayland_output.cpp:191
#14 KWin::Wayland::WaylandOutput::setCursor (this=0x55aeb46c5e40, source=0x55aeb4d9ac40)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/backends/wayland/wayland_output.cpp:184
#15 0x00007fa8735f0a42 in operator() (__closure=__closure@entry=0x7ffd47965fd0)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/composite.cpp:455
#16 0x00007fa8735f4686 in KWin::Compositor::addOutput (this=this@entry=0x55aeb46d5000, output=0x55aeb46c5e40)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/composite.cpp:471
#17 0x00007fa8735f4988 in KWin::Compositor::startupWithWorkspace (this=0x55aeb46d5000)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/composite.cpp:383
#18 0x00007fa871cdf03b in QObject::event (this=0x55aeb46d5000, e=0x55aeb4714ba0) at kernel/qobject.cpp:1347
#19 0x00007fa8713aece5 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /lib64/libQt5Widgets.so.5
#20 0x00007fa871cb3648 in QCoreApplication::notifyInternal2 (receiver=0x55aeb46d5000, event=0x55aeb4714ba0)
    at kernel/qcoreapplication.cpp:1064
#21 0x00007fa871cb6af5 in QCoreApplicationPrivate::sendPostedEvents (receiver=receiver@entry=0x0,
--Type <RET> for more, q to quit, c to continue without paging--
    event_type=event_type@entry=0, data=data@entry=0x55aeb467e900) at kernel/qcoreapplication.cpp:1821
#22 0x00007fa871d03371 in QEventDispatcherUNIX::processEvents (this=0x55aeb46817d0, flags=...)
    at kernel/qeventdispatcher_unix.cpp:468
#23 0x000055aeb41480c2 in QUnixEventDispatcherQPA::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
#24 0x00007fa871cb201b in QEventLoop::exec (this=this@entry=0x7ffd479663f0, flags=..., flags@entry=...)
    at ../../include/QtCore/../../src/corelib/global/qflags.h:69
#25 0x00007fa871cba29b in QCoreApplication::exec () at ../../include/QtCore/../../src/corelib/global/qflags.h:121
#26 0x000055aeb4063c04 in main (argc=<optimized out>, argv=<optimized out>)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/main_wayland.cpp:628

KWin::Wayland::WaylandEglLayerBuffer::WaylandEglLayerBuffer in frame 1 had the null pointer gbmDevice, which might've been why gbm was null in gbm_bo_create_with_modifiers.

(gdb) frame 1
#1  0x00007fa8737d49e6 in KWin::Wayland::WaylandEglLayerBuffer::WaylandEglLayerBuffer (modifiers=...,
    backend=0x55aeb4759160, format=<optimized out>, size=..., this=0x55aeb4dbf930)
    at /usr/src/debug/kwin-5.27.0-2.fc38.x86_64/src/backends/wayland/wayland_egl_backend.cpp:53
53              m_bo = gbm_bo_create_with_modifiers(gbmDevice,
(gdb) p m_bo
$1 = (gbm_bo *) 0x0
(gdb) p gbmDevice
$2 = (gbm_device *) 0x0

The error "kwin_wayland_backend: Failed to open drm render node /dev/dri/renderD128" in the output might be related to why gbm was null. This problem happened 4/4 times I tried to run a nested kwin_wayland session on bare metal in basic graphics mode using the llvmpipe driver. kwin_wayland crashed with different traces each of a few times I tried to run a nested kwin_wayland session in VMs using the llvmpipe driver as I reported at https://bugs.kde.org/show_bug.cgi?id=466281 Nested kwin_wayland started normally in VMs with the same image using 3D acceleration enabled using the virgl mesa driver and on bare metal using the radeonsi mesa driver. The problem might be specific to the use of llvmpipe on bare metal with the simpledrm kernel driver.

STEPS TO REPRODUCE
1. Boot a Fedora 37 KDE Plasma installation updated to 2023-2-22 with updates-testing enabled
2. Log in to Plasma 5.27.0 on Wayland from sddm
3. Download Fedora-KDE-Live-x86_64-38-20230221.n.1.iso from https://koji.fedoraproject.org/koji/buildinfo?buildID=2157026
4. Install Fedora Media Writer if it isn't already with sudo dnf install mediawriter
5. Start Fedora Media Writer
6. write the Fedora 38 KDE Plasma live image Fedora-KDE-Live-x86_64-38-20230221.n.1.iso to a USB flash drive with Fedora Media Writer
7. Reboot
8. In grub, Select Troubleshooting > Start Fedora-KDE-Live 38 in basic graphics mode
7. Start Konsole in Plasma
8. In Konsole, run
export $(dbus-launch)
kwin_wayland --xwayland

OBSERVED RESULT
Nested kwin_wayland crashed when starting in gbm_bo_create_with_modifiers on bare metal using the llvmpipe driver

EXPECTED RESULT
Nested kwin_wayland wouldn't have crashed when starting on bare metal using the llvmpipe driver

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Fedora 38
(available in About System)
KDE Plasma Version: 5.27.0
KDE Frameworks Version: 5.103.0
Qt Version: 5.15.8

ADDITIONAL INFORMATION

I'm attaching the full trace of all threads. I ran nested kwin_wayland under valgrind in konsole with valgrind --log-file=valgrind-kwin_wayland-5.27.0-llvmpipe-bare-metal-1.txt --enable-debuginfod=no kwin_wayland --xwayland

The valgrind log showed the syscall param waitid(infop) pointed to unaddressable byte(s) 0x0 and an invalid read of the address 0x10 at gbm_bo_create_with_modifiers (gbm.c:518) causing the segmentation fault.

==3098== Memcheck, a memory error detector
==3098== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==3098== Using Valgrind-3.20.0 and LibVEX; rerun with -h for copyright info
==3098== Command: kwin_wayland --xwayland
==3098== Parent PID: 2795
==3098== 
==3098== Syscall param waitid(infop) points to unaddressable byte(s)
==3098==    at 0x78A3D2D: syscall (syscall.S:38)
==3098==    by 0x666F85E: sys_waitid (forkfd_linux.c:65)
==3098==    by 0x666F85E: detect_clone_pidfd_support (forkfd_linux.c:126)
==3098==    by 0x666F85E: system_forkfd (forkfd_linux.c:142)
==3098==    by 0x666F85E: forkfd (forkfd.c:651)
==3098==    by 0x6655118: QProcessPrivate::startProcess() (qprocess_unix.cpp:466)
==3098==    by 0x2137EE: KWin::Xwl::XwaylandLauncher::startInternal() [clone .isra.0] (xwaylandlauncher.cpp:186)
==3098==    by 0x66FBF50: call (qobjectdefs_impl.h:398)
==3098==    by 0x66FBF50: void doActivate<false>(QObject*, int, void**) (qobject.cpp:3923)
==3098==    by 0x4CB93BE: KWin::Compositor::setupStart() [clone .part.0] (composite.cpp:335)
==3098==    by 0x4CBAE27: KWin::WaylandCompositor::start() (composite.cpp:799)
==3098==    by 0x66F303A: QObject::event(QEvent*) (qobject.cpp:1347)
==3098==    by 0x6E34CE4: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.15.8)
==3098==    by 0x66C7647: QCoreApplication::notifyInternal2(QObject*, QEvent*) (qcoreapplication.cpp:1064)
==3098==    by 0x66CAAF4: QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (qcoreapplication.cpp:1821)
==3098==    by 0x6717370: QEventDispatcherUNIX::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (qeventdispatcher_unix.cpp:468)
==3098==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==3098== 
==3098== Invalid read of size 4
==3098==    at 0x8213365: gbm_bo_create_with_modifiers (gbm.c:518)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (wayland_egl_backend.cpp:53)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (stl_construct.h:119)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (alloc_traits.h:660)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr_base.h:604)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr_base.h:971)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr_base.h:1712)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr.h:464)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr.h:1010)
==3098==    by 0x4E9A9E5: KWin::Wayland::WaylandEglLayerSwapchain::WaylandEglLayerSwapchain(QSize const&, unsigned int, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend*) (wayland_egl_backend.cpp:127)
==3098==    by 0x4E9D6EA: UnknownInlinedFun (unique_ptr.h:1071)
==3098==    by 0x4E9D6EA: KWin::Wayland::WaylandEglCursorLayer::beginFrame() (wayland_egl_backend.cpp:284)
==3098==    by 0x4EA42E1: KWin::Wayland::WaylandOutput::renderCursorOpengl(KWin::Wayland::WaylandEglBackend*, KWin::CursorSource*) (wayland_output.cpp:217)
==3098==    by 0x4EA4804: UnknownInlinedFun (wayland_output.cpp:191)
==3098==    by 0x4EA4804: KWin::Wayland::WaylandOutput::setCursor(KWin::CursorSource*) (wayland_output.cpp:184)
==3098==    by 0x4CB6A41: KWin::Compositor::addOutput(KWin::Output*)::{lambda()#2}::operator()() const (composite.cpp:455)
==3098==    by 0x4CBA685: KWin::Compositor::addOutput(KWin::Output*) (composite.cpp:471)
==3098==    by 0x4CBA987: KWin::Compositor::startupWithWorkspace() (composite.cpp:383)
==3098==    by 0x66F303A: QObject::event(QEvent*) (qobject.cpp:1347)
==3098==    by 0x6E34CE4: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.15.8)
==3098==    by 0x66C7647: QCoreApplication::notifyInternal2(QObject*, QEvent*) (qcoreapplication.cpp:1064)
==3098==    by 0x66CAAF4: QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (qcoreapplication.cpp:1821)
==3098==  Address 0x10 is not stack'd, malloc'd or (recently) free'd
==3098== 
==3098== 
==3098== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==3098==  Access not within mapped region at address 0x10
==3098==    at 0x8213365: gbm_bo_create_with_modifiers (gbm.c:518)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (wayland_egl_backend.cpp:53)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (stl_construct.h:119)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (alloc_traits.h:660)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr_base.h:604)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr_base.h:971)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr_base.h:1712)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr.h:464)
==3098==    by 0x4E9A9E5: UnknownInlinedFun (shared_ptr.h:1010)
==3098==    by 0x4E9A9E5: KWin::Wayland::WaylandEglLayerSwapchain::WaylandEglLayerSwapchain(QSize const&, unsigned int, QVector<unsigned long> const&, KWin::Wayland::WaylandEglBackend*) (wayland_egl_backend.cpp:127)
==3098==    by 0x4E9D6EA: UnknownInlinedFun (unique_ptr.h:1071)
==3098==    by 0x4E9D6EA: KWin::Wayland::WaylandEglCursorLayer::beginFrame() (wayland_egl_backend.cpp:284)
==3098==    by 0x4EA42E1: KWin::Wayland::WaylandOutput::renderCursorOpengl(KWin::Wayland::WaylandEglBackend*, KWin::CursorSource*) (wayland_output.cpp:217)
==3098==    by 0x4EA4804: UnknownInlinedFun (wayland_output.cpp:191)
==3098==    by 0x4EA4804: KWin::Wayland::WaylandOutput::setCursor(KWin::CursorSource*) (wayland_output.cpp:184)
==3098==    by 0x4CB6A41: KWin::Compositor::addOutput(KWin::Output*)::{lambda()#2}::operator()() const (composite.cpp:455)
==3098==    by 0x4CBA685: KWin::Compositor::addOutput(KWin::Output*) (composite.cpp:471)
==3098==    by 0x4CBA987: KWin::Compositor::startupWithWorkspace() (composite.cpp:383)
==3098==    by 0x66F303A: QObject::event(QEvent*) (qobject.cpp:1347)
==3098==    by 0x6E34CE4: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib64/libQt5Widgets.so.5.15.8)
==3098==    by 0x66C7647: QCoreApplication::notifyInternal2(QObject*, QEvent*) (qcoreapplication.cpp:1064)
==3098==    by 0x66CAAF4: QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (qcoreapplication.cpp:1821)
==3098==  If you believe this happened as a result of a stack
==3098==  overflow in your program's main thread (unlikely but
==3098==  possible), you can try to increase the size of the
==3098==  main thread stack using the --main-stacksize= flag.
==3098==  The main thread stack size used in this run was 8388608.
==3098== 
==3098== HEAP SUMMARY:
==3098==     in use at exit: 9,041,623 bytes in 44,454 blocks
==3098==   total heap usage: 250,646 allocs, 206,192 frees, 88,517,079 bytes allocated
==3098== 
==3098== LEAK SUMMARY:
==3098==    definitely lost: 72 bytes in 1 blocks
==3098==    indirectly lost: 0 bytes in 0 blocks
==3098==      possibly lost: 83,136 bytes in 777 blocks
==3098==    still reachable: 8,956,399 bytes in 43,655 blocks
==3098==                       of which reachable via heuristic:
==3098==                         newarray           : 9,648 bytes in 6 blocks
==3098==         suppressed: 0 bytes in 0 blocks
==3098== Rerun with --leak-check=full to see details of leaked memory
==3098== 
==3098== For lists of detected and suppressed errors, rerun with: -s
==3098== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)