Bug 505758

Summary: (Wayland/NVIDIA) Fail when starting RDP session: ERRINFO_GRAPHICS_SUBSYSTEM_FAILED (0x0000112F)
Product: [Plasma] KRdp Reporter: Andrés Cabezas <andrescabezasharo>
Component: generalAssignee: Unassigned bugs <unassigned-bugs-null>
Status: REPORTED ---    
Severity: normal CC: ahiemstra, nate
Priority: NOR    
Version First Reported In: 6.3.5   
Target Milestone: ---   
Platform: Arch Linux   
OS: Linux   
Latest Commit: Version Fixed/Implemented In:
Sentry Crash Report:

Description Andrés Cabezas 2025-06-18 23:22:55 UTC
SUMMARY
When trying to connect to my computer (running Wayland KDE with an NVIDIA graphics card), the client connects, shows a black screen for fraction of a second and the connection gets interrupted. When trying to use krdpserver manually, the following errors appear

```
org.kde.krdp: Listening for connections on QHostAddress(QHostAddress::Any) 3389
org.kde.krdp: Initializing Freedesktop Portal Session
org.kde.krdp: Session setup completed, start processing...
[19:04:38:345] [26458:26466] [INFO][com.freerdp.core.connection] - Client Security: NLA:1 TLS:1 RDP:0
[19:04:38:346] [26458:26466] [INFO][com.freerdp.core.connection] - Server Security: NLA:1 TLS:0 RDP:0
[19:04:38:346] [26458:26466] [INFO][com.freerdp.core.connection] - Negotiated Security: NLA:1 TLS:0 RDP:0
[19:04:38:406] [26458:26466] [WARN][com.winpr.negotiate] - AcceptSecurityContext status SEC_I_CONTINUE_NEEDED [0x00090312]
[19:04:38:413] [26458:26466] [WARN][com.winpr.negotiate] - AcceptSecurityContext status SEC_I_COMPLETE_NEEDED [0x00090313]
[19:04:38:417] [26458:26466] [INFO][com.freerdp.core.connection] - Accepted client: SM-P613
[19:04:38:417] [26458:26466] [INFO][com.freerdp.core.connection] - Accepted channels:
[19:04:38:417] [26458:26466] [INFO][com.freerdp.core.connection] -  rdpdr
[19:04:38:417] [26458:26466] [INFO][com.freerdp.core.connection] -  rdpsnd
[19:04:38:417] [26458:26466] [INFO][com.freerdp.core.connection] -  cliprdr
[19:04:38:417] [26458:26466] [INFO][com.freerdp.core.connection] -  drdynvc
[19:04:38:417] [26458:26466] [INFO][com.freerdp.core.gcc] - Active rdp encryption level: NONE
[19:04:38:417] [26458:26466] [INFO][com.freerdp.core.gcc] - Selected rdp encryption method: NONE
org.kde.krdp: New client connected:  Android platform Unspecified version
org.kde.krdp: Video stream initialized
org.kde.krdp: Received caps:
org.kde.krdp:   RDPGFX_CAPVERSION_8 AVC: false YUV420: false
org.kde.krdp:   RDPGFX_CAPVERSION_81 AVC: false YUV420: false
org.kde.krdp:   RDPGFX_CAPVERSION_10 AVC: false YUV420: false
org.kde.krdp:   RDPGFX_CAPVERSION_102 AVC: false YUV420: false
org.kde.krdp:   RDPGFX_CAPVERSION_103 AVC: false YUV420: false
org.kde.krdp:   RDPGFX_CAPVERSION_104 AVC: false YUV420: true
org.kde.krdp:   UNKNOWN_VERSION AVC: false YUV420: false
org.kde.krdp: Client does not support H.264 in YUV420 mode!
[19:04:38:497] [26458:26476] [INFO][com.freerdp.core] - ERRINFO_GRAPHICS_SUBSYSTEM_FAILED (0x0000112F):The server-side graphics subsystem is in an error state and unable to continue graphics encoding.
[19:04:38:497] [26458:26476] [ERROR][com.freerdp.core] - rdp_set_error_info:freerdp_set_last_error_ex ERRINFO_GRAPHICS_SUBSYSTEM_FAILED [0x0001112F]
[19:04:38:497] [26458:26476] [ERROR][com.freerdp.channels.rdpgfx.server] - context->CapsAdvertise failed with error 20
[19:04:38:497] [26458:26476] [ERROR][com.freerdp.channels.rdpgfx.server] - rdpgfx_recv_caps_advertise_pdu failed with error 20!
[19:04:38:497] [26458:26476] [ERROR][com.freerdp.channels.rdpgfx.server] - Error while parsing GFX cmdId: RDPGFX_CMDID_CAPSADVERTISE (0x0012)
[19:04:38:497] [26458:26476] [ERROR][com.freerdp.channels.rdpgfx.server] - rdpgfx_server_receive_pdu failed with error 20!
[19:04:38:497] [26458:26476] [ERROR][com.freerdp.channels.rdpgfx.server] - rdpgfx_server_handle_messages failed with error 20
[19:04:38:501] [26458:26466] [ERROR][com.freerdp.core.transport] - BIO_read returned a system error 0: Conseguido
org.kde.krdp: Unable to check file descriptor
org.kde.krdp: Closing session
org.kde.krdp: Closing Freedesktop Portal Session
```

The first error to appear in the log is "ERRINFO_GRAPHICS_SUBSYSTEM_FAILED (0x0000112F):The server-side graphics subsystem is in an error state and unable to continue graphics encoding.".

According to the README available in KRdp's GitLab (https://invent.kde.org/plasma/krdp), "[...] hardware encoding on NVidia hardware can not be used and software encoding will be used instead.", however it doesn't seem that software encoding is working, since a warning from KRdp says "Client does not support H.264 in YUV420 mode!"

STEPS TO REPRODUCE
1. Start KRdp using the command krdpserver -u user -p password
2. Try to connect with an RDP client (example used: Microsoft's Windows App, other alternatives also fail)

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Arch Linux (6.15.2-arch1-1)
KDE Plasma Version: 6.3.5
KDE Frameworks Version: 6.15.0
Qt Version: 6.9.1

ADDITIONAL INFORMATION
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 7 5700X 8-Core Processor
Memory: 15.5 GiB of RAM
Graphics Processor: NVIDIA GeForce RTX 3070