515342 – Server fails to initialize hardware encoding on AMD hardware, falling back to CPU due to missing hw_frames_ctx

Bug 515342 - Server fails to initialize hardware encoding on AMD hardware, falling back to CPU due to missing hw_frames_ctx

Summary: Server fails to initialize hardware encoding on AMD hardware, falling back to...

Status:	REPORTED

Alias:	None

Product:	KRdp
Classification:	Plasma
Component:	general (other bugs)
Version First Reported In:	6.5.5
Platform:	CachyOS Linux

Importance:	NOR normal
Target Milestone:	---
Assignee:	Unassigned bugs

URL:
Keywords:

Depends on:
Blocks:

Reported:	2026-01-31 12:28 UTC by Zack
Modified:	2026-01-31 12:31 UTC (History)
CC List:	1 user (show)

See Also:
Latest Commit:
Version Fixed/Implemented In:
Sentry Crash Report:

Attachments
Add an attachment

Note You need to log in before you can comment on or make changes to this bug.

Description Zack 2026-01-31 12:28:03 UTC

SUMMARY
The KRDP server fails to utilize GPU hardware encoding on AMD hardware (Ryzen 5 5800H), defaulting to CPU-based libx264 encoding. The logs indicate a failure to initialize the hardware frames context (hw_frames_ctx), preventing the video stream from using the hardware-accelerated path.

STEPS TO REPRODUCE
Enable the KRDP server on a machine with AMD hardware encoding support (e.g., Ryzen 5 5800H).
Connect to the server using any of the following clients: KRDC (Linux), aRDP (Android), or Thincast (Windows).
Inspect the server logs using journalctl -t krdpserver.

OBSERVED RESULT The server identifies hardware capabilities but fails during filter creation, resulting in a fallback to software encoding regardless of the client used.

Relevant Log Output:
Jan 31 23:25:10 cachyos-server krdpserver[1840]: Session setup completed, start processing...
Jan 31 23:25:10 cachyos-server krdpserver[1840]: [23:25:10:221] [1840:0006f623] [ERROR][com.freerdp.core.transport] - [transport_read_layer]: BIO_read returned a system error 104: Connection reset by peer
Jan 31 23:25:10 cachyos-server krdpserver[1840]: [23:25:10:221] [1840:0006f623] [ERROR][com.freerdp.core.peer] - [transport_read_layer]: ERRCONNECT_CONNECT_TRANSPORT_FAILED [0x0002000D]
Jan 31 23:25:10 cachyos-server krdpserver[1840]: [23:25:10:221] [1840:00000730] [WARN][com.freerdp.core.rdp] - [rdp_send_deactivate_all][0x559f0a8f9de0]: rdpMcs::userId == 0, skip sending PDU_TYPE_DEACTIVATE_ALL
Jan 31 23:25:10 cachyos-server krdpserver[1840]: Unable to check file descriptor
Jan 31 23:25:10 cachyos-server krdpserver[1840]: [23:25:10:221] [1840:00000730] [ERROR][com.freerdp.core.transport] - [transport_default_write]: BIO_should_retry returned an error: error:80000020:system library::Broken pipe
Jan 31 23:25:10 cachyos-server krdpserver[1840]: Closing session
Jan 31 23:25:10 cachyos-server krdpserver[1840]: QDBusConnection: error: could not send message to service "org.freedesktop.portal.Desktop" path "/org/freedesktop/portal/desktop" interface "org.freedesktop.portal.RemoteDesktop" member "NotifyKeyboardKeycode": Marshalling failed: Invalid object path passed in arguments
Jan 31 23:25:10 cachyos-server krdpserver[1840]: QDBusConnection: error: could not send message to service "org.freedesktop.portal.Desktop" path "/org/freedesktop/portal/desktop" interface "org.freedesktop.portal.RemoteDesktop" member "NotifyKeyboardKeycode": Marshalling failed: Invalid object path passed in arguments
Jan 31 23:25:10 cachyos-server krdpserver[1840]: QDBusConnection: error: could not send message to service "org.freedesktop.portal.Desktop" path "/org/freedesktop/portal/desktop" interface "org.freedesktop.portal.RemoteDesktop" member "NotifyKeyboardKeycode": Marshalling failed: Invalid object path passed in arguments
Jan 31 23:25:10 cachyos-server krdpserver[1840]: QDBusConnection: error: could not send message to service "org.freedesktop.portal.Desktop" path "/org/freedesktop/portal/desktop" interface "org.freedesktop.portal.RemoteDesktop" member "NotifyKeyboardKeycode": Marshalling failed: Invalid object path passed in arguments
Jan 31 23:25:10 cachyos-server krdpserver[1840]: QDBusConnection: error: could not send message to service "org.freedesktop.portal.Desktop" path "/org/freedesktop/portal/desktop" interface "org.freedesktop.portal.RemoteDesktop" member "NotifyKeyboardKeycode": Marshalling failed: Invalid object path passed in arguments
Jan 31 23:25:10 cachyos-server krdpserver[1840]: QDBusConnection: error: could not send message to service "org.freedesktop.portal.Desktop" path "/org/freedesktop/portal/desktop" interface "org.freedesktop.portal.RemoteDesktop" member "NotifyKeyboardKeycode": Marshalling failed: Invalid object path passed in arguments
Jan 31 23:25:10 cachyos-server krdpserver[1840]: QDBusConnection: error: could not send message to service "org.freedesktop.portal.Desktop" path "/org/freedesktop/portal/desktop" interface "org.freedesktop.portal.RemoteDesktop" member "NotifyKeyboardKeycode": Marshalling failed: Invalid object path passed in arguments
Jan 31 23:25:10 cachyos-server krdpserver[1840]: QDBusConnection: error: could not send message to service "org.freedesktop.portal.Desktop" path "/org/freedesktop/portal/desktop" interface "org.freedesktop.portal.RemoteDesktop" member "NotifyKeyboardKeycode": Marshalling failed: Invalid object path passed in arguments
Jan 31 23:25:10 cachyos-server krdpserver[1840]: QDBusConnection: error: could not send message to service "org.freedesktop.portal.Desktop" path "" interface "org.freedesktop.portal.Session" member "Close": Object path cannot be empty
Jan 31 23:25:10 cachyos-server krdpserver[1840]: Closing Freedesktop Portal Session
Jan 31 23:25:10 cachyos-server krdpserver[1840]: Session setup completed, start processing...
Jan 31 23:25:13 cachyos-server krdpserver[1840]: New client connected:  Windows platform Windows NT
Jan 31 23:25:13 cachyos-server krdpserver[1840]: Attempting authenticating user with PAM
Jan 31 23:25:13 cachyos-server krdpserver[456473]: pam_unix(login:account): setuid failed: Operation not permitted
Jan 31 23:25:13 cachyos-server krdpserver[1840]: PAM authentication succeeded for user "zack"
Jan 31 23:25:13 cachyos-server krdpserver[1840]: Video stream initialized
Jan 31 23:25:13 cachyos-server krdpserver[1840]: Initializing Freedesktop Portal Session
Jan 31 23:25:13 cachyos-server krdpserver[1840]: Started Freedesktop Portal session
Jan 31 23:25:13 cachyos-server krdpserver[1840]: Received caps:
Jan 31 23:25:13 cachyos-server krdpserver[1840]:   RDPGFX_CAPVERSION_8 AVC: false YUV420: false
Jan 31 23:25:13 cachyos-server krdpserver[1840]:   RDPGFX_CAPVERSION_81 AVC: false YUV420: false
Jan 31 23:25:13 cachyos-server krdpserver[1840]:   RDPGFX_CAPVERSION_10 AVC: true YUV420: false
Jan 31 23:25:13 cachyos-server krdpserver[1840]:   RDPGFX_CAPVERSION_102 AVC: true YUV420: false
Jan 31 23:25:13 cachyos-server krdpserver[1840]:   RDPGFX_CAPVERSION_103 AVC: true YUV420: false
Jan 31 23:25:13 cachyos-server krdpserver[1840]:   RDPGFX_CAPVERSION_104 AVC: true YUV420: true
Jan 31 23:25:13 cachyos-server krdpserver[1840]:   RDPGFX_CAPVERSION_105 AVC: true YUV420: true
Jan 31 23:25:13 cachyos-server krdpserver[1840]:   RDPGFX_CAPVERSION_106 AVC: true YUV420: true
Jan 31 23:25:13 cachyos-server krdpserver[1840]: Selected caps: RDPGFX_CAPVERSION_106
Jan 31 23:25:13 cachyos-server krdpserver[1840]: [in @ 0x7f7210380b80] Setting BufferSourceContext.pix_fmt to a HW format requires hw_frames_ctx to be non-NULL!
Jan 31 23:25:13 cachyos-server krdpserver[1840]: Failed to create the buffer filter
Jan 31 23:25:13 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] -qscale is ignored, -crf is recommended.
Jan 31 23:25:13 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
Jan 31 23:25:13 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] profile Constrained Baseline, level 6.1, 4:2:0, 8-bit
Jan 31 23:25:13 cachyos-server krdpserver[1840]: eglChooseConfig returned this many configs: 1
Jan 31 23:25:16 cachyos-server krdpserver[1840]: suspend frame ack
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [23:25:22:787] [1840:0006f625] [WARN][com.freerdp.api] - [peer_recv_data_pdu]: disconnect provider ultimatum sent to peer, closing connection
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [23:25:22:787] [1840:0006f625] [ERROR][com.freerdp.core.transport] - [transport_check_fds]: transport_check_fds: transport->ReceiveCallback() - STATE_RUN_QUIT_SESSION [-2]
Jan 31 23:25:22 cachyos-server krdpserver[1840]: Unable to check file descriptor
Jan 31 23:25:22 cachyos-server krdpserver[1840]: Closing session
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [23:25:22:796] [1840:00000730] [ERROR][com.freerdp.core.transport] - [transport_default_write]: BIO_should_retry returned an error: error:80000068:system library::Connection reset by peer
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [23:25:22:796] [1840:00000730] [ERROR][com.freerdp.core.peer] - [transport_default_write]: ERRCONNECT_CONNECT_TRANSPORT_FAILED [0x0002000D]
Jan 31 23:25:22 cachyos-server krdpserver[1840]: Closing Freedesktop Portal Session
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] frame I:2     Avg QP:17.50  size: 76022
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] frame P:187   Avg QP:17.47  size:  1283
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] mb I  I16..4: 100.0%  0.0%  0.0%
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] mb P  I16..4:  0.7%  0.0%  0.0%  P16..4:  2.0%  0.0%  0.0%  0.0%  0.0%    skip:97.4%
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] coded y,uvDC,uvAC intra: 10.2% 11.8% 4.5% inter: 0.2% 1.6% 0.0%
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] i16 v,h,dc,p: 53% 23% 17%  7%
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] i8c dc,h,v,p: 79% 14%  6%  1%
Jan 31 23:25:22 cachyos-server krdpserver[1840]: [libx264 @ 0x7f72112f7d80] kb/s:16587.85

EXPECTED RESULT 
KRDP should correctly initialize the hw_frames_ctx for the FFmpeg/VA-API pipeline, allowing the server to utilize GPU-based hardware encoding for the RDP stream across all supported clients.

SOFTWARE/OS VERSIONS 
Windows: Windows 11 Pro (Tested with Thincast client) Android: Tested with aRDP client Linux/KDE Plasma: CachyOS (Tested with KRDC client) KDE Plasma Version: 6.5.5 KDE Frameworks Version: 6.22.0 Qt Version: 6.10.1

ADDITIONAL INFORMATION
Kernel Version: 6.18.7-2-cachyos
KRDP Version: 6.5.5-1.1
Graphics Platform: Wayland
Hardware: Beelink Mini PC (Ryzen 5 5800H with Radeon Graphics)
Notes: The behavior is consistent across Linux, Android, and Windows clients. Hardware encoding (VA-API) is confirmed functional on this machine in other applications like Jellyfin and FFmpeg.

Comment 1 Zack 2026-01-31 12:31:51 UTC

vainfo - output from Beelink Mini PC (Ryzen 5 5800H with Radeon Graphics)
Trying display: wayland
vainfo: VA-API version: 1.22 (libva 2.22.0)
vainfo: Driver version: Mesa Gallium driver 25.3.4-arch1.2 for AMD Radeon Graphics (radeonsi, renoir, ACO, DRM 3.64, 6.18.7-2-cachyos)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSlice
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc