Bug 507949

Summary:	RDP sessions that are not closed client side but stop receiving data cause OOM condition
Product:	[Plasma] KRdp	Reporter:	Cody Wilson <cody>
Component:	general	Assignee:	Unassigned bugs <unassigned-bugs-null>
Status:	REPORTED ---
Severity:	major	CC:	ahiemstra, jackyzy823, nate
Priority:	NOR	Keywords:	efficiency-and-performance
Version First Reported In:	6.4.3
Target Milestone:	---
Platform:	CachyOS
OS:	Linux
Latest Commit:		Version Fixed In:
Sentry Crash Report:
Attachments:	pmap of krdpserver process while experiencing the high memory event

Description Cody Wilson 2025-08-06 16:54:47 UTC

SUMMARY
RDP sessions that are not closed client side but stop receiving data cause OOM condition 

STEPS TO REPRODUCE
1. Connect to the kRDP server as normal 
2. Allow the client's operating system to sleep without closing the RDP client session

OBSERVED RESULT
RAM utilization on the system will slowly increase until the system becomes unresponsive, requiring either manual intervention or being killed by oomkiller and the like. 

EXPECTED RESULT
A timeout or buffer size monitor threshold is triggered when the server can not send data to the client after an appropriate length, closing the connection server-side. 

SOFTWARE/OS VERSIONS
Client Side: 
Thincast Remote Desktop Client
1.1.634 stable (tcc-stable-1.1.634)
OpenSSL
OpenSSL 1.1.1w 11 Sep 2023
OрenH264 2.6.0
CJSON: 1.7.16
FreeRDP: 3.16.1-devO (3.16.1)
WinPR: 3.16.1-devO (3.16.1)
Qt: 5.15.15

macOS: 15.6

Server Side:
Operating System: CachyOS Linux
KDE Plasma Version: 6.4.3
KDE Frameworks Version: 6.16.0
Qt Version: 6.9.1
Kernel Version: 6.16.0-5-cachyos (64-bit)
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 7 9800X3D 8-Core Processor
Memory: 64 GiB of RAM (60.4 GiB usable)
Graphics Processor 1: AMD Radeon RX 6800 XT
Graphics Processor 2: AMD Radeon Graphics

ADDITIONAL INFORMATION
I believe this to be caused by either krdpserver buffering video, or the network socket buffering the data to be sent to the client. I'm in the process of collecting additional data about this should it be valuable. 

I have not been able to test this with other clients as this particular client is the only one I currently have that supports h.264 enabled clients appropriately. I have however been able to reproduce the issue several times successfully. It is easier to reproduce this when the server side is producing complex video data to send over the wire. In my case, this is usually triggered when I forget to close an RDP client session before switching to the RDP server and playing a game. I imagine watching a video would also produce sufficiently complex data to be encoded and trigger the condition.

Comment 1 Cody Wilson 2025-08-12 22:35:46 UTC

Created attachment 184002 [details]
pmap of krdpserver process while experiencing the high memory event

Output of other memory diagnostic commands against krdpserver during a high memory event. 

Repro steps are identical to what was reported in the original bug submission. 
❯ ps aux | grep krdpserver | grep -v grep
belthes+   51830  0.0  2.7 57349244 1754824 ?    SLsl Aug07   6:17 /usr/bin/krdpserver
❯ grep -i vmrss /proc/51830/status
VmRSS:   1525944 kB
❯ grep -e '^Rss' /proc/51830/smaps | awk '{sum += $2} END {print sum, "kB"}'
1355268 kB
               total        used        free      shared  buff/cache   available
Mem:           61885       56668         543         154        5520        5216
Swap:          61884       56238        5646

After stopping the session  via the tray icon: 

               total        used        free      shared  buff/cache   available
Mem:           61885       52626        4579         179        5552        9259
Swap:          61884       53285        8599


After restarting krdpserver:
❯ ps aux | grep krdpserver | grep -v grep
belthes+   51830  0.0  0.4 57180280 274420 ?     SLsl Aug07   6:22 /usr/bin/krdpserver
❯ grep -i vmrss /proc/51830/status
VmRSS:    271936 kB
❯ grep -e '^Rss' /proc/51830/smaps | awk '{sum += $2} END {print sum, "kB"}'
271936 kB

Log output: 

❯ journalctl --user -f -u app-org.kde.krdpserver
Aug 12 17:43:50 fate krdpserver[51830]: QObject::killTimer: Timers cannot be stopped from another thread
Aug 12 17:43:50 fate krdpserver[51830]: QObject::startTimer: Timers cannot be started from another thread
Aug 12 18:29:32 fate krdpserver[51830]: org.kde.krdp: Portal session was closed!
Aug 12 18:29:33 fate krdpserver[51830]: [18:29:33:610] [51830:000497f9] [ERROR][com.freerdp.core.transport] - [transport_default_write]: BIO_should_retry returned an error: error:80000020:system library::Broken pipe
Aug 12 18:29:33 fate krdpserver[51830]: [18:29:33:610] [51830:000497f9] [ERROR][com.freerdp.core.peer] - [transport_default_write]: ERRCONNECT_CONNECT_TRANSPORT_FAILED [0x0002000D]
Aug 12 18:29:33 fate krdpserver[51830]: org.kde.krdp: Unable to check Virtual Channel Manager file descriptor, closing connection
Aug 12 18:29:33 fate krdpserver[51830]: org.kde.krdp: Closing session
Aug 12 18:29:33 fate krdpserver[51830]: [18:29:33:634] [51830:0000ca76] [ERROR][com.freerdp.core.transport] - [transport_default_write]: BIO_should_retry returned an error: error:0A00010F:SSL routines::bad length
Aug 12 18:29:33 fate krdpserver[51830]: org.kde.krdp: Closing Freedesktop Portal Session
Aug 12 18:29:33 fate krdpserver[51830]: QObject::~QObject: Timers cannot be stopped from another thread

After restarting krdpserver:

❯ free -m
               total        used        free      shared  buff/cache   available
Mem:           61885       12837       44389         179        5531       49047
Swap:          61884       12696       49188