Bug 486081

Summary:	Frame swapping in WebRTC screensharing while moving the mouse
Product:	[Plasma] kwin	Reporter:	fililip <team>
Component:	screencasting	Assignee:	KWin default assignee <kwin-bugs-null>
Status:	RESOLVED UPSTREAM
Severity:	normal	CC:	aleksander, b2c, linx.system.adm, miranda, phantomshift, s+kde, scooterdelta, sophisticatedbystander, xaver.hugl
Priority:	NOR
Version:	6.0.4
Target Milestone:	---
Platform:	Arch Linux
OS:	Linux
Latest Commit:		Version Fixed In:
Sentry Crash Report:

Description fililip 2024-04-24 16:45:11 UTC

SUMMARY
Screensharing using browsers (Firefox, Chromium, as well as things like Electron) / WebRTC swaps frames on the stream when mouse movement is performed. However, OBS Studio and Spectacle work fine regardless if the pointer is being moved or not.

STEPS TO REPRODUCE
1. Go to a website like https://webrtc.github.io/samples/src/content/getusermedia/getdisplaymedia and share the entire desktop
2. Open an app that constantly updates in the background (like vkcube or VRRTest)
3. Look at the browser preview and move the cursor around

OBSERVED RESULT
The preview in the browser has weird frame pacing issues

EXPECTED RESULT
The preview in the browser displays frames in proper order, just like OBS and Spectacle do

SOFTWARE/OS VERSIONS
Operating System: Arch Linux 
KDE Plasma Version: 6.0.4
KDE Frameworks Version: 6.1.0
Qt Version: 6.7.0
Kernel Version: 6.8.4 (64-bit)
Graphics Platform: Wayland
Graphics Processor: AMD Radeon RX 6600 XT

ADDITIONAL INFORMATION
KWin 6.0.2 does not have this issue, this seems to be a regression in 6.0.3 or 6.0.4.
Additionally, if by any chance the cursor does not get shown in the preview after screencasting is turned on (happened to me once), it seems to work fine too.

Comment 1 Aleksander 2024-04-24 18:41:55 UTC

Can confirm.
Operating System: Arch Linux 
KDE Plasma Version: 6.0.3
KDE Frameworks Version: 6.0.0
Qt Version: 6.7.0
Kernel Version: 6.9.0-rc3-1-mainline (64-bit)
Graphics Platform: Wayland
Processors: 24 × AMD Ryzen 9 5900X 12-Core Processor
Memory: 31,3 GiB of RAM
Graphics Processor: AMD Radeon RX 7800 XT

Comment 2 fililip 2024-04-26 09:39:17 UTC

I've noticed one more thing.

Going to https://mozilla.github.io/webrtc-landing/gum_test.html and setting a high FPS value (like 60 or even 120) results in the "FPS now" metric severely dropping while moving the mouse.

Comment 3 sophisticatedbystander 2024-05-04 17:01:31 UTC

*** Bug 486578 has been marked as a duplicate of this bug. ***

Comment 4 phantomshift 2024-05-08 17:30:02 UTC

It appears this issue is specific to WebRTC (or any client checking the size of the buffer instead of the `SPA_CHUNK_FLAG_CORRUPTED` flag), since as of 6.0.3 (in this commit in particular https://invent.kde.org/plasma/kwin/-/commit/6000cca75640595385d4627735669be23119224f) kwin dropped the `spa_buffer->datas[0].chunk->size = 0;` assignment again, while as of writing WebRTC is still checking if the size of the chunk is 0 rather than the `SPA_CHUNK_FLAG_CORRUPTED` flag.
https://issues.webrtc.org/issues/338232699

As mentioned in their report, OBS already started taking this into account a year ago (https://github.com/obsproject/obs-studio/pull/8793/commits/949f0b5209d8c4c0f879a9d7df3211eb83c0945e), which explains why OBS continues to work perfectly fine whereas anything using WebRTC is experiencing the bug.

Comment 5 fililip 2024-05-08 18:22:20 UTC

Thanks a lot for finding relevant information! I'll try to patch kwin and report back.

Comment 6 fililip 2024-05-13 20:05:01 UTC

It looks like https://webrtc-review.googlesource.com/c/src/+/349881 has been merged.
Now we need to wait for Firefox and Chromium to make use of this WebRTC fix.

Comment 7 fililip 2024-05-21 15:57:05 UTC

This is now fixed in the latest Chromium build, you can check it out yourself by grabbing a build from https://download-chromium.appspot.com

Comment 8 phantomshift 2024-05-26 03:57:01 UTC

The fix is also merged in Firefox, expected to be included in the 128 release
https://bugzilla.mozilla.org/show_bug.cgi?id=1896575

Comment 9 Zamundaaa 2024-07-17 14:08:59 UTC

*** Bug 489959 has been marked as a duplicate of this bug. ***