Bug 486081 - Frame swapping in WebRTC screensharing while moving the mouse
Summary: Frame swapping in WebRTC screensharing while moving the mouse
Status: RESOLVED UPSTREAM
Alias: None
Product: kwin
Classification: Plasma
Component: screencasting (show other bugs)
Version: 6.0.4
Platform: Arch Linux Linux
: NOR normal
Target Milestone: ---
Assignee: KWin default assignee
URL:
Keywords:
: 486578 489959 (view as bug list)
Depends on:
Blocks:
 
Reported: 2024-04-24 16:45 UTC by fililip
Modified: 2024-07-17 14:08 UTC (History)
9 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description fililip 2024-04-24 16:45:11 UTC
SUMMARY
Screensharing using browsers (Firefox, Chromium, as well as things like Electron) / WebRTC swaps frames on the stream when mouse movement is performed. However, OBS Studio and Spectacle work fine regardless if the pointer is being moved or not.

STEPS TO REPRODUCE
1. Go to a website like https://webrtc.github.io/samples/src/content/getusermedia/getdisplaymedia and share the entire desktop
2. Open an app that constantly updates in the background (like vkcube or VRRTest)
3. Look at the browser preview and move the cursor around

OBSERVED RESULT
The preview in the browser has weird frame pacing issues

EXPECTED RESULT
The preview in the browser displays frames in proper order, just like OBS and Spectacle do

SOFTWARE/OS VERSIONS
Operating System: Arch Linux 
KDE Plasma Version: 6.0.4
KDE Frameworks Version: 6.1.0
Qt Version: 6.7.0
Kernel Version: 6.8.4 (64-bit)
Graphics Platform: Wayland
Graphics Processor: AMD Radeon RX 6600 XT

ADDITIONAL INFORMATION
KWin 6.0.2 does not have this issue, this seems to be a regression in 6.0.3 or 6.0.4.
Additionally, if by any chance the cursor does not get shown in the preview after screencasting is turned on (happened to me once), it seems to work fine too.
Comment 1 Aleksander 2024-04-24 18:41:55 UTC
Can confirm.
Operating System: Arch Linux 
KDE Plasma Version: 6.0.3
KDE Frameworks Version: 6.0.0
Qt Version: 6.7.0
Kernel Version: 6.9.0-rc3-1-mainline (64-bit)
Graphics Platform: Wayland
Processors: 24 × AMD Ryzen 9 5900X 12-Core Processor
Memory: 31,3 GiB of RAM
Graphics Processor: AMD Radeon RX 7800 XT
Comment 2 fililip 2024-04-26 09:39:17 UTC
I've noticed one more thing.

Going to https://mozilla.github.io/webrtc-landing/gum_test.html and setting a high FPS value (like 60 or even 120) results in the "FPS now" metric severely dropping while moving the mouse.
Comment 3 sophisticatedbystander 2024-05-04 17:01:31 UTC
*** Bug 486578 has been marked as a duplicate of this bug. ***
Comment 4 phantomshift 2024-05-08 17:30:02 UTC
It appears this issue is specific to WebRTC (or any client checking the size of the buffer instead of the `SPA_CHUNK_FLAG_CORRUPTED` flag), since as of 6.0.3 (in this commit in particular https://invent.kde.org/plasma/kwin/-/commit/6000cca75640595385d4627735669be23119224f) kwin dropped the `spa_buffer->datas[0].chunk->size = 0;` assignment again, while as of writing WebRTC is still checking if the size of the chunk is 0 rather than the `SPA_CHUNK_FLAG_CORRUPTED` flag.
https://issues.webrtc.org/issues/338232699

As mentioned in their report, OBS already started taking this into account a year ago (https://github.com/obsproject/obs-studio/pull/8793/commits/949f0b5209d8c4c0f879a9d7df3211eb83c0945e), which explains why OBS continues to work perfectly fine whereas anything using WebRTC is experiencing the bug.
Comment 5 fililip 2024-05-08 18:22:20 UTC
Thanks a lot for finding relevant information! I'll try to patch kwin and report back.
Comment 6 fililip 2024-05-13 20:05:01 UTC
It looks like https://webrtc-review.googlesource.com/c/src/+/349881 has been merged.
Now we need to wait for Firefox and Chromium to make use of this WebRTC fix.
Comment 7 fililip 2024-05-21 15:57:05 UTC
This is now fixed in the latest Chromium build, you can check it out yourself by grabbing a build from https://download-chromium.appspot.com
Comment 8 phantomshift 2024-05-26 03:57:01 UTC
The fix is also merged in Firefox, expected to be included in the 128 release
https://bugzilla.mozilla.org/show_bug.cgi?id=1896575
Comment 9 Zamundaaa 2024-07-17 14:08:59 UTC
*** Bug 489959 has been marked as a duplicate of this bug. ***