Created attachment 185734 [details] Missing damage events, framebuffer is corrupt SUMMARY When using the ScreenCast/RemoteDesktop portal, buffers can be "dropped" if pw_stream_events.process() does not finish processing quick enough. This is problematic when using SPA_META_VideoDamage, as some damage events are never propagated, which breaks damage tracking. STEPS TO REPRODUCE 1. Use ScreenCast/RemoteDesktop Portal & get a PipeWire remote. 2. Enable damage tracking with SPA_META_VideoDamage. 3. Start processing the stream and track damage. OBSERVED RESULT In some cases, frames are dropped and there are missing SPA_META_VideoDamage events. This will result in an incorrect/corrupt image. EXPECTED RESULT I should not get a corrupt image, and damage events should include everything that was changed since the last buffer was processed. SOFTWARE/OS VERSIONS Linux/KDE Plasma: Fedora 42 KDE Plasma Version: 6.4.5 KDE Frameworks Version: 6.18.0 Qt Version: 6.9.2 xdg-desktop-portal-kde Version: 6.4.5 TigerVNC w0vncserver: dae6ab944af151a512efec477356b98d9146f583 ADDITIONAL INFORMATION I could not find any information/specification that says how this should be handled. Is it mine or the compositors responsibility to handle the missed damage events? Or is there some other mechanism to avoid this? I noticed this issue when testing TigerVNC's Portal remote desktop called w0vncserver. I would (rarely) see artifacts indicative of incorrect damage tracking. To confirm that this is the case, I added a 500ms sleep in the pw_stream_events.process() handler. As long as there were no damage events more than every 500ms, there were no issues. But, if I were to for example maximize and minimize a window within 500ms of each other, the damage tracking would always break. This was also confirmed by adding debug logging in the pw_stream_events.process() handler, where I could see that the damage reported did not match what was happening on screen. The current workaround is to detect dropped frames [1] by looking at the spa_buffer's header->seq. If there was a sequence missed, we can no longer rely on damage tracking and have to mark the entire display as damaged. [1] https://github.com/TigerVNC/tigervnc/pull/2001
>I could not find any information/specification that says how this should be handled. Is it mine or the compositors responsibility to handle the missed damage events? Or is there some other mechanism to avoid this? I've spent some time reading around and I don't think there's anything we can do at a KDE level. My understanding is that what TigerVNC now does is correct. There is a SPA_META_HEADER_FLAG_DISCONT which might be better. Though pipewire documentation could definitely do with being a lot clearer. In our RDP server we have some backpressure handling where we track the latency and adjust the requested framerate from the server If you have the opportunity to test on another platform and confirm if it behaves differently please reopen
Thanks for looking into this. > If you have the opportunity to test on another platform and confirm if it behaves differently please reopen Tested this on GNOME, where they do damage reporting when using virtual displays. I got the same behaviour as on KDE (damage events get dropped/lost), and the workaround we have is needed there as well. Had a look at SPA_META_HEADER_FLAG_DISCONT and could not see that it is used anywhere really, and confirmed it is never set on KDE/GNOME.