Bug 488896 - Huge frame drops when recording WebM/VP9 with spectacle
Summary: Huge frame drops when recording WebM/VP9 with spectacle
Status: RESOLVED FIXED
Alias: None
Product: KPipeWire
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Arch Linux Linux
: HI normal
Target Milestone: ---
Assignee: Plasma Bugs List
URL:
Keywords:
: 485410 492283 498691 498692 (view as bug list)
Depends on:
Blocks:
 
Reported: 2024-06-21 14:20 UTC by Yujiro Hanma
Modified: 2025-03-27 22:42 UTC (History)
14 users (show)

See Also:
Latest Commit:
Version Fixed In: 6.4.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Yujiro Hanma 2024-06-21 14:20:43 UTC
***
If you're not sure this is actually a bug, instead post about it at https://discuss.kde.org

If you're reporting a crash, attach a backtrace with debug symbols; see https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports
***

SUMMARY
When recording a video with spectacle on my intel i5-1235U laptop with onboard Intel Iris Xe graphics, there are huge frame drops and progressively more and more frames are dropped.

STEPS TO REPRODUCE
1. Record a video with spectacle

OBSERVED RESULT
There are high number of dropped frames

EXPECTED RESULT
There should be no dropped frames.

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: Arch Linux
(available in About System)
KDE Plasma Version: 6.1.0
KDE Frameworks Version: 6.3.0
Qt Version: 6.7.1


ADDITIONAL INFORMATION
Looking at the logs, it seems like spectacle is using vaapi for encoding the video, and the journal is spammed with these messages:
kpipewire_vaapi_logging: VAAPI: entrypoint 8 of profile 14 is not supported by the device "/dev/dri/renderD128"
kpipewire_vaapi_logging: VAAPI: entrypoint 6 of profile 14 is not supported by the device "/dev/dri/renderD128"
kpipewire_record_logging: Filter queue is full, dropping frame 2484... (hundreds of messages of dropping frames)
Comment 1 Yujiro Hanma 2024-06-21 14:32:41 UTC
I forgot to add this, but I am using the WebM/VP9 codec for encoding.
Comment 2 Bogdan Dragoiu 2024-06-21 18:52:54 UTC
Same exact problem with the same log messages after the Plasma 6.1 update.
Work for a couple of seconds then a lot of the filter queue messages show up:
```
kpipewire_record_logging: Filter queue is full, dropping frame <frame number>
```

Operating System: Fedora Linux 40
KDE Plasma Version: 6.1.0
KDE Frameworks Version: 6.3.0
Qt Version: 6.7.1
Kernel Version: 6.9.4-200.fc40.x86_64 (64-bit)
Graphics Platform: Wayland
Processors: 24 × AMD Ryzen 9 3900X 12-Core Processor
Memory: 125.7 GiB of RAM
Graphics Processor: AMD Radeon RX 6600
Comment 3 Noah Davis 2024-06-21 18:57:05 UTC
What it means is that for some reason, the system cannot encode fast enough to keep the frames from piling up in memory. When frames start piling up without being dropped, you may quickly run out of memory and have your whole system freeze. The reason why this happens could simply be that your computer is too slow or has too many other things competing for system resources.
Comment 4 Yujiro Hanma 2024-06-21 19:16:33 UTC
(In reply to Noah Davis from comment #3)
> What it means is that for some reason, the system cannot encode fast enough
> to keep the frames from piling up in memory. When frames start piling up
> without being dropped, you may quickly run out of memory and have your whole
> system freeze. The reason why this happens could simply be that your
> computer is too slow or has too many other things competing for system
> resources.

I don't think this is the reason. I tested it on a fresh boot and the same thing occurred. Also it used to work perfectly before.
Comment 5 Bogdan Dragoiu 2024-06-21 19:52:05 UTC
Here's a recording: https://imgur.com/a/f09s7Cg
It seems even initially it starts at over 1.5 core, then ramps up to over 3 cores and I hear the fan increasing speed.

I am on 12c/24t 128GB DDR4 so the overall system is not starved for resources.

Spectacle is likely impacted by changes upstream as I got spectacle-24.05.0-2 from Fedora repos on 2024-06-08, but my update to Plasma 6.1 was almost 2 weeks later on 2024-06-20.

I use Spectacle almost daily to record small clips of devlogs to share with friends, that is why I noticed instantly.
Comment 6 Noah Davis 2024-06-21 20:40:16 UTC
It is possible for there to be a bug. As you said, this probably is an upstream issue since Spectacle does not do the frame dropping itself. The issue is probably in KWin, KPipeWire or in something that either of those rely on.
Comment 7 Yujiro Hanma 2024-06-22 04:34:32 UTC
(In reply to Noah Davis from comment #6)
> It is possible for there to be a bug. As you said, this probably is an
> upstream issue since Spectacle does not do the frame dropping itself. The
> issue is probably in KWin, KPipeWire or in something that either of those
> rely on.

I did some more testing, and it seems only the libvpx and libvpx-vp9 encoders face this issue.
libx264, h264_vaapi and their baseline profiles seem to work fine.
Comment 8 Yujiro Hanma 2024-06-23 07:12:40 UTC
I think we should perhaps move this to kpipewire, this does not seem like a bug of spectacle.
Comment 9 Victor Ryzhykh 2024-06-28 12:49:39 UTC
(In reply to Noah Davis from comment #6)
> It is possible for there to be a bug. As you said, this probably is an
> upstream issue since Spectacle does not do the frame dropping itself. The
> issue is probably in KWin, KPipeWire or in something that either of those
> rely on.

Problems with writing to spectacle began on all distributions that updated Pipewire to a version >= 1.1.80
Comment 10 Yujiro Hanma 2024-06-29 13:38:24 UTC
I can confirm the issue still persists with pipewire 1.2.0
Comment 11 Yujiro Hanma 2024-07-20 03:31:29 UTC
I do not know if this is relevant, but kpipewire always seems to drop frames starting around the 3200th frame number. It also drops every 16th or 17th frame. For example look at this

ideapad spectacle[3957]: kpipewire_record_logging: Filter queue is full, dropping frame 6633
ideapad spectacle[3957]: kpipewire_record_logging: Filter queue is full, dropping frame 6650
ideapad spectacle[3957]: kpipewire_record_logging: Filter queue is full, dropping frame 6666
ideapad spectacle[3957]: kpipewire_record_logging: Filter queue is full, dropping frame 6683
ideapad spectacle[3957]: kpipewire_record_logging: Filter queue is full, dropping frame 6700 

Also please note that this only happens on webm/vp9 codec.
Comment 12 Alejandro M. 2024-07-29 04:02:51 UTC
Same problem here in openSUSE Tumbleweed with codecs installed by opi. Using libKPipeWire6 6.1.3-1.1 and pipewire 1.2.1-1.1. I also cannot save video using MP4 and get no error logs, but I think that may not be relevant in this case.
Comment 13 ase1590 2024-11-12 18:00:06 UTC
Spectacle Version: 24.08.3
KDE Plasma Version: 6.23
KDE Frameworks Version: 6.8.0
Qtversion: 6.8.0
Kernel Version: 6.11.6-arch1-1 (64-bit)
Graphics Platform: Wayland
Processors: Ryzen 5 3600X
Memory: 31.3 GiB of RAM
Graphics Processor: AMD Radeon RX 7900 XTX


WebM  still drops frames after a few seconds when recording with a rectangular frame at 966x721
This results in the following repeated error with kpipewire:

`kpipewire_record_logging: Filter queue is full, dropping frame`
Comment 14 Avraham Hollander 2024-12-08 16:24:14 UTC
Same here.
Operating System: Gentoo Linux 2.17
KDE Plasma Version: 6.2.4
KDE Frameworks Version: 6.8.0
Qt Version: 6.8.1
Kernel Version: 6.12.3-20R5-sched-ext (64-bit)
Graphics Platform: Wayland
Processors: 8 × Intel® Core™ i5-10310U CPU @ 1.70GHz
Memory: 7.5 GiB of RAM
Graphics Processor: Mesa Intel® UHD Graphics
Comment 15 Dmitry Nezhevenko 2025-01-22 10:38:14 UTC
Same here. And only with VP9. For me spectacle can record up to ~35 seconds of video. And starts dropping frames immediately.

I think that system has enough power to handle it. i7-12700, 64GB RAM.

```
The cached device pixel ratio value was stale on window update.  Please file a QTBUG which explains how to reproduce.
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so 
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
kpipewire_vaapi_logging: VAAPI: profile 14 is not supported by the device "/dev/dri/renderD128"
kpipewire_vaapi_logging: VAAPI: profile 14 is not supported by the device "/dev/dri/renderD128"
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so 
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so 
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
[libvpx-vp9 @ 0x7fd9208b7c80] Application has requested 20 threads. Using a thread count greater than 16 is not recommended.
[libvpx-vp9 @ 0x7fd9208b7c80] v1.15.0
kpipewire_dmabuf_logging: eglChooseConfig returned this many configs: 1
kpipewire_record_logging: Filter queue is full, dropping frame 25217
kpipewire_record_logging: Filter queue is full, dropping frame 25450
kpipewire_record_logging: Filter queue is full, dropping frame 25717
kpipewire_record_logging: Filter queue is full, dropping frame 25734
kpipewire_record_logging: Filter queue is full, dropping frame 25934
kpipewire_record_logging: Filter queue is full, dropping frame 26067
kpipewire_record_logging: Filter queue is full, dropping frame 26217
kpipewire_record_logging: Filter queue is full, dropping frame 26234
```
h.264 works fine.
Comment 16 herms123 2025-02-19 09:12:55 UTC
This was working for me until recently.
Can confirm I have the same issue when using the webm/vp9 codec for screen recording.
h264 is working fine.

Spectacle Version: 24.12.2
KDE Plasma Version: 6.3.0
KDE Frameworks Version: 6.10.0
Qtversion: 6.8.2
Kernel Version: 6.13.2-arch1-1 (64-bit)
Graphics Platform: Wayland
Processors:  i5-1135G7
Memory: 24GB of RAM
Graphics Processor: Intel® Iris® Xe Graphics

kpipewire_vaapi_logging: VAAPI: Intel iHD driver for Intel(R) Gen Graphics - 24.4.4 () in use for device "/dev/dri/renderD128"
kpipewire_vaapi_logging: VAAPI: entrypoint 6 of profile 14 is not supported by the device "/dev/dri/renderD128"
kpipewire_vaapi_logging: VAAPI: entrypoint 8 of profile 14 is not supported by the device "/dev/dri/renderD128"
kpipewire_vaapi_logging: VAAPI: entrypoint 6 of profile 14 is not supported by the device "/dev/dri/renderD128"
kpipewire_vaapi_logging: VAAPI: entrypoint 8 of profile 14 is not supported by the device "/dev/dri/renderD128"
qt.multimedia.ffmpeg: Using Qt multimedia with FFmpeg version n7.1 GPL version 3 or later
qt.multimedia.ffmpeg: Available HW decoding frameworks:
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
qt.multimedia.ffmpeg:      vaapi
qt.multimedia.ffmpeg:      vulkan
qt.multimedia.ffmpeg: Available HW encoding frameworks:
qt.multimedia.ffmpeg:      vaapi
qt.multimedia.ffmpeg:      vulkan
kpipewire_vaapi_logging: VAAPI: entrypoint 6 of profile 14 is not supported by the device "/dev/dri/renderD128"
kpipewire_vaapi_logging: VAAPI: entrypoint 8 of profile 14 is not supported by the device "/dev/dri/renderD128"
kpipewire_vaapi_logging: VAAPI: entrypoint 6 of profile 14 is not supported by the device "/dev/dri/renderD128"
kpipewire_vaapi_logging: VAAPI: entrypoint 8 of profile 14 is not supported by the device "/dev/dri/renderD128"
[libvpx-vp9 @ 0x7e8d20468bc0] v1.15.0
kpipewire_dmabuf_logging: eglChooseConfig returned this many configs: 1
kpipewire_record_logging: Filter queue is full, dropping frame 36700
kpipewire_record_logging: Filter queue is full, dropping frame 36717
Comment 17 Nate Graham 2025-02-19 21:13:06 UTC
Quite possibly because of Bug 492844.
Comment 18 Nate Graham 2025-02-19 21:13:11 UTC
*** Bug 498692 has been marked as a duplicate of this bug. ***
Comment 19 Nate Graham 2025-02-19 21:14:10 UTC
*** Bug 498691 has been marked as a duplicate of this bug. ***
Comment 20 Nate Graham 2025-03-07 14:41:14 UTC
*** Bug 477788 has been marked as a duplicate of this bug. ***
Comment 21 Nate Graham 2025-03-07 14:52:38 UTC
*** Bug 492283 has been marked as a duplicate of this bug. ***
Comment 22 Nate Graham 2025-03-07 14:54:47 UTC
*** Bug 485410 has been marked as a duplicate of this bug. ***
Comment 23 Bug Janitor Service 2025-03-20 13:32:02 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/kpipewire/-/merge_requests/194
Comment 24 Arjen Hiemstra 2025-03-24 21:16:51 UTC
Git commit 52911b7060fc3341c4ab4b308be1e47f985e8576 by Arjen Hiemstra.
Committed on 21/03/2025 at 09:44.
Pushed by ahiemstra into branch 'master'.

Change Encoder::applyEncodingPreference() to buildEncodingOptions()

As it turns out, FFmpeg does something to the pointers here that causes
the dictionary to never be properly used. This means we were never
applying the encoding options, as avcodec_open() was passed a nullptr
for options.

Fix this by changing the function to return an AVDictionary* instead of
trying to modify a passed-in pointer. This results in a proper dict
being returned, that can then be passed to avcodec_open().

The main result of this is that we now properly apply the encoding
options for VP9 encoding, which means we can now encode VP9 at realtime
speeds instead of it massively lagging behind.

M  +5    -1    src/encoder.cpp
M  +1    -1    src/encoder_p.h
M  +0    -7    src/gifencoder.cpp
M  +0    -1    src/gifencoder_p.h
M  +5    -5    src/h264vaapiencoder.cpp
M  +1    -1    src/h264vaapiencoder_p.h
M  +5    -5    src/libopenh264encoder.cpp
M  +1    -1    src/libopenh264encoder_p.h
M  +6    -4    src/libvpxencoder.cpp
M  +1    -1    src/libvpxencoder_p.h
M  +6    -4    src/libvpxvp9encoder.cpp
M  +1    -1    src/libvpxvp9encoder_p.h
M  +0    -1    src/libwebpencoder_p.h
M  +6    -5    src/libx264encoder.cpp
M  +1    -1    src/libx264encoder_p.h

https://invent.kde.org/plasma/kpipewire/-/commit/52911b7060fc3341c4ab4b308be1e47f985e8576