Bug 476186

> Screen recording quality in Spectacle is bad to the point of being unusable for anything more than casual use.

Is this the case for all codecs pr just x264? I personally thought VP9 quality was generally OK, but it might vary based on hardware and usecase.

Are you using a screen with a scale factor other than 100%? I'm wondering if that could be a source of the bug. I noticed that rectangle recording has abysmally bad quality when recording my 200% scale screen (my other screen has 100% scale). However, if I do a screen recording of the 200% scale screen (I select the screen instead of making a rectangle), then the quality is fine.

I am in fact (150%), but the bad quality looks like compression artifacts, so it shouldn't be related to scaling/resolution, but rather a codec issue. It also happens when recording the full screen.

@Noah yes, with fine detail like lots of text (especially colored) it falls apart. Looking at the printed config, `rc_end_usage` is 0 which means VBR, which is the same problem as libx264: you are telling the decoder to target a specific bitrate regardless of picture complexity, so if things get too complex, the whole thing becomes a blockfest since it's physically impossible to encode in that few bits. Screen recording *really* needs constant quality mode to be useful for offline recording (not streaming where you have constraints).

Do you have this issue with the beta version of Spectacle? If you do, can you post a short video demonstrating the bad quality? Also, do you you get better or worse quality with larger resolution videos or different recording types (e.g., screen recording vs region recording)? We've fixed the issue with high DPI screens being scaled down, so we can rule out that issue for now, although I'm not sure if the latest beta release has that change.

I can't reproduce this with today's git master, FWIW. The resource usage while recording goes bananas, but the final video has reasonable quality and a very small file size.

Created attachment 165924 [details]
screencast illustrating poor recording quality

I see this on the latest beta (kpipewire 5.92.3, spectacle 24.01.95) with VP9 recording. As requested I'm attaching a video, which is a short recording I made of Crosswords [1] to illustrate a bug with that program.

These are clearly encoding artifacts stemming from the very low CRF used by default (31) for VP9. I was able to mostly fix the issue by changing the default quality to 20, which seems like a much more reasonable default.

[1] https://gitlab.gnome.org/jrb/crosswords

(In reply to Adam Fontenot from comment #6)
> Created attachment 165924 [details]
> screencast illustrating poor recording quality
> 
> I see this on the latest beta (kpipewire 5.92.3, spectacle 24.01.95) with
> VP9 recording. As requested I'm attaching a video, which is a short
> recording I made of Crosswords [1] to illustrate a bug with that program.
> 
> These are clearly encoding artifacts stemming from the very low CRF used by
> default (31) for VP9. I was able to mostly fix the issue by changing the
> default quality to 20, which seems like a much more reasonable default.
> 
> [1] https://gitlab.gnome.org/jrb/crosswords

From all the reading of encoder documentation I've done in the past, 31 is not low, it's average and actually looks decent when doing fullscreen recording. In my own tests, there was not a significant difference in quality between 20 CRF and 31 CRF when doing fullscreen recording either. For me, fullscreen is 1080p@60Hz or 4K@60Hz. For some reason, recording a smaller region has a strong negative effect on recording quality, so we may need to scale quality with region size inversely.

(In reply to Noah Davis from comment #7)
> From all the reading of encoder documentation I've done in the past, 31 is
> not low, it's average and actually looks decent when doing fullscreen
> recording. In my own tests, there was not a significant difference in
> quality between 20 CRF and 31 CRF when doing fullscreen recording either.
> For me, fullscreen is 1080p@60Hz or 4K@60Hz. For some reason, recording a
> smaller region has a strong negative effect on recording quality, so we may
> need to scale quality with region size inversely.

I think the issue is more likely to lie elsewhere. There are very few actual frames in my screencast, which is an issue not only for seeking, but also for the efficiency of the encoder.

This is probably not as noticeable with full motion 24 fps or 60 fps video, but in the case of my screencast, there are only 90 actual frames in the file, because KPipeWire seems to be dropping duplicate frames rather than sending them to the encoder. In a 15.35 second clip with a nominal framerate of 60 fps, you expect to see 921 frames. So ffmpeg has only encoded 1 in 10 frames, due to the missing duplicates.

The low number of frames means that the quality of the video depends on the quality of the individual frames much more. At a relatively low quality CRF like 31, the encoder is probably cheating too much on these frames. This is combined with the fact that video encoders are optimized for full motion video, not screencasts, so their quality metrics cheat heavily on flat fields (regions of nearly solid color) because they can be smoothed out and encoded at a very small size. That's clearly visible in my screencast.

The recommendation for CRF 31 seems to come from Google, and I'm skeptical of their figures. They consistently underestimate bitrates needed for reasonable quality, as is readily apparent if you watch their encodes for YouTube. Interestingly, they recommend a CRF of 15 for 4K video, so VP9's CRF metric does not seem particularly consistent. For what it's worth, their example encode of "Tears of Steel" (the open source Blender film) at CRF 31 and 1080p strikes me as rather poor, and that has the advantage of being a two-pass encode. [1]

I'm not trying to be overly debate-y here; there's not going to be a single one-size-fits-all value for all resolutions and all use cases. The correct approach is probably to allow setting the CRF directly in the Spectacle settings and passing the value to KPipeWire.

[1] https://developers.google.com/media/vp9/settings/vod/

Created attachment 165948 [details]
illustration of crf 20 vs 31 in a fullscreen recording

As a quick illustration, I've taken two fullscreen (60 fps 1080p) screencasts with comparable details (I scrolled up and down this page in my browser for a bit), at both CRF 31 (the default) and CRF 20 (with my patch).

The result illustrates the way that VP9 cheats on flat fields quite well. This area of the screen shown in the image is basically static in the video, and contains large regions of a single color. The encoder cheats a lot here, with a result that is visibly much worse with CRF 31.

For common screencast use cases, like recording bugs and features, this is a worst case scenario of sorts, since you'd like to see relatively crisp looking text and icons.

Wouldn't it be possible to just add some slider to Spectacle to allow to specify the bitrate? After all this is possible for the jpeg quality as well. I've also just run into this with a higher DPI UI setting, and the resulting recording is both extremely blurry and so full of artifacts that it's hard to use it for anything. On top of things it's not even that small, but if the initial quality is already so bad this also prevents re-encoding for a more reasonable file size. It therefore seems to me like it would be a better idea to err on the side of too high quality rather than too low.

My discussion above was about VP9, I've been investigating quality issues with H.264 today. Rather, the lack thereof, because the results are pretty okay for me.

`m_avCodecContext->global_quality` is equivalent to setting the `-q` option on the ffmpeg command line. Unfortunately, as is clear from the documentation [1], libx264 ignores the global quality setting *entirely*. So on most ffmpeg versions it's equivalent to the default of CRF 23, which is pretty tolerable quality for screen recordings.

As evidence of this, when recording the screen with libx264, Spectacle prints the following to the terminal:

[libx264 @ 0x77274430b640] -qscale is ignored, -crf is recommended.

As a result the file I get is encoded with `rc=crf`, since that's the default. I'm not sure how Hector got `rc=abr` in their files, my understanding is that this is only applicable to two pass encodes, not for streaming encoding as is used here.

At any rate, what's clear is that the current H.264 code needs to be replaced. At present the quality settings are a complete noop.

See also: https://trac.ffmpeg.org/ticket/3238

[1] https://ffmpeg.org/ffmpeg-codecs.html#Options-35

I hope this question is still relevant.

I also notice that the quality when recording in h264 is indeed terrible. It’s acceptable for personal use, but if you want to record video for documentation, it looks awful. Overall, I understand that it's done this way to ensure quick performance and general functionality.

I'll attach some examples of window captures recorded in h264 and vp9:
https://drive.google.com/drive/folders/1Qo5bKJvvQLNpFmeZhx_-7B4CfpI5hgL9?usp=sharing

You can see that vp9 records much better, but my laptop is fairly old and starts to lag on vp9, resulting in jerky screen-captured videos. I’ve also included a screenshot showing CPU utilization during video encoding.

Some information about my system:
- **Operating System:** openSUSE Tumbleweed 20241025
- **KDE Plasma Version:** 6.2.2
- **KDE Frameworks Version:** 6.7.0
- **Qt Version:** 6.8.0
- **Kernel Version:** 6.11.5-1-default (64-bit)
- **Graphics Platform:** Wayland
- **Processors:** 8 × Intel® Core™ i5-8350U CPU @ 1.70GHz
- **Memory:** 23.3 GiB of RAM
- **Graphics Processor:** Mesa Intel® UHD Graphics 620
- **Manufacturer:** LENOVO
- **Product Name:** 20L8A03KMZ
- **System Version:** ThinkPad T480s

Perhaps we should try detecting hardware acceleration support, and if available, use it.

For example, in openSUSE Tumbleweed, ffmpeg is compiled with support for all popular hardware acceleration technologies.

ffmpeg -hide_banner -hwaccels
Hardware acceleration methods:
vdpau
cuda
vaapi
qsv
drm
vulkan


ffmpeg -hide_banner -encoders 2>&1| grep 264
 V....D libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
 V....D libx264rgb           libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)
 V....D libopenh264          OpenH264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
 V....D h264_amf             AMD AMF H.264 Encoder (codec h264)
 V....D h264_nvenc           NVIDIA NVENC H.264 encoder (codec h264)
 V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)
 V....D h264_vaapi           H.264/AVC (VAAPI) (codec h264)


So, we only need to detect if ffmpeg supports it, and if it does, whether there’s a graphics card available for the required encoder. In the case of VAAPI/Intel, we could perhaps call `vainfo`. For AMD or NVIDIA, unfortunately, I don’t have experience or equipment to investigate further.

Then, we could write a switch case for each scenario, ensuring stable and fast performance on systems without HW acceleration and good quality without performance loss on systems with HW acceleration. In the settings, we could note whether hardware acceleration is detected, such as VAAPI/Intel, etc., or indicate if it’s unavailable and that video quality may be poor.

What do you think?