Bug 476187 - OpenH264 codec support
Summary: OpenH264 codec support
Status: RESOLVED FIXED
Alias: None
Product: KPipeWire
Classification: Frameworks and Libraries
Component: general (show other bugs)
Version: unspecified
Platform: Fedora RPMs Linux
: NOR wishlist
Target Milestone: ---
Assignee: Plasma Bugs List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-10-28 08:20 UTC by Hector Martin
Modified: 2024-07-20 10:31 UTC (History)
10 users (show)

See Also:
Latest Commit:
Version Fixed In: 6.2.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Hector Martin 2023-10-28 08:20:03 UTC
KPipeWire currently only supports software h.264 encoding via libx264. libx264 cannot be safely distributed by default in countries with restrictive software patent legislation. OpenH264 is an open source h.264 encoder distributed by Cisco, making use of their H.264 license (https://www.openh264.org/). It allows distributions like Fedora to support h.264 without much fuss, and in particular for the Fedora Asahi Remix we have hooked it up to be installed automatically, so H.264 works out of the box in most apps.

Since KPipeWire uses libavcodec, and libavcodec has openh264 integration, it should be pretty easy to generalize the code to not explicitly hardcode libx264 but rather allow openh264 as well.

The codec name in this case is `libopenh264`. Quality controls have to be validated to make sure they work well on both codecs (see bug 476186 for the story on libx264, I don't know off the top of my head what the appropriate quality controls are for openh264 but I can investigate). Where libx264 is available, it should be preferred, since x264 is widely considered to be the best h.264 encoder in the world and particularly well optimized.

In the future there will be more h.264 encoder options, e.g. for Fedora Asahi we plan to expose the internal hardware encoder as a h264_v4l2m2m implementation. This is also supported by ffmpeg. So it might be worth setting things up so that, at the very least as a fallback, "any codec that can encode h.264" is selected. I believe this should be possible with ffmpeg by requesting the `h264` codec and letting it pick an appropriate encoder. That will make h.264 encoding at least work (if perhaps without ideal quality control) on any machine that has a usable codec, without explicitly handling them all in KPipeWire.
Comment 1 Eileen Yoon 2023-11-28 13:28:43 UTC
(In reply to Hector Martin from comment #0)

> at the very least as a fallback, "any codec that can encode h.264" is selected.
I think we should be a bit more cautious since this is lossy encoding we are talking about. I found out about a week ago that openh264 only supports Constrained Baseline Profile, which is the lowest class intended for low-cost mobile applications like, a decade ago. And I'm not even talking about fancy alpha channels or high bitdepths, Baseline Profile very notably lacks B-frames. It's misleading to claim openh264 is a drop-in h264 replacement, not in this day and age. I'd be pressed to find out my screen recording was terribly compressed and full of artifacts (especially for text). The encoder's constraints should at least be clear to the end-user and other available codecs should be favored.

https://github.com/cisco/openh264/issues/2949#issuecomment-388658110

Also hwaccel is part of the same native/in-house ffmpeg encoder/decoder pipeline (it passes it off to hwaccel after parsing headers), so once we do our part IIUC avcodec_find_decoder() should default to hwaccel else it will fallback to native sw. So the generic avcodec init call sounds like a good idea, and no one should have no worry about V4L2M2M and whatnot.
Comment 2 Neal Gompa 2023-11-28 22:03:37 UTC
OpenH264 supports constrained baseline, main, and high profiles. I know this because I implemented it for OBS Studio: https://github.com/obsproject/obs-studio/pull/8529
Comment 3 Eileen Yoon 2023-11-28 23:19:17 UTC
(In reply to Neal Gompa from comment #2)
> OpenH264 supports constrained baseline, main, and high profiles
https://github.com/cisco/openh264#encoder-features
> Encoder Features
> Constrained Baseline Profile up to Level 5.2 (Max frame size is 36864 macro-blocks)
Comment 4 Hector Martin 2023-11-29 00:47:16 UTC
Considering it also only lists CBP for *decoding* and yet it can obviously decode higher profiles (otherwise it would be useless for the web), I think that feature list is clearly not exhaustive.
Comment 5 Bug Janitor Service 2024-07-06 16:42:57 UTC
A possibly relevant merge request was started @ https://invent.kde.org/plasma/kpipewire/-/merge_requests/152
Comment 6 Fabian Vogt 2024-07-16 11:31:31 UTC
Git commit e17793a3b023f26411001093bb2d5934adf715c7 by Fabian Vogt.
Committed on 16/07/2024 at 11:24.
Pushed by fvogt into branch 'master'.

Add encoder using libopenh264

On some distributions, libopenh264 is the only encoder available OOTB.
Add support for it and use it as fallback.

M  +1    -0    src/CMakeLists.txt
A  +106  -0    src/libopenh264encoder.cpp     [License: LGPL(3+eV) LGPL(v3.0) LGPL(v2.1)]
A  +28   -0    src/libopenh264encoder_p.h     [License: LGPL(3+eV) LGPL(v3.0) LGPL(v2.1)]
M  +1    -1    src/pipewirebaseencodedstream.cpp
M  +11   -0    src/pipewireproduce.cpp

https://invent.kde.org/plasma/kpipewire/-/commit/e17793a3b023f26411001093bb2d5934adf715c7
Comment 7 Fabian Vogt 2024-07-20 10:31:50 UTC
Git commit bdb1152e85fdfb7456c6aa65cf41fe0f375fe30b by Fabian Vogt.
Committed on 17/07/2024 at 17:55.
Pushed by fvogt into branch 'Plasma/6.1'.

Add encoder using libopenh264

On some distributions, libopenh264 is the only encoder available OOTB.
Add support for it and use it as fallback.
(cherry picked from commit e17793a3b023f26411001093bb2d5934adf715c7)

M  +1    -0    src/CMakeLists.txt
A  +106  -0    src/libopenh264encoder.cpp     [License: LGPL(3+eV) LGPL(v3.0) LGPL(v2.1)]
A  +28   -0    src/libopenh264encoder_p.h     [License: LGPL(3+eV) LGPL(v3.0) LGPL(v2.1)]
M  +1    -1    src/pipewirebaseencodedstream.cpp
M  +11   -0    src/pipewireproduce.cpp

https://invent.kde.org/plasma/kpipewire/-/commit/bdb1152e85fdfb7456c6aa65cf41fe0f375fe30b