H.265 (HEVC) Codec Support

[afgarcia86]

Is your feature request related to a problem? Please describe.
Currently, our application supports H.264 video codec, but we encounter limitations in handling high-resolution video files efficiently. H.265 (HEVC) offers superior compression efficiency over H.264, allowing for smaller file sizes and better quality at the same bitrate.

Describe the solution you'd like
Enable encoding and decoding capabilities for H.265 in the SDK.

Additional context
After speaking with @hiroshihorie on slack he suggested the following:

I've browsed the WebRTC code and I think we need to implement a encoder/decoder for that codec.
We can re-use the RTCVideoEncoderH264 . Since I think it shares much of the code.
I would probably make a RTCVideoEncoderH26X parent class and make 2 children for h246 and h265.

[hiroshihorie]

I'm hoping it's not too hard since VideoToolbox already supports it. (Unless we run into other issues 😅)

Also will probably need to implement backup/fallback-codec logic for subscribers that don't support HEVC.

I'm also interested in how it performs on a iPhone device. CPU usage / battery drain etc.

[pblazej]

There shouldn't be any significant difference in the interface, apart from codec initialization (setting magic cookies, extradata, etc.). Not sure if we care about HEVC with alpha.

[davidliu]

For reference, Shiguredo has H265 support patches for iOS:

https://github.com/shiguredo-webrtc-build/webrtc-build/blob/master/patches/h265.patch

https://github.com/shiguredo-webrtc-build/webrtc-build/blob/master/patches/h265_ios.patch

Looks like they're originally from Intel's libwebrtc fork here: open-webrtc-toolkit/owt-deps-webrtc@35a8ddd

That said, Intel also mentioned that they didn't merge it to upstream due to legal review here: https://groups.google.com/g/discuss-webrtc/c/NO2j-BhEPO0/m/bpWENb52BAAJ

We may need to check on this with dz/legal just in case.

[alinekborges]

From my understanding, Apple has h265/hvec support in AVFoundation classes. If those are used, there wouldn't be problems with licensing. Any ideas why this route can't be used?

[davidliu]

From my understanding, Apple has h265/hvec support in AVFoundation classes. If those are used, there wouldn't be problems with licensing. Any ideas why this route can't be used?

No idea. It might be that it can be used fine, but that's something for lawyers to determine. It sounds like it would be fine, but if Intel found something that stopped them from merging the changes, we should at least do the due diligence to make sure what stopped them won't be an issue for us.

[charlie-n-hill]

Just circling back to see if there’s been any update on exposing H.265 in the iOS SDK.

From our perspective, this should be similar to H.264 in terms of legal and technical surface area — we’re not looking to bundle any codecs or introduce custom encoding logic, just to expose what’s already natively available through AVFoundation and iOS.

If this isn’t on the current roadmap, would it be possible to jump on a quick call so our engineers can get pointed in the right direction to implement it ourselves? We’d love to contribute and make sure it aligns with LiveKit’s architecture.

[hiroshihorie]

Hi, i'm starting to implement this. Will post an update with progress😁

[charlie-n-hill]

@hiroshihorie 🙌 🚀

Any chance someone is working on the Android SDK as well?

[afgarcia86]

Not sure if this is related but it seems like someone else is working on this webrtc-sdk/webrtc#183

I see that the swift SDK version is built on a custom release of m125 but figured I would share incase it helps. 🙏

[charlie-n-hill]

@hiroshihorie will this include e2ee support?