VP9: switch packetizer to non-flexible mode (F=0)#868
Open
farit2000 wants to merge 2 commits intoalgesten:mainfrom
Open
VP9: switch packetizer to non-flexible mode (F=0)#868farit2000 wants to merge 2 commits intoalgesten:mainfrom
farit2000 wants to merge 2 commits intoalgesten:mainfrom
Conversation
Safari and older Chrome versions drop inter-frames when the VP9 RTP packetizer uses flexible mode (F=1). This rewrites the VP9 packetizer to use non-flexible mode (F=0) with proper layer indices and tl0picidx, matching what browsers expect. Changes: - Switch from F=1 (flexible) to F=0 (non-flexible) packetization - Add layer indices (L=1) with TID=0, SID=0, tl0picidx to every packet - Add scalability structure (V=1) on keyframe first-packets (GOF) - Detect keyframes from VP9 bitstream to set P flag correctly - Increment tl0picidx on every frame (all TID=0 per spec Section 6.3) - Add detect_vp9_keyframe_bitstream() for raw bitstream inspection - VP9HEADER_SIZE increased to 8 (max header with SS on keyframes) The depacketizer is unchanged — it already handles both modes correctly.
Owner
|
@farit2000 Great find! A little cargo fmt and we merge! |
Owner
|
@farit2000 thinking about this. Safari limitations might guide our default to @davibe thoughts? |
Contributor
Author
Makes sense — we could keep both and add a config flag. In our SFU we always use F=0 because Chrome/Safari both handle it well, but having flexible mode as an option is reasonable. I can refactor this so Vp9Packetizer has a mode field (Flexible / NonFlexible) if you'd like. |
Owner
|
@farit2000 perfect! Let's keep |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We've been running str0m as a WebRTC SFU in rtp_mode and hit an issue: Safari drops all VP9 inter-frames when the packetizer uses flexible mode (F=1). After debugging, we found that Safari's VP9 decoder expects non-flexible mode with
tl0picidxfor temporal reference tracking.Switching to F=0 (non-flexible mode) fixes Safari and is also what other SFUs like LiveKit and mediasoup use in practice.
What changed
tl0picidxon every packetdetect_vp9_keyframe_bitstream()to correctly set the P bit from raw VP9 datatl0picidxincrements on every frame (all TID=0) and wraps at 255→0, per the specVP9HEADER_SIZEbumped to 8 to account for the larger header on keyframesThe depacketizer already handles both modes, so no changes needed there.
Testing