Skip to content

feat(recorder): add MP4 muxer#8

Open
RealKeyboardWarrior wants to merge 8 commits intomainfrom
video/muxer
Open

feat(recorder): add MP4 muxer#8
RealKeyboardWarrior wants to merge 8 commits intomainfrom
video/muxer

Conversation

@RealKeyboardWarrior
Copy link
Owner

@RealKeyboardWarrior RealKeyboardWarrior commented Sep 6, 2023

This PR adds a MP4 muxer to store the video files. The Zoom h264 stream is encoded using AnnexB so its recoded to using AVCC so it is MP4 compatible.

TODO:

  • The sample builder is currently dropping packets. It seems like some RTP packets are just not arriving, this can be noticed in the logs with the lacking of certain sequence numbers. I don't know what the culprit is yet but I haven't figured out the PLI/NACK/FLI/SR/RR RTCP logic which would be used to request a retransmission of RTP packets so that is my main suspect. Given that this all occurs over a WebSocket, we shouldn't even need these if the server automatically takes care of this. Perhaps these missing RTP packets are automatically retransmitted with PT 110. Either way, this is causing fuckups on decoding, because we can't assemble certain keyframes. Another potential problems with dropping samples is that the sample builder will still attempt to build a sample/frame, this may cause it to mess up the FU-A depacketizers because it aggregates packets.
  • Add flushMoof to joy4 fragmented mp4 (this is because our fragmented mp4 player only flushes a moof when a new keyframe has arrived.

Fixes #2

@RealKeyboardWarrior RealKeyboardWarrior changed the base branch from master to main September 6, 2023 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

screenshare: missing pps (& sps?)

1 participant