View Source Membrane.MP4.Muxer.CMAF (Membrane MP4 plugin v0.22.1)
Puts a payloaded stream into Common Media Application Format, an MP4-based container commonly used in adaptive streaming over HTTP.
The element supports up to 2 input tracks that can result in one output:
- audio input -> audio output
- video input -> video output
- video input + audio input -> muxed audio and video output
media-objects
Media objects
Accordingly to the spec, the Membrane.MP4.Muxer.CMAF
is able to assemble the following media entities:
headers
- media initialization object. Contains information necessary to play the media segments (when binary concatenated with a media segment it creates a valid MP4 file). The media header is sent as an output pad's stream format.segments
- media data that when combined with media headers can be played independently to other segments. Segments due to their nature (video decoding) must start with a key frame (doesn't apply to audio-only tracks) which is a main driver when collecting video samples and deciding when a segment should be createdchunks
- fragmented media data that when binary concatenated should make up a regular segment. Chunks no longer have the requirement to start with a key frame (except for the first chunk that starts a new segment) and their main goal is to reduce the latency of creating the media segments (chunks can be delivered to a client faster so it can start playing them before a full segment gets assembled)
segment-chunk-metadata
Segment/Chunk metadata
Each outgoing buffer containing a segment/chunk contains the following fields in the buffer's metadata:
duration
- the duration of the underlying segment/chunkindependent?
- tells if a segment/chunk can be independently played (starts with a keyframe), it is always true for segmentslast_chunk?
- tells if the underlying chunk is the last one of the segment currently being assembled, for segments this flag is always true and has no real meaning
segment-creation
Segment creation
A segment gets created based on the duration of currently collected media samples and
:segment_min_duration
options passed when initializing Membrane.MP4.Muxer.CMAF
.
It is expected that the segment will not be shorter than the specified minimum duration value and the aim is to end the segment as soon as the next key frames arrives that will become a part of a new segment.
If a user prefers to have segments of unified durations then he needs to take into consideration the incoming keyframes interval. For instance, if a keyframe interval is 2 seconds and the goal is to have 6 seconds segments then the minimum segment duration should be lower than 6 seconds (the key frame at the 6-second mark will force the segment finalization).
note
NoteIf a key frame comes at irregular intervals, the segment could be much longer than expected as after the minimum duration muxer will always look for a key frame to finish the segment.
forcing-segment-creation
Forcing segment creation
It may happen that one may need to create a segment before it reaches the minimum duration (for purposes such as fast AD insertion).
To instruct the muxer to finalize the current segment as soon as possible one can send Membrane.MP4.Muxer.CMAF.RequestMediaFinalization
event on the :output
pad. The event will enforce the muxer to end the current segment as soon as possible (usually on the nearest key frame).
After the segment gets generated, the muxer will go back to its normal behaviour of creating segments.
chunk-creation
Chunk creation
As previously mentioned, chunks are not required to start with a key frame except for a first chunk of a new segment. Those are once again created based on the duration of the collected samples but this time the process needs to be smarter as we can't allow the chunk to significantly exceed their target duration.
Exceeding the chunk's target duration can cause unrecoverable player stalls e.g. when playing LL-HLS on Safari, same goes if the chunk's duration is lower than 85% of the target duration when the chunk is the not last of its parent segment (Safari again). This is why proper duration MUST get collected. The limitation does not apply to the last chunk of a given regular segment.
The behaviour of creating chunk is as follows:
if the duration of the regular segment currently being assembled is lower than the minimum then try to collect chunk with its given
target
duration value no matter whatif the duration of the regular segment currently being assembled is greater than the minimum then try to finish the chunk as fast as possible (without exceeding the chunk's target) when encountering a key frame. When such chunk gets created it also means that its parent segment is also done.
Note that once the Membrane.MP4.Muxer.CMAF
is in a phase of finalizing a regular segment, more than one
chunk could get created until a key frame is encountered.
important-for-video
Important for video
:chunk_target_duration
should be chosen with special care and appropriately for its use case. It is unnecessary to create chunks when the target use case is not live streaming.The chunk duration usability may depend on its use case e.g. for live streaming there is very little value for having duration higher than 1s/2s, also having really short duration may add a communication overhead for a client (a necessity for downloading many small chunks).
note-1
NoteIf a stream contains non-key frames (like H264 P or B frames), they should be marked with a
mp4_payload: %{key_frame?: false}
metadata entry.
element-options
Element options
Passed via struct Membrane.MP4.Muxer.CMAF.t/0
segment_min_duration
Membrane.Time.t()
Default value:
2000000000
Minimum duration of a regular media segment.When the minimum duration is reached the muxer will try to finalize the segment as soon as a new key frame arrives which will start a new segment.
chunk_target_duration
Membrane.Time.t() | nil
Default value:
nil
pads
Pads
input
:input
Accepted formats:
Membrane.MP4.Payload
Direction: | :input |
Availability: | :on_request |
Mode: | :pull |
Demand mode: | :manual |
Demand unit: | :buffers |
output
:output
Accepted formats:
Membrane.CMAF.Track
Direction: | :output |
Availability: | :always |
Mode: | :pull |
Demand mode: | :manual |
Link to this section Summary
Types
Struct containing options for Membrane.MP4.Muxer.CMAF
Link to this section Types
@type t() :: %Membrane.MP4.Muxer.CMAF{ chunk_target_duration: Membrane.Time.t() | nil, segment_min_duration: Membrane.Time.t() }
Struct containing options for Membrane.MP4.Muxer.CMAF
Link to this section Functions
@spec options() :: keyword()
Returns description of options available for this module