Membrane.RTP.SessionBin (Membrane RTP plugin v0.9.0) View Source

Bin handling one RTP session, that may consist of multiple incoming and outgoing RTP streams.

Incoming streams

Incoming RTP streams can be connected via :rtp_input pads. As each pad can provide multiple RTP streams, they are distinguished basing on SSRC. Once a new stream is received, bin sends new_stream_notification_t/0 notification, meaning the parent should link Pad.ref(:output, ssrc) pad to consuming components. The stream is then depayloaded and forwarded via said pad.

Outgoing streams

To create an RTP stream, the source stream needs to be connected via Pad.ref(:input, ssrc) pad and the sink - via Pad.ref(:rtp_output, ssrc). At least one of :encoding or :payload_type options of :rtp_output pad must be provided too.

Payloaders and depayloaders

Payloaders are Membrane elements that transform stream so that it can be put into RTP packets, while depayloaders work the other way round. Different codecs require different payloaders and depayloaders.

By default SessionBin will neither payload nor depayload incoming/outgoing streams, to do so a payloader/depayloader needs to be passed via Pad.ref(:input, ssrc) and Pad.ref(:output, ssrc) pads options.

Payloading/Depayloading is necessary if we need to somehow transform the streams. If SessionBins main role is to route packets then depayloading and payloading processes are redundant.

Payloaders and depayloaders can be found in membrane_rtp_X_plugin packages, where X stands for codec name. It's enough when such a plugin is added to dependencies. To determine which payloader/depayloader to use, one can use Membrane.RTP.PayloadFormatResolver which given an encoding name should resolve to proper payloader/depayloader modules (if those previously have been registered via mentioned plugins).

For payloading and depayloading, SessionBin uses respective bins Membrane.RTP.PayloaderBin and Membrane.RTP.DepayloaderBin which will be spawned once payloader/depayloader are passed explicitly via pads' options.

Important note

Payloaders and depayloaders are mostly needed when working with external media sources (in different formats than RTP). For applications such as an SFU it is not needed to either payload or depayload the RTP stream as we are always dealing with RTP format. In such a case, SessionBin will receive payloaded packets and work as a simple proxy just forwarding the packets (and decrypting them if necessary). Therefore it is possible to specify in newly added pads if payloaders/depayloaders should be used for the certain stream.

RTCP

RTCP packets for inbound stream can be provided either in-band or via a separate rtp_input pad instance. Corresponding receiver report packets will be sent back through rtcp_output with the same id as rtp_input for the RTP stream. RTCP for outbound stream is not yet supported.

Bin options

Passed via struct Membrane.RTP.SessionBin.t/0

  • fmt_mapping

    %{RTP.payload_type_t() => {RTP.encoding_name_t(), RTP.clock_rate_t()}}

    Default value: %{}
    Mapping of the custom payload types ( > 95)

  • rtcp_report_interval

    Membrane.Time.t() | nil

    Default value: nil
    Interval between sending subseqent RTCP receiver reports.

  • receiver_ssrc_generator

    (local_ssrcs :: [pos_integer], remote_ssrcs :: [pos_integer] -> ssrc :: pos_integer)

    Default value: &Membrane.RTP.SessionBin.generate_receiver_ssrc/2
    Function generating receiver SSRCs. Default one generates random SSRC that is not in local_ssrcs nor remote_ssrcs.

  • secure?

    boolean

    Default value: false
    Specifies whether to use SRTP. Requires adding srtp dependency to work.

  • srtp_policies

    [ExLibSRTP.Policy.t()]

    Default value: []
    List of SRTP policies to use for decrypting packets. Used only when secure? is set to true. See ExLibSRTP.Policy.t/0 for details.

  • receiver_srtp_policies

    [ExLibSRTP.Policy.t()] | nil

    Default value: nil
    List of SRTP policies to use for encrypting receiver reports and other receiver RTCP packets. Used only when secure? is set to true. Defaults to the value of srtp_policies. See ExLibSRTP.Policy.t/0 for details.

Pads

:rtp_input

Availability:on_request
CapsMembrane.RemoteStream, restrictions:<br />&nbsp;&nbsp;type: :packetized,<br />&nbsp;&nbsp;content_format: one_of([nil, Membrane.RTP])
Demand unit:buffers
Direction:input
Mode:pull
Name:rtp_input

:input

Availability:on_request
Caps:any
Demand unit:buffers
Direction:input
Mode:pull
Name:input

Options:

  • payloader

    module() | nil

    Default value: nil
    Payloader's module that should be used for a media stream flowing through the pad. If set to nil then the payloading process gets skipped.

:rtcp_output

Availability:on_request
CapsMembrane.RemoteStream, restrictions:<br />&nbsp;&nbsp;type: :packetized,<br />&nbsp;&nbsp;content_format: Membrane.RTCP
Demand unit:buffers
Direction:output
Mode:pull
Name:rtcp_output

:rtp_output

Availability:on_request
CapsMembrane.RemoteStream, restrictions:<br />&nbsp;&nbsp;type: :packetized,<br />&nbsp;&nbsp;content_format: Membrane.RTP
Demand unit:buffers
Direction:output
Mode:pull
Name:rtp_output

Options:

  • payload_type

    RTP.payload_type_t() | nil

    Default value: nil
    Payload type of output stream. If not provided, determined from :encoding.

  • encoding

    RTP.encoding_name_t() | nil

    Default value: nil
    Encoding name of output stream. If not provided, determined from :payload_type.

  • clock_rate

    integer() | nil

    Default value: nil
    Clock rate to use. If not provided, determined from :payload_type.

:output

Availability:on_request
Caps:any
Demand unit:buffers
Direction:output
Mode:pull
Name:output

Options:

  • depayloader

    module() | nil

    Default value: nil
    Depayloader's module that should be used for an outgoing media stream flowing through the pad. If set to nil then the depayloading process gets skipped.

  • clock_rate

    integer() | nil

    Default value: nil
    Clock rate to use. If not provided, determined from fmt_mapping or defaults registered by proper plugins i.e. Membrane.RTP.X.Plugin where X is the name of codec corresponding to encoding.

  • extensions

    [extension_t()]

    Default value: []
    List of extensions. Currently :vad is only supported.

    • :vad will turn on Voice Activity Detection mechanism firing appropriate notifications when needed. Should be set only for audio tracks. For more information refer to Membrane.RTP.VAD module documentation. Extensions are applied in the same order as passed to the pad options.
  • packet_filters

    [packet_filter_t()]

    Default value: []
    A list of filter elements that will be attached to the packets flow (added inside Membrane.RTP.StreamReceiveBin). In case of SRTP filters are placed before the Decryptor. The order of provided elements is important as the filters are applied in FIFO order. A filter can be responsible e.g. for dropping silent audio packets when encountered VAD extension data in the packet header.

  • rtcp_fir_interval

    Membrane.Time.t() | nil

    Default value: 1000000000
    Interval between sending subseqent RTCP Full Intra Request packets.

Link to this section Summary

Types

A module that will be spawned and linked just before a newly created :output pad representing a single RTP stream.

Options for pad :input

Options for pad :output

A definition of a child that will be responsible for arbitrary packet filtering inside Membrane.RTP.StreamReceiveBin. Each filter should have just a single input and output pad named accordingly.

Options for pad :rtp_output

t()

Struct containing options for Membrane.RTP.SessionBin

Functions

Returns pads descriptions for Membrane.RTP.SessionBin

Returns description of options available for this module

Link to this section Types

Specs

extension_t() ::
  {extension_name :: atom(),
   extension_config :: Membrane.ParentSpec.child_spec_t()}

A module that will be spawned and linked just before a newly created :output pad representing a single RTP stream.

Given extension config must be a valid Membrane.Filter.

An extension will be spawned inside the bin under {extension_name :: atom(), ssrc} name.

Currently supported extensions are:

Example usage

{:vad, %Mebrane.RTP.VAD{time_window: 1_000_000}}

Specs

input_pad_opts_t() :: [{:payloader, module() | nil}]

Options for pad :input

Link to this type

new_stream_notification_t()

View Source

Specs

new_stream_notification_t() ::
  Membrane.RTP.SSRCRouter.new_stream_notification_t()

Specs

output_pad_opts_t() :: [
  depayloader: module() | nil,
  clock_rate: integer() | nil,
  extensions: [extension_t()],
  packet_filters: [packet_filter_t()],
  rtcp_fir_interval: Membrane.Time.t() | nil
]

Options for pad :output

Specs

A definition of a child that will be responsible for arbitrary packet filtering inside Membrane.RTP.StreamReceiveBin. Each filter should have just a single input and output pad named accordingly.

A filter can be responsible e.g. for dropping silent audio packets when encountered VAD extension data in the packet header.

Link to this type

rtp_output_pad_opts_t()

View Source

Specs

rtp_output_pad_opts_t() :: [
  payload_type: Membrane.RTP.payload_type_t() | nil,
  encoding: Membrane.RTP.encoding_name_t() | nil,
  clock_rate: integer() | nil
]

Options for pad :rtp_output

Specs

t() :: %Membrane.RTP.SessionBin{
  fmt_mapping: %{
    required(Membrane.RTP.payload_type_t()) =>
      {Membrane.RTP.encoding_name_t(), Membrane.RTP.clock_rate_t()}
  },
  receiver_srtp_policies: [ExLibSRTP.Policy.t()] | nil,
  receiver_ssrc_generator:
    (local_ssrcs :: [pos_integer()], remote_ssrcs :: [pos_integer()] ->
       ssrc :: pos_integer()),
  rtcp_report_interval: Membrane.Time.t() | nil,
  secure?: boolean(),
  srtp_policies: [ExLibSRTP.Policy.t()]
}

Struct containing options for Membrane.RTP.SessionBin

Link to this section Functions

Specs

membrane_pads() :: [{Membrane.Pad.name_t(), Membrane.Pad.description_t()}]

Returns pads descriptions for Membrane.RTP.SessionBin

Specs

options() :: keyword()

Returns description of options available for this module