Tracks audio playback progress for handling interruptions.
When you have custom playback logic or expect audio to be played with delays or at different speeds, use this tracker to inform the model about the actual playback state. This is important for proper interruption handling.
Usage
tracker = PlaybackTracker.new()
|> PlaybackTracker.set_audio_format(:pcm16)
|> PlaybackTracker.on_play_bytes("item_123", 0, audio_data)
state = PlaybackTracker.get_state(tracker)
# %{current_item_id: "item_123", current_item_content_index: 0, elapsed_ms: 500.0}Why This Matters
The model generates audio much faster than realtime playback speed. If there's an interruption, the model needs to know how much audio has actually been played to the user. In low-latency scenarios, assuming immediate playback is fine. In scenarios like phone calls or remote interactions, use this tracker to provide accurate playback state.
Summary
Functions
Get the current playback state.
Create a new playback tracker.
Reset the tracker when audio playback has been interrupted.
Record audio bytes that have been played.
Record milliseconds of audio that have been played.
Set the audio format for duration calculations.
Types
@type t() :: %Codex.Realtime.PlaybackTracker{ current_item: {String.t(), non_neg_integer()} | nil, elapsed_ms: float() | nil, format: Codex.Realtime.Audio.format() | nil }
Functions
@spec get_state(t()) :: %{ current_item_id: String.t() | nil, current_item_content_index: non_neg_integer() | nil, elapsed_ms: float() | nil }
Get the current playback state.
Returns a map with the current item ID, content index, and elapsed milliseconds. If no audio has been played, all values are nil.
Examples
tracker = PlaybackTracker.new()
|> PlaybackTracker.on_play_ms("item_123", 0, 500.0)
PlaybackTracker.get_state(tracker)
# %{current_item_id: "item_123", current_item_content_index: 0, elapsed_ms: 500.0}@spec new() :: %Codex.Realtime.PlaybackTracker{
current_item: nil,
elapsed_ms: nil,
format: nil
}
Create a new playback tracker.
Examples
iex> tracker = Codex.Realtime.PlaybackTracker.new()
iex> tracker.format
nilReset the tracker when audio playback has been interrupted.
This is called by the model when an interruption occurs.
Examples
tracker = PlaybackTracker.new()
|> PlaybackTracker.on_play_ms("item_123", 0, 500.0)
|> PlaybackTracker.on_interrupted()
# current_item and elapsed_ms are now nil@spec on_play_bytes(t(), String.t(), non_neg_integer(), binary()) :: t()
Record audio bytes that have been played.
This calculates the duration in milliseconds based on the audio format
and calls on_play_ms/4.
Parameters
tracker- The playback trackeritem_id- The item ID of the audio being playedcontent_index- The index of the audio content initem.contentbytes- The audio bytes that have been fully played
Examples
tracker = PlaybackTracker.new()
|> PlaybackTracker.set_audio_format(:pcm16)
|> PlaybackTracker.on_play_bytes("item_123", 0, audio_data)@spec on_play_ms(t(), String.t(), non_neg_integer(), float()) :: t()
Record milliseconds of audio that have been played.
If the item/content changes, the elapsed time is reset. Otherwise, the elapsed time is accumulated.
Parameters
tracker- The playback trackeritem_id- The item ID of the audio being playedcontent_index- The index of the audio content initem.contentms- The number of milliseconds of audio that have been played
Examples
tracker = PlaybackTracker.new()
|> PlaybackTracker.on_play_ms("item_123", 0, 500.0)
|> PlaybackTracker.on_play_ms("item_123", 0, 250.0)
# elapsed_ms is now 750.0@spec set_audio_format(t(), Codex.Realtime.Audio.format()) :: t()
Set the audio format for duration calculations.
This is called by the model when the audio format is configured.
Examples
tracker = PlaybackTracker.new()
|> PlaybackTracker.set_audio_format(:pcm16)