Orchestrates STT → Workflow → TTS voice pipelines.

A voice pipeline provides an opinionated three-step flow for voice agents:

1. Transcribe - Convert audio input to text using a speech-to-text model
2. Process - Run the text through a workflow to generate a response
3. Synthesize - Convert the response text to audio using a text-to-speech model

Workflows

A workflow is any module that implements a run/2 function that takes the workflow struct and input text, returning an enumerable of response text chunks.

For multi-turn conversations with StreamedAudioInput, workflows can optionally implement on_start/1 to generate an initial greeting.

Example

# Define a simple workflow
defmodule EchoWorkflow do
  defstruct []

  def run(_workflow, text) do
    ["You said: #{text}"]
  end

  def on_start(_workflow) do
    ["Hello! I'm an echo bot. Say something!"]
  end
end

# Create and run the pipeline
workflow = %EchoWorkflow{}
pipeline = Pipeline.new(workflow: workflow)

audio = AudioInput.new(audio_bytes)
{:ok, result} = Pipeline.run(pipeline, audio)

# Stream the results
result
|> Result.stream()
|> Enum.each(fn event ->
  case event do
    %{type: :voice_stream_event_audio, data: data} ->
      play_audio(data)

    %{type: :voice_stream_event_lifecycle, event: :turn_ended} ->
      IO.puts("Turn complete!")

    %{type: :voice_stream_event_lifecycle, event: :session_ended} ->
      IO.puts("Session ended")

    %{type: :voice_stream_event_error, error: error} ->
      Logger.error("Error: #{inspect(error)}")
  end
end)

Input Types

• AudioInput - A static audio buffer for single-turn interactions
• StreamedAudioInput - A streaming audio input for multi-turn conversations

Configuration

The pipeline can be configured with custom STT and TTS models, or it will use the default OpenAI models:

config = Config.new(
  workflow_name: "Customer Support",
  tts_settings: TTSSettings.new(voice: :nova)
)

pipeline = Pipeline.new(
  workflow: workflow,
  config: config,
  stt_model: "whisper-1",
  tts_model: "tts-1-hd"
)