Gemini.Live.Session (GeminiEx v0.9.1)

Copy Markdown View Source

GenServer managing a Live API WebSocket session.

Provides a high-level interface for real-time bidirectional communication with Gemini models for voice, video, and text interactions.

Usage

# Start a session
{:ok, pid} = Session.start_link(
  model: "gemini-2.5-flash-native-audio-preview-12-2025",
  auth: :gemini,
  on_message: fn msg -> IO.inspect(msg) end,
  on_error: fn err -> Logger.error(inspect(err)) end
)

# Connect to the Live API
:ok = Session.connect(pid)

# Send text content
:ok = Session.send_client_content(pid, "Hello!")

# Send realtime audio
:ok = Session.send_realtime_input(pid, audio: audio_blob)

# Close when done
:ok = Session.close(pid)

Callbacks

  • on_message - Called for each server message
  • on_error - Called on errors
  • on_close - Called when session closes
  • on_tool_call - Called when model requests tool execution (may return tool responses)
  • on_transcription - Called for audio transcriptions
  • on_voice_activity - Called for voice activity signals

Session State

The session tracks:

  • Connection status
  • Setup completion
  • Active tool calls
  • Session resumption handle
  • Usage metadata

Audio Format

  • Input: 16-bit PCM, 16kHz, mono
  • Output: 16-bit PCM, 24kHz, mono

Summary

Types

callback()
session_status()
state()
tool_call_callback()
tool_call_callback_result()
tool_response()
tool_responses()

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

close(session)

Closes the session gracefully.

connect(session)

Connects to the Live API and sends setup configuration.

get_session_handle(session)

Returns the session resumption handle (if available).

send_client_content(session, content, opts \\ [])

Sends client content (text turns) to the model.

send_realtime_input(session, opts)

Sends realtime input (audio, video, text) to the model.

send_tool_response(session, responses)

Sends tool/function responses to the model.

start_link(opts)

Starts a new Live session process.

status(session)

Returns the current session status.

Types

callback()

@type callback() :: (term() -> any())

session_status()

@type session_status() ::
  :disconnected | :connecting | :setup_pending | :ready | :closing

state()

@type state() :: %{
  websocket: Gemini.Client.WebSocket.t() | term() | nil,
  websocket_module: module(),
  websocket_opts: keyword(),
  status: session_status(),
  config: map(),
  callbacks: map(),
  pending_setup: Gemini.Types.Live.Setup.t() | nil,
  session_handle: String.t() | nil,
  usage_metadata: map() | nil,
  owner: pid()
}

tool_call_callback()

@type tool_call_callback() :: (Gemini.Types.Live.ToolCall.t() ->
                           tool_call_callback_result())

tool_call_callback_result()

@type tool_call_callback_result() ::
  :ok
  | {:tool_response, tool_responses()}
  | {:send_tool_response, tool_responses()}
  | tool_responses()

tool_response()

@type tool_response() :: map()

tool_responses()

@type tool_responses() :: [tool_response()]

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

close(session)

@spec close(GenServer.server()) :: :ok

Closes the session gracefully.

Returns

  • :ok - Session closed

connect(session)

@spec connect(GenServer.server()) :: :ok | {:error, term()}

Connects to the Live API and sends setup configuration.

Must be called after start_link/1 to establish the WebSocket connection. Waits for the setup_complete response before returning.

Returns

  • :ok - Connected and setup complete
  • {:error, reason} - Connection failed

get_session_handle(session)

@spec get_session_handle(GenServer.server()) :: String.t() | nil

Returns the session resumption handle (if available).

The handle can be used to resume a session after disconnection. Only available if session_resumption was enabled and the server has provided a handle.

send_client_content(session, content, opts \\ [])

@spec send_client_content(GenServer.server(), String.t() | list(), keyword()) ::
  :ok | {:error, term()}

Sends client content (text turns) to the model.

Parameters

  • session - Session PID
  • content - String or list of turn maps
  • opts - Options:
    • :turn_complete - Whether this completes the turn (default: true)

Returns

  • :ok - Content sent
  • {:error, reason} - Send failed

Examples

# Simple text
Session.send_client_content(pid, "What is 2+2?")

# With turn control
Session.send_client_content(pid, "Part 1", turn_complete: false)
Session.send_client_content(pid, "Part 2", turn_complete: true)

# Multi-turn context
Session.send_client_content(pid, [
  %{role: "user", parts: [%{text: "Hello"}]},
  %{role: "model", parts: [%{text: "Hi!"}]},
  %{role: "user", parts: [%{text: "How are you?"}]}
])

send_realtime_input(session, opts)

@spec send_realtime_input(
  GenServer.server(),
  keyword()
) :: :ok | {:error, term()}

Sends realtime input (audio, video, text) to the model.

Parameters

  • session - Session PID
  • opts - Input options:
    • :audio - Audio blob (16-bit PCM, 16kHz mono)
    • :video - Video blob
    • :text - Text string
    • :activity_start - Signal start of user activity
    • :activity_end - Signal end of user activity
    • :audio_stream_end - Signal audio stream ended

Returns

  • :ok - Input sent
  • {:error, reason} - Send failed

Examples

# Send audio chunk
Session.send_realtime_input(pid, audio: %{data: pcm_data, mime_type: "audio/pcm;rate=16000"})

# Signal manual activity
Session.send_realtime_input(pid, activity_start: true)
Session.send_realtime_input(pid, audio: audio_chunk)
Session.send_realtime_input(pid, activity_end: true)

send_tool_response(session, responses)

@spec send_tool_response(GenServer.server(), list()) :: :ok | {:error, term()}

Sends tool/function responses to the model.

Parameters

  • session - Session PID
  • responses - List of function response maps with :id, :name, and :response keys

Returns

  • :ok - Response sent
  • {:error, reason} - Send failed

Example

Session.send_tool_response(pid, [
  %{id: "call_123", name: "get_weather", response: %{temp: 72}}
])

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

Starts a new Live session process.

Options

  • :model - Required. Model name (e.g., "gemini-2.5-flash-native-audio-preview-12-2025")
  • :auth - Auth strategy (:gemini or :vertex_ai, default: auto-detect)
  • :project_id - Required for Vertex AI
  • :location - Vertex AI location (default: "us-central1")
  • :api_version - Live API version (default: "v1beta")
  • :generation_config - Generation configuration
  • :system_instruction - System instruction content
  • :tools - Tool declarations
  • :proactivity - Proactivity configuration (v1alpha)
  • :enable_affective_dialog - Enable affective dialog (v1alpha)
  • :realtime_input_config - Realtime input configuration
  • :on_message - Callback for server messages
  • :on_error - Callback for errors
  • :on_close - Callback for session close
  • :on_tool_call - Callback for tool call requests (may return tool responses)
    • Return {:tool_response, responses} or a list of responses to send automatically
  • :on_tool_call_cancellation - Callback for tool call cancellation
  • :on_transcription - Callback for transcriptions
  • :on_voice_activity - Callback for voice activity signals
  • :on_session_resumption - Callback for session resumption updates
  • :on_go_away - Callback for GoAway notices (impending disconnection)
  • :session_resumption - Enable session resumption
  • :resume_handle - Handle from previous session to resume
  • :context_window_compression - Enable context compression
  • :websocket_module - Advanced: override WebSocket client module (useful for testing)
  • :websocket_opts - Advanced: extra options passed to WebSocket.connect/2

Returns

  • {:ok, pid} - Session started
  • {:error, reason} - Start failed

Examples

{:ok, session} = Session.start_link(
  model: "gemini-2.5-flash-native-audio-preview-12-2025",
  auth: :gemini,
  generation_config: %{response_modalities: ["TEXT"]},
  on_message: fn msg -> IO.inspect(msg) end
)

status(session)

@spec status(GenServer.server()) :: session_status()

Returns the current session status.

Status Values

  • :disconnected - Not connected
  • :connecting - Connection in progress
  • :setup_pending - Connected, waiting for setup_complete
  • :ready - Connected and ready for messages
  • :closing - Received GoAway, closing soon