Gemini.Live.Session (GeminiEx v0.8.2)

View Source

GenServer for managing WebSocket connections to the Gemini Live API.

The Live API provides bidirectional streaming communication with Gemini models, supporting real-time text, audio, and video interactions.

Features

  • WebSocket-based real-time communication
  • Automatic reconnection with exponential backoff
  • Message queuing during connection setup
  • Callback-based event handling
  • Graceful connection lifecycle management

Usage

# Start a session
{:ok, session} = LiveSession.start_link(
  model: "gemini-2.0-flash-exp",
  generation_config: %{temperature: 0.8},
  on_message: fn message -> IO.inspect(message, label: "Received") end
)

# Connect to the Live API
:ok = LiveSession.connect(session)

# Send a text message
:ok = LiveSession.send(session, "Hello, how are you?")

# Send client content with tools
:ok = LiveSession.send_client_content(session, [
  %{role: "user", parts: [%{text: "What's the weather?"}]}
])

# Send tool response
:ok = LiveSession.send_tool_response(session, [
  %{name: "get_weather", response: %{temperature: 72, condition: "sunny"}}
])

# Close the session
:ok = LiveSession.close(session)

Callbacks

The session supports several callbacks for handling events:

  • :on_message - Called when a message is received from the server
  • :on_connect - Called when successfully connected
  • :on_disconnect - Called when disconnected
  • :on_error - Called when an error occurs

Example with Callbacks

LiveSession.start_link(
  model: "gemini-2.0-flash-exp",
  on_message: fn message ->
    case message do
      %{server_content: content} ->
        IO.puts("Model: #{inspect(content)}")
      %{tool_call: calls} ->
        IO.puts("Function calls: #{inspect(calls)}")
      _ ->
        IO.puts("Other: #{inspect(message)}")
    end
  end,
  on_connect: fn -> IO.puts("Connected!") end,
  on_error: fn error -> IO.puts("Error: #{inspect(error)}") end
)

Summary

Types

callback()
message()
session()
t()

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

close(session)

Close the Live API session.

connect(session)

Connect to the Live API WebSocket endpoint.

send(session, text)

Send a text message to the model.

send_client_content(session, turns, turn_complete \\ true)

Send client content (turns) to the model.

send_realtime_input(session, media_chunks, opts \\ [])

Send real-time input (audio/video) to the model.

send_tool_response(session, function_responses, opts \\ [])

Send tool/function response to the model.

start_link(opts)

Start a Live API session.

status(session)

Get the current status of the session.

Types

callback()

@type callback() :: (term() -> any())

message()

@type message() :: String.t() | map()

session()

@type session() :: pid() | atom()

t()

@type t() :: %Gemini.Live.Session{
  auth_headers: [{String.t(), String.t()}] | nil,
  auth_strategy: :gemini | :vertex_ai,
  config: map(),
  conn_pid: pid() | nil,
  message_queue: [map()],
  model: String.t(),
  on_connect: callback() | nil,
  on_disconnect: callback() | nil,
  on_error: callback() | nil,
  on_message: callback() | nil,
  ping_timer: reference() | nil,
  reconnect_delay: non_neg_integer(),
  reconnect_timer: reference() | nil,
  setup_complete: boolean(),
  setup_sent: boolean(),
  status: :disconnected | :connecting | :connected | :error,
  stream_ref: reference() | nil,
  websocket_url: String.t() | nil
}

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

close(session)

@spec close(session()) :: :ok

Close the Live API session.

Examples

:ok = LiveSession.close(session)

connect(session)

@spec connect(session()) :: :ok | {:error, term()}

Connect to the Live API WebSocket endpoint.

This must be called after start_link to establish the connection.

Examples

{:ok, session} = LiveSession.start_link(model: "gemini-2.0-flash-exp")
:ok = LiveSession.connect(session)

send(session, text)

@spec send(session(), String.t()) :: :ok | {:error, term()}

Send a text message to the model.

Examples

:ok = LiveSession.send(session, "What is the weather today?")

send_client_content(session, turns, turn_complete \\ true)

@spec send_client_content(
  session(),
  [map()] | Gemini.Live.Message.ClientContent.t(),
  boolean()
) ::
  :ok | {:error, term()}

Send client content (turns) to the model.

Parameters

  • session - Session PID or name
  • turns - List of content turns or ClientContent struct
  • turn_complete - Whether the turn is complete (default: true)

Examples

# Send simple content
:ok = LiveSession.send_client_content(session, [
  %{role: "user", parts: [%{text: "Hello"}]}
])

# Send ClientContent struct
content = %ClientContent{
  turns: [...],
  turn_complete: true
}
:ok = LiveSession.send_client_content(session, content)

send_realtime_input(session, media_chunks, opts \\ [])

@spec send_realtime_input(session(), [map()], keyword()) :: :ok | {:error, term()}

Send real-time input (audio/video) to the model.

Parameters

  • session - Session PID or name
  • media_chunks - List of media chunks
  • opts - Additional options

Examples

# Send audio chunk
:ok = LiveSession.send_realtime_input(session, [
  %{data: audio_data, mime_type: "audio/pcm"}
])

send_tool_response(session, function_responses, opts \\ [])

@spec send_tool_response(session(), [map()], keyword()) :: :ok | {:error, term()}

Send tool/function response to the model.

Parameters

  • session - Session PID or name
  • function_responses - List of function responses
  • opts - Additional options

Examples

:ok = LiveSession.send_tool_response(session, [
  %{
    name: "get_weather",
    response: %{temperature: 72, condition: "sunny"}
  }
])

start_link(opts)

@spec start_link(keyword()) :: GenServer.on_start()

Start a Live API session.

Options

  • :model - Model to use (required)
  • :auth - Authentication strategy (:gemini or :vertex_ai, auto-detected if not provided)
  • :generation_config - Generation configuration
  • :system_instruction - System instruction
  • :tools - Available tools/functions
  • :tool_config - Tool configuration
  • :on_message - Callback when message received
  • :on_connect - Callback when connected
  • :on_disconnect - Callback when disconnected
  • :on_error - Callback when error occurs
  • :name - Optional name for the GenServer

Examples

{:ok, session} = LiveSession.start_link(
  model: "gemini-2.0-flash-exp",
  on_message: &handle_message/1
)

status(session)

@spec status(session()) :: atom()

Get the current status of the session.

Returns one of: :disconnected, :connecting, :connected, :error

Examples

status = LiveSession.status(session)
# => :connected