Universal Elixir client for LLM APIs. Streaming text generation, tool use, and structured output.

Omni provides a single API for text generation that works with many LLM providers. All requests are streaming-first — generate_text/3 is built on top of stream_text/3.

Setup

Add Omni to your dependencies:

{:omni, "~> 1.2"}

Each built-in provider reads its API key from a standard environment variable by default — if your keys are set, no configuration is needed:

To override a key — for example, to use a different variable name or load from a vault at runtime — use per-provider application config:

# config/runtime.exs
config :omni, Omni.Providers.Anthropic, api_key: {:system, "MY_ANTHROPIC_KEY"}
config :omni, Omni.Providers.OpenAI, api_key: {MyApp.Secrets, :fetch, ["openai"]}

You can also pass :api_key directly to generate_text/3 or stream_text/3.

Anthropic, OpenAI, and Google are loaded by default. To add others or limit what loads at startup:

config :omni, :providers, [:anthropic, :openai, :openrouter]

Text generation

The simplest way to use Omni — pass a model tuple and a string:

{:ok, response} = Omni.generate_text({:anthropic, "claude-sonnet-4-5-20250514"}, "Hello!")

response.message
#=> %Omni.Message{role: :assistant, content: [%Omni.Content.Text{text: "Hello! How can..."}]}

response.usage
#=> %Omni.Usage{input_tokens: 10, output_tokens: 25, total_cost: 0.0003, ...}

For multi-turn conversations, build a context with a system prompt and messages:

context = Omni.context(
  system: "You are a helpful assistant.",
  messages: [
    Omni.message(role: :user, content: "What is Elixir?"),
    Omni.message(role: :assistant, content: "Elixir is a functional programming language..."),
    Omni.message(role: :user, content: "How does it handle concurrency?")
  ]
)

{:ok, response} = Omni.generate_text({:anthropic, "claude-sonnet-4-5-20250514"}, context)

Streaming

stream_text/3 returns a StreamingResponse that you can consume with event handlers:

{:ok, stream} = Omni.stream_text({:anthropic, "claude-sonnet-4-5-20250514"}, "Tell me a story")

{:ok, response} =
  stream
  |> StreamingResponse.on(:text_delta, fn %{delta: text} -> IO.write(text) end)
  |> StreamingResponse.complete()

For simple cases where you just need the text chunks:

stream
|> StreamingResponse.text_stream()
|> Enum.each(&IO.write/1)

See Omni.StreamingResponse for the full event taxonomy.

Structured output

Pass an Omni.Schema map via the :output option to get validated, decoded output:

schema = Omni.Schema.object(%{
  name: Omni.Schema.string(description: "The capital city"),
  population: Omni.Schema.integer(description: "Approximate population")
}, required: [:name, :population])

{:ok, response} =
  Omni.generate_text(
    {:anthropic, "claude-sonnet-4-5-20250514"},
    "What is the capital of France?",
    output: schema
  )

response.output
#=> %{name: "Paris", population: 2161000}

The schema is sent to the provider for constrained decoding. The response text is validated against the schema and decoded into response.output, with automatic retries on validation failure.

Tool use

Define tools with schemas and handlers — the loop automatically executes tool uses and feeds results back to the model:

weather_tool = Omni.tool(
  name: "get_weather",
  description: "Gets the current weather for a city",
  input_schema: Omni.Schema.object(
    %{city: Omni.Schema.string(description: "City name")},
    required: [:city]
  ),
  handler: fn input -> "72°F and sunny in #{input.city}" end
)

context = Omni.context(
  messages: [Omni.message("What's the weather in London?")],
  tools: [weather_tool]
)

{:ok, response} = Omni.generate_text({:anthropic, "claude-sonnet-4-5-20250514"}, context)

When tools have handlers, the loop automatically executes tool uses and feeds results back to the model until it produces a final text response.

For reusable tools with validation, define a tool module — see Omni.Tool.

Providers

Omni ships with built-in providers for Anthropic, OpenAI, Google, OpenRouter, and OpenCode Zen. Each provider is paired with one or more dialects that handle wire format translation — most providers use a single dialect, while multi-model gateways like OpenCode Zen use different dialects per model.

Models are referenced as {provider_id, model_id} tuples. To list available models for a provider:

{:ok, models} = Omni.list_models(:anthropic)

To add a custom provider, implement the Omni.Provider behaviour and load it at runtime — see Omni.Provider for details.