Behaviour and shared logic for LLM providers.

A provider represents a specific LLM service — its endpoint, authentication mechanism, and any service-specific adaptations. It is the authenticated HTTP layer: it knows where to send requests and how to authenticate them, but not what the request body should contain (that's the dialect's job).

Omni separates providers from dialects because the mapping is many-to-one. There are ~4–5 wire formats but ~20–30 services. Groq, Together, Fireworks, and OpenRouter all speak the OpenAI Chat Completions format — their request bodies and streaming events are identical. Only endpoint, authentication, and the occasional quirk differ. A provider captures those differences; a dialect (see Omni.Dialect) handles the wire format shared across many providers.

Most providers speak a single dialect, but multi-model gateways (like OpenCode Zen) route to different upstream APIs depending on the model. These providers omit the :dialect option — the dialect is resolved per-model from the JSON data files at load time. See "Multi-dialect providers" below.

Defining a provider

Use use Omni.Provider with an optional :dialect option. Most providers declare a dialect — the only required callback is config/0:

defmodule MyApp.Providers.Acme do
  use Omni.Provider, dialect: Omni.Dialects.OpenAICompletions

  @impl true
  def config do
    %{
      base_url: "https://api.acme.ai",
      api_key: {:system, "ACME_API_KEY"}
    }
  end

  @impl true
  def models do
    [
      Omni.Model.new(
        id: "acme-7b",
        name: "Acme 7B",
        provider: __MODULE__,
        dialect: dialect(),
        context_size: 128_000,
        max_output_tokens: 4096
      )
    ]
  end
end

The use macro generates a dialect/0 accessor from the provided module (or nil when omitted) and default implementations for all optional callbacks. Override only what your provider needs — most providers are just config/0 and models/0.

The request pipeline

Understanding the request pipeline clarifies when each callback runs and why. Internally, Omni orchestrates the flow by calling dialect and provider callbacks via the module references on the %Model{} struct:

# 1. Dialect builds the request body from Omni types
body = dialect.handle_body(model, context, opts)

# 2. Provider adjusts the body for service-specific quirks
body = provider.modify_body(body, context, opts)

# 3. Dialect determines the URL path for this API
path = dialect.handle_path(model, opts)

# 4. Provider builds the full URL (base_url + path)
url = provider.build_url(path, opts)

# 5. Provider adds authentication to the request
{:ok, req} = provider.authenticate(req, opts)

On the response side, streaming events flow through a similar pipeline:

# 1. Dialect parses raw SSE JSON into normalized delta tuples
deltas = dialect.handle_event(raw_event)

# 2. Provider adjusts deltas for service-specific data
deltas = provider.modify_events(deltas, raw_event)

The dialect does the heavy lifting (full type translation); the provider makes small, targeted adjustments. Most providers don't override modify_body/3 or modify_events/2 at all.

Callbacks

Authentication

The default authenticate/2 resolves opts.api_key via resolve_auth/1 and sets the appropriate header. When no :auth_header is configured, it sends a Bearer token on the "authorization" header (the most common scheme). When a custom :auth_header is set in config/0 (e.g. "x-api-key"), the raw key is sent on that header instead.

Override authenticate/2 only for unusual schemes like request signing or token refresh:

# Custom authentication
@impl true
def authenticate(req, opts) do
  with {:ok, key} <- Omni.Provider.resolve_auth(opts.api_key) do
    {:ok, Req.Request.put_header(req, "x-custom-auth", sign(key))}
  end
end

API keys are resolved in priority order:

1. Call-site opts — :api_key passed directly to generate_text/3 or stream_text/3
2. Application config — config :omni, MyProvider, api_key: ...
3. Provider default — the :api_key value from config/0

All three tiers accept the same value types — see resolve_auth/1.

Config keys

config/0 returns a map with the following keys:

• :base_url (required) — the service's base URL
• :api_key — default API key (see resolve_auth/1 for accepted formats)
• :auth_header — custom header name for the API key; when set, the raw key is sent on this header instead of as a Bearer token on "authorization"
• :headers — additional headers to include on every request (map)

These values serve as defaults. Users can override :base_url, :api_key, and :headers at the application config level or at the call site.

Multi-dialect providers

Some providers act as gateways to multiple upstream APIs, each with its own wire format. For example, OpenCode Zen routes Claude models through the Anthropic Messages format and GPT models through OpenAI Responses.

These providers omit the :dialect option:

defmodule MyApp.Providers.Gateway do
  use Omni.Provider

  @impl true
  def config do
    %{base_url: "https://gateway.example.com", api_key: {:system, "GW_KEY"}}
  end

  @impl true
  def models do
    Omni.Provider.load_models(__MODULE__, "priv/models/gateway.json")
  end
end

When dialect/0 returns nil, load_models/2 reads the "dialect" string from each model's JSON entry and resolves it via Omni.Dialect.get!/1. If a model is missing the "dialect" field, loading raises at startup.

Choosing a dialect

Pick the dialect that matches your provider's wire format:

• Omni.Dialects.OpenAICompletions — OpenAI Chat Completions format, used by the majority of providers (Groq, Together, Fireworks, DeepSeek, etc.)
• Omni.Dialects.OpenAIResponses — OpenAI's newer Responses API format
• Omni.Dialects.AnthropicMessages — Anthropic Messages format
• Omni.Dialects.GoogleGemini — Google Gemini format
• Omni.Dialects.OllamaChat — Ollama native chat format (NDJSON streaming)

If your provider speaks a format not listed here, you'll need to implement a new dialect — see Omni.Dialect for the behaviour specification.

Registering a provider

Providers are loaded at startup from application config. Built-in providers use shorthand atoms; custom providers use {id, module} tuples:

config :omni, :providers, [
  :anthropic,
  :openai,
  acme: MyApp.Providers.Acme
]

To load a provider at runtime without restarting:

Omni.Provider.load(acme: MyApp.Providers.Acme)

The provider's models are then available via Omni.get_model(:acme, "acme-7b").