Ex Outlines

Structured LLM output validation for Elixir. Guarantee valid, type-safe data from any LLM through automatic validation and repair.

# Define what you expect
schema = Schema.new(%{
  sentiment: %{type: {:enum, ["positive", "negative", "neutral"]}},
  confidence: %{type: :number, min: 0, max: 1},
  summary: %{type: :string, max_length: 100}
})

# Generate and validate
{:ok, result} = ExOutlines.generate(schema, backend: HTTP, backend_opts: opts)

# Use validated data
result.sentiment  # Guaranteed to be "positive", "negative", or "neutral"
result.confidence # Guaranteed to be 0-1
result.summary    # Guaranteed to be ≤100 characters

Why Ex Outlines?

The Problem: LLMs generate unpredictable outputs. You ask for JSON, you might get:

• Invalid JSON: {name: Alice}
• Wrong types: {"age": "thirty"}
• Missing fields: {"name": "Alice"} (no age)
• Extra text: ```json\n{"name": "Alice"}\n```

The Solution: Ex Outlines validates LLM outputs against your schema and automatically repairs errors through a retry loop:

1. Define Schema → Specify exact structure and constraints
2. Generate → LLM creates output
3. Validate → Check against schema
4. Repair → If invalid, send diagnostics back to LLM and retry
5. Guarantee → Return {:ok, validated_data} or {:error, reason}

Result: No more parsing failures. No more invalid data. Just validated, type-safe outputs.

Features

Core Capabilities

• Rich Type System - Strings, integers, booleans, numbers, enums, arrays, nested objects, union types
• Comprehensive Constraints - Length limits, min/max values, regex patterns, unique items
• Automatic Retry-Repair - Failed validations trigger repair prompts with clear diagnostics
• Backend Agnostic - Works with OpenAI, Anthropic, or any LLM API
• Batch Processing - Concurrent generation using BEAM lightweight processes
• Ecto Integration - Convert Ecto schemas automatically (optional)
• Telemetry Built-In - Observable with Phoenix.LiveDashboard
• Testing First-Class - Deterministic Mock backend for tests

Elixir-Specific Advantages

• BEAM Concurrency - Process 100s of requests concurrently
• Phoenix Integration - Works seamlessly in controllers and LiveView
• Type Safety - Dialyzer type specifications throughout
• Battle-Tested - 364 tests, 93% coverage, production-grade

Installation

Add to your mix.exs:

def deps do
  [
    {:ex_outlines, "~> 0.1.0"}
  ]
end

Optional: Add Ecto for schema adapter:

def deps do
  [
    {:ex_outlines, "~> 0.1.0"},
    {:ecto, "~> 3.11"}  # Optional
  ]
end

Run mix deps.get

Quick Start

1. Define a Schema

alias ExOutlines.Spec.Schema

schema = Schema.new(%{
  name: %{
    type: :string,
    required: true,
    min_length: 2,
    max_length: 50
  },
  age: %{
    type: :integer,
    required: true,
    min: 0,
    max: 120
  },
  email: %{
    type: :string,
    required: true,
    pattern: ~r/@/
  }
})

2. Generate Structured Output

{:ok, user} = ExOutlines.generate(schema,
  backend: ExOutlines.Backend.HTTP,
  backend_opts: [
    api_key: System.get_env("OPENAI_API_KEY"),
    model: "gpt-4o-mini",
    messages: [
      %{role: "system", content: "Extract user data."},
      %{role: "user", content: "My name is Alice, I'm 30 years old, email alice@example.com"}
    ]
  ]
)

# Result is validated and typed
user.name   # "Alice"
user.age    # 30
user.email  # "alice@example.com"

3. Handle Errors

case ExOutlines.generate(schema, opts) do
  {:ok, data} ->
    # Use validated data
    process_user(data)

  {:error, :max_retries_exceeded} ->
    # LLM couldn't produce valid output after all retries
    Logger.error("Generation failed after retries")

  {:error, {:backend_error, reason}} ->
    # API error (rate limit, timeout, etc.)
    Logger.error("Backend error: #{inspect(reason)}")
end

Type System

Primitive Types

# String
%{type: :string}
%{type: :string, min_length: 3, max_length: 100}
%{type: :string, pattern: ~r/^[A-Z][a-z]+$/}

# Integer
%{type: :integer}
%{type: :integer, min: 0, max: 100}
%{type: :integer, positive: true}  # Shorthand for min: 1

# Boolean
%{type: :boolean}

# Number (integer or float)
%{type: :number, min: 0.0, max: 1.0}

Composite Types

# Enum (multiple choice)
%{type: {:enum, ["red", "green", "blue"]}}

# Array
%{type: {:array, %{type: :string}}}
%{type: {:array, %{type: :integer, min: 0}}, min_items: 1, max_items: 10, unique_items: true}

# Nested Object
address_schema = Schema.new(%{
  street: %{type: :string, required: true},
  city: %{type: :string, required: true},
  zip: %{type: :string, pattern: ~r/^\d{5}$/}
})

person_schema = Schema.new(%{
  name: %{type: :string, required: true},
  address: %{type: {:object, address_schema}, required: true}
})

# Union Types (optional/nullable fields)
%{type: {:union, [%{type: :string}, %{type: :null}]}}
%{type: {:union, [%{type: :string}, %{type: :integer}]}}  # Either string or int

Backends

HTTP Backend (OpenAI-Compatible)

Works with OpenAI, Azure OpenAI, and compatible APIs:

alias ExOutlines.Backend.HTTP

ExOutlines.generate(schema,
  backend: HTTP,
  backend_opts: [
    api_key: System.get_env("OPENAI_API_KEY"),
    model: "gpt-4o-mini",
    api_url: "https://api.openai.com/v1/chat/completions",
    temperature: 0.0
  ]
)

Anthropic Backend

Native Claude API support:

alias ExOutlines.Backend.Anthropic

ExOutlines.generate(schema,
  backend: Anthropic,
  backend_opts: [
    api_key: System.get_env("ANTHROPIC_API_KEY"),
    model: "claude-3-5-sonnet-20241022",
    max_tokens: 1024
  ]
)

Mock Backend (Testing)

Deterministic responses for tests:

alias ExOutlines.Backend.Mock

# Single response
mock = Mock.new([{:ok, ~s({"name": "Alice", "age": 30})}])

# Multiple responses (for retry testing)
mock = Mock.new([
  {:ok, ~s({"name": "Alice", "age": "invalid"})},  # Invalid (will retry)
  {:ok, ~s({"name": "Alice", "age": 30})}          # Valid (succeeds)
])

# Always same response
mock = Mock.always({:ok, ~s({"status": "ok"})})

ExOutlines.generate(schema, backend: Mock, backend_opts: [mock: mock])

Batch Processing

Process multiple schemas concurrently using BEAM's lightweight processes:

# Define tasks
tasks = [
  {schema1, [backend: HTTP, backend_opts: opts1]},
  {schema2, [backend: HTTP, backend_opts: opts2]},
  {schema3, [backend: HTTP, backend_opts: opts3]}
]

# Process concurrently
results = ExOutlines.generate_batch(tasks, max_concurrency: 5)

# Results: [{:ok, data1}, {:ok, data2}, {:error, reason3}]

Example: Classify 100 messages in parallel

messages = get_messages(100)

tasks = Enum.map(messages, fn msg ->
  {classification_schema, [
    backend: HTTP,
    backend_opts: build_opts(msg)
  ]}
end)

# Process 10 at a time to respect rate limits
results = ExOutlines.generate_batch(tasks, max_concurrency: 10)

Ecto Integration

Automatically convert Ecto schemas to Ex Outlines schemas:

defmodule User do
  use Ecto.Schema
  import Ecto.Changeset

  embedded_schema do
    field :email, :string
    field :age, :integer
    field :username, :string
  end

  def changeset(user, params) do
    user
    |> cast(params, [:email, :age, :username])
    |> validate_required([:email, :username])
    |> validate_format(:email, ~r/@/)
    |> validate_number(:age, greater_than_or_equal_to: 0, less_than: 150)
    |> validate_length(:username, min: 3, max: 20)
  end
end

# Convert automatically
schema = Schema.from_ecto_schema(User, changeset_function: :changeset)

# Now use with LLM
{:ok, user} = ExOutlines.generate(schema, backend: HTTP, backend_opts: opts)

See Ecto Schema Adapter Guide for details.

Phoenix Integration

Use in Phoenix controllers:

defmodule MyAppWeb.TicketController do
  use MyAppWeb, :controller

  def create(conn, %{"message" => message}) do
    case classify_ticket(message) do
      {:ok, classification} ->
        {:ok, ticket} = Tickets.create(classification)

        conn
        |> put_flash(:info, "Ticket created with #{ticket.priority} priority")
        |> redirect(to: ~p"/tickets/#{ticket.id}")

      {:error, _reason} ->
        conn
        |> put_flash(:error, "Failed to classify ticket")
        |> render("new.html")
    end
  end

  defp classify_ticket(message) do
    schema = Schema.new(%{
      priority: %{type: {:enum, ["low", "medium", "high", "critical"]}},
      category: %{type: {:enum, ["technical", "billing", "account"]}}
    })

    ExOutlines.generate(schema,
      backend: HTTP,
      backend_opts: [
        api_key: Application.get_env(:my_app, :openai_api_key),
        model: "gpt-4o-mini",
        messages: build_messages(message)
      ]
    )
  end
end

See Phoenix Integration Guide for more patterns.

Examples

Browse examples/ for production-ready examples:

• Classification - Customer support triage with priority, category, sentiment
• E-commerce Categorization - Product classification with features and tags
• Document Metadata - Extract structured metadata from documents
• Customer Support Triage - Automated ticket routing

Run examples:

elixir examples/classification.exs

Documentation

Guides

• Getting Started - Installation, first schema, validation basics
• Core Concepts - Deep dive into schemas, validation, retry-repair loop
• Phoenix Integration - Controllers, LiveView, Oban patterns
• Ecto Schema Adapter - Automatic Ecto schema conversion
• Testing Strategies - Testing with Mock backend
• Error Handling - Robust error handling patterns

API Reference

Complete API documentation: hexdocs.pm/ex_outlines

Interactive Tutorials

14 comprehensive Livebook tutorials available in livebooks/ directory:

Beginner Level:

• Getting Started - Introduction to ExOutlines fundamentals

Intermediate Level:

• Named Entity Extraction - Extract structured entities from text
• Dating Profile Generation - Creative content with EEx templates
• Question Answering with Citations - Build trustworthy Q&A systems
• Sampling and Self-Consistency - Multi-sample generation strategies

Advanced Level:

• Models Playing Chess - Constrained move generation game
• SimToM: Theory of Mind - Perspective-taking with Mermaid diagrams
• Chain of Thought - Step-by-step reasoning patterns
• ReAct Agent - Build agents with tool integration
• Structured Generation Workflow - Multi-stage pipelines

Vision & Document Processing:

• PDF Reading - Extract data from PDFs with vision models
• Earnings Reports - Financial data extraction and analysis
• Receipt Digitization - Process receipt images for expenses
• Extract Event Details - Natural language to calendar events

Open with Livebook for interactive learning.

Telemetry & Observability

Ex Outlines emits telemetry events for monitoring:

:telemetry.attach(
  "ex-outlines-monitor",
  [:ex_outlines, :generate, :stop],
  fn _event, measurements, metadata, _config ->
    duration_ms = System.convert_time_unit(measurements.duration, :native, :millisecond)

    Logger.info("""
    LLM Generation:
      Duration: #{duration_ms}ms
      Attempts: #{measurements.attempt_count}
      Status: #{metadata.status}
    """)
  end,
  nil
)

Integrate with Phoenix.LiveDashboard:

# lib/my_app_web/telemetry.ex
def metrics do
  [
    summary("ex_outlines.generate.stop.duration",
      unit: {:native, :millisecond},
      tags: [:backend, :status]
    ),

    summary("ex_outlines.generate.stop.attempt_count",
      tags: [:backend]
    )
  ]
end

How It Works

The Retry-Repair Loop

┌─────────────┐
│ User Prompt │
└──────┬──────┘
       │
       v
┌─────────────┐
│ LLM Generate│
└──────┬──────┘
       │
       v
┌─────────────┐     Valid      ┌────────┐
│  Validate   │───────────────>│ Return │
└──────┬──────┘                └────────┘
       │ Invalid
       │
       v
┌─────────────┐
│Build Repair │
│   Prompt    │
└──────┬──────┘
       │
       └──────────────────┘ (back to LLM Generate)

Example:

1. Generate: LLM returns {"age": 150}
2. Validate: Fails (age > 120)
3. Repair: "Field 'age' must be at most 120. Please fix."
4. Retry: LLM returns {"age": 30}
5. Validate: Success
6. Return: {:ok, %{age: 30}}

Comparison to Python Outlines

When to use Ex Outlines:

• Building in Elixir/Phoenix
• Need backend flexibility (any LLM API)
• Want explicit error handling and diagnostics
• Value BEAM concurrency for batch processing

When to use Python Outlines:

• Building in Python
• Have logit-level API access
• Need absolute minimum LLM calls
• Require context-free grammars

Both tools serve different ecosystems and constraints.

Next Steps

Active development priorities for v0.2.0:

1. Google Gemini Backend (Complete) - Native Google Gemini API support for fast, cost-effective generation
2. EEx Template Integration - Reusable prompt templates with variable interpolation using Elixir's built-in EEx
3. Streaming Support - Real-time generation with incremental validation for responsive UIs and LiveView
4. Vision Model Support - Multimodal image input for structured data extraction from invoices and documents
5. Ollama Native Backend - Local model support for privacy-focused and cost-free generation
6. Bumblebee Integration - Run Transformers models locally within the BEAM for offline generation
7. Advanced Numeric Constraints - Add exclusive min/max and multipleOf constraints from JSON Schema
8. Tuple Type Support - Fixed-length arrays with different types per position for precise validation
9. Conditional Fields - Schema dependencies and conditional requirements for complex validation logic
10. Production Examples - Expand example library with legal, medical, financial, and code analysis use cases

Roadmap

v0.3 (Planned)

• [ ] Template system (EEx-based prompt templates)
• [ ] Streaming support (incremental validation)
• [ ] Generator abstraction (reusable model + schema)
• [ ] Additional backends (Ollama, vLLM)
• [x] 14 comprehensive Livebook tutorials (completed in v0.1)

v0.4+ (Future)

• [ ] Context-free grammar support
• [ ] Local model integration (Bumblebee)
• [ ] Function calling DSL
• [ ] Advanced caching layer

See CHANGELOG.md for full version history.

Testing

Run the test suite:

mix test

With coverage:

mix test --cover

Strict checks:

mix format --check-formatted
mix credo --strict
mix dialyzer

Contributing

Contributions are welcome! Please:

1. Open an issue for discussion before major changes
2. Add tests for new functionality
3. Follow existing code style (mix format)
4. Ensure mix credo --strict passes
5. Update documentation
6. Add type specs for public functions

License

MIT License - see LICENSE for details.

Credits

Inspired by Python Outlines by dottxt-ai.

Built using:

• Elixir
• Jason - JSON parsing
• Telemetry - Observability
• Ecto - Optional schema adapter

Links

• Documentation
• Hex.pm
• GitHub
• Changelog
• Issues

Made with Elixir. Powered by BEAM.