Rag.Router (rag v0.3.4)

Multi-LLM provider router.

The Router manages multiple LLM providers and routes requests to the most appropriate provider based on the configured strategy.

Configuration

{:ok, router} = Router.new(
  providers: [:gemini, :codex, :claude],
  strategy: :specialist,  # or :fallback, :round_robin, :auto
  fallback_order: [:gemini, :codex, :claude]
)

Strategies

:fallback - Try providers in order until success (default with 1-2 providers)
:round_robin - Distribute load across providers
:specialist - Route by task type to best provider (default with 3 providers)
:auto - Auto-select strategy based on available providers

Usage

# Route a request
{:ok, provider, router} = Router.route(router, :text, "Hello", [])

# Execute with the provider
{:ok, result, router} = Router.execute(router, :text, "Hello", [])

# Report result manually
router = Router.report_result(router, :gemini, {:ok, "response"})

Auto-Detection

# Auto-detect available providers based on loaded modules and credentials
{:ok, router} = Router.new(auto_detect: true)

Summary

Types

t()

Functions

available_providers(router)

Get list of currently available providers.

execute(router, type, prompt, opts)

Execute a request with automatic provider selection and retry.

get_provider(router, provider)

Get capabilities for a specific provider.

new(opts \\ [])

Create a new router with the given configuration.

next_provider(router, failed_provider)

Get the next provider after a failure.

report_result(router, provider, result)

Report the result of a provider call.

route(router, type, prompt, opts)

Route a request to select the best provider.

Types

t()

@type t() :: %Rag.Router{
  provider_instances: %{required(atom()) => struct()},
  providers: [atom()],
  strategy_module: module(),
  strategy_state: term()
}

Functions

available_providers(router)

@spec available_providers(t()) :: [atom()]

Get list of currently available providers.

execute(router, type, prompt, opts)

@spec execute(t(), atom(), term(), keyword()) :: {:ok, term(), t()} | {:error, term()}

Execute a request with automatic provider selection and retry.

Routes the request, executes with the selected provider, and handles retries on failure.

get_provider(router, provider)

@spec get_provider(t(), atom()) :: {:ok, map()} | {:error, :not_found}

Get capabilities for a specific provider.

new(opts \\ [])

@spec new(keyword()) :: {:ok, t()} | {:error, term()}

Create a new router with the given configuration.

Options

:providers - List of provider atoms (required unless auto_detect)
:strategy - Strategy atom or :auto (default: auto-selected)
:auto_detect - Auto-detect available providers (default: false)
:fallback_order - Order for fallback attempts
Strategy-specific options passed through

Returns

{:ok, router} on success
{:error, reason} on failure

next_provider(router, failed_provider)

@spec next_provider(t(), atom()) :: {:ok, atom(), t()} | {:error, term()}

Get the next provider after a failure.

report_result(router, provider, result)

@spec report_result(t(), atom(), {:ok, term()} | {:error, term()}) :: t()

Report the result of a provider call.

Updates the router's strategy state based on success or failure.

route(router, type, prompt, opts)

@spec route(t(), atom(), term(), keyword()) :: {:ok, atom(), t()} | {:error, term()}

Route a request to select the best provider.

Parameters

router - The router struct
type - Request type (:text or :embeddings)
prompt - The prompt or list of texts for embeddings
opts - Additional options

Returns

{:ok, provider, updated_router} - Selected provider
{:error, reason} - No provider available