ReqLLM.Provider.Options (ReqLLM v1.0.0)

View Source

Runtime generation options processing for ReqLLM providers.

This module handles only the core generation options that are truly universal across providers, plus the orchestration logic for validation, translation, and provider-specific option handling.

Design Principles

  1. Universal Core: Only include options supported by most major providers
  2. Provider Extensions: Allow providers to define their own options via provider_schema/0
  3. Clean Separation: Metadata (capabilities, costs) belongs in separate modules

Core Options

The following options are considered universal:

  • temperature, max_tokens - Basic sampling control
  • top_p, top_k - Advanced sampling
  • frequency_penalty, presence_penalty - Repetition control
  • seed, stop - Deterministic generation and control
  • tools, tool_choice - Function calling
  • reasoning_effort, reasoning_token_budget - Reasoning controls
  • n, stream - Output control
  • user - Tracking/identification

Provider-Specific Options

Providers can extend the schema via:

def provider_schema do
  NimbleOptions.new!([
    dimensions: [type: :pos_integer, doc: "Embedding dimensions"],
    custom_param: [type: :string, doc: "Provider-specific parameter"]
  ])
end

Usage

The main entry point is process/4 which handles the complete pipeline:

{:ok, processed_opts} = Options.process(MyProvider, :chat, model, user_opts)

Summary

Functions

Returns a list of all generation option keys.

Builds a dynamic schema by composing base schema with provider-specific options.

Returns the effective base URL for the provider based on precedence rules.

Extracts only generation options from a mixed options list.

Extracts provider-specific options from a mixed options list.

Filters generation options to only include supported keys.

Returns the core generation options schema.

Returns a NimbleOptions schema that contains only the requested generation keys.

Merges options with defaults, respecting user-provided overrides.

Main processing function - validates, translates, and composes options.

Same as process/4 but raises on error.

Validates generation options against a subset of supported keys.

Functions

all_generation_keys()

Returns a list of all generation option keys.

compose_schema(base_schema, provider_mod)

Builds a dynamic schema by composing base schema with provider-specific options.

This function takes a base schema and provider module, creating a unified schema where provider-specific options are nested under the :provider_options key with proper validation.

This is the public API for schema composition and should be used by external modules that need to validate options with provider-specific extensions.

Parameters

  • base_schema - Base NimbleOptions schema (usually generation_options_schema/0)
  • provider_mod - Provider module that may implement provider_schema/0

Examples

schema = ReqLLM.Provider.Options.compose_schema(
  ReqLLM.Provider.Options.generation_schema(),
  MyProvider
)

effective_base_url(provider_mod, model, opts)

@spec effective_base_url(module(), ReqLLM.Model.t(), keyword()) :: String.t()

Returns the effective base URL for the provider based on precedence rules.

The base URL is determined by the following precedence order (highest to lowest):

  1. opts[:base_url] - Explicitly passed in options
  2. Application config - Application.get_env(:req_llm, model.provider)[:base_url]
  3. Provider registry metadata - Loaded from provider's JSON metadata file
  4. Provider default - provider_mod.default_base_url()

Parameters

  • provider_mod - Provider module implementing the Provider behavior
  • model - ReqLLM.Model struct containing provider information
  • opts - Options keyword list that may contain :base_url

Examples

iex> model = %ReqLLM.Model{provider: :openai, model: "gpt-4"}
iex> ReqLLM.Provider.Options.effective_base_url(ReqLLM.Providers.OpenAI, model, [])
"https://api.openai.com/v1"

iex> opts = [base_url: "https://custom.example.com"]
iex> ReqLLM.Provider.Options.effective_base_url(ReqLLM.Providers.OpenAI, model, opts)
"https://custom.example.com"

extract_generation_opts(opts)

Extracts only generation options from a mixed options list.

Unlike extract_provider_options/1, this returns only the generation options without the unused remainder.

Examples

iex> mixed_opts = [temperature: 0.7, custom_param: "value", max_tokens: 100]
iex> ReqLLM.Provider.Options.extract_generation_opts(mixed_opts)
[temperature: 0.7, max_tokens: 100]

extract_provider_options(opts)

Extracts provider-specific options from a mixed options list.

This is useful for separating standard options from provider-specific ones.

Examples

iex> opts = [temperature: 0.7, max_tokens: 100, custom_param: "value"]
iex> ReqLLM.Provider.Options.extract_provider_options(opts)
{[temperature: 0.7, max_tokens: 100], [custom_param: "value"]}

filter_generation_options(opts, keys)

Filters generation options to only include supported keys.

This is a pure filter function that doesn't validate - it just removes unsupported keys from the options.

Examples

iex> opts = [temperature: 0.7, unsupported_key: "value", max_tokens: 100]
iex> ReqLLM.Provider.Options.filter_generation_options(opts, [:temperature, :max_tokens])
[temperature: 0.7, max_tokens: 100]

generation_schema()

Returns the core generation options schema.

generation_subset_schema(keys)

Returns a NimbleOptions schema that contains only the requested generation keys.

Examples

iex> schema = ReqLLM.Provider.Options.generation_subset_schema([:temperature, :max_tokens])
iex> NimbleOptions.validate([temperature: 0.7], schema)
{:ok, [temperature: 0.7]}

merge_with_defaults(opts, defaults)

Merges options with defaults, respecting user-provided overrides.

Examples

iex> defaults = [temperature: 0.7, max_tokens: 1000]
iex> user_opts = [temperature: 0.9]
iex> result = ReqLLM.Provider.Options.merge_with_defaults(user_opts, defaults)
iex> result[:temperature]
0.9
iex> result[:max_tokens]
1000

process(provider_mod, operation, model, opts)

@spec process(module(), atom(), ReqLLM.Model.t(), keyword()) ::
  {:ok, keyword()} | {:error, term()}

Main processing function - validates, translates, and composes options.

This is the primary public API for option processing. It handles:

  1. Provider key collision detection (prevents shadowing core options)
  2. Validation against composed schema (core + provider options)
  3. Provider-specific option translation
  4. Internal option preservation
  5. Error wrapping for consistency

Parameters

  • provider_mod - Provider module implementing the Provider behavior
  • operation - Operation type (:chat, :embedding, :object, etc.)
  • model - ReqLLM.Model struct
  • opts - Raw user options keyword list

Returns

{:ok, processed_opts} or {:error, wrapped_error}

Examples

model = %ReqLLM.Model{provider: :openai, model: "gpt-4"}

opts = [
  temperature: 0.7,
  provider_options: [dimensions: 512, encoding_format: "float"]
]
{:ok, processed} = Options.process(MyProvider, :chat, model, opts)

process!(provider_mod, operation, model, opts)

@spec process!(module(), atom(), ReqLLM.Model.t(), keyword()) :: keyword()

Same as process/4 but raises on error.

validate_generation_options(opts, list)

Validates generation options against a subset of supported keys.

Examples

iex> ReqLLM.Provider.Options.validate_generation_options(
...>   [temperature: 0.7, max_tokens: 100],
...>   only: [:temperature, :max_tokens]
...> )
{:ok, [temperature: 0.7, max_tokens: 100]}