ReqLLM.Provider.Options (ReqLLM v1.0.0)
View SourceRuntime generation options processing for ReqLLM providers.
This module handles only the core generation options that are truly universal across providers, plus the orchestration logic for validation, translation, and provider-specific option handling.
Design Principles
- Universal Core: Only include options supported by most major providers
- Provider Extensions: Allow providers to define their own options via
provider_schema/0 - Clean Separation: Metadata (capabilities, costs) belongs in separate modules
Core Options
The following options are considered universal:
temperature,max_tokens- Basic sampling controltop_p,top_k- Advanced samplingfrequency_penalty,presence_penalty- Repetition controlseed,stop- Deterministic generation and controltools,tool_choice- Function callingreasoning_effort,reasoning_token_budget- Reasoning controlsn,stream- Output controluser- Tracking/identification
Provider-Specific Options
Providers can extend the schema via:
def provider_schema do
NimbleOptions.new!([
dimensions: [type: :pos_integer, doc: "Embedding dimensions"],
custom_param: [type: :string, doc: "Provider-specific parameter"]
])
endUsage
The main entry point is process/4 which handles the complete pipeline:
{:ok, processed_opts} = Options.process(MyProvider, :chat, model, user_opts)
Summary
Functions
Returns a list of all generation option keys.
Builds a dynamic schema by composing base schema with provider-specific options.
Returns the effective base URL for the provider based on precedence rules.
Extracts only generation options from a mixed options list.
Extracts provider-specific options from a mixed options list.
Filters generation options to only include supported keys.
Returns the core generation options schema.
Returns a NimbleOptions schema that contains only the requested generation keys.
Merges options with defaults, respecting user-provided overrides.
Main processing function - validates, translates, and composes options.
Same as process/4 but raises on error.
Validates generation options against a subset of supported keys.
Functions
Returns a list of all generation option keys.
Builds a dynamic schema by composing base schema with provider-specific options.
This function takes a base schema and provider module, creating a unified schema where provider-specific options are nested under the :provider_options key with proper validation.
This is the public API for schema composition and should be used by external modules that need to validate options with provider-specific extensions.
Parameters
base_schema- Base NimbleOptions schema (usually generation_options_schema/0)provider_mod- Provider module that may implement provider_schema/0
Examples
schema = ReqLLM.Provider.Options.compose_schema(
ReqLLM.Provider.Options.generation_schema(),
MyProvider
)
@spec effective_base_url(module(), ReqLLM.Model.t(), keyword()) :: String.t()
Returns the effective base URL for the provider based on precedence rules.
The base URL is determined by the following precedence order (highest to lowest):
opts[:base_url]- Explicitly passed in options- Application config -
Application.get_env(:req_llm, model.provider)[:base_url] - Provider registry metadata - Loaded from provider's JSON metadata file
- Provider default -
provider_mod.default_base_url()
Parameters
provider_mod- Provider module implementing the Provider behaviormodel- ReqLLM.Model struct containing provider informationopts- Options keyword list that may contain :base_url
Examples
iex> model = %ReqLLM.Model{provider: :openai, model: "gpt-4"}
iex> ReqLLM.Provider.Options.effective_base_url(ReqLLM.Providers.OpenAI, model, [])
"https://api.openai.com/v1"
iex> opts = [base_url: "https://custom.example.com"]
iex> ReqLLM.Provider.Options.effective_base_url(ReqLLM.Providers.OpenAI, model, opts)
"https://custom.example.com"
Extracts only generation options from a mixed options list.
Unlike extract_provider_options/1, this returns only the generation
options without the unused remainder.
Examples
iex> mixed_opts = [temperature: 0.7, custom_param: "value", max_tokens: 100]
iex> ReqLLM.Provider.Options.extract_generation_opts(mixed_opts)
[temperature: 0.7, max_tokens: 100]
Extracts provider-specific options from a mixed options list.
This is useful for separating standard options from provider-specific ones.
Examples
iex> opts = [temperature: 0.7, max_tokens: 100, custom_param: "value"]
iex> ReqLLM.Provider.Options.extract_provider_options(opts)
{[temperature: 0.7, max_tokens: 100], [custom_param: "value"]}
Filters generation options to only include supported keys.
This is a pure filter function that doesn't validate - it just removes unsupported keys from the options.
Examples
iex> opts = [temperature: 0.7, unsupported_key: "value", max_tokens: 100]
iex> ReqLLM.Provider.Options.filter_generation_options(opts, [:temperature, :max_tokens])
[temperature: 0.7, max_tokens: 100]
Returns the core generation options schema.
Returns a NimbleOptions schema that contains only the requested generation keys.
Examples
iex> schema = ReqLLM.Provider.Options.generation_subset_schema([:temperature, :max_tokens])
iex> NimbleOptions.validate([temperature: 0.7], schema)
{:ok, [temperature: 0.7]}
Merges options with defaults, respecting user-provided overrides.
Examples
iex> defaults = [temperature: 0.7, max_tokens: 1000]
iex> user_opts = [temperature: 0.9]
iex> result = ReqLLM.Provider.Options.merge_with_defaults(user_opts, defaults)
iex> result[:temperature]
0.9
iex> result[:max_tokens]
1000
Main processing function - validates, translates, and composes options.
This is the primary public API for option processing. It handles:
- Provider key collision detection (prevents shadowing core options)
- Validation against composed schema (core + provider options)
- Provider-specific option translation
- Internal option preservation
- Error wrapping for consistency
Parameters
provider_mod- Provider module implementing the Provider behavioroperation- Operation type (:chat, :embedding, :object, etc.)model- ReqLLM.Model structopts- Raw user options keyword list
Returns
{:ok, processed_opts} or {:error, wrapped_error}
Examples
model = %ReqLLM.Model{provider: :openai, model: "gpt-4"}
opts = [
temperature: 0.7,
provider_options: [dimensions: 512, encoding_format: "float"]
]
{:ok, processed} = Options.process(MyProvider, :chat, model, opts)
@spec process!(module(), atom(), ReqLLM.Model.t(), keyword()) :: keyword()
Same as process/4 but raises on error.
Validates generation options against a subset of supported keys.
Examples
iex> ReqLLM.Provider.Options.validate_generation_options(
...> [temperature: 0.7, max_tokens: 100],
...> only: [:temperature, :max_tokens]
...> )
{:ok, [temperature: 0.7, max_tokens: 100]}