Model inference options.
Provides typed, validated options for controlling model behavior. All options are optional - only set what you need to change from defaults.
Summary
Functions
Build options from keyword list with validation.
Build options, raising on validation error.
Set frequency penalty (0.0-2.0).
Create options from a map.
Set mirostat mode (0 = disabled, 1 = v1, 2 = v2).
Create a new empty options struct.
Set context window size.
Set number of GPU layers (-1 = all, 0 = CPU only).
Set max tokens to generate (-1 = unlimited).
Enable/disable NUMA.
Set presence penalty (0.0-2.0).
Set repeat penalty (1.0 = no penalty).
Set random seed for reproducibility.
Set stop sequences.
Set temperature (>= 0.0).
Convert options struct to map for API request.
Set top_k sampling (0 = disabled).
Set top_p/nucleus sampling (0.0-1.0).
Enable/disable memory mapping.
Types
@type t() :: %Ollixir.Options{ embedding_only: boolean() | nil, extra: map(), f16_kv: boolean() | nil, frequency_penalty: float() | nil, logits_all: boolean() | nil, low_vram: boolean() | nil, main_gpu: non_neg_integer() | nil, mirostat: 0 | 1 | 2 | nil, mirostat_eta: float() | nil, mirostat_tau: float() | nil, num_batch: pos_integer() | nil, num_ctx: pos_integer() | nil, num_gpu: integer() | nil, num_keep: integer() | nil, num_predict: integer() | nil, num_thread: pos_integer() | nil, numa: boolean() | nil, penalize_newline: boolean() | nil, presence_penalty: float() | nil, repeat_last_n: integer() | nil, repeat_penalty: float() | nil, seed: integer() | nil, stop: [String.t()] | nil, temperature: float() | nil, tfs_z: float() | nil, top_k: non_neg_integer() | nil, top_p: float() | nil, typical_p: float() | nil, use_mlock: boolean() | nil, use_mmap: boolean() | nil, vocab_only: boolean() | nil }
Functions
Build options from keyword list with validation.
Build options, raising on validation error.
Set frequency penalty (0.0-2.0).
Create options from a map.
Set mirostat mode (0 = disabled, 1 = v1, 2 = v2).
@spec new() :: t()
Create a new empty options struct.
@spec num_ctx(t(), pos_integer()) :: t()
Set context window size.
Set number of GPU layers (-1 = all, 0 = CPU only).
Set max tokens to generate (-1 = unlimited).
Enable/disable NUMA.
Set presence penalty (0.0-2.0).
Set repeat penalty (1.0 = no penalty).
Set random seed for reproducibility.
Set stop sequences.
Set temperature (>= 0.0).
Convert options struct to map for API request.
Excludes nil values and merges custom keys.
@spec top_k(t(), non_neg_integer()) :: t()
Set top_k sampling (0 = disabled).
Set top_p/nucleus sampling (0.0-1.0).
Enable/disable memory mapping.