Ollama provider – self-hosted OpenAI-compatible Chat Completions API.
Implementation
Uses built-in OpenAI-style encoding/decoding defaults.
Ollama exposes an OpenAI-compatible API at /v1, so no custom
request/response handling is needed.
Self-Hosted Configuration
Ollama is a self-hosted inference server. Users must:
- Install and run Ollama (https://ollama.com)
- Pull a model (e.g.,
ollama pull llama3.2) - Optionally set a custom
base_urlif not running on localhost
Authentication
Ollama does not require authentication by default.
Set OLLAMA_API_KEY to any non-empty value (it is required by the
provider interface but not validated by Ollama).
Configuration
# Add to .env file (automatically loaded)
OLLAMA_API_KEY=ollamaExamples
# Basic usage with default localhost
Condukt.run(agent, "Hello!",
model: "ollama:llama3.2"
)
# With custom base_url for a remote Ollama instance
MyAgent.start_link(
model: "ollama:llama3.2",
base_url: "http://my-server:11434/v1"
)Summary
Functions
Default implementation of attach/3.
Default implementation of attach_stream/4.
Default implementation of build_body/1.
Default implementation of decode_response/1.
Default implementation of decode_stream_event/2.
Callback implementation for ReqLLM.Provider.default_env_key/0.
Default implementation of encode_body/1.
Default implementation of extract_usage/2.
Default implementation of prepare_request/4.
Default implementation of translate_options/3.
Functions
Default implementation of attach/3.
Sets up Bearer token authentication and standard pipeline steps.
Default implementation of attach_stream/4.
Builds complete streaming requests using OpenAI-compatible format.
Default implementation of build_body/1.
Builds request body using OpenAI-compatible format for chat and embedding operations.
Default implementation of decode_response/1.
Handles success/error responses with standard ReqLLM.Response creation.
Default implementation of decode_stream_event/2.
Decodes SSE events using OpenAI-compatible format.
Callback implementation for ReqLLM.Provider.default_env_key/0.
Default implementation of encode_body/1.
Encodes request body using OpenAI-compatible format for chat and embedding operations.
Default implementation of extract_usage/2.
Extracts usage data from standard usage field in response body.
Default implementation of prepare_request/4.
Handles :chat, :object, and :embedding operations using OpenAI-compatible patterns.
Default implementation of translate_options/3.
Pass-through implementation that returns options unchanged.