ReqLLM.Providers.OpenAI (ReqLLM v1.0.0-rc.8)

View Source

OpenAI provider implementation with dual-driver architecture for Chat and Responses APIs.

Architecture

This provider uses a metadata-driven routing system to dispatch requests to specialized API drivers:

The provider automatically routes requests based on the "api" field in model metadata:

  • "api": "chat" → uses ChatAPI driver (default)
  • "api": "responses" → uses ResponsesAPI driver

Capabilities

Chat Completions API (ChatAPI)

  • Text generation with GPT models
  • Streaming responses with usage tracking
  • Tool calling (function calling)
  • Multi-modal inputs (text and images)
  • Embeddings generation
  • Full OpenAI Chat API compatibility

Responses API (ResponsesAPI)

  • Extended reasoning for o1/o3/o4/GPT-4.1/GPT-5 models
  • Reasoning effort control (minimal, low, medium, high)
  • Streaming with reasoning token tracking
  • Tool calling with responses-specific format
  • Enhanced usage metrics including :reasoning_tokens

Usage Normalization

Both drivers normalize usage metrics to provide consistent field names:

  • :reasoning_tokens - Primary field for reasoning token count (ResponsesAPI)
  • :reasoning - Backward-compatibility alias (deprecated, use :reasoning_tokens)

Deprecation Notice: The :reasoning usage key is deprecated in favor of :reasoning_tokens and will be removed in a future version.

Configuration

Set your OpenAI API key via environment variable or application config:

# Option 1: Environment variable (automatically loaded from .env via dotenvy)
OPENAI_API_KEY=sk-...

# Option 2: Store in application config
ReqLLM.put_key(:openai_api_key, "sk-...")

Examples

# Simple text generation (ChatAPI)
model = ReqLLM.Model.from("openai:gpt-4")
{:ok, response} = ReqLLM.generate_text(model, "Hello!")

# Reasoning model (ResponsesAPI)
model = ReqLLM.Model.from("openai:o1")
{:ok, response} = ReqLLM.generate_text(model, "Solve this problem...")
response.usage.reasoning_tokens  # Reasoning tokens used

# Streaming with reasoning
{:ok, stream} = ReqLLM.stream_text(model, "Complex question", stream: true)

# Tool calling (both APIs)
tools = [%ReqLLM.Tool{name: "get_weather", ...}]
{:ok, response} = ReqLLM.generate_text(model, "What's the weather?", tools: tools)

# Embeddings (ChatAPI)
{:ok, embedding} = ReqLLM.generate_embedding("openai:text-embedding-3-small", "Hello world")

# Reasoning effort (ResponsesAPI)
{:ok, response} = ReqLLM.generate_text(
  "openai:gpt-5",
  "Hard problem",
  provider_options: [reasoning_effort: :high]
)

Summary

Functions

attach(request, model_input, user_opts)

Default implementation of attach/3.

attach_stream(model, context, opts, finch_name)

Custom attach_stream to route reasoning models to /v1/responses endpoint for streaming.

decode_response(request_response)

Custom decode_response to delegate to the selected API module.

decode_sse_event(event, model)

Custom decode_sse_event to route based on model API type.

default_base_url()
default_env_key()

Callback implementation for ReqLLM.Provider.default_env_key/0.

default_provider_opts()
encode_body(request)

Custom body encoding that delegates to the selected API module.

extract_usage(body, model)

Default implementation of extract_usage/2.

metadata()
prepare_request(operation, model_spec, input, opts)

Custom prepare_request to route reasoning models to /v1/responses endpoint.

provider_extended_generation_schema()
provider_id()
provider_schema()
supported_provider_options()
translate_options(operation, model, opts)

Translates provider-specific options for different model types.

Functions

attach(request, model_input, user_opts)

Default implementation of attach/3.

Sets up Bearer token authentication and standard pipeline steps.

attach_stream(model, context, opts, finch_name)

Custom attach_stream to route reasoning models to /v1/responses endpoint for streaming.

decode_response(request_response)

Custom decode_response to delegate to the selected API module.

Auto-detects the API type from the response body if not already set. This is important for fixture replay where api_mod isn't set during prepare_request.

decode_sse_event(event, model)

Custom decode_sse_event to route based on model API type.

default_base_url()

default_env_key()

Callback implementation for ReqLLM.Provider.default_env_key/0.

default_provider_opts()

encode_body(request)

Custom body encoding that delegates to the selected API module.

extract_usage(body, model)

Default implementation of extract_usage/2.

Extracts usage data from standard usage field in response body.

metadata()

prepare_request(operation, model_spec, input, opts)

Custom prepare_request to route reasoning models to /v1/responses endpoint.

  • :chat operations detect model type and route to appropriate endpoint
  • :object operations maintain OpenAI-specific token handling

provider_extended_generation_schema()

provider_id()

provider_schema()

supported_provider_options()

translate_options(operation, model, opts)

Translates provider-specific options for different model types.

Uses a profile-based system to apply model-specific parameter transformations. Profiles are resolved from model metadata and capabilities, making it easy to add new model-specific rules without modifying this function.

Reasoning Models

Models with reasoning capabilities (o1, o3, o4, gpt-5, etc.) have special parameter requirements:

  • max_tokens is renamed to max_completion_tokens
  • temperature may be unsupported or restricted depending on the specific model

Returns

{translated_opts, warnings} where warnings is a list of transformation messages.