ReqLLM.Provider.Defaults (ReqLLM v1.0.0-rc.8)

View Source

Default implementations for common provider behavior patterns.

This module extracts shared logic between OpenAI-compatible providers (OpenAI, Groq, etc.) into reusable runtime functions and a __using__ macro that provides default callback implementations.

Usage

defmodule MyProvider do
  @behaviour ReqLLM.Provider
  use ReqLLM.Provider.DSL, [...]
  use ReqLLM.Provider.Defaults

  # All default implementations are available and overridable
  # Override only what you need to customize
end

Design Principles

  • Runtime functions are pure and testable
  • Provider module is passed as first argument to access attributes
  • All defaults are defoverridable for selective customization
  • Providers can override individual methods or use helper functions directly

Default Implementations

The following methods get default implementations:

  • prepare_request/4 - Standard chat/object/embedding request preparation
  • attach/3 - OAuth Bearer authentication and standard pipeline steps
  • encode_body/1 - OpenAI-compatible request body encoding
  • decode_response/1 - Standard response decoding with error handling
  • extract_usage/2 - Usage extraction from standard usage field
  • translate_options/3 - No-op translation (pass-through)
  • decode_sse_event/2 - OpenAI-compatible SSE event decoding
  • attach_stream/4 - OpenAI-compatible streaming request building
  • display_name/0 - Human-readable provider name from provider_id

Runtime Functions

All default implementations delegate to pure runtime functions that can be called independently:

Customization Examples

# Override just the body encoding while keeping everything else
def encode_body(request) do
  request
  |> ReqLLM.Provider.Defaults.default_encode_body()
  |> add_custom_headers()
end

# Use runtime functions directly for testing
test "encoding produces correct format" do
  request = build_test_request()
  encoded = ReqLLM.Provider.Defaults.default_encode_body(request)
  assert encoded.body =~ ~s("model":")
end

Summary

Functions

__using__(opts)

Provides default implementations for common provider patterns.

build_openai_chat_body(request)

Build a complete OpenAI-style chat body from a Req request.

decode_response_body_openai_format(data, model)

Decodes OpenAI-format response body to ReqLLM.Response.

default_attach(provider_mod, request, model_input, user_opts)

Default attachment implementation with Bearer token auth and standard pipeline steps.

default_attach_stream(provider_mod, model, context, opts, finch_name)

Default implementation of attach_stream/4.

default_decode_response(arg)

Default response decoding with success/error handling.

default_decode_sse_event(arg1, model)

Default SSE event decoding for OpenAI-compatible providers.

default_display_name(provider_mod)

Default display name implementation.

default_encode_body(request)

Default body encoding for OpenAI-compatible APIs.

default_extract_usage(body, arg2)

Default usage extraction from standard usage field.

default_translate_options(operation, model, opts)

Default options translation (pass-through).

encode_context_to_openai_format(context, model_name)

Encodes ReqLLM.Context to OpenAI-compatible format.

prepare_chat_request(provider_mod, model_spec, prompt, opts)

Prepares a chat completion request.

prepare_embedding_request(provider_mod, model_spec, text, opts)

Prepares an embedding generation request.

prepare_object_request(provider_mod, model_spec, prompt, opts)

Prepares an object generation request using tool calling.

prepare_request(provider_mod, operation, model_spec, input, opts)

Runtime implementation of prepare_request/4.

Functions

__using__(opts)

(macro)

Provides default implementations for common provider patterns.

All methods are defoverridable, so providers can selectively override only the methods they need to customize.

build_openai_chat_body(request)

Build a complete OpenAI-style chat body from a Req request.

This helper function encodes context, adds common options (temperature, max_tokens, etc.), converts tools to OpenAI schema, and handles stream flags. Providers can use this as a starting point and add provider-specific fields.

Example

def encode_body(req) do
  body = Defaults.build_openai_chat_body(req)
  |> Map.put(:my_provider_field, req.options[:my_provider_field])

  req
  |> Req.Request.put_header("content-type", "application/json")
  |> Map.put(:body, Jason.encode!(body))
end

decode_response_body_openai_format(data, model)

@spec decode_response_body_openai_format(map(), ReqLLM.Model.t()) ::
  {:ok, ReqLLM.Response.t()} | {:error, term()}

Decodes OpenAI-format response body to ReqLLM.Response.

This function moves the logic from ReqLLM.Response.Codec.Map directly into Provider.Defaults for the protocol removal refactoring.

default_attach(provider_mod, request, model_input, user_opts)

@spec default_attach(module(), Req.Request.t(), term(), keyword()) :: Req.Request.t()

Default attachment implementation with Bearer token auth and standard pipeline steps.

default_attach_stream(provider_mod, model, context, opts, finch_name)

@spec default_attach_stream(
  module(),
  ReqLLM.Model.t(),
  ReqLLM.Context.t(),
  keyword(),
  atom()
) :: {:ok, Finch.Request.t()} | {:error, Exception.t()}

Default implementation of attach_stream/4.

Builds complete streaming requests using OpenAI-compatible format and returns a complete Finch.Request.t() ready for streaming execution.

default_decode_response(arg)

@spec default_decode_response({Req.Request.t(), Req.Response.t()}) ::
  {Req.Request.t(), Req.Response.t() | Exception.t()}

Default response decoding with success/error handling.

default_decode_sse_event(arg1, model)

@spec default_decode_sse_event(map(), ReqLLM.Model.t()) :: [ReqLLM.StreamChunk.t()]

Default SSE event decoding for OpenAI-compatible providers.

This function moves the logic from ReqLLM.Response.Codec.Map directly into Provider.Defaults for the protocol removal refactoring.

default_display_name(provider_mod)

@spec default_display_name(module()) :: String.t()

Default display name implementation.

Returns a human-readable display name based on the provider_id from DSL, or falls back to capitalizing the module name.

default_encode_body(request)

@spec default_encode_body(Req.Request.t()) :: Req.Request.t()

Default body encoding for OpenAI-compatible APIs.

default_extract_usage(body, arg2)

@spec default_extract_usage(term(), ReqLLM.Model.t() | nil) ::
  {:ok, map()} | {:error, term()}

Default usage extraction from standard usage field.

default_translate_options(operation, model, opts)

@spec default_translate_options(atom(), ReqLLM.Model.t(), keyword()) ::
  {keyword(), [String.t()]}

Default options translation (pass-through).

encode_context_to_openai_format(context, model_name)

@spec encode_context_to_openai_format(ReqLLM.Context.t(), String.t()) :: map()

Encodes ReqLLM.Context to OpenAI-compatible format.

This function moves the logic from ReqLLM.Context.Codec.Map directly into Provider.Defaults for the protocol removal refactoring.

prepare_chat_request(provider_mod, model_spec, prompt, opts)

@spec prepare_chat_request(module(), term(), term(), keyword()) ::
  {:ok, Req.Request.t()} | {:error, Exception.t()}

Prepares a chat completion request.

prepare_embedding_request(provider_mod, model_spec, text, opts)

@spec prepare_embedding_request(module(), term(), term(), keyword()) ::
  {:ok, Req.Request.t()} | {:error, Exception.t()}

Prepares an embedding generation request.

prepare_object_request(provider_mod, model_spec, prompt, opts)

@spec prepare_object_request(module(), term(), term(), keyword()) ::
  {:ok, Req.Request.t()} | {:error, Exception.t()}

Prepares an object generation request using tool calling.

prepare_request(provider_mod, operation, model_spec, input, opts)

@spec prepare_request(module(), atom(), term(), term(), keyword()) ::
  {:ok, Req.Request.t()} | {:error, Exception.t()}

Runtime implementation of prepare_request/4.

Delegates to operation-specific preparation functions.