Get all supported capabilities for a model.

Returns a list of capability atoms that the model supports based on provider metadata.

Parameters

• model_spec - Model specification in various formats

Examples

ReqLLM.capabilities("anthropic:claude-3-haiku")
#=> [:max_tokens, :system_prompt, :temperature, :tools, :streaming]

model = %ReqLLM.Model{provider: :anthropic, model: "claude-3-sonnet"}
ReqLLM.capabilities(model)
#=> [:max_tokens, :system_prompt, :temperature, :tools, :streaming, :reasoning]

@spec context([struct()] | struct() | String.t()) :: ReqLLM.Context.t()

Creates a context from a list of messages, a single message struct, or a string.

Parameters

• messages - List of Message structs, a single Message struct, or a string

Examples

messages = [
  ReqLLM.Context.system("You are helpful"),
  ReqLLM.Context.user("Hello!")
]
ctx = ReqLLM.context(messages)
# Now you can use Enum functions on the context
user_msgs = ctx |> Enum.filter(&(&1.role == :user))

# Single message struct
ctx = ReqLLM.context(ReqLLM.Context.user("Hello!"))

# String prompt
ctx = ReqLLM.context("Hello!")

@spec cosine_similarity([number()], [number()]) :: float()

Calculates cosine similarity between two embedding vectors.

Equivalent to Vercel AI SDK's cosineSimilarity() function. Returns a similarity score between -1 and 1, where:

• 1.0 indicates identical vectors (maximum similarity)
• 0.0 indicates orthogonal vectors (no similarity)
• -1.0 indicates opposite vectors (maximum dissimilarity)

Parameters

• embedding_a - First embedding vector (list of numbers)
• embedding_b - Second embedding vector (list of numbers)

Examples

# Identical vectors
ReqLLM.cosine_similarity([1.0, 0.0, 0.0], [1.0, 0.0, 0.0])
#=> 1.0

# Orthogonal vectors
ReqLLM.cosine_similarity([1.0, 0.0], [0.0, 1.0])
#=> 0.0

# Opposite vectors
ReqLLM.cosine_similarity([1.0, 0.0], [-1.0, 0.0])
#=> -1.0

# Similar vectors
ReqLLM.cosine_similarity([0.5, 0.8, 0.3], [0.6, 0.7, 0.4])
#=> 0.9487...

Generates embeddings for single or multiple text inputs.

Accepts either a single string or a list of strings, automatically handling both cases using pattern matching.

Parameters

• model_spec - Model specification in various formats
• input - Text string or list of text strings to generate embeddings for
• opts - Additional options (keyword list)

Options

• :dimensions - Number of dimensions for embeddings
• :provider_options - Provider-specific options

Examples

# Single text input
{:ok, embedding} = ReqLLM.embed("openai:text-embedding-3-small", "Hello world")
#=> {:ok, [0.1, -0.2, 0.3, ...]}

# Multiple text inputs
{:ok, embeddings} = ReqLLM.embed(
  "openai:text-embedding-3-small",
  ["Hello", "World"]
)
#=> {:ok, [[0.1, -0.2, ...], [0.3, 0.4, ...]]}

Generates structured data using an AI model with schema validation.

Equivalent to Vercel AI SDK's generateObject() function, this method generates structured data according to a provided schema and validates the output against that schema.

Parameters

• model_spec - Model specification in various formats
• messages - Text prompt or list of messages
• schema - Schema definition for structured output (NimbleOptions schema or JSON Schema map)
• opts - Additional options (keyword list)

Options

• :temperature - Control randomness in responses (0.0 to 2.0)
• :max_tokens - Limit the length of the response
• :provider_options - Provider-specific options

Examples

# Generate a structured object
schema = [
  name: [type: :string, required: true],
  age: [type: :pos_integer, required: true]
]
{:ok, object} = ReqLLM.generate_object("anthropic:claude-3-sonnet", "Generate a person", schema)
#=> {:ok, %{name: "John Doe", age: 30}}

# Generate an array of objects (requires JSON Schema-capable provider like OpenAI)
person_schema = ReqLLM.Schema.to_json([
  name: [type: :string, required: true],
  age: [type: :pos_integer, required: true]
])

array_schema = %{"type" => "array", "items" => person_schema}

{:ok, response} = ReqLLM.generate_object(
  "openai:gpt-4o",
  "Generate 3 heroes",
  array_schema
)
# Note: Array outputs currently require manual extraction from the response

# Recommended: Use Zoi for cleaner array schema definition
person = Zoi.object(%{
  name: Zoi.string(),
  age: Zoi.number()
})

array_schema = Zoi.array(person) |> ReqLLM.Schema.to_json()

{:ok, response} = ReqLLM.generate_object(
  "openai:gpt-4o",
  "Generate 3 heroes",
  array_schema
)

Note: Top-level non-object outputs (arrays, enums) require raw JSON Schema and are only supported by providers with native JSON Schema capabilities (e.g., OpenAI). Most providers only support object-type schemas. For cleaner array schema definitions, consider using the Zoi library as shown above.

Generates structured data using an AI model, returning only the object content.

This is a convenience function that extracts just the object from the response. For access to usage metadata and other response data, use generate_object/4.

Parameters

Same as generate_object/4.

Examples

ReqLLM.generate_object!("anthropic:claude-3-sonnet", "Generate a person", schema)
#=> %{name: "John Doe", age: 30}

Generates text using an AI model with full response metadata.

Returns a canonical ReqLLM.Response which includes usage data, context, and metadata. For simple text-only results, use generate_text!/3.

Parameters

• model_spec - Model specification in various formats
• messages - Text prompt or list of messages
• opts - Additional options (keyword list)

Options

• :temperature - Control randomness in responses (0.0 to 2.0)
• :max_tokens - Limit the length of the response
• :top_p - Nucleus sampling parameter
• :presence_penalty - Penalize new tokens based on presence
• :frequency_penalty - Penalize new tokens based on frequency
• :tools - List of tool definitions
• :tool_choice - Tool choice strategy
• :system_prompt - System prompt to prepend
• :provider_options - Provider-specific options

Examples

{:ok, response} = ReqLLM.generate_text("anthropic:claude-3-sonnet", "Hello world")
ReqLLM.Response.text(response)
#=> "Hello! How can I assist you today?"

# Access usage metadata
ReqLLM.Response.usage(response)
#=> %{input_tokens: 10, output_tokens: 8}

Generates text using an AI model, returning only the text content.

This is a convenience function that extracts just the text from the response. For access to usage metadata and other response data, use generate_text/3. Raises on error.

Parameters

Same as generate_text/3.

Examples

ReqLLM.generate_text!("anthropic:claude-3-sonnet", "Hello world")
#=> "Hello! How can I assist you today?"

@spec get_key(atom() | String.t()) :: String.t() | nil

Gets an API key from application config or system environment.

Keys from .env files are automatically loaded via dotenvy at startup.

Parameters

• key - The configuration key (atom or string)

Examples

ReqLLM.get_key(:anthropic_api_key)
ReqLLM.get_key("ANTHROPIC_API_KEY")  # Auto-loaded from .env

@spec json_schema(keyword(), keyword()) :: map()

Creates a JSON schema object compatible with ReqLLM.

Equivalent to Vercel AI SDK's jsonSchema() helper, this function creates schema objects for structured data generation and validation.

Parameters

• schema - NimbleOptions schema definition (keyword list)
• opts - Additional options (optional)

Options

• :validate - Custom validation function (optional)

Examples

# Basic schema
schema = ReqLLM.json_schema([
  name: [type: :string, required: true, doc: "User name"],
  age: [type: :integer, doc: "User age"]
])

# Schema with custom validation
schema = ReqLLM.json_schema(
  [email: [type: :string, required: true]],
  validate: fn value ->
    if String.contains?(value["email"], "@") do
      {:ok, value}
    else
      {:error, "Invalid email format"}
    end
  end
)

@spec model(String.t() | {atom(), keyword()} | struct()) ::
  {:ok, struct()} | {:error, term()}

Creates a model struct from various specifications.

Parameters

• model_spec - Model specification in various formats:
  • String format: "anthropic:claude-3-sonnet"
  • Tuple format: {:anthropic, "claude-3-sonnet", temperature: 0.7}
  • Model struct: %ReqLLM.Model{}

Examples

ReqLLM.model("anthropic:claude-3-sonnet")
#=> {:ok, %ReqLLM.Model{provider: :anthropic, model: "claude-3-sonnet"}}

ReqLLM.model({:anthropic, "claude-3-sonnet", temperature: 0.5})
#=> {:ok, %ReqLLM.Model{provider: :anthropic, model: "claude-3-sonnet", temperature: 0.5}}

@spec provider(atom()) ::
  {:ok, module()}
  | {:error,
     ReqLLM.Error.Invalid.Provider.t()
     | ReqLLM.Error.Invalid.Provider.NotImplemented.t()}

Gets a provider module from the registry.

Parameters

• provider - Provider identifier (atom)

Examples

ReqLLM.provider(:anthropic)
#=> {:ok, ReqLLM.Providers.Anthropic}

ReqLLM.provider(:unknown)
#=> {:error, %ReqLLM.Error.Invalid.Provider{provider: :unknown}}

@spec put_key(atom(), term()) :: :ok

Stores an API key in application configuration.

Keys from .env files are automatically loaded via dotenvy at startup. This function is useful for programmatic key management in tests or at runtime.

Parameters

• key - The configuration key (atom)
• value - The value to store

Examples

ReqLLM.put_key(:anthropic_api_key, "sk-ant-...")

Streams structured data generation using an AI model with schema validation.

Equivalent to Vercel AI SDK's streamObject() function, this method streams structured data generation according to a provided schema.

Parameters

• model_spec - Model specification in various formats
• messages - Text prompt or list of messages
• schema - Schema definition for structured output
• opts - Additional options (keyword list)

Options

Same as generate_object/4.

Examples

# Stream structured object generation
schema = [
  name: [type: :string, required: true],
  description: [type: :string, required: true]
]
{:ok, stream} = ReqLLM.stream_object("anthropic:claude-3-sonnet", "Generate a character", schema)
stream |> Enum.each(&IO.inspect/1)

DEPRECATED: This function will be removed in a future version.

The streaming API has been redesigned to return a composite StreamResponse struct that provides both the stream and metadata. Use stream_object/4 instead:

{:ok, response} = ReqLLM.stream_object(model, messages, schema)
response.stream |> Enum.each(&IO.inspect/1)

For simple object extraction, use:

object = ReqLLM.StreamResponse.object(response)

Legacy Parameters

Same as stream_object/4.

Legacy Examples

ReqLLM.stream_object!("anthropic:claude-3-sonnet", "Generate a character", schema)
|> Enum.each(&IO.inspect/1)

Streams text generation using an AI model with concurrent metadata collection.

Returns a ReqLLM.StreamResponse that provides both real-time token streaming and asynchronous metadata collection (usage, finish_reason). This enables zero-latency content delivery while collecting billing/usage data concurrently.

The streaming implementation uses Finch directly for production-grade performance with HTTP/2 multiplexing and automatic connection pooling.

Parameters

Same as generate_text/3.

Returns

• {:ok, stream_response} - StreamResponse with stream and metadata task
• {:error, reason} - Request failed or invalid parameters

Examples

# Real-time streaming
{:ok, response} = ReqLLM.stream_text("anthropic:claude-3-sonnet", "Tell me a story")
response
|> ReqLLM.StreamResponse.tokens()
|> Stream.each(&IO.write/1)
|> Stream.run()

# Concurrent metadata collection
usage = ReqLLM.StreamResponse.usage(response)
#=> %{input_tokens: 15, output_tokens: 42, total_cost: 0.087}

# Simple text collection
text = ReqLLM.StreamResponse.text(response)

# Backward compatibility
{:ok, legacy_response} = ReqLLM.StreamResponse.to_response(response)

StreamResponse Fields

• stream - Lazy enumerable of StreamChunk structs for real-time consumption
• metadata_task - Concurrent Task collecting usage and finish_reason
• cancel - Function to terminate streaming and cleanup resources
• model - Model specification that generated this response
• context - Updated conversation context including assistant's response

Performance Notes

The stream is lazy and supports backpressure. Metadata collection happens concurrently and won't block token delivery. Use cancellation for early termination to free resources.

DEPRECATED: This function will be removed in a future version.

The streaming API has been redesigned to return a composite StreamResponse struct that provides both the stream and metadata. Use stream_text/3 instead:

{:ok, response} = ReqLLM.stream_text(model, messages)
response.stream |> Enum.each(&IO.write/1)

For simple text extraction, use:

text = ReqLLM.StreamResponse.text(response)

Legacy Behavior

This function currently returns :ok and logs a deprecation warning. It will be formally removed in the next major version.

@spec tool(keyword()) :: ReqLLM.Tool.t()

Creates a Tool struct for AI model function calling.

Equivalent to Vercel AI SDK's tool() helper, providing type-safe tool definitions with parameter validation. This is a convenience function for creating ReqLLM.Tool structs.

Parameters

• opts - Tool definition options (keyword list)

Options

• :name - Tool name (required, must be valid identifier)
• :description - Tool description for AI model (required)
• :parameters - Parameter schema as NimbleOptions keyword list (optional)
• :callback - Callback function or MFA tuple (required)

Examples

# Simple tool with no parameters
tool = ReqLLM.tool(
  name: "get_time",
  description: "Get the current time",
  callback: fn _args -> {:ok, DateTime.utc_now()} end
)

# Tool with parameters
weather_tool = ReqLLM.tool(
  name: "get_weather",
  description: "Get current weather for a location",
  parameters: [
    location: [type: :string, required: true, doc: "City name"],
    units: [type: :string, default: "metric", doc: "Temperature units"]
  ],
  callback: {WeatherAPI, :fetch_weather}
)