ReqLLM.Providers.XAI (ReqLLM v1.0.0)

xAI (Grok) provider – OpenAI Chat Completions compatible with xAI's models and features.

Implementation

Uses built-in OpenAI-style encoding/decoding defaults with xAI-specific enhancements.

Structured Outputs

The provider supports two modes for structured outputs via ReqLLM.generate_object/4:

Native Structured Outputs (Recommended)

For models >= grok-2-1212, uses xAI's native response_format with json_schema:

Guaranteed schema compliance enforced at generation time
Lower token overhead (no tool definitions in prompt)
More reliable than tool calling approach

Supported models:

grok-2-1212 and grok-2-vision-1212
grok-beta
All grok-3 and grok-4 variants
All future models

Tool Calling Fallback

For legacy models or when other tools are present:

Uses a synthetic structured_output tool with forced tool_choice
Works on all models as fallback
Automatically used for grok-2 and grok-2-vision (pre-1212 versions)

The mode is automatically selected based on model capabilities, or can be explicitly controlled via :xai_structured_output_mode option.

xAI-Specific Extensions

Beyond standard OpenAI parameters, xAI supports:

max_completion_tokens - Preferred over max_tokens for Grok-4 models
reasoning_effort - Reasoning level (low, medium, high) for Grok-3 mini models only
search_parameters - Live Search configuration with web search capabilities
parallel_tool_calls - Allow parallel function calls (default: true)
stream_options - Streaming configuration (include_usage)
xai_structured_output_mode - Control structured output implementation (:auto, :json_schema, :tool_strict)

Model Compatibility Notes

Native structured outputs supported on models >= grok-2-1212 and grok-2-vision-1212
reasoning_effort is only supported for grok-3-mini and grok-3-mini-fast models
Grok-4 models do not support stop, presence_penalty, or frequency_penalty
Live Search via search_parameters incurs additional costs per source

Schema Constraints (Native Mode)

xAI's native structured outputs have these JSON Schema limitations:

No minLength/maxLength for strings
No minItems/maxItems/minContains/maxContains for arrays
No pattern constraints
No allOf (must be expanded/flattened)
anyOf is supported

The provider automatically sanitizes schemas by removing unsupported constraints and enforcing additionalProperties: false on root objects.

See provider_schema/0 for the complete xAI-specific schema and ReqLLM.Provider.Options for inherited OpenAI parameters.

Configuration

# Add to .env file (automatically loaded)
XAI_API_KEY=xai-...

Examples

# Automatic mode selection (recommended)
{:ok, response} =
  ReqLLM.generate_object(
    "xai:grok-4",
    "Generate a person profile",
    %{
      type: :object,
      properties: %{
        name: %{type: :string},
        age: %{type: :integer}
      }
    }
  )

# Explicit mode control
{:ok, response} =
  ReqLLM.generate_object(
    "xai:grok-4",
    "Generate a person profile",
    schema,
    provider_options: [xai_structured_output_mode: :json_schema]
  )

Summary

Functions

attach(request, model_input, user_opts)

Default implementation of attach/3.

attach_stream(model, context, opts, finch_name)

Custom attach_stream that ensures translate_options is called for streaming requests.

decode_response(request_response)

Decodes xAI API responses based on operation type and streaming mode.

decode_stream_event(event, model)

Default implementation of decode_stream_event/2.

default_base_url()

default_env_key()

Callback implementation for ReqLLM.Provider.default_env_key/0.

default_provider_opts()

determine_output_mode(model, opts)

Determine the structured output mode for a model and options.

encode_body(request)

Custom body encoding that adds xAI-specific extensions to the default OpenAI-compatible format.

extract_usage(body, model)

Default implementation of extract_usage/2.

metadata()

prepare_request(operation, model_spec, input, opts)

Custom prepare_request for :object operations using xAI native structured outputs.

provider_extended_generation_schema()

provider_id()

provider_schema()

supported_provider_options()

supports_native_structured_outputs?(model)

Check if a model supports native structured outputs via response_format with json_schema.

supports_strict_tools?(model)

Check if a model supports strict tool calling.

translate_options(operation, model, opts)

Default implementation of translate_options/3.

Functions

attach(request, model_input, user_opts)

Default implementation of attach/3.

Sets up Bearer token authentication and standard pipeline steps.

attach_stream(model, context, opts, finch_name)

Custom attach_stream that ensures translate_options is called for streaming requests.

This is necessary because the default streaming path doesn't call translate_options, which means xAI-specific option normalization (max_tokens -> max_completion_tokens, reasoning_effort translation, etc.) wouldn't be applied to streaming requests.

decode_response(request_response)

Decodes xAI API responses based on operation type and streaming mode.

Response Handling

Chat operations: Converts to ReqLLM.Response struct
Streaming: Creates response with chunk stream
Non-streaming: Merges context with assistant response

Error Handling

Non-200 status codes are converted to ReqLLM.Error.API.Response exceptions.

decode_stream_event(event, model)

Default implementation of decode_stream_event/2.

Decodes SSE events using OpenAI-compatible format.

default_base_url()

default_env_key()

Callback implementation for ReqLLM.Provider.default_env_key/0.

default_provider_opts()

determine_output_mode(model, opts)

@spec determine_output_mode(
  ReqLLM.Model.t(),
  keyword()
) :: :json_schema | :tool_strict

Determine the structured output mode for a model and options.

Mode Selection Logic

If explicit xai_structured_output_mode is set, validate and use it
If response_format with json_schema is present in options, force :json_schema
If :auto:
- Use :json_schema when model supports it AND no other tools present
- Otherwise use :tool_strict

Examples

iex> determine_output_mode(%ReqLLM.Model{model: "grok-3"}, [])
:json_schema

iex> determine_output_mode(%ReqLLM.Model{model: "grok-2"}, [])
:tool_strict

iex> determine_output_mode(%ReqLLM.Model{model: "grok-3"}, tools: [%{name: "other"}])
:tool_strict

encode_body(request)

Custom body encoding that adds xAI-specific extensions to the default OpenAI-compatible format.

Adds support for:

max_completion_tokens (preferred over max_tokens for Grok-4)
reasoning_effort (low, medium, high) for grok-3-mini models
search_parameters (Live Search configuration)
parallel_tool_calls (with skip for true default)
stream_options (streaming configuration)

extract_usage(body, model)

Default implementation of extract_usage/2.

Extracts usage data from standard usage field in response body.

metadata()

prepare_request(operation, model_spec, input, opts)

Custom prepare_request for :object operations using xAI native structured outputs.

Determines the appropriate mode (:json_schema or :tool_strict) and delegates to the corresponding preparation function. Ensures adequate token limits for structured outputs.

provider_extended_generation_schema()

provider_id()

provider_schema()

supported_provider_options()

supports_native_structured_outputs?(model)

@spec supports_native_structured_outputs?(ReqLLM.Model.t() | binary()) :: boolean()

Check if a model supports native structured outputs via response_format with json_schema.

Prefers metadata flags when available, with heuristic fallback for models without metadata.

Heuristic Rules

Exact "grok-2" or "grok-2-vision" → false (legacy models)
Model starting with "grok-2-" or "grok-2-vision-" with suffix < "1212" → false
Everything else → true (grok-3+, grok-4+, grok-beta, grok-2-1212+)

Examples

iex> supports_native_structured_outputs?(%ReqLLM.Model{model: "grok-2"})
false

iex> supports_native_structured_outputs?(%ReqLLM.Model{model: "grok-2-1212"})
true

iex> supports_native_structured_outputs?("grok-3")
true

supports_strict_tools?(model)

@spec supports_strict_tools?(ReqLLM.Model.t() | binary()) :: boolean()

Check if a model supports strict tool calling.

All xAI models support strict tools.

translate_options(operation, model, opts)

Default implementation of translate_options/3.

Pass-through implementation that returns options unchanged.