Represents the OpenAI ChatModel.

Parses and validates inputs for making a requests from the OpenAI Chat API.

Converts responses into more specialized LangChain data structures.

• https://github.com/openai/openai-cookbook/blob/main/examples/How_to_call_functions_with_chat_models.ipynb

ContentPart Types

OpenAI supports several types of content parts that can be combined in a single message:

Text Content

Basic text content is the default and most common type:

Message.new_user!("Hello, how are you?")copy

Image Content

OpenAI supports both base64-encoded images and image URLs:

# Using a base64 encoded image
Message.new_user!([
  ContentPart.text!("What's in this image?"),
  ContentPart.image!("base64_encoded_image_data", media: :jpg)
])

# Using an image URL
Message.new_user!([
  ContentPart.text!("Describe this image:"),
  ContentPart.image_url!("https://example.com/image.jpg")
])copy

For images, you can specify the detail level which affects token usage:

• detail: "low" - Lower resolution, fewer tokens
• detail: "high" - Higher resolution, more tokens
• detail: "auto" - Let the model decide

File Content

OpenAI supports both base64-encoded files and file IDs:

# Using a base64 encoded file
Message.new_user!([
  ContentPart.text!("Process this file:"),
  ContentPart.file!("base64_encoded_file_data",
    type: :base64,
    filename: "document.pdf"
  )
])

# Using a file ID (after uploading to OpenAI)
Message.new_user!([
  ContentPart.text!("Process this file:"),
  ContentPart.file!("file-1234", type: :file_id)
])copy

Callbacks

See the set of available callbacks: LangChain.Chains.ChainCallbacks

Rate Limit API Response Headers

OpenAI returns rate limit information in the response headers. Those can be accessed using the LLM callback on_llm_ratelimit_info like this:

handlers = %{
  on_llm_ratelimit_info: fn _model, headers ->
    IO.inspect(headers)
  end
}

{:ok, chat} = ChatOpenAI.new(%{callbacks: [handlers]})copy

When a request is received, something similar to the following will be output to the console.

%{
  "x-ratelimit-limit-requests" => ["5000"],
  "x-ratelimit-limit-tokens" => ["160000"],
  "x-ratelimit-remaining-requests" => ["4999"],
  "x-ratelimit-remaining-tokens" => ["159973"],
  "x-ratelimit-reset-requests" => ["12ms"],
  "x-ratelimit-reset-tokens" => ["10ms"],
  "x-request-id" => ["req_1234"]
}copy

Token Usage

OpenAI returns token usage information as part of the response body. That data can be accessed using the LLM callback on_llm_token_usage like this:

handlers = %{
  on_llm_token_usage: fn _model, usage ->
    IO.inspect(usage)
  end
}

{:ok, chat} = ChatOpenAI.new(%{
  callbacks: [handlers],
  stream: true,
  stream_options: %{include_usage: true}
})copy

When a request is received, something similar to the following will be output to the console.

%LangChain.TokenUsage{input: 15, output: 3}copy

The OpenAI documentation instructs to provide the stream_options with the include_usage: true for the information to be provided.

Tool Choice

OpenAI's ChatGPT API supports forcing a tool to be used.

• https://platform.openai.com/docs/api-reference/chat/create#chat-create-tool_choice

This is supported through the tool_choice options. It takes a plain Elixir map to provide the configuration.

By default, the LLM will choose a tool call if a tool is available and it determines it is needed. That's the "auto" mode.

Example

For the LLM's response to make a tool call of the "get_weather" function.

ChatOpenAI.new(%{
  model: "...",
  tool_choice: %{"type" => "function", "function" => %{"name" => "get_weather"}}
})copy

Azure OpenAI Support

To use ChatOpenAI with Microsoft's Azure hosted OpenAI models, the endpoint must be overridden and the API key needs to be provided in some way. The MS Quickstart guide for REST access may be helpful.

In order to use it, you must have an Azure account and from the console, a model must be deployed for your account. Use the Azure AI Foundry and Azure OpenAI Service to deploy the model you want to use. The entire URL is used as the endpoint and the provided key is used as the api_key.

The following is an example of setting up ChatOpenAI for use with an Azure hosted model.

endpoint = System.fetch_env!("AZURE_OPENAI_ENDPOINT")
api_key = System.fetch_env!("AZURE_OPENAI_KEY")

llm =
  ChatOpenAI.new!(%{
    endpoint: endpoint,
    api_key: api_key,
    seed: 0,
    temperature: 1,
    stream: false
  })copy

The URL itself specifies the model to use and the model attribute is disregarded.

A fake example URL for the endpoint value:

https://some-subdomain.cognitiveservices.azure.com/openai/deployments/gpt-4o-mini/chat/completions?api-version=2024-08-01-preview"

Reasoning Model Support

OpenAI made some significant API changes with the introduction of their "reasoning" models. This includes the o1 and o1-mini models.

To enable this mode, set :reasoning_mode to true:

model = ChatOpenAI.new!(%{reasoning_mode: true})copy

Setting reasoning_mode to true does at least the two following things:

• Set :developer as the role for system messages. The OpenAI documentation says API calls to o1 and newer models must use the role: :developer instead of role: :system and errors if not set correctly.
• The :reasoning_effort option included in LLM requests. This setting is only permitted on a reasoning model. The :reasoning_effort values support the "low", "medium" (default), and "high" options specified in the OpenAI documentation. This instructs the LLM on how much time, and tokens, should be spent on thinking through and reasoning about the request and the response.