Module for interacting with Anthropic models.

Parses and validates inputs for making requests to Anthropic's messages API.

Converts responses into more specialized LangChain data structures.

Callbacks

See the set of available callbacks: LangChain.Chains.ChainCallbacks

Rate Limit API Response Headers

Anthropic returns rate limit information in the response headers. Those can be accessed using an LLM callback like this:

handler = %{
  on_llm_ratelimit_info: fn _chain, headers ->
    IO.inspect(headers)
  end
}

%{llm: ChatAnthropic.new!(%{model: "..."})}
|> LLMChain.new!()
# ... add messages ...
|> LLMChain.add_callback(handler)
|> LLMChain.run()

When a request is received, something similar to the following will be output to the console.

%{
  "anthropic-ratelimit-requests-limit" => ["50"],
  "anthropic-ratelimit-requests-remaining" => ["49"],
  "anthropic-ratelimit-requests-reset" => ["2024-06-08T04:28:30Z"],
  "anthropic-ratelimit-tokens-limit" => ["50000"],
  "anthropic-ratelimit-tokens-remaining" => ["50000"],
  "anthropic-ratelimit-tokens-reset" => ["2024-06-08T04:28:30Z"],
  "request-id" => ["req_1234"]
}

Token Usage

Anthropic returns token usage information as part of the response body. The LangChain.TokenUsage is added to the metadata of the LangChain.Message and LangChain.MessageDelta structs that are processed under the :usage key.

%LangChain.MessageDelta{
  content: [],
  status: :incomplete,
  index: nil,
  role: :assistant,
  tool_calls: nil,
  metadata: %{
          usage: %LangChain.TokenUsage{
            input: 55,
            output: 4,
            raw: %{
              "cache_creation_input_tokens" => 0,
              "cache_read_input_tokens" => 0,
              "input_tokens" => 55,
              "output_tokens" => 4
            }
          }
  }
}

The TokenUsage data is accumulated for MessageDelta structs and the final usage information will be on the LangChain.Message.

Tool Choice

Anthropic supports forcing a tool to be used.

• https://docs.anthropic.com/en/docs/build-with-claude/tool-use#forcing-tool-use

This is supported through the tool_choice options. It takes a plain Elixir map to provide the configuration.

By default, the LLM will choose a tool call if a tool is available and it determines it is needed. That's the "auto" mode.

Example

Force the LLM's response to make a tool call of the "get_weather" function.

ChatAnthropic.new(%{
  model: "...",
  tool_choice: %{"type" => "tool", "name" => "get_weather"}
})

AWS Bedrock Support

Anthropic Claude is supported in AWS Bedrock.

To configure ChatAnthropic for use on AWS Bedrock:

1. Request Model Access to get access to the Anthropic models you intend to use.

2. Using your AWS Console, create an Access Key for your application.

3. Set the key values in your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY ENVs.

4. Get the Model ID for the model you intend to use. Base Models

5. Refer to LangChain.Utils.BedrockConfig for setting up the Bedrock authentication credentials for your environment.

6. Setup your ChatAnthropic similar to the following:

   alias LangChain.ChatModels.ChatAnthropic

   ChatAnthropic.new!(%{ model: "anthropic.claude-3-5-sonnet-20241022-v2:0", bedrock: BedrockConfig.from_application_env!() })

Thinking

Models like Claude 3.7 Sonnet introduced a hybrid approach which allows for "thinking" and reasoning. See the Anthropic thinking documentation for up-to-date instructions on the usage.

For instance, enabling thinking may require the temperature to be set to 1 and other settings like topP may not be allowed.

The model supports a :thinking attribute where the data is a map that matches the structure in the Anthropic documentation. It is passed along as-is.

Example:

# Enable thinking and budget 2,000 tokens for the thinking space.
model = ChatAnthropic.new!(%{
  model: "claude-3-7-sonnet-latest",
  thinking: %{type: "enabled", budget_tokens: 2000}
})

# Disable thinking
model = ChatAnthropic.new!(%{
  model: "claude-3-7-sonnet-latest",
  thinking: %{type: "disabled"}
})

As of the documentation for Claude 3.7 Sonnet, the minimum budget for thinking is 1024 tokens.