View Source LangChain.ChatModels.ChatAnthropic (LangChain v0.3.3)

Module for interacting with Anthropic models.

Parses and validates inputs for making requests to Anthropic's messages API.

Converts responses into more specialized LangChain data structures.

Callbacks

See the set of available callbacks: LangChain.Chains.ChainCallbacks

Rate Limit API Response Headers

Anthropic returns rate limit information in the response headers. Those can be accessed using an LLM callback like this:

handler = %{
  on_llm_ratelimit_info: fn _chain, headers ->
    IO.inspect(headers)
  end
}

%{llm: ChatAnthropic.new!(%{model: "..."})}
|> LLMChain.new!()
# ... add messages ...
|> LLMChain.add_callback(handler)
|> LLMChain.run()

When a request is received, something similar to the following will be output to the console.

%{
  "anthropic-ratelimit-requests-limit" => ["50"],
  "anthropic-ratelimit-requests-remaining" => ["49"],
  "anthropic-ratelimit-requests-reset" => ["2024-06-08T04:28:30Z"],
  "anthropic-ratelimit-tokens-limit" => ["50000"],
  "anthropic-ratelimit-tokens-remaining" => ["50000"],
  "anthropic-ratelimit-tokens-reset" => ["2024-06-08T04:28:30Z"],
  "request-id" => ["req_1234"]
}

Tool Choice

Anthropic supports forcing a tool to be used.

https://docs.anthropic.com/en/docs/build-with-claude/tool-use#forcing-tool-use

This is supported through the tool_choice options. It takes a plain Elixir map to provide the configuration.

By default, the LLM will choose a tool call if a tool is available and it determines it is needed. That's the "auto" mode.

Example

Force the LLM's response to make a tool call of the "get_weather" function.

ChatAnthropic.new(%{
  model: "...",
  tool_choice: %{"type" => "tool", "name" => "get_weather"}
})

AWS Bedrock Support

Anthropic Claude is supported in AWS Bedrock.

To configure ChatAnthropic for use on AWS Bedrock:

Request Model Access to get access to the Anthropic models you intend to use.
Using your AWS Console, create an Access Key for your application.
Set the key values in your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY ENVs.
Get the Model ID for the model you intend to use. Base Models
Refer to LangChain.Utils.BedrockConfig for setting up the Bedrock authentication credentials for your environment.
Setup your ChatAnthropic similar to the following:
alias LangChain.ChatModels.ChatAnthropic
ChatAnthropic.new!(%{ model: "anthropic.claude-3-5-sonnet-20241022-v2:0", bedrock: BedrockConfig.from_application_env!() })

Summary

Functions

call(anthropic, prompt, functions \\ [])

Calls the Anthropic API passing the ChatAnthropic struct with configuration, plus either a simple message or the list of messages to act as the prompt.

for_api(msg)

Convert a LangChain structure to the expected map of data for the Anthropic API.

for_api(anthropic, messages, tools)

Return the params formatted for an API request.

new(attrs \\ %{})

Setup a ChatAnthropic client configuration.

new!(attrs \\ %{})

Setup a ChatAnthropic client configuration and return it or raise an error if invalid.

post_process_and_combine_messages(messages)

After all the messages have been converted using for_api/1, this combines multiple sequential tool response messages. The Anthropic API is very strict about user, assistant, user, assistant sequenced messages.

restore_from_map(data)

Restores the model from the config.

serialize_config(model)

Generate a config map that can later restore the model's configuration.

Types

t()

@type t() :: %LangChain.ChatModels.ChatAnthropic{
  api_key: term(),
  api_version: term(),
  bedrock: term(),
  beta_headers: term(),
  callbacks: term(),
  endpoint: term(),
  max_tokens: term(),
  model: term(),
  receive_timeout: term(),
  stream: term(),
  temperature: term(),
  tool_choice: term(),
  top_k: term(),
  top_p: term()
}

Functions

call(anthropic, prompt, functions \\ [])

Calls the Anthropic API passing the ChatAnthropic struct with configuration, plus either a simple message or the list of messages to act as the prompt.

Optionally pass in a callback function that can be executed as data is received from the API.

NOTE: This function can be used directly, but the primary interface should be through LangChain.Chains.LLMChain. The ChatAnthropic module is more focused on translating the LangChain data structures to and from the Anthropic API.

Another benefit of using LangChain.Chains.LLMChain is that it combines the storage of messages, adding functions, adding custom context that should be passed to functions, and automatically applying LangChain.MessageDelta structs as they are are received, then converting those to the full LangChain.Message once fully complete.

for_api(msg)

@spec for_api(
  LangChain.Message.t()
  | LangChain.Message.ContentPart.t()
  | LangChain.Function.t()
) ::
  %{required(String.t()) => any()} | no_return()

Convert a LangChain structure to the expected map of data for the Anthropic API.

for_api(anthropic, messages, tools)

@spec for_api(t(), message :: [map()], LangChain.ChatModels.ChatModel.tools()) :: %{
  required(atom()) => any()
}

Return the params formatted for an API request.

new(attrs \\ %{})

@spec new(attrs :: map()) :: {:ok, t()} | {:error, Ecto.Changeset.t()}

Setup a ChatAnthropic client configuration.

new!(attrs \\ %{})

@spec new!(attrs :: map()) :: t() | no_return()

Setup a ChatAnthropic client configuration and return it or raise an error if invalid.

post_process_and_combine_messages(messages)

restore_from_map(data)

Restores the model from the config.

serialize_config(model)

@spec serialize_config(t()) :: %{required(String.t()) => any()}

Generate a config map that can later restore the model's configuration.