LangChain.TokenUsage (LangChain v0.6.0)

Contains token usage information returned from an LLM.

Example

%TokenUsage{
  input: 30,
  output: 15,
  raw: %{
    "total_tokens" => 29
  }
}

Input is the tokens from the prompt. Output is the completion or generated tokens returned.

Refer to the raw token usage information for access to LLM-specific information that may be available.

Summary

Types

t()

Functions

add(usage, usage)

Combines two TokenUsage structs by adding their respective input and output values. The raw maps are merged, with values being added if they are numeric.

get(arg1)

Extracts token usage information from a LangChain.Message or LangChain.MessageDelta struct's metadata. Returns nil if no token usage information is found.

new(attrs \\ %{})

Build a new TokenUsage and return an :ok/:error tuple with the result.

new!(attrs \\ %{})

Build a new TokenUsage and return it or raise an error if invalid.

set(message, usage)

Sets the token usage information on a LangChain.Message or LangChain.MessageDelta struct in the metadata under the :usage key.

set_wrapped(message, usage)

Sets the token usage information on a LangChain.Message or LangChain.MessageDelta struct when wrapped in an :ok,:error tuple in the metadata under the :usage key.

total(usage)

Return the total token usage amount. The total is the sum of input and output.

Types

t()

@type t() :: %LangChain.TokenUsage{
  cumulative: term(),
  input: term(),
  output: term(),
  raw: term()
}

Functions

add(usage, usage)

@spec add(t() | nil, t() | nil) :: t() | nil

Combines two TokenUsage structs by adding their respective input and output values. The raw maps are merged, with values being added if they are numeric.

If both arguments are nil, returns nil. If one argument is nil, returns the non-nil argument.

Example

iex> usage1 = LangChain.TokenUsage.new!(%{input: 10, output: 20, raw: %{"total_tokens" => 30}})
iex> usage2 = LangChain.TokenUsage.new!(%{input: 5, output: 15, raw: %{"total_tokens" => 20}})
iex> combined = LangChain.TokenUsage.add(usage1, usage2)
iex> combined.input
15
iex> combined.output
35
iex> combined.raw["total_tokens"]
50

get(arg1)

@spec get(%{metadata: %{usage: t()}} | %{metadata: map() | nil}) :: t() | nil

Extracts token usage information from a LangChain.Message or LangChain.MessageDelta struct's metadata. Returns nil if no token usage information is found.

Example

iex> message = %LangChain.Message{metadata: %{usage: %LangChain.TokenUsage{input: 10, output: 20}}}
iex> LangChain.TokenUsage.get(message)
%LangChain.TokenUsage{input: 10, output: 20}

iex> message = %LangChain.Message{metadata: %{}}
iex> LangChain.TokenUsage.get(message)
nil

new(attrs \\ %{})

@spec new(attrs :: map()) :: {:ok, t()} | {:error, Ecto.Changeset.t()}

Build a new TokenUsage and return an :ok/:error tuple with the result.

new!(attrs \\ %{})

@spec new!(attrs :: map()) :: t() | no_return()

Build a new TokenUsage and return it or raise an error if invalid.

set(message, usage)

@spec set(t() | %{metadata: %{usage: t()}}, nil | t()) ::
  t() | %{metadata: %{usage: t()}}

Sets the token usage information on a LangChain.Message or LangChain.MessageDelta struct in the metadata under the :usage key.

Example

iex> message = %LangChain.Message{metadata: %{}}
iex> token_usage = %LangChain.TokenUsage{input: 10, output: 20}
iex> LangChain.TokenUsage.set(message, token_usage)
%LangChain.Message{metadata: %{usage: %LangChain.TokenUsage{input: 10, output: 20}}}

set_wrapped(message, usage)

@spec set_wrapped(
  {:ok, %{metadata: nil | map()}},
  nil | t()
) :: {:ok, %{metadata: %{usage: t()}} | {:error, any()}}

Sets the token usage information on a LangChain.Message or LangChain.MessageDelta struct when wrapped in an :ok,:error tuple in the metadata under the :usage key.

total(usage)

@spec total(t()) :: integer()

Return the total token usage amount. The total is the sum of input and output.