View Source LangChain.MessageDelta (LangChain v0.4.0-rc.2)

Models a "delta" message from a chat LLM. A delta is a small chunk, or piece of a much larger complete message. A series of deltas are used to construct the complete message.

Delta messages must be applied in order for them to be valid. Delta messages can be combined and transformed into a LangChain.Message once the final piece is received.

Roles

  • :unknown - The role data is missing for the delta.
  • :assistant - Responses coming back from the LLM.

Tool Usage

Tools can be used or called by the assistant (LLM). A tool call is also split across many message deltas and must be fully assembled before it can be executed.

Metadata

The metadata field is a map that can contain any additional information about the message delta. It is used to store token usage, model, and other LLM-specific information.

Content Fields

The content field may contain:

  • A string (for backward compatibility)
  • A LangChain.Message.ContentPart struct
  • An empty list [] that is received from some services like Anthropic, which is a signal that the content will be a list of content parts

The module uses two content-related fields:

  • content - The raw content received from the LLM. This can be either a string (for backward compatibility), a LangChain.Message.ContentPart struct, or a [] indicating it will be a list of content parts. This field is cleared (set to nil) after merging into merged_content.

  • merged_content - The accumulated list of ContentParts after merging deltas. This is the source of truth for the message content and is used when converting to a LangChain.Message. When merging deltas:

    • For string content, it's converted to a ContentPart of type :text
    • For ContentPart content, it's merged based on the index field
    • Multiple content parts can be maintained in the list to support multi-modal responses (text, images, audio) or separate thinking content from final text

Summary

Functions

Accumulates token usage from delta messages. Uses LangChain.TokenUsage.add/2 to combine the usage data from both deltas.

Convert the MessageDelta's merged content to a string. Specify the type of content to convert so it can return just the text parts or thinking parts, etc. Defaults to :text.

Merge two MessageDelta structs. The first MessageDelta is the primary one that smaller deltas are merged into.

Merges a list of MessageDeltas into a single MessageDelta. The deltas are merged in order, with each delta being merged into the result of the previous merge.

Migrates a MessageDelta's string content to use LangChain.Message.ContentPart. This is for backward compatibility with models that don't yet support ContentPart streaming.

Create a new MessageDelta that represents a message chunk.

Create a new MessageDelta that represents a message chunk and return it or raise an error if invalid.

Convert the MessageDelta to a Message. Can only convert a fully complete MessageDelta.

Types

@type t() :: %LangChain.MessageDelta{
  content: term(),
  index: term(),
  merged_content: term(),
  metadata: term(),
  role: term(),
  status: term(),
  tool_calls: term()
}

Functions

Link to this function

accumulate_token_usage(primary, delta_part)

View Source
@spec accumulate_token_usage(t(), t()) :: t()

Accumulates token usage from delta messages. Uses LangChain.TokenUsage.add/2 to combine the usage data from both deltas.

Example

iex> alias LangChain.TokenUsage
iex> alias LangChain.MessageDelta
iex> delta1 = %MessageDelta{
...>   metadata: %{
...>     usage: %TokenUsage{input: 10, output: 5}
...>   }
...> }
iex> delta2 = %MessageDelta{
...>   metadata: %{
...>     usage: %TokenUsage{input: 5, output: 15}
...>   }
...> }
iex> result = MessageDelta.accumulate_token_usage(delta1, delta2)
iex> result.metadata.usage.input
15
iex> result.metadata.usage.output
20
Link to this function

content_to_string(delta, type \\ :text)

View Source
@spec content_to_string(t(), type :: atom()) :: nil | String.t()

Convert the MessageDelta's merged content to a string. Specify the type of content to convert so it can return just the text parts or thinking parts, etc. Defaults to :text.

Link to this function

merge_delta(primary, delta_part)

View Source
@spec merge_delta(nil | t(), t()) :: t()

Merge two MessageDelta structs. The first MessageDelta is the primary one that smaller deltas are merged into.

The merging process:

  1. Migrates any string content to ContentParts for backward compatibility
  2. Merges the content into merged_content based on the index field
  3. Clears the content field (sets to nil) after merging
  4. Updates other fields (tool_calls, status, etc.)

Examples

iex> delta_1 =
...>   %LangChain.MessageDelta{
...>     content: nil,
...>     index: 0,
...>     tool_calls: [],
...>     role: :assistant,
...>     status: :incomplete
...>   }
iex> delta_2 =
...>   %LangChain.MessageDelta{
...>     content: "Hello",
...>     index: 0,
...>     tool_calls: [],
...>     role: :unknown,
...>     status: :incomplete
...>   }
iex> LangChain.MessageDelta.merge_delta(delta_1, delta_2)
%LangChain.MessageDelta{
  content: nil,
  merged_content: [%LangChain.Message.ContentPart{type: :text, content: "Hello"}],
  status: :incomplete,
  index: 0,
  role: :assistant,
  tool_calls: []
}

A set of deltas can be easily merged like this:

  MessageDelta.merge_deltas(list_of_delta_messages)
@spec merge_deltas([t()]) :: t()

Merges a list of MessageDeltas into a single MessageDelta. The deltas are merged in order, with each delta being merged into the result of the previous merge.

Examples

iex> deltas = [
...>   %LangChain.MessageDelta{content: "Hello", role: :assistant},
...>   %LangChain.MessageDelta{content: " world", role: :assistant},
...>   %LangChain.MessageDelta{content: "!", role: :assistant, status: :complete}
...> ]
iex> LangChain.MessageDelta.merge_deltas(deltas)
%LangChain.MessageDelta{
  content: nil,
  merged_content: [%LangChain.Message.ContentPart{type: :text, content: "Hello world!"}],
  status: :complete,
  role: :assistant
}
Link to this function

migrate_to_content_parts(delta)

View Source
@spec migrate_to_content_parts(t()) :: t()

Migrates a MessageDelta's string content to use LangChain.Message.ContentPart. This is for backward compatibility with models that don't yet support ContentPart streaming.

Examples

iex> delta = %LangChain.MessageDelta{content: "Hello world"}
iex> upgraded = migrate_to_content_parts(delta)
iex> upgraded.content
%LangChain.Message.ContentPart{type: :text, content: "Hello world"}
@spec new(attrs :: map()) :: {:ok, t()} | {:error, Ecto.Changeset.t()}

Create a new MessageDelta that represents a message chunk.

@spec new!(attrs :: map()) :: t() | no_return()

Create a new MessageDelta that represents a message chunk and return it or raise an error if invalid.

@spec to_message(t()) :: {:ok, LangChain.Message.t()} | {:error, String.t()}

Convert the MessageDelta to a Message. Can only convert a fully complete MessageDelta.

This is assumed to be the result of merging all the received MessageDeltas. An error is returned if the status is :incomplete.

If the MessageDelta fails to convert to a LangChain.Message, an error is returned with the reason.