View Source LangChain.MessageDelta (LangChain v0.4.0-rc.2)
Models a "delta" message from a chat LLM. A delta is a small chunk, or piece of a much larger complete message. A series of deltas are used to construct the complete message.
Delta messages must be applied in order for them to be valid. Delta messages
can be combined and transformed into a LangChain.Message
once the final
piece is received.
Roles
:unknown
- The role data is missing for the delta.:assistant
- Responses coming back from the LLM.
Tool Usage
Tools can be used or called by the assistant (LLM). A tool call is also split across many message deltas and must be fully assembled before it can be executed.
Metadata
The metadata
field is a map that can contain any additional information
about the message delta. It is used to store token usage, model, and other
LLM-specific information.
Content Fields
The content
field may contain:
- A string (for backward compatibility)
- A
LangChain.Message.ContentPart
struct - An empty list
[]
that is received from some services like Anthropic, which is a signal that the content will be a list of content parts
The module uses two content-related fields:
content
- The raw content received from the LLM. This can be either a string (for backward compatibility), aLangChain.Message.ContentPart
struct, or a[]
indicating it will be a list of content parts. This field is cleared (set tonil
) after merging intomerged_content
.merged_content
- The accumulated list ofContentPart
s after merging deltas. This is the source of truth for the message content and is used when converting to aLangChain.Message
. When merging deltas:- For string content, it's converted to a
ContentPart
of type:text
- For
ContentPart
content, it's merged based on theindex
field - Multiple content parts can be maintained in the list to support multi-modal responses (text, images, audio) or separate thinking content from final text
- For string content, it's converted to a
Summary
Functions
Accumulates token usage from delta messages. Uses LangChain.TokenUsage.add/2
to combine
the usage data from both deltas.
Convert the MessageDelta's merged content to a string. Specify the type of
content to convert so it can return just the text parts or thinking parts,
etc. Defaults to :text
.
Merge two MessageDelta
structs. The first MessageDelta
is the primary
one that smaller deltas are merged into.
Merges a list of MessageDelta
s into a single MessageDelta
. The deltas
are merged in order, with each delta being merged into the result of the
previous merge.
Migrates a MessageDelta's string content to use LangChain.Message.ContentPart
.
This is for backward compatibility with models that don't yet support ContentPart streaming.
Create a new MessageDelta
that represents a message chunk.
Create a new MessageDelta
that represents a message chunk and return it or
raise an error if invalid.
Convert the MessageDelta to a Message. Can only convert a fully complete MessageDelta.
Types
Functions
Accumulates token usage from delta messages. Uses LangChain.TokenUsage.add/2
to combine
the usage data from both deltas.
Example
iex> alias LangChain.TokenUsage
iex> alias LangChain.MessageDelta
iex> delta1 = %MessageDelta{
...> metadata: %{
...> usage: %TokenUsage{input: 10, output: 5}
...> }
...> }
iex> delta2 = %MessageDelta{
...> metadata: %{
...> usage: %TokenUsage{input: 5, output: 15}
...> }
...> }
iex> result = MessageDelta.accumulate_token_usage(delta1, delta2)
iex> result.metadata.usage.input
15
iex> result.metadata.usage.output
20
Convert the MessageDelta's merged content to a string. Specify the type of
content to convert so it can return just the text parts or thinking parts,
etc. Defaults to :text
.
Merge two MessageDelta
structs. The first MessageDelta
is the primary
one that smaller deltas are merged into.
The merging process:
- Migrates any string content to
ContentPart
s for backward compatibility - Merges the content into
merged_content
based on theindex
field - Clears the
content
field (sets tonil
) after merging - Updates other fields (tool_calls, status, etc.)
Examples
iex> delta_1 =
...> %LangChain.MessageDelta{
...> content: nil,
...> index: 0,
...> tool_calls: [],
...> role: :assistant,
...> status: :incomplete
...> }
iex> delta_2 =
...> %LangChain.MessageDelta{
...> content: "Hello",
...> index: 0,
...> tool_calls: [],
...> role: :unknown,
...> status: :incomplete
...> }
iex> LangChain.MessageDelta.merge_delta(delta_1, delta_2)
%LangChain.MessageDelta{
content: nil,
merged_content: [%LangChain.Message.ContentPart{type: :text, content: "Hello"}],
status: :incomplete,
index: 0,
role: :assistant,
tool_calls: []
}
A set of deltas can be easily merged like this:
MessageDelta.merge_deltas(list_of_delta_messages)
Merges a list of MessageDelta
s into a single MessageDelta
. The deltas
are merged in order, with each delta being merged into the result of the
previous merge.
Examples
iex> deltas = [
...> %LangChain.MessageDelta{content: "Hello", role: :assistant},
...> %LangChain.MessageDelta{content: " world", role: :assistant},
...> %LangChain.MessageDelta{content: "!", role: :assistant, status: :complete}
...> ]
iex> LangChain.MessageDelta.merge_deltas(deltas)
%LangChain.MessageDelta{
content: nil,
merged_content: [%LangChain.Message.ContentPart{type: :text, content: "Hello world!"}],
status: :complete,
role: :assistant
}
Migrates a MessageDelta's string content to use LangChain.Message.ContentPart
.
This is for backward compatibility with models that don't yet support ContentPart streaming.
Examples
iex> delta = %LangChain.MessageDelta{content: "Hello world"}
iex> upgraded = migrate_to_content_parts(delta)
iex> upgraded.content
%LangChain.Message.ContentPart{type: :text, content: "Hello world"}
@spec new(attrs :: map()) :: {:ok, t()} | {:error, Ecto.Changeset.t()}
Create a new MessageDelta
that represents a message chunk.
Create a new MessageDelta
that represents a message chunk and return it or
raise an error if invalid.
@spec to_message(t()) :: {:ok, LangChain.Message.t()} | {:error, String.t()}
Convert the MessageDelta to a Message. Can only convert a fully complete MessageDelta.
This is assumed to be the result of merging all the received MessageDelta
s.
An error is returned if the status
is :incomplete
.
If the MessageDelta
fails to convert to a LangChain.Message
, an error is
returned with the reason.