Mojentic.LLM.ChatSession (Mojentic v1.2.0)

Copy Markdown View Source

Manages stateful conversation sessions with an LLM.

ChatSession maintains conversation history with automatic context window management based on token counts. When the context exceeds the maximum token limit, the oldest messages are removed (preserving the system prompt).

Features

  • Automatic message history tracking
  • Token-based context window management
  • Tool support through broker integration
  • Configurable system prompt and temperature
  • Tokenizer integration for accurate token counting

Examples

alias Mojentic.LLM.{Broker, ChatSession}
alias Mojentic.LLM.Gateways.Ollama

broker = Broker.new("qwen3:32b", Ollama)
session = ChatSession.new(broker)

{:ok, response} = ChatSession.send(session, "Hello!")
# => {:ok, "Hello! How can I help you?", updated_session}

# Continue conversation
{:ok, response, session} = ChatSession.send(session, "Tell me a joke")

With Tools

alias Mojentic.LLM.Tools.DateResolver

session = ChatSession.new(broker, tools: [DateResolver])
{:ok, response, session} = ChatSession.send(session, "What day is tomorrow?")

Summary

Types

sized_message()
stream_handle()
t()

Functions

finalize_stream(arg)

Finalizes a streaming send by recording the accumulated response in the session.

messages(session)

Returns the current message history.

new(broker, opts \\ [])

Creates a new ChatSession.

send(session, query)

Sends a query to the LLM and returns the response.

send_stream(session, query)

Sends a query to the LLM and returns a stream of response chunks.

token_count(session)

Returns the total token count of all messages.

Types

sized_message()

@type sized_message() :: %{
  message: Mojentic.LLM.Message.t(),
  token_length: non_neg_integer()
}

stream_handle()

@type stream_handle() :: {t(), pid()}

t()

@type t() :: %Mojentic.LLM.ChatSession{
  broker: Mojentic.LLM.Broker.t(),
  max_context: non_neg_integer(),
  messages: [sized_message()],
  system_prompt: String.t(),
  temperature: float(),
  tokenizer: Mojentic.LLM.Gateways.TokenizerGateway.t(),
  tools: [module()] | nil
}

Functions

finalize_stream(arg)

@spec finalize_stream(stream_handle()) :: t()

Finalizes a streaming send by recording the accumulated response in the session.

Must be called after the stream returned by send_stream/2 has been fully consumed.

Parameters

Returns

The updated session with the assistant's response recorded in history.

Examples

{:ok, stream, handle} = ChatSession.send_stream(session, "Tell me a story")
stream |> Stream.each(&IO.write/1) |> Stream.run()
session = ChatSession.finalize_stream(handle)

messages(session)

@spec messages(t()) :: [sized_message()]

Returns the current message history.

Messages are returned with their token lengths for debugging and monitoring context usage.

Examples

messages = ChatSession.messages(session)
total_tokens = Enum.reduce(messages, 0, fn m, acc -> acc + m.token_length end)

new(broker, opts \\ [])

@spec new(
  Mojentic.LLM.Broker.t(),
  keyword()
) :: t()

Creates a new ChatSession.

Options

  • :system_prompt - System prompt for the conversation (default: "You are a helpful assistant.")
  • :tools - List of tool modules to make available to the LLM (default: nil)
  • :max_context - Maximum token count for context window (default: 32,768)
  • :tokenizer - TokenizerGateway instance (default: auto-created with gpt2)
  • :temperature - Temperature for response generation (default: 1.0)

Examples

broker = Broker.new("qwen3:32b", Ollama)
session = ChatSession.new(broker)

# With custom options
session = ChatSession.new(broker,
  system_prompt: "You are a coding assistant.",
  max_context: 16_384,
  temperature: 0.7
)

# With tools
session = ChatSession.new(broker, tools: [MyTool])

send(session, query)

@spec send(t(), String.t()) :: {:ok, String.t(), t()} | {:error, term()}

Sends a query to the LLM and returns the response.

The query is added as a user message, the LLM generates a response, and the response is added as an assistant message. Both are tracked in the conversation history.

Parameters

  • session - The ChatSession instance
  • query - The user's query text

Returns

  • {:ok, response, updated_session} - Success with response text and updated session
  • {:error, reason} - Error from broker

Examples

{:ok, response, session} = ChatSession.send(session, "What is 2+2?")
# => {:ok, "2+2 equals 4.", updated_session}

# Continue conversation
{:ok, response, session} = ChatSession.send(session, "And what about 3+3?")

send_stream(session, query)

@spec send_stream(t(), String.t()) :: {:ok, Enumerable.t(), stream_handle()}

Sends a query to the LLM and returns a stream of response chunks.

Because Elixir data is immutable, streaming requires a two-phase approach:

  1. send_stream/2 returns a stream and a handle
  2. After consuming the stream, call finalize_stream/1 with the handle to get the updated session

The user message is added to the session before streaming begins. An Agent process accumulates chunks as they flow through the stream.

Parameters

  • session - The ChatSession instance
  • query - The user's query text

Returns

  • {:ok, stream, handle} where stream is an Enumerable.t() of string chunks and handle is passed to finalize_stream/1

Examples

{:ok, stream, handle} = ChatSession.send_stream(session, "Tell me a story")
stream |> Stream.each(&IO.write/1) |> Stream.run()
session = ChatSession.finalize_stream(handle)

token_count(session)

@spec token_count(t()) :: non_neg_integer()

Returns the total token count of all messages.

Examples

token_count = ChatSession.token_count(session)
# => 1234