Normandy.Context.Summarizer (normandy v0.2.0)

Handles conversation summarization for context window management.

When conversations exceed token limits, this module can summarize older messages to preserve context while reducing token usage.

Example

# Summarize old messages
{:ok, summary} = Summarizer.summarize_messages(client, agent, messages)

# Replace old messages with summary
{:ok, updated_agent} = Summarizer.compress_conversation(client, agent, keep_recent: 5)

Summary

Functions

compress_conversation(client, agent, opts \\ [])

Compresses a conversation by summarizing old messages.

estimate_savings(messages, opts \\ [])

Estimates token savings from summarization.

summarize_messages(client, agent, messages, opts \\ [])

Summarizes a list of messages using the LLM.

Functions

compress_conversation(client, agent, opts \\ [])

@spec compress_conversation(struct(), struct(), keyword()) ::
  {:ok, struct()} | {:error, term()}

Compresses a conversation by summarizing old messages.

Keeps recent messages intact and replaces older messages with a summary.

Options

:keep_recent - Number of recent messages to keep (default: 10)
:summary_role - Role for summary message (default: "system")
:max_summary_tokens - Max tokens for summary (default: 500)

Example

{:ok, updated_agent} = Summarizer.compress_conversation(
  client,
  agent,
  keep_recent: 5
)

estimate_savings(messages, opts \\ [])

@spec estimate_savings(
  list(),
  keyword()
) :: {:ok, map()}

Estimates token savings from summarization.

Example

{:ok, savings} = Summarizer.estimate_savings(messages, summary_tokens: 200)
#=> %{original: 1500, summary: 200, savings: 1300, savings_percent: 86.7}

summarize_messages(client, agent, messages, opts \\ [])

@spec summarize_messages(struct(), struct(), list(), keyword()) ::
  {:ok, String.t()} | {:error, term()}

Summarizes a list of messages using the LLM.

Options

:prompt - Custom summarization prompt (default: built-in prompt)
:model - Model to use for summarization (default: from agent config)
:max_tokens - Maximum tokens for summary (default: 500)

Example

messages = [
  %{role: "user", content: "Hello"},
  %{role: "assistant", content: "Hi there!"}
]

{:ok, summary} = Summarizer.summarize_messages(client, agent, messages)
#=> "User greeted, assistant responded"