Sagents.Middleware.ConversationTitle (Sagents v0.4.0)

Copy Markdown

Middleware that automatically generates conversation titles based on the initial user message.

Spawns an async task before the first LLM call to generate a concise title using an LLM. By running in before_model, title generation happens in parallel with the main execution, so titles appear quickly even when long-running tools are involved. The title is stored in state metadata and broadcast to subscribers.

Usage

{:ok, agent} = Agent.new(
  llm: main_model,
  middleware: [
    {ConversationTitle, [
      chat_model: ChatAnthropic.new!(%{model: "claude-3-5-haiku-latest"}),
      fallbacks: [backup_model]
    ]}
  ]
)

Configuration Options

  • :chat_model (required) - The LLM model to use for title generation
  • :fallbacks - List of fallback models if primary fails (default: [])
  • :prompt_template - Custom prompt template (default: uses TextToTitleChain defaults)
  • :examples - List of example titles to guide LLM (default: provided examples)
  • :id - Custom middleware ID for multiple instances (default: module name)

Events

Broadcasts the following events to PubSub:

  • {:conversation_title_generated, title, agent_id} - For backward compatibility with LiveView
  • {:agent_state_update, %{conversation_title: title}} - Generic agent state update notification

Metadata

Stores the generated title in state metadata:

  • "conversation_title" - The generated title string (presence indicates title has been generated)

Summary

Functions

before_model(state, config)

Called before the LLM model call.

handle_message(arg, state, config)

Handle messages from async title generation tasks.

init(opts)

Initialize the ConversationTitle middleware with configuration options.

Functions

before_model(state, config)

Called before the LLM model call.

Checks if a conversation title has already been generated. If not, spawns an async task to generate one based on the user's message content. Running in before_model allows title generation to happen in parallel with the main LLM call and any subsequent tool execution.

The async task will send a message back to the AgentServer when complete, which will be handled by handle_message/3.

Parameters

  • state - The current agent state before model call
  • config - The middleware configuration from init/1

Returns

  • {:ok, state} - Always returns the unchanged state (title generation happens asynchronously)

handle_message(arg, state, config)

Handle messages from async title generation tasks.

This callback receives messages sent by the async task spawned in after_model/2. It handles both success and failure cases.

Success Message: {:title_generated, title}

When a title is successfully generated:

  • Stores the title in state metadata under the key "conversation_title"
  • Requests a broadcast to notify subscribers (LiveViews, external clients)

Failure Message: {:title_generation_failed, reason}

When title generation fails:

  • Logs a warning with the failure reason
  • Does not update state or broadcast

Parameters

  • message - The message tuple from the async task
  • state - The current agent state
  • config - The middleware configuration

Returns

  • {:ok, updated_state} - For successful title generation
  • {:ok, state} - For failures (no state changes)

init(opts)

Initialize the ConversationTitle middleware with configuration options.

Required Options

  • :chat_model - The ChatModel to use for title generation

Optional Options

  • :fallbacks - List of fallback ChatModels if primary fails (default: [])
  • :prompt_template - Custom prompt template for title generation (default: uses TextToTitleChain defaults)
  • :examples - List of example titles to guide the LLM (default: provided examples)
  • :id - Custom middleware ID for multiple instances (default: module name)

Returns

  • {:ok, config} - Configuration map to be passed to other callbacks
  • {:error, reason} - Initialization failed

Example

{:ok, config} = ConversationTitle.init([
  chat_model: ChatAnthropic.new!(%{model: "claude-3-5-haiku-latest"}),
  fallbacks: [backup_model],
  examples: ["Debug API error", "Plan sprint goals"]
])