PtcRunner.SubAgent (PtcRunner v0.7.0)

Copy Markdown View Source

Agentic loop for LLM-driven PTC-Lisp execution.

A SubAgent prompts an LLM to write programs, executes them in a sandbox, and loops until completion. Define agents with new/1, execute with run/2.

Execution Modes

ModeConditionBehavior
Single-shotmax_turns == 1 and tools == %{}One LLM call, expression returned
LoopOtherwiseMulti-turn with tools until return or fail

Examples

# Simple single-shot
{:ok, step} = SubAgent.run("What's 2 + 2?", llm: my_llm, max_turns: 1)
step.return  #=> 4

# With tools and signature
agent = SubAgent.new(
  prompt: "Find expensive products",
  signature: "{name :string, price :float}",
  tools: %{"list_products" => &MyApp.list/0}
)
{:ok, step} = SubAgent.run(agent, llm: my_llm)

See Also

Summary

Types

Compression strategy configuration.

Output format options for truncation and display.

Language spec for system prompts.

LLM response format.

Output mode for SubAgent execution.

Plan step definition.

t()

Functions

Wraps a SubAgent as a tool callable by other agents.

Returns the default format options.

Returns the agent's tools with builtin tools injected.

Expands a list of builtin tool family atoms to [{name, sentinel}] pairs.

Creates a SubAgent struct from keyword options.

Preview the system and user prompts that would be sent to the LLM.

Executes a SubAgent with the given options.

Bang variant of run/2 that raises on failure.

Chains SubAgent/CompiledAgent executions with error propagation.

Chains agents in a pipeline, passing the previous step as context.

Unwraps internal sentinel values from a search result.

Types

compression_opts()

@type compression_opts() :: nil | false | true | module() | {module(), keyword()}

Compression strategy configuration.

Can be:

  • nil or false - Compression disabled (default)
  • true - Use default strategy (SingleUserCoalesced) with default options
  • Module - Use custom strategy module with default options
  • {Module, opts} - Use custom strategy module with custom options

See PtcRunner.SubAgent.Compression for details.

format_options()

@type format_options() :: [
  feedback_limit: pos_integer(),
  feedback_max_chars: pos_integer(),
  history_max_bytes: pos_integer(),
  result_limit: pos_integer(),
  result_max_chars: pos_integer(),
  max_print_length: pos_integer()
]

Output format options for truncation and display.

Fields:

  • feedback_limit - Max collection items in turn feedback (default: 10)
  • feedback_max_chars - Max chars in turn feedback (default: 512)
  • history_max_bytes - Truncation limit for *1/*2/*3 history (default: 512)
  • result_limit - Inspect :limit for final result (default: 50)
  • result_max_chars - Final string truncation (default: 500)
  • max_print_length - Max chars per println call (default: 2000)

language_spec()

@type language_spec() :: String.t() | atom() | (map() -> String.t())

Language spec for system prompts.

Can be:

  • String: used as-is
  • Atom: resolved via PtcRunner.Lisp.LanguageSpec.get!/1 (e.g., :minimal, :default)
  • Function: callback receiving context map with :turn, :model, :memory, :messages

llm_callback()

@type llm_callback() :: (map() -> {:ok, llm_response()} | {:error, term()})

llm_registry()

@type llm_registry() :: %{required(atom()) => llm_callback()}

llm_response()

@type llm_response() ::
  String.t()
  | %{
      :content => String.t(),
      optional(:tokens) => %{
        optional(:input) => pos_integer(),
        optional(:output) => pos_integer()
      }
    }

LLM response format.

Can be either a plain string (backward compatible) or a map with content and optional tokens. When tokens are provided, they are included in telemetry measurements and accumulated in Step.usage.

output_mode()

@type output_mode() :: :ptc_lisp | :json

Output mode for SubAgent execution.

  • :ptc_lisp - Default. LLM generates PTC-Lisp code that is executed.
  • :json - LLM generates JSON directly matching the signature's return type.

plan_step()

@type plan_step() :: {String.t(), String.t()}

Plan step definition.

Each step is a {id, description} tuple where:

  • id is a string identifier (used as key in summaries)
  • description is a human-readable description of the step

system_prompt_opts()

@type system_prompt_opts() ::
  %{
    optional(:prefix) => String.t(),
    optional(:suffix) => String.t(),
    optional(:language_spec) => language_spec(),
    optional(:output_format) => String.t()
  }
  | (String.t() -> String.t())
  | String.t()

t()

@type t() :: %PtcRunner.SubAgent{
  builtin_tools: [atom()],
  compression: compression_opts(),
  context_descriptions: map() | nil,
  description: String.t() | nil,
  field_descriptions: map() | nil,
  float_precision: non_neg_integer(),
  format_options: format_options(),
  llm: atom() | (map() -> {:ok, llm_response()} | {:error, term()}) | nil,
  llm_query: boolean(),
  llm_retry: map() | nil,
  max_depth: pos_integer(),
  max_heap: pos_integer() | nil,
  max_turns: pos_integer(),
  memory_limit: pos_integer() | nil,
  memory_strategy: :strict | :rollback,
  mission_timeout: pos_integer() | nil,
  output: output_mode(),
  parsed_signature: {:signature, list(), term()} | nil,
  plan: [plan_step()],
  pmap_timeout: term(),
  prompt: String.t(),
  prompt_limit: map() | nil,
  retry_turns: non_neg_integer(),
  schema: term(),
  signature: String.t() | nil,
  system_prompt: system_prompt_opts() | nil,
  thinking: boolean(),
  timeout: pos_integer(),
  tools: map(),
  turn_budget: pos_integer()
}

Functions

as_tool(agent, opts \\ [])

@spec as_tool(
  t(),
  keyword()
) :: PtcRunner.SubAgent.SubAgentTool.t()

Wraps a SubAgent as a tool callable by other agents.

Returns a SubAgentTool struct that parent agents can include in their tools map. When called, the wrapped agent inherits LLM and registry from the parent unless overridden.

Options

  • :llm - Bind specific LLM (atom or function). Overrides parent inheritance.
  • :description - Override agent's description (falls back to agent.description)
  • :name - Suggested tool name (informational, not enforced by the struct)
  • :cache - Cache results by input args (default: false). Only use for deterministic agents where same inputs always produce same outputs.

Description Requirement

A description is required for tools. It can be provided either:

  • On the SubAgent via new(description: "..."), or
  • Via the :description option when calling as_tool/2

Raises ArgumentError if neither is provided.

LLM Resolution

When the tool is called, the LLM is resolved in priority order:

  1. agent.llm - The agent's own LLM override (highest priority)
  2. bound_llm - LLM bound via the :llm option
  3. Parent's llm - Inherited from the calling agent (lowest priority)

Examples

iex> child = PtcRunner.SubAgent.new(
...>   prompt: "Double {{n}}",
...>   signature: "(n :int) -> {result :int}",
...>   description: "Doubles a number"
...> )
iex> tool = PtcRunner.SubAgent.as_tool(child)
iex> tool.signature
"(n :int) -> {result :int}"
iex> tool.description
"Doubles a number"

iex> child = PtcRunner.SubAgent.new(prompt: "Process data", description: "Default desc")
iex> tool = PtcRunner.SubAgent.as_tool(child, llm: :haiku, description: "Processes data")
iex> tool.bound_llm
:haiku
iex> tool.description
"Processes data"

iex> child = PtcRunner.SubAgent.new(prompt: "Analyze {{text}}", signature: "(text :string) -> :string", description: "Analyzes text")
iex> tool = PtcRunner.SubAgent.as_tool(child, name: "analyzer")
iex> tool.signature
"(text :string) -> :string"

iex> child = PtcRunner.SubAgent.new(prompt: "No description")
iex> PtcRunner.SubAgent.as_tool(child)
** (ArgumentError) as_tool requires description to be set - pass description: option or set description on the SubAgent

compile(agent, opts)

See PtcRunner.SubAgent.Compiler.compile/2.

default_format_options()

@spec default_format_options() :: format_options()

Returns the default format options.

effective_tools(agent)

@spec effective_tools(t()) :: map()

Returns the agent's tools with builtin tools injected.

Merges llm_query and builtin_tools families into the tools map. User-defined tools are never overwritten by builtins.

Examples

iex> agent = PtcRunner.SubAgent.new(prompt: "test", builtin_tools: [:grep])
iex> tools = PtcRunner.SubAgent.effective_tools(agent)
iex> Map.has_key?(tools, "grep")
true

iex> agent = PtcRunner.SubAgent.new(prompt: "test", builtin_tools: [:grep], tools: %{"grep" => fn _ -> :custom end})
iex> tools = PtcRunner.SubAgent.effective_tools(agent)
iex> is_function(tools["grep"])
true

expand_builtin_tools(families)

@spec expand_builtin_tools([atom()]) :: [{String.t(), atom()}]

Expands a list of builtin tool family atoms to [{name, sentinel}] pairs.

Useful for external modules (e.g., PlanExecutor) that need to generate tool descriptions for builtins without reaching into SubAgent internals.

Examples

iex> PtcRunner.SubAgent.expand_builtin_tools([:grep])
[{"grep", :builtin_grep}, {"grep-n", :builtin_grep_n}]

iex> PtcRunner.SubAgent.expand_builtin_tools([])
[]

new(opts)

@spec new(keyword()) :: t()

Creates a SubAgent struct from keyword options.

Raises ArgumentError if validation fails (missing required fields or invalid types).

Parameters

  • opts - Keyword list of options

Required Options

  • prompt - String template describing what to accomplish (supports {{placeholder}} expansion)

Optional Options

  • signature - String contract defining expected inputs and outputs
  • tools - Map of callable tools (default: %{})
  • max_turns - Positive integer for maximum LLM calls (default: 5)
  • retry_turns - Non-negative integer for extra turns in must-return mode (default: 0)
  • prompt_limit - Map with truncation config for LLM view
  • timeout - Positive integer for max milliseconds per Lisp execution (default: 5000)
  • max_heap - Positive integer for max heap size in words per Lisp execution (default: app config or 1,250,000 ~10MB)
  • mission_timeout - Positive integer for max milliseconds for entire execution
  • llm_retry - Map with infrastructure retry config
  • llm - Atom or function for optional LLM override
  • system_prompt - System prompt customization (map, function, or string)
  • memory_limit - Positive integer for max bytes for memory map (default: 1MB = 1,048,576 bytes)
  • memory_strategy - How to handle memory limit exceeded: :strict (fatal, default) or :rollback (roll back memory, feed error to LLM)
  • description - String describing the agent's purpose (for external docs)
  • field_descriptions - Map of field names to descriptions for signature fields
  • context_descriptions - Map of context variable names to descriptions (shown in Data Inventory)
  • format_options - Keyword list controlling output truncation (merged with defaults)
  • float_precision - Non-negative integer for decimal places in floats (default: 2)
  • compression - Compression strategy for turn history (see compression_opts/0)
  • pmap_timeout - Positive integer for max milliseconds per pmap parallel operation (default: 5000)
  • max_depth - Positive integer for maximum recursion depth in nested agents (default: 3)
  • turn_budget - Positive integer for total turn budget across retries (default: 20)
  • output - Output mode: :ptc_lisp (default) or :json
  • thinking - Boolean enabling thinking section in output format (default: false)
  • llm_query - Boolean enabling LLM query mode (default: false)
  • builtin_tools - List of builtin tool families to enable (default: []). Available: :grep (adds grep and grep-n tools)
  • plan - List of plan steps (strings, {id, description} tuples, or keyword list)

Returns

A %SubAgent{} struct.

Raises

  • ArgumentError - if prompt is missing or not a string, max_turns is not positive, tools is not a map, any optional field has an invalid type, or prompt placeholders don't match signature parameters (when signature is provided)

Examples

iex> agent = PtcRunner.SubAgent.new(prompt: "Analyze the data")
iex> agent.prompt
"Analyze the data"

iex> email_tools = %{"list_emails" => fn _args -> [] end}
iex> agent = PtcRunner.SubAgent.new(
...>   prompt: "Find urgent emails for {{user}}",
...>   signature: "(user :string) -> {count :int, _ids [:int]}",
...>   tools: email_tools,
...>   max_turns: 10
...> )
iex> agent.max_turns
10

preview_prompt(agent, opts \\ [])

@spec preview_prompt(
  t(),
  keyword()
) :: %{
  system: String.t(),
  user: String.t(),
  tool_schemas: [map()],
  schema: map() | nil
}

Preview the system and user prompts that would be sent to the LLM.

This function generates and returns the prompts without executing the agent, useful for debugging prompt generation, verifying template expansion, and reviewing what the LLM will see.

Parameters

  • agent - A %SubAgent{} struct
  • opts - Keyword list with:
    • context - Context map for template expansion (default: %{})

Returns

A map with:

  • :system - The static system prompt (cacheable - does NOT include mission)
  • :user - The full first user message (context sections + mission)
  • :tool_schemas - List of tool schema maps with name, signature, and description fields
  • :schema - JSON schema for the return type (JSON mode only, nil for PTC-Lisp)

Examples

iex> agent = PtcRunner.SubAgent.new(
...>   prompt: "Find emails for {{user}}",
...>   signature: "(user :string) -> {count :int}",
...>   tools: %{"list_emails" => fn _ -> [] end}
...> )
iex> preview = PtcRunner.SubAgent.preview_prompt(agent, context: %{user: "alice"})
iex> preview.user =~ "Find emails for alice"
true
iex> preview.user =~ "# Mission"
true
iex> preview.system =~ "PTC-Lisp"
true
iex> preview.system =~ "# Mission"
false

run(agent_or_prompt, opts \\ [])

@spec run(
  t() | String.t(),
  keyword()
) :: {:ok, PtcRunner.Step.t()} | {:error, PtcRunner.Step.t()}

Executes a SubAgent with the given options.

Returns a Step struct containing the result, metrics, and execution trace.

Parameters

  • agent - A %SubAgent{} struct or a string prompt (for convenience)
  • opts - Keyword list of runtime options

Runtime Options

  • llm - Required. LLM callback function (map() -> {:ok, String.t()} | {:error, term()}) or atom

  • llm_registry - Map of atom to LLM callback for atom-based LLM references (default: %{})
  • context - Map of input data (default: %{})
  • debug - Deprecated, no longer needed. Turn structs always capture raw_response. Use SubAgent.Debug.print_trace(step, raw: true) to view full LLM output.
  • trace - Trace collection mode (default: true):
    • true - Always collect trace in Step
    • false - Never collect trace
    • :on_error - Only include trace when execution fails
  • llm_retry - Optional map to configure retry behavior for transient LLM failures:
    • max_attempts - Maximum retry attempts (default: 1, meaning no retries unless explicitly configured)
    • backoff - Backoff strategy: :exponential, :linear, or :constant (default: :exponential)
    • base_delay - Base delay in milliseconds (default: 1000)
    • retryable_errors - List of error types to retry (default: [:rate_limit, :timeout, :server_error])
  • collect_messages - Capture full conversation history in Step.messages (default: false). When enabled, messages are in OpenAI format: [%{role: :system | :user | :assistant, content: String.t()}]
  • Other options from agent definition can be overridden

LLM Registry

When using atom LLMs (like :haiku or :sonnet), provide an llm_registry map:

registry = %{
  haiku: fn input -> MyApp.LLM.haiku(input) end,
  sonnet: fn input -> MyApp.LLM.sonnet(input) end
}

SubAgent.run(agent, llm: :sonnet, llm_registry: registry)

The registry is automatically inherited by all child SubAgents, so you only need to provide it once at the top level.

Returns

  • {:ok, Step.t()} on success
  • {:error, Step.t()} on failure

Examples

# Using a SubAgent struct
iex> agent = PtcRunner.SubAgent.new(prompt: "Calculate {{x}} + {{y}}", max_turns: 1)
iex> llm = fn %{messages: [%{content: _prompt}]} -> {:ok, "```clojure\n(+ data/x data/y)\n```"} end
iex> {:ok, step} = PtcRunner.SubAgent.run(agent, llm: llm, context: %{x: 5, y: 3})
iex> step.return
8

# Using string convenience form
iex> llm = fn %{messages: [%{content: _prompt}]} -> {:ok, "```clojure\n42\n```"} end
iex> {:ok, step} = PtcRunner.SubAgent.run("Return 42", max_turns: 1, llm: llm)
iex> step.return
42

# Using atom LLM with registry
iex> registry = %{test: fn %{messages: [%{content: _}]} -> {:ok, "```clojure\n100\n```"} end}
iex> {:ok, step} = PtcRunner.SubAgent.run("Test", max_turns: 1, llm: :test, llm_registry: registry)
iex> step.return
100

run!(agent, opts \\ [])

@spec run!(
  t() | String.t(),
  keyword()
) :: PtcRunner.Step.t()

Bang variant of run/2 that raises on failure.

Returns the Step struct directly instead of {:ok, step}. Raises SubAgentError if execution fails.

Examples

iex> agent = PtcRunner.SubAgent.new(prompt: "Say hello", max_turns: 1)
iex> mock_llm = fn _ -> {:ok, "```clojure\n\"Hello!\"\n```"} end
iex> step = PtcRunner.SubAgent.run!(agent, llm: mock_llm)
iex> step.return
"Hello!"

# Failure case (using loop mode)
iex> agent = PtcRunner.SubAgent.new(prompt: "Fail", max_turns: 2)
iex> mock_llm = fn _ -> {:ok, ~S|(fail {:reason :test :message "Error"})|} end
iex> PtcRunner.SubAgent.run!(agent, llm: mock_llm)
** (PtcRunner.SubAgentError) SubAgent failed: failed - %{message: "Error", reason: :test}

then(result, agent, opts \\ [])

Chains SubAgent/CompiledAgent executions with error propagation.

See PtcRunner.SubAgent.Chaining.then/3 for full documentation.

then!(step, agent, opts \\ [])

Chains agents in a pipeline, passing the previous step as context.

See PtcRunner.SubAgent.Chaining.then!/3 for full documentation.

unwrap_sentinels(step)

@spec unwrap_sentinels(PtcRunner.Step.t()) ::
  {:ok, PtcRunner.Step.t()} | {:error, PtcRunner.Step.t()}

Unwraps internal sentinel values from a search result.

Handles:

  • {:__ptc_return__, value} -> {:ok, step_with_raw_value}
  • {:__ptc_fail__, value} -> {:error, error_step}

Used by single-shot mode and compiled agents to provide clean results.