PtcRunner.PlanCritic (PtcRunner v0.9.0)

Copy Markdown View Source

Adversarial review of execution plans before they run.

The PlanCritic acts as an "SRE" that validates plans for:

  • Missing synthesis gates (context window risk)
  • Loose coupling (tasks expecting data that may not exist)
  • Optimism bias (wrong error handling for flaky operations)
  • Structural issues (circular deps, orphan tasks)

Usage

{:ok, critique} = PlanCritic.review(plan, llm: my_llm)

if critique.score < 8 do
  {:ok, refined_plan} = MetaPlanner.refine(plan, critique)
end

Static vs LLM Analysis

The critic performs both:

  • Static analysis: Structural checks that don't need an LLM
  • LLM analysis: Semantic checks that require understanding intent

Summary

Types

critique()
issue()

Functions

review(plan, opts \\ [])

Review a plan and return a structured critique.

static_review(plan)

Quick static-only review (no LLM needed).

Types

critique()

@type critique() :: %{
  score: 1..10,
  issues: [issue()],
  summary: String.t(),
  recommendations: [String.t()]
}

issue()

@type issue() :: %{
  category: atom(),
  severity: :critical | :warning | :info,
  task_id: String.t() | nil,
  message: String.t(),
  recommendation: String.t()
}

Functions

review(plan, opts \\ [])

@spec review(
  PtcRunner.Plan.t(),
  keyword()
) :: {:ok, critique()} | {:error, term()}

Review a plan and return a structured critique.

Combines static analysis (fast, deterministic) with optional LLM analysis (slower, semantic understanding).

Options

  • llm - LLM callback for semantic analysis (optional)
  • static_only - Skip LLM analysis, only do structural checks (default: false)

Returns

A critique map with:

  • score - Overall quality score 1-10
  • issues - List of specific issues found
  • summary - Human-readable summary
  • recommendations - List of actionable fixes

static_review(plan)

@spec static_review(PtcRunner.Plan.t()) :: {:ok, critique()}

Quick static-only review (no LLM needed).