Captures trajectory from agent execution via telemetry.
The Collector attaches telemetry handlers during execution to automatically
capture all Puck.call/4 and Puck.stream/4 invocations and build a
Puck.Eval.Trajectory.
Usage
{output, trajectory} = Puck.Eval.Collector.collect(fn ->
MyAgent.run("Find John's email")
end)
trajectory.total_steps # => 2
trajectory.total_tokens # => 385How It Works
- Attaches handlers to call and stream telemetry events
- Runs the provided function
- Collects telemetry events from this process and any spawned child processes
- Matches start/stop events by emitting process to build Steps
- Returns the result and the captured Trajectory
The Collector uses process isolation - each collect/1 call has its own
unique handler ID, so concurrent collections don't interfere with each other.
Child processes spawned during collection (via Task.async, etc.) are
automatically tracked as long as they inherit the $ancestors process
dictionary key (which OTP processes do by default).
Requirements
Requires the :telemetry dependency to be installed.
Summary
Functions
Collects trajectory from the provided function.
Functions
Collects trajectory from the provided function.
Wraps the function, capturing all Puck.call/4 and Puck.stream/4 invocations
made during its execution. Returns a tuple of {result, trajectory}.
Streaming calls are captured with step.metadata[:streamed] == true and the
concatenated stream content as step.output. Note that streaming steps have
zero token counts since usage isn't available during streaming.
Example
{output, trajectory} = Collector.collect(fn ->
client = Puck.Client.new({Puck.Backends.ReqLLM, "anthropic:claude-sonnet-4-5"})
{:ok, response, _ctx} = Puck.call(client, "Hello!")
response.content
end)
IO.puts("Output: #{output}")
IO.puts("Steps: #{trajectory.total_steps}")
IO.puts("Tokens: #{trajectory.total_tokens}")Options
:timeout- Maximum time to wait for events after function completes (default: 100ms)