Nous.Eval.Metrics (nous v0.13.3)

Metrics collected during evaluation runs.

Tracks token usage, latency, tool calls, and costs.

Example

metrics = Metrics.new()
metrics = Metrics.from_usage(agent_result.usage)
IO.puts("Total tokens: #{metrics.total_tokens}")

Summary

Types

t()

Functions

from_agent_result(result, duration_ms)

Create metrics from an agent result.

from_usage(usage)

Create metrics from a Nous.Usage struct.

merge(m1, m2)

Merge two metrics structs.

new()

Create empty metrics.

with_cost(metrics, provider)

Add cost estimation to metrics.

Types

t()

@type t() :: %Nous.Eval.Metrics{
  estimated_cost: float() | nil,
  first_token_ms: non_neg_integer() | nil,
  input_tokens: non_neg_integer(),
  iterations: non_neg_integer(),
  model_latency_ms: non_neg_integer(),
  output_tokens: non_neg_integer(),
  requests: non_neg_integer(),
  retries: non_neg_integer(),
  tool_calls: non_neg_integer(),
  tool_errors: non_neg_integer(),
  tool_latency_ms: non_neg_integer(),
  tools_used: %{required(String.t()) => non_neg_integer()},
  total_duration_ms: non_neg_integer(),
  total_tokens: non_neg_integer()
}

Functions

from_agent_result(result, duration_ms)

@spec from_agent_result(map(), non_neg_integer()) :: t()

Create metrics from an agent result.

from_usage(usage)

@spec from_usage(Nous.Usage.t()) :: t()

Create metrics from a Nous.Usage struct.

merge(m1, m2)

@spec merge(t(), t()) :: t()

Merge two metrics structs.

new()

@spec new() :: t()

Create empty metrics.

with_cost(metrics, provider)

@spec with_cost(t(), String.t()) :: t()

Add cost estimation to metrics.