Observability

View Source

OpenResponses emits structured telemetry events at every significant point in the request lifecycle. These events power Prometheus metrics via the built-in PromEx plugin and can be consumed by any :telemetry handler.

Telemetry events

All events are emitted under the [:open_responses, ...] namespace.

Request events

EventMetadata
[:open_responses, :request, :start]%{model: model}
[:open_responses, :request, :stop]%{model: model, status: status}

Loop events

EventMetadata
[:open_responses, :loop, :iteration, :start]%{model: model, iteration: n}
[:open_responses, :loop, :iteration, :stop]%{model: model, iteration: n, action: action}

Adapter events

EventMetadata
[:open_responses, :adapter, :stream, :start]%{model: model, iteration: n}
[:open_responses, :adapter, :stream, :stop]%{model: model}
[:open_responses, :adapter, :stream, :chunk]%{type: event_type}

Tool events

EventMetadata
[:open_responses, :tool, :dispatch, :start]%{tool: name}

| [:open_responses, :tool, :dispatch, :stop] | %{tool: name, result: :ok | :error} |

Stream events

EventMetadata
[:open_responses, :stream, :event]%{type: event_type, response_id: id}

Prometheus metrics

OpenResponses ships a PromEx plugin that wires the above telemetry to Prometheus metrics.

Setup

Add OpenResponses.PromEx to your supervision tree:

# application.ex
children = [
  OpenResponses.PromEx,
  # ... other children
]

Mount the metrics endpoint in your router:

# router.ex
get "/metrics", PromEx.Plug, prom_ex_module: OpenResponses.PromEx

Add to config/config.exs:

config :open_responses, OpenResponses.PromEx,
  manual_metrics_start: :no_async,
  drop_metrics_groups: [],
  grafana: :disabled,
  metrics_server: :disabled

Available metrics

MetricTypeDescription
open_responses_loop_iterations_totalCounterTotal agentic loop iterations, tagged by model
open_responses_tool_dispatches_totalCounterTotal internal tool calls, tagged by tool
open_responses_stream_events_totalCounterTotal SSE events emitted, tagged by type
open_responses_requests_totalCounterTotal requests received, tagged by model

Scraping

Metrics are available at GET /metrics in Prometheus text format. Configure your Prometheus instance:

scrape_configs:
  - job_name: open_responses
    static_configs:
      - targets: ['your-host:4000']
    metrics_path: /metrics

Custom telemetry handlers

Attach your own handler to any event:

:telemetry.attach(
  "my-loop-logger",
  [:open_responses, :loop, :iteration, :start],
  fn _event, _measurements, %{model: model, iteration: n}, _config ->
    Logger.info("Loop iteration #{n} started", model: model)
  end,
  nil
)

Or attach to multiple events at once:

:telemetry.attach_many(
  "my-handler",
  [
    [:open_responses, :request, :start],
    [:open_responses, :request, :stop]
  ],
  &MyApp.Telemetry.handle/4,
  %{}
)

Logger metadata

Each loop process sets Logger metadata for its lifetime:

%{
  response_id: "resp_01",
  model: "gpt-4o",
  loop_iteration: 2
}

All log lines emitted during a loop carry this context automatically, making it straightforward to trace a single request through your logs.

LiveDashboard

If you include phoenix_live_dashboard (enabled by default in Phoenix apps), you can view telemetry metrics in the LiveDashboard at /dev/dashboard. The OpenResponsesWeb.Telemetry module registers the relevant metrics for the dashboard.