Telemetry

Copy Markdown View Source

TimelessTraces emits telemetry events for monitoring and observability.

Events

Flush

Emitted when the buffer flushes spans to a block.

Event: [:timeless_traces, :flush, :stop]

MeasurementTypeDescription
durationinteger (native time)Time taken to write the block
entry_countintegerNumber of spans in the block
byte_sizeintegerSize of the written block in bytes
MetadataTypeDescription
block_idintegerID of the written block

Query

Emitted when a query completes.

Event: [:timeless_traces, :query, :stop]

MeasurementTypeDescription
durationinteger (native time)Total query execution time
totalintegerTotal matching spans (before pagination)
blocks_readintegerNumber of blocks decompressed
MetadataTypeDescription
filterskeywordThe query filters used

Retention

Emitted when a retention run completes.

Event: [:timeless_traces, :retention, :stop]

MeasurementTypeDescription
durationinteger (native time)Time taken for retention enforcement
blocks_deletedintegerNumber of blocks removed

Block error

Emitted when a block fails to read or decompress.

Event: [:timeless_traces, :block, :error]

MetadataTypeDescription
file_pathstringPath to the block file
reasontermError reason

Attaching handlers

Basic logging

:telemetry.attach_many(
  "timeless-traces-logger",
  [
    [:timeless_traces, :flush, :stop],
    [:timeless_traces, :query, :stop],
    [:timeless_traces, :retention, :stop],
    [:timeless_traces, :block, :error]
  ],
  fn event, measurements, metadata, _config ->
    require Logger
    Logger.info("#{inspect(event)}: #{inspect(measurements)} #{inspect(metadata)}")
  end,
  nil
)

With telemetry_metrics

defmodule MyApp.Telemetry do
  use Supervisor
  import Telemetry.Metrics

  def start_link(arg) do
    Supervisor.start_link(__MODULE__, arg, name: __MODULE__)
  end

  def init(_arg) do
    children = [
      {Telemetry.Metrics.ConsoleReporter, metrics: metrics()}
    ]

    Supervisor.init(children, strategy: :one_for_one)
  end

  defp metrics do
    [
      summary("timeless_traces.flush.stop.duration",
        unit: {:native, :millisecond}),
      summary("timeless_traces.query.stop.duration",
        unit: {:native, :millisecond}),
      counter("timeless_traces.flush.stop.entry_count"),
      counter("timeless_traces.block.error")
    ]
  end
end

With TimelessMetrics

If you're also running TimelessMetrics, you can write telemetry events as metrics:

:telemetry.attach(
  "timeless-traces-to-metrics",
  [:timeless_traces, :flush, :stop],
  fn _event, %{entry_count: count, byte_size: bytes}, _metadata, _config ->
    TimelessMetrics.write(:metrics, "timeless_traces_flush_spans", %{}, count / 1)
    TimelessMetrics.write(:metrics, "timeless_traces_flush_bytes", %{}, bytes / 1)
  end,
  nil
)

With TimelessLogs

If you're also running TimelessLogs, telemetry events are automatically captured via Logger calls.