LLM Analytics is an observability product for LLM-powered applications.

LLM Analytics works by capturing special types of events: traces ($ai_trace) and spans ($ai_generation, $ai_span, and $ai_embedding). They organize into a tree structure that are grouped into sessions:

flowchart TD

    S["<strong>$ai_session_id</strong><br/>(optional)"]

    A[<strong>$ai_trace</strong>]

    A2[<strong>$ai_trace</strong>]

    B[<strong>$ai_generation</strong>]

    C@{ shape: processes, label: "<strong>$ai_spans</strong>" }

    D[<strong>$ai_generation</strong>]

    E@{ shape: processes, label: "<strong>$ai_spans</strong>" }

    F[<strong>$ai_generation</strong>]

    S -.-> A
    S -.-> A2
    A --> B
    A --> C
    C --> D
    C --> E
    E --> F

This module provides an interface for instrumenting your application with traces and spans.

Traces

Traces define how spans are grouped together. You can capture them explicitly with $ai_trace event if you want, but it's not required. As long as all your spans include $ai_trace_id property, PostHog will group them automatically. set_trace/2 function will generate a random UUIDv7 trace id and set it in the context for relevant events automatically:

iex> PostHog.LLMAnalytics.set_trace()
"019a69ad-a9e9-7a20-9540-40101e01a364"
iex> PostHog.get_event_context("$ai_span")
%{"$ai_trace_id": "019a69ad-a9e9-7a20-9540-40101e01a364"}

Sessions

Sessions group multiple traces together using the $ai_session_id property.

In this SDK, sessions work the same way as traces – you set them for the current process and LLM-related events captured in this process will have the same $ai_session_id property.

iex> PostHog.LLMAnalytics.set_session()
"019a88b5-d2e0-75df-8601-7c1101717959"
iex> PostHog.get_event_context("$ai_span")
%{"$ai_session_id": "019a88b5-d2e0-75df-8601-7c1101717959"}

Spans

$ai_generation, $ai_span and $ai_embedding are all span events. To capture them, use the capture_span/2 function:

iex> PostHog.LLMAnalytics.capture_span("$ai_generation", %{"$ai_span_name": "user message"})
{:ok, "019a69b8-c465-7981-99cc-5578ae10f55b"}

It automatically generates and returns a span id, which can be used as a parent span id later:

iex> PostHog.LLMAnalytics.capture_span("$ai_span", %{"$ai_span_name": "tool call", "$ai_parent_id": "019a69b8-c465-7981-99cc-5578ae10f55b"})
{:ok, "019a69bb-4ba1-7cdc-8287-3425e4e7033f"}

Nested Spans

Very often, it's not practical to carry all span properties to the place that actually captures the event. In this case, use start_span/2 to start a span and capture_current_span/3 to capture it:

def generate_response(user_message) do
  LLMAnalytics.start_span(%{"$ai_span_name": "LLM call", "$ai_input_state": user_message})
  
  Req.post!("https://api.openai.com/v1/responses", json: %{input: user_message})
  |> handle_response()
end

defp handle_response(%{status: 200, body: %{"output" => output}}) do
  LLMAnalytics.capture_current_span("$ai_generation", %{"$ai_output_choices": output})
  ...
end

You can also start nested spans and SDK will automatically take care of setting parent span IDs:

iex> PostHog.LLMAnalytics.start_span(%{"$ai_span_name": "parent"})
"019a69de-0d29-7160-bea2-c93124109de6"
iex> PostHog.LLMAnalytics.capture_span("$ai_span", %{"$ai_span_name": "child"})
{:ok, "019a69de-38a9-7975-ac51-97e056cee6bf"}
iex> PostHog.LLMAnalytics.capture_current_span("$ai_span")
{:ok, "019a69de-0d29-7160-bea2-c93124109de6"}

Think of capture_span as a way to capture "leaf" nodes of the tree.

Asynchronous Environment

Just as with Context, LLMAnalytics tracks the current trace and span in the process dictionary. Any time you spawn a new process, you'll need to propagate this information. Use set_session/2, set_trace/2 and set_root_span/2:

def generate_response(user_message) do
  session_id = LLMAnalytics.set_session()
  trace_id = LLMAnalytics.set_trace()
  {:ok, span_id} = LLMAnalytics.capture_span("$ai_span", %{"$ai_span_name": "top level", "$ai_input_state": user_message})
  
  Task.async(fn -> 
    LLMAnalytics.set_session(session)
    LLMAnalytics.set_trace(trace_id)
    LLMAnalytics.set_root_span(span_id)
    
    resp = Req.post!("https://api.openai.com/v1/responses", json: %{input: "Check if this message violates our policies: " <> user_message})
    LLMAnalytics.capture_span("$ai_generation", %{"$ai_span_name": "railguard check", ...})
    ...
  end)
  
  Req.post!("https://api.openai.com/v1/responses, json: %{input: user_message})
  ...
end