View Source Erlang/Elixir OpenTelemetry API

EEF Observability WG project Hex.pm

This is the API portion of OpenTelemetry for Erlang and Elixir Applications, implementing the API portion of the specification.

This is a library, it does not start any processes, and should be the only OpenTelemetry dependency of Erlang/Elixir Applications.

Use

There are both Erlang and Elixir macros that make use of the current module's name to lookup a Named Tracer -- a Named Tracer is created for each Application loaded in the system at start time -- for you and can be used for Trace and Span operations:

-include_lib("opentelemetry_api/include/otel_tracer.hrl").

some_fun() ->
    ?with_span(<<"some_fun/0">>, #{}, 
        fun(_SpanCtx) -> 
            ...
            ?set_attribute(<<"key">>, <<"value">>),
            ...
        end),
require OpenTelemetry.Tracer
      
def some_fun() do
    OpenTelemetry.Tracer.with_span "some-span" do
      ...
      OpenTelemetry.Tracer.set_attribute("key", "value")
      ...
    end
end

Tracing API

The macros and functions available for Elixir in OpenTelemetry.Tracer and the Erlang macros in otel_tracer.hrl are the best way to work with Spans. They will automatically use the Tracer named for the Application the module using the macro is in. For example, the Spans created in opentelemetry_oban use the with_span macro resulting in the Span being created with the opentelemetry_oban named Tracer and associated with the Instrumentation Library of the same name and version of the Tracer -- the version also matches the opentelemetry_oban Application version.

Context

Context is used to pass values associated with the current execution unit. At this time the only values kept in the Context by this OpenTelemetry library are the Span Context for the currently active Span and the Baggage

When a Context variable is not an explicit argument in the API macros or functions the Context from the process dictionary is used. If no Context is found in the current process's pdict then one is created.

Starting and Ending Spans

A Span represents a single operation in a Trace. It has a start and end time, can have a single parent and one or more children. The easiest way to create Spans is to wrap the operation you want a Span to represent in the with_span macro. The macro handles getting a Tracer associated with the OTP Application the module is in, starting the Span, setting it as the currently active Span in the Context stored in the process dictionary and ending the Span when the Fun or body of the Elixir macro finish, even if an exception is thrown -- however, the exception is not caught, so it does not change how user code should deal with raised exceptions. After the Span is ended the Context in the process dictionary is reset to its value before the newly started Span was set as the active Span. This handling of the active Span in the process dictionary ensures proper lineage of Spans is kept when starting and ending child Spans.

?with_span(SpanName, StartOpts, Fun)
OpenTelemetry.Tracer.with_span name, start_opts do
    ...
end

StartOpts/start_opts is a map of Span creation options:

  • kind: SpanKind defines the relationship between the Span, its parents, and its children in a Trace. Possible values: internal, server, client, producer and consumer. Defaults to internal if not specified.
  • attributes: See Attributes for details about Attributes. Default is an empty list of attributes.
  • links: List of Links to causally related Spans from the same or a different Trace.
  • start_time: The start time of the Span operation. Defaults to the current time. The option should only be set if the start of the operation described by the Span has already passed.

current_span_ctx(ctx)

set_current_span(span_ctx)

When using start_span instead of with_span there must be a corresponding call to the end Span API to signal that the operation described by the Span has ended. end_span optionally takes a timestamp to use as the end time of the Span.

?end_span()
?end_span(Timestamp)
OpenTelemetry.Tracer.end_span(timestamp \\ :undefined)

Sampling

Sampling is performed at span creation time by the Sampler configured on the Tracer, see Samplers.

To pass attributes for use by the sampler, use the attributes field of StartOpts/start_opts

example:

OpenTelemetry.Tracer.start_span(span_name, %{attributes: %{my_attribute: "my value"}})

Setting Attributes

Setting Attributes can be done with a single key and value passed to set_attribute or through a map of Attributes all at once. Setting an attribute with a key that already exists in the Span's map of attributes will result in that key's value being overwritten.

?set_attribute(Key, Value)
?set_attributes(Attributes)
OpenTelemetry.Tracer.set_attribute(key, value)
OpenTelemetry.Tracer.set_attributes(attributes)

Be aware that there are configurable limits on the number and size of Attributes per Span.

Adding Events

Adding Events can be done by passing the name of the event and the Attributes to associate with it or as a list of Events. Each Event in the list of Events is a map containing the timestamp, name, and Attributes which can be created with the function event/2 and event/3 in the opentelemetry and OpenTelemetry modules.

?add_event(Name, Attributes)
?add_events(Events)
OpenTelemetry.Tracer.add_event(event, attributes)
OpenTelemetry.Tracer.add_events(events)

Setting the Status

Set Status will override the default Span Status of Unset. A Status is a code (ok, error or unset) and, only if the code is error, an optional message string that describes the error.

?set_status(Code, Message)
OpenTelemetry.Tracer.set_status(code, message)

Update Span Name

Updating the Span name can be done after starting the Span but must be done before the Span is end'ed.

?update_name(Name)
OpenTelemetry.Tracer.update_name(name)

Including the OpenTelemetry SDK

When only the API is available at runtime a no-op Tracer is used and no Traces are exported. The OpenTelemetry SDK provides the functionality of Tracers, Span Processors and Exporters and should be included as part of a Release and not as a dependency of any individual Application.

Exporters

Included in the same Github repo as the API and SDK are an exporter for the OpenTelemetry Protocol (OTLP) and Zipkin:

Log Correlation

When a Span is made active in a process, for example when the with_span macro is used, it is added to the logger metadata. The metadata is under the key otel_span_ctx. Example usage:

{kernel,
  [{logger_level, debug},
    {logger,
     [{handler, default, logger_std_h,
       #{formatter => {logger_formatter,
                        #{template => [time, " ", file, ":", line, " ", level, ": ",
                                       {otel_trace_id, ["trace_id=",otel_trace_id," "], []},
                                       {otel_span_id, ["span_id=",otel_span_id," "], []},
                                       msg,"\n"]}}}}]}]}

Integrations

Instrumentations of many popular Erlang and Elixir projects can be found in the contrib repo and on hex.pm under the OpenTelemetry organization.

Contributing

Read OpenTelemetry project contributing guide for general information about the project.