View Source Datadog.Sketch (Data Streams Ex v1.2.2)
A minimal implementation of the distributed quantile sketch (DDSketch)
algorithm as implemented in the sketches-go library. For more
information, please read the sketches-go README.
This is a minimal implementation needed to support DataDog data streams.
Some initial work was taken from the great dog_sketch Elixir library.
This library includes some DataDog specific work like Protobuf encoding. It's
worth mentioning that some of this code looks out of place in Elixir. That is
because it's pulled directly from the sketches-go library and kept
similar for ease of debugging and backporting fixes.
Summary
Functions
Adds a value to the sketch.
Adds a bin type to the sketch.
Adds multiple bin types to the sketch.
Adds a value to the sketch with a float count.
Returns true if no value has been added to this sketch.
Return the total number of values that have been added to this sketch.
Returns the maximum value that has been added to this sketch.
Returns the minimum value that has been added to this sketch.
Returns an approximation of the sum of the values that have been added to the sketch. If the values that have been added to the sketch all have the same sign, the approximation error has the relative accuracy guarantees of the mapping used for this sketch.
Return the value at the specified quantile.
Return the values at the respective specified quantiles.
Returns the number of zero values that have been added to this sketch.
Creates a new Sketch.
Creates a new Sketch with separate stores for positive and negative values.
Creates a new Sketch with the default index mapping and store values.
This equates to using the IndexMapping.Logarithmic module with a
0.01 relative accuracy and the Store.Dense store.
Reweight multiples all values from the sketch by weight, but keeps
the same global distribution. weight has to be strictly greater
than zero.
Returns a Protobuf-able struct for the sketch. Used for sending data to Datadog.
Types
@type t() :: %Datadog.Sketch{ index_mapping: Datadog.Sketch.IndexMapping.t(), negative_value_store: Datadog.Sketch.Store.t(), positive_value_store: Datadog.Sketch.Store.t(), zero_count: float() }
Functions
Adds a value to the sketch.
Examples
iex> %Sketch{} = Sketch.add(Sketch.new_default(), 4293.7)
iex> %Sketch{} = Sketch.add(Sketch.new_default(), -4592.3)
iex> %Sketch{} = Sketch.add(Sketch.new_default(), 0.0)@spec add_bin(t(), Datadog.Sketch.Store.bin()) :: t()
Adds a bin type to the sketch.
Examples
iex> %Sketch{} = Sketch.add_bin(Sketch.new_default(), %{index: 100, count: 13.13})@spec add_bins(t(), [Datadog.Sketch.Store.bin()]) :: t()
Adds multiple bin types to the sketch.
Examples
iex> %Sketch{} = Sketch.add_bins(Sketch.new_default(), [
...> %{index: 100, count: 13.13},
...> %{index: 20, count: 2342.4}
...> ])Adds a value to the sketch with a float count.
Examples
iex> %Sketch{} = Sketch.add_with_count(Sketch.new_default(), 4293.7, 294)
iex> %Sketch{} = Sketch.add_with_count(Sketch.new_default(), -4592.3, 23)
iex> %Sketch{} = Sketch.add_with_count(Sketch.new_default(), 0.0, 10)@spec empty?(t()) :: bool()
Returns true if no value has been added to this sketch.
Examples
iex> Sketch.empty?(Sketch.new_default())
true
iex> Sketch.new_default()
...> |> Sketch.add_with_count(42, 482.23)
...> |> Sketch.empty?()
false
iex> Sketch.new_default()
...> |> Sketch.add_with_count(-75, 157)
...> |> Sketch.empty?()
falseReturn the total number of values that have been added to this sketch.
Examples
iex> sketch = Sketch.add_bins(Sketch.new_default(), [
...> %{index: 12, count: 423.43},
...> %{index: 244, count: 1238.123},
...> %{index: 124, count: 2184.124}
...> ])
...> Sketch.get_count(sketch)
3845.6769999999997Returns the maximum value that has been added to this sketch.
Examples
iex> sketch = Sketch.add_bins(Sketch.new_default(), [
...> %{index: 12, count: 34},
...> %{index: -423, count: 571},
...> %{index: 0, count: 27.1}
...> ])
...> Sketch.get_max_value(sketch)
12.061674179039226Returns the minimum value that has been added to this sketch.
Examples
iex> sketch = Sketch.add_bins(Sketch.new_default(), [
...> %{index: 12, count: 34},
...> %{index: -423, count: 571},
...> %{index: 0, count: 27.1}
...> ])
...> Sketch.get_min_value(sketch)
-424.1773628048435Returns an approximation of the sum of the values that have been added to the sketch. If the values that have been added to the sketch all have the same sign, the approximation error has the relative accuracy guarantees of the mapping used for this sketch.
Examples
# Verified with golang implementation (within accuracy)
iex> sketch = Sketch.add_bins(Sketch.new_default(), [
...> %{index: 12, count: 34.0},
...> %{index: -24, count: 84.0},
...> %{index: 0, count: 2.4}
...> ])
...> Sketch.get_sum(sketch)
-1589.8430984284082Return the value at the specified quantile.
Examples
# Validated with golang implementation
iex> sketch = Sketch.add_bins(Sketch.new_default(), [
...> %{index: 12, count: 423.43},
...> %{index: 244, count: 1238.123},
...> %{index: 124, count: 2184.124}
...> ])
...> Sketch.get_value_at_quantile(sketch, 0.4)
125.2248607394614Return the values at the respective specified quantiles.
Examples
# Validated with golang implementation (within accuracy)
iex> sketch = Sketch.add_bins(Sketch.new_default(), [
...> %{index: 12, count: 423.43},
...> %{index: 244, count: 1238.123},
...> %{index: 124, count: 2184.124}
...> ])
...> Sketch.get_values_at_quantiles(sketch, [0.1, 0.25, 0.5])
[12.061674179039226, 125.2248607394614, 125.2248607394614]Returns the number of zero values that have been added to this sketch.
Examples
iex> sketch = Sketch.add_bins(Sketch.new_default(), [
...> %{index: 12, count: 34},
...> %{index: -423, count: 571},
...> %{index: 0, count: 27.1}
...> ])
...> Sketch.get_zero_count(sketch)
27.1@spec new(Datadog.Sketch.IndexMapping.t(), Datadog.Sketch.Store.t()) :: t()
Creates a new Sketch.
Examples
iex> index_mapping = Sketch.IndexMapping.Logarithmic.new(0.01)
...> store = Sketch.Store.Dense.new()
...> %Sketch{} = Sketch.new(index_mapping, store)@spec new( Datadog.Sketch.IndexMapping.t(), Datadog.Sketch.Store.t(), Datadog.Sketch.Store.t() ) :: t()
Creates a new Sketch with separate stores for positive and negative values.
Examples
iex> index_mapping = Sketch.IndexMapping.Logarithmic.new(0.01)
...> positive_store = Sketch.Store.Dense.new()
...> negative_store = Sketch.Store.Dense.new()
...> %Sketch{} = Sketch.new(index_mapping, positive_store, negative_store)@spec new_default() :: t()
Creates a new Sketch with the default index mapping and store values.
This equates to using the IndexMapping.Logarithmic module with a
0.01 relative accuracy and the Store.Dense store.
Examples
iex> %Sketch{} = Sketch.new_default()Reweight multiples all values from the sketch by weight, but keeps
the same global distribution. weight has to be strictly greater
than zero.
Examples
# Verified with golang implementation (within accuracy)
iex> sketch = Sketch.add_bins(Sketch.new_default(), [
...> %{index: -7, count: 10.0},
...> %{index: 24, count: 20.0},
...> %{index: 3, count: 30.0}
...> ])
...> sketch = Sketch.reweight(sketch, 2.5)
...> Sketch.get_sum(sketch)
1237.7881696246109Returns a Protobuf-able struct for the sketch. Used for sending data to Datadog.
Examples
iex> %Sketch.Protobuf.DDSketch{} = Sketch.to_proto(Sketch.new_default())