ExDataSketch.Quantiles (ExDataSketch v0.7.1)

Copy Markdown View Source

Facade for quantile sketch algorithms.

Provides a unified API for creating and querying quantile sketches.

Supported Types

Examples

iex> sketch = ExDataSketch.Quantiles.new(type: :kll)
iex> sketch.__struct__
ExDataSketch.KLL

iex> sketch = ExDataSketch.Quantiles.new()
iex> sketch = ExDataSketch.Quantiles.update(sketch, 42.0)
iex> ExDataSketch.Quantiles.count(sketch)
1

Summary

Types

sketch()

Functions

cdf(sketch, split_points)

Returns the CDF at the given split points.

count(sketch)

Returns the total number of items inserted into the sketch.

max_value(sketch)

Returns the maximum value seen by the sketch, or nil if empty.

merge(a, b)

Merges two quantile sketches of the same type.

min_value(sketch)

Returns the minimum value seen by the sketch, or nil if empty.

new(opts \\ [])

Creates a new quantile sketch.

pmf(sketch, split_points)

Returns the PMF at the given split points.

quantile(sketch, rank)

Returns the approximate value at the given normalized rank.

quantiles(sketch, ranks)

Returns the approximate values at the given normalized ranks.

rank(sketch, value)

Returns the approximate normalized rank of a given value.

update(sketch, value)

Updates the sketch with a single numeric value.

update_many(sketch, items)

Updates the sketch with multiple numeric values.

Types

sketch()

@type sketch() ::
  ExDataSketch.KLL.t() | ExDataSketch.DDSketch.t() | ExDataSketch.REQ.t()

Functions

cdf(sketch, split_points)

@spec cdf(sketch(), [number()]) :: [float()] | nil

Returns the CDF at the given split points.

Given split points [s1, s2, ..., sm], returns [rank(s1), rank(s2), ..., rank(sm)]. Returns nil if the sketch is empty.

Examples

iex> sketch = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update_many(1..100)
iex> cdf = ExDataSketch.Quantiles.cdf(sketch, [25.0, 75.0])
iex> length(cdf)
2

count(sketch)

@spec count(sketch()) :: non_neg_integer()

Returns the total number of items inserted into the sketch.

Examples

iex> ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.count()
0

max_value(sketch)

@spec max_value(sketch()) :: float() | nil

Returns the maximum value seen by the sketch, or nil if empty.

Examples

iex> sketch = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update(5.0)
iex> ExDataSketch.Quantiles.max_value(sketch)
5.0

merge(a, b)

@spec merge(sketch(), sketch()) :: sketch()

Merges two quantile sketches of the same type.

Examples

iex> a = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update(1.0)
iex> b = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update(2.0)
iex> merged = ExDataSketch.Quantiles.merge(a, b)
iex> ExDataSketch.Quantiles.count(merged)
2

min_value(sketch)

@spec min_value(sketch()) :: float() | nil

Returns the minimum value seen by the sketch, or nil if empty.

Examples

iex> sketch = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update(5.0)
iex> ExDataSketch.Quantiles.min_value(sketch)
5.0

new(opts \\ [])

@spec new(keyword()) :: sketch()

Creates a new quantile sketch.

Options

  • :type - sketch type, :kll (default), :ddsketch, or :req.
  • All other options are passed to the underlying sketch constructor.

Examples

iex> sketch = ExDataSketch.Quantiles.new(type: :kll, k: 200)
iex> sketch.opts
[k: 200]

pmf(sketch, split_points)

@spec pmf(sketch(), [number()]) :: [float()] | nil

Returns the PMF at the given split points.

Given split points [s1, s2, ..., sm], returns m+1 values representing the approximate fraction of items in each interval. Returns nil if empty.

Examples

iex> sketch = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update_many(1..100)
iex> pmf = ExDataSketch.Quantiles.pmf(sketch, [50.0])
iex> length(pmf)
2

quantile(sketch, rank)

@spec quantile(sketch(), float()) :: float() | nil

Returns the approximate value at the given normalized rank.

Examples

iex> sketch = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update_many(1..100)
iex> q = ExDataSketch.Quantiles.quantile(sketch, 0.5)
iex> is_float(q)
true

quantiles(sketch, ranks)

@spec quantiles(sketch(), [float()]) :: [float() | nil]

Returns the approximate values at the given normalized ranks.

Examples

iex> sketch = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update_many(1..100)
iex> qs = ExDataSketch.Quantiles.quantiles(sketch, [0.25, 0.5, 0.75])
iex> length(qs)
3

rank(sketch, value)

@spec rank(sketch(), number()) :: float() | nil

Returns the approximate normalized rank of a given value.

The rank is the fraction of items in the sketch that are less than or equal to the given value. Returns nil if the sketch is empty.

Examples

iex> sketch = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update_many(1..100)
iex> r = ExDataSketch.Quantiles.rank(sketch, 50.0)
iex> is_float(r)
true

update(sketch, value)

@spec update(sketch(), number()) :: sketch()

Updates the sketch with a single numeric value.

Examples

iex> sketch = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update(1.0)
iex> ExDataSketch.Quantiles.count(sketch)
1

update_many(sketch, items)

@spec update_many(sketch(), Enumerable.t()) :: sketch()

Updates the sketch with multiple numeric values.

Examples

iex> sketch = ExDataSketch.Quantiles.new() |> ExDataSketch.Quantiles.update_many([1.0, 2.0])
iex> ExDataSketch.Quantiles.count(sketch)
2