View Source Benchee.Scenario (Benchee v1.3.0)

Core data structure representing one particular case (combination of function and input).

Represents the combination of a particular function to benchmark (also called "job" defined by job_name and function) in combination with a specific input (input_name and input). When no input is given, the combined value is representative of "no input".

A scenario then further gathers all data collected for this particular combination during Benchee.Benchmark.collect/3, which are then used later in the process by Benchee.Statistics to compute the relevant statistics which are then also added to the scenario. It is the home of the aggregated knowledge regarding this particular case/scenario.

name is the name that should be used by formatters to display scenarios as it potentially includes the tag present when loading scenarios that were saved before. See display_name/1.

Summary

Types

benchmarking_function()

The main function executed while benchmarking.

t()

All the data collected for a scenario (combination of function and input)

to_benchmark()

What shall be benchmarked, mostly a function but can contain options.

Functions

data_processed?(scenario, atom)

Returns true if data of the provided type has been fully procsessed, false otherwise.

display_name(map)

Returns the correct name to display of the given scenario data.

measurement_data(scenario, measurement_type)

Given the measurement type name given by measurement_types/0, get the associated data.

measurement_types()

Returns the different measurement types supported.

Types

benchmarking_function()

@type benchmarking_function() :: (-> any()) | (any() -> any())

The main function executed while benchmarking.

No arguments if no inputs are used, one argument if inputs are used.

t()

@type t() :: %Benchee.Scenario{
  after_each: Benchee.Benchmark.Hooks.hook_function() | nil,
  after_scenario: Benchee.Benchmark.Hooks.hook_function() | nil,
  before_each: Benchee.Benchmark.Hooks.hook_function() | nil,
  before_scenario: Benchee.Benchmark.Hooks.hook_function() | nil,
  function: benchmarking_function(),
  input: any() | nil,
  input_name: String.t() | nil,
  job_name: String.t(),
  memory_usage_data: Benchee.CollectionData.t(),
  name: String.t(),
  reductions_data: Benchee.CollectionData.t(),
  run_time_data: Benchee.CollectionData.t(),
  tag: String.t() | nil
}

All the data collected for a scenario (combination of function and input)

Among all the data required to execute the scenario (function, input, all the hooks aka after*/before*), data needed to display (name, job_name, input_name, tag) and of course run_time_data and memory_data with all the samples and computed statistics.

to_benchmark()

@type to_benchmark() :: benchmarking_function() | {benchmarking_function(), keyword()}

What shall be benchmarked, mostly a function but can contain options.

Options are there for hooks (after_each, before_each etc.)

Functions

data_processed?(scenario, atom)

@spec data_processed?(t(), :run_time | :memory) :: boolean()

Returns true if data of the provided type has been fully procsessed, false otherwise.

Current available types are run_time and memory. Reasons they might not have been processed yet are:

Suite wasn't configured to collect them at all
Benchee.statistics/1 hasn't been called yet so that data was collected but statistics aren't present yet

Examples

iex> scenario = %Benchee.Scenario{run_time_data: %Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 100}}}
iex> data_processed?(scenario, :run_time)
true

iex> scenario = %Benchee.Scenario{memory_usage_data: %Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 1}}}
iex> data_processed?(scenario, :memory)
true

iex> scenario = %Benchee.Scenario{reductions_data: %Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 1}}}
iex> data_processed?(scenario, :reductions)
true

iex> scenario = %Benchee.Scenario{memory_usage_data: %Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 0}}}
iex> data_processed?(scenario, :memory)
false

display_name(map)

@spec display_name(t()) :: String.t()

Returns the correct name to display of the given scenario data.

In the normal case this is job_name, however when scenarios are loaded they are tagged and these tags should be shown for disambiguation.

Examples

iex> display_name(%Benchee.Scenario{job_name: "flat_map"})
"flat_map"

iex> display_name(%Benchee.Scenario{job_name: "flat_map", tag: "main"})
"flat_map (main)"

iex> display_name(%{job_name: "flat_map"})
"flat_map"

measurement_data(scenario, measurement_type)

@spec measurement_data(t(), :memory | :reductions | :run_time) ::
  Benchee.CollectionData.t()

Given the measurement type name given by measurement_types/0, get the associated data.

Raises if no correct measurement type was specified.

Examples

iex> scenario = %Benchee.Scenario{run_time_data: %Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 1}}}
iex> measurement_data(scenario, :run_time)
%Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 1}}

iex> scenario = %Benchee.Scenario{memory_usage_data: %Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 2}}}
iex> measurement_data(scenario, :memory)
%Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 2}}

iex> scenario = %Benchee.Scenario{reductions_data: %Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 3}}}
iex> measurement_data(scenario, :reductions)
%Benchee.CollectionData{statistics: %Benchee.Statistics{sample_size: 3}}

iex> measurement_data(%Benchee.Scenario{}, :memory)
%Benchee.CollectionData{}

iex> measurement_data(%Benchee.Scenario{}, :invalid)
** (FunctionClauseError) no function clause matching in Benchee.Scenario.measurement_data/2

measurement_types()

@spec measurement_types() :: [:memory | :reductions | :run_time, ...]

Returns the different measurement types supported.

Examples

iex> measurement_types()
[:run_time, :memory, :reductions]