-type concurrency_result() :: {QPS :: non_neg_integer(), Concurrency :: non_neg_integer()}.

Basic concurrency estimation report

-type concurrency_test() ::
    #{threshold => pos_integer(),
      min => pos_integer(),
      step => pos_integer(),
      max => pos_integer()}.

Concurrency estimation mode options.

• min: initial number of workers, default is 1
• step: increase the number of workers by this value on each iteration, default is 1
• max: maximum number of workers, defaults to erlang:system_info(process_limit) - 1000
• threshold: stop concurrency run when adding this amount of workers does not result in further total throughput increase. Default is 3

-type concurrency_test_result() ::
    concurrency_result() | {Max :: concurrency_result(), [concurrency_result()]}.

Concurrency estimation mode result

-type isolation() :: #{host => string()}.

Node isolation settings.

-type report() ::
    #{mode := timed | continuous | concurrency,
      result := run_statistics(),
      history => [{Concurrency :: pos_integer(), Result :: run_statistics()}],
      code := erlperf_job:code_map(),
      run_options := run_options(),
      concurrency_options => concurrency_test(),
      system => system_information(),
      sleep => sleep | busy_wait}.

Full benchmark report, containing all collected samples and statistics

• mode: benchmark run mode
• result: benchmark result. Concurrency estimation mode contains the best result (with the highest average throughput recorded)
• code: original code
• run_options: full set of options, with all defaults filled in
• system: information about the system benchmark is running on
• history: returned only for concurrency estimation mode, contains a list of all runs with their results
• concurrency_options: returned only for concurrency estimation mode, with all defaults filled in
• sleep: method used for waiting for a specified amount of time. Normally set to sleep, but may be reported as busy_wait if erlperf scheduling is impacted by lock contention or another problem preventing it from using precise timing

-type run_options() ::
    #{concurrency => pos_integer(),
      sample_duration => pos_integer() | undefined | {timed, pos_integer()},
      warmup => non_neg_integer(),
      samples => pos_integer(),
      cv => float() | undefined,
      priority => erlang:priority_level(),
      report => basic | extended | full,
      isolation => isolation()}.

Benchmarking mode selection and parameters of the benchmark run.

• concurrency: number of workers to run, applies only for the continuous benchmarking mode
• cv: coefficient of variation. Acceptable standard deviation for the test to conclude. Not applicable for timed mode. When the value is set, benchmark will continue running until standard deviation for the last collected samples divided by average value (arithmetic mean) is smaller than cv specified.
• isolation: request separate Erlang VM instance for each job. Some benchmarks may lead to internal VM structures corruption, or change global structures affecting other benchmarks when running in the same VM. host sub-option is currently ignored.
• priority: sets the job controller process priority (defaults to high). Running with normal or lower priority may prevent the controller from timely starting ot stopping workers.
• report: applies only for continuous mode. basic report contains only the average value. Specify extended to get the list of actual samples, to calculate exotic statistics. Pass full to receive full report, including benchmark settings and extra statistics for continuous mode - minimum, maximum, average, median and 99th percentile (more metrics may be added in future releases)
• samples: number of measurements to take. Default is 3. For continuous mode it results in a 3 second run, when sample_duration is set to default 1000 ms.
• sample_duration: time, milliseconds, between taking iteration counter samples. Multiplied by samples, this parameter defines the total benchmark run duration. Default is 1000 ms. Passing {timed, Counter}` engages timed mode with `Counter iterations taken samples times
• warmup: how many extra samples are collected and discarded at the beginning of the continuous run

-type run_result() :: non_neg_integer() | [non_neg_integer()].

Benchmark results.

-type run_statistics() ::
    #{average => non_neg_integer(),
      variance => float(),
      stddev => float(),
      median => non_neg_integer(),
      p99 => non_neg_integer(),
      best => non_neg_integer(),
      worst => non_neg_integer(),
      samples => [non_neg_integer()],
      time => non_neg_integer(),
      iteration_time => non_neg_integer()}.

Results reported by a single benchmark run.

• best: highest throughput for continuous mode, or lowest time for timed
• worst: lowest throughput, or highest time
• average: arithmetic mean, iterations for continuous mode and microseconds for timed
• stddev: standard deviation
• median: median (50% percentile)
• p99: 99th percentile
• samples: raw samples from the run, monotonic counter for continuous mode, and times measured for timed run
• time: total benchmark duration (us), may exceed sample * sample_duration when cv is specified and results are not immediately stable
• iteration_time: approximate single iteration time (of one runner)