Supertester.PerformanceHelpers (Supertester v0.6.0)

Performance testing and regression detection for OTP systems.

Integrates performance measurement with Supertester's isolation patterns and provides performance assertions for CI/CD pipelines.

Key Features

Performance assertions (max time, memory, reductions)
Memory leak detection
Mailbox growth monitoring
Operation measurement and comparison

Usage

import Supertester.PerformanceHelpers

test "critical path performance" do
  assert_performance(
    fn -> CriticalServer.operation(server) end,
    max_time_ms: 100,
    max_memory_bytes: 1_000_000
  )
end

Summary

Functions

assert_expectations(measurements, expectations)

Asserts that the provided measurement map meets the expectations.

assert_mailbox_stable(server, opts)

Asserts GenServer mailbox remains stable during operation.

assert_no_memory_leak(iterations, operation, opts \\ [])

Verifies operation doesn't leak memory over many iterations.

assert_performance(operation, expectations)

Asserts operation meets performance expectations.

compare_performance(functions)

Compares performance of multiple functions.

measure_mailbox_growth(server, operation, opts \\ [])

Measures mailbox growth during operation.

measure_operation(operation)

Measures operation performance metrics.

Functions

assert_expectations(measurements, expectations)

@spec assert_expectations(
  map(),
  keyword()
) :: :ok

Asserts that the provided measurement map meets the expectations.

assert_mailbox_stable(server, opts)

@spec assert_mailbox_stable(
  pid(),
  keyword()
) :: :ok

Asserts GenServer mailbox remains stable during operation.

Options

:during - Function to execute while monitoring (required)
:max_size - Maximum mailbox size allowed (default: 100)
Additional options forwarded to measure_mailbox_growth/3

Examples

assert_mailbox_stable(server,
  during: fn ->
    perform_operations(server, 1000)
  end,
  max_size: 50
)

assert_no_memory_leak(iterations, operation, opts \\ [])

@spec assert_no_memory_leak(pos_integer(), (-> any()), keyword()) :: :ok

Verifies operation doesn't leak memory over many iterations.

Parameters

iterations - Number of times to run the operation
operation - Function to execute
opts - Options (:threshold for acceptable growth rate, default: 0.1 = 10%)

Examples

assert_no_memory_leak(10_000, fn ->
  handle_message(server, message())
end)

assert_performance(operation, expectations)

@spec assert_performance(
  (-> any()),
  keyword()
) :: :ok

Asserts operation meets performance expectations.

Expectations

:max_time_ms - Maximum execution time in milliseconds
:max_memory_bytes - Maximum memory consumption in bytes
:max_reductions - Maximum reductions (CPU work)

Examples

assert_performance(
  fn -> expensive_operation() end,
  max_time_ms: 100,
  max_memory_bytes: 500_000,
  max_reductions: 100_000
)

compare_performance(functions)

@spec compare_performance(map()) :: map()

Compares performance of multiple functions.

Parameters

functions - Map of function name to function

Returns

Map of function name to performance metrics

Examples

results = compare_performance(%{
  "approach_a" => fn -> approach_a() end,
  "approach_b" => fn -> approach_b() end
})

# Find fastest
fastest = Enum.min_by(results, fn {_name, metrics} -> metrics.time_us end)

measure_mailbox_growth(server, operation, opts \\ [])

@spec measure_mailbox_growth(pid(), (-> any()), keyword()) :: map()

Measures mailbox growth during operation.

Parameters

server - The GenServer PID to monitor
operation - Function to execute while monitoring

Options

:sampling_interval - Interval in ms between mailbox samples (default: 10)

Returns

Map with:

:initial_size - Mailbox size before operation
:final_size - Mailbox size after operation
:max_size - Maximum mailbox size observed
:avg_size - Average mailbox size
:result - The result returned by the wrapped operation

Examples

report = measure_mailbox_growth(server, fn ->
  send_many_messages(server, 1000)
end)

assert report.max_size < 100

measure_operation(operation)

@spec measure_operation((-> any())) :: map()

Measures operation performance metrics.

Returns

Map with:

:time_us - Execution time in microseconds
:memory_bytes - Memory used in bytes
:reductions - Reductions (CPU work)
:result - Operation result

Examples

metrics = measure_operation(fn -> complex_calculation() end)
IO.puts "Time: #{metrics.time_us}μs, Memory: #{metrics.memory_bytes} bytes"