View Source Benchee.Formatter behaviour (Benchee v1.3.0)

Defines a behaviour for formatters in Benchee, and functions to work with these.

When implementing a Benchee formatter as a behaviour please adopt this behaviour, as it helps with uniformity and also allows at least the .format function of formatters to be run in parallel.

The module itself then has functions to deal with formatters defined in this way allowing for parallel output through output/1 or just output a single formatter through output/3.

Summary

Types

Options given to formatters, entirely defined by formatter authors.

A suite scrubbed of heavy data.

Callbacks

Takes a suite and returns a representation write/2 can use.

Takes the return value of format/1 and then performs some I/O for the user to actually see the formatted data (UI, File IO, HTTP, ...)

Functions

Format and output all configured formatters and formatting functions.

Output a suite with a given formatter and options.

Types

@type options() :: any()

Options given to formatters, entirely defined by formatter authors.

@type scrubbed_suite() :: Benchee.Suite.t()

A suite scrubbed of heavy data.

Type to bring awareness to the fact that format/2 doesn't have access to all data in Benchee.Suite - please read the docs for format/2 to learn more.

Callbacks

@callback format(Benchee.Suite.t(), options()) :: any()

Takes a suite and returns a representation write/2 can use.

Takes the suite and returns whatever representation the formatter wants to use to output that information. It is important that this function needs to be pure (aka have no side effects) as Benchee will run format/1 functions of multiple formatters in parallel. The result will then be passed to write/2.

Note: Due to memory consumption issues in benchmarks with big inputs, the suite passed to the formatters is missing anything referencing big input data to avoid huge memory consumption and run time. Namely this constitutes:

Technically speaking this "scrubbing" of Benchee.Suite only occurs when formatters are run in parallel, you still shouldn't rely on those values (and they should not be needed). If you do need them for some reason, please get in touch/open an issue.

@callback write(any(), options()) :: :ok | {:error, String.t()}

Takes the return value of format/1 and then performs some I/O for the user to actually see the formatted data (UI, File IO, HTTP, ...)

Functions

@spec output(Benchee.Suite.t()) :: Benchee.Suite.t()

Format and output all configured formatters and formatting functions.

Expects a suite that already has been run through all previous functions so has the aggregated statistics etc. that the formatters rely on.

Works by invoking the format/2 and write/2 functions defined in this module. The format/2 functions are actually called in parallel (as they should be pure) - due to potential interference the write/2 functions are called serial.

Also handles pure functions that will then be called with the suite.

You can't rely on the formatters being called in pre determined order.

Link to this function

output(suite, formatter, options \\ %{})

View Source
@spec output(Benchee.Suite.t(), module(), options()) :: Benchee.Suite.t()

Output a suite with a given formatter and options.

Replacement for the old MyFormatter.output/1 - calls format/2 and write/2 one after another to create the output defined by the given formatter module. For the given options please refer to the documentation of the formatters you use.