benchee_json

v0.3.1

benchee_json v0.3.1 Benchee.Formatters.JSON

Functionality for converting Benchee benchmarking results to JSON so that they can be written to file or just generated for your usage.

The most basic use case is to configure it as one of the formatters to be used by Benchee.run/2.

list = Enum.to_list(1..10_000)
map_fun = fn(i) -> [i, i * i] end

Benchee.run(%{
  "flat_map"    => fn -> Enum.flat_map(list, map_fun) end,
  "map.flatten" => fn -> list |> Enum.map(map_fun) |> List.flatten end
},
  formatters: [
    &Benchee.Formatters.JSON.output/1,
    &Benchee.Formatters.Console.output/1
  ],
  formatter_options: [json: [file: "my.json"]]
)

Summary

Functions

encode!(benchee_structure)

Simple wrapper for encoding a map/benchee structure to JSON

format(map)

Formats the output of benchee to a map from input names to their associated JSON with run_times and statistics

format_measurements(statistics, run_times)

Transforms the benchmarking results of run times and staistics (for one input) to JSON

output(map)

Uses Benchee.Formatters.JSON.format/1 to transform the statistics output to a JSON, but also already writes it to a file defined in the initial configuration under formatter_options: %{json: %{file: "my.json"}}

Functions

encode!(benchee_structure)

Simple wrapper for encoding a map/benchee structure to JSON.

Decouples it from the actual JSON library (currently Poison) used by benchee_json so that other plugins can rely on it just working with a general Benchee suite structure.

format(map) 
format(Benchee.Suite.t) :: %{optional(Benchee.Suite.key) => String.t}

Formats the output of benchee to a map from input names to their associated JSON with run_times and statistics.

Examples 

iex> suite = %{
...>   run_times: %{"Some Input" =>
...>     %{"My Job" => [200, 400, 400, 400, 500, 500, 700, 900]}},
...>   statistics: %{"Some Input" =>
...>     %{"My Job" => %{average: 500.0, ips: 2000.0, std_dev: 200.0,
...>       std_dev_ratio: 0.4, std_dev_ips: 800.0, median: 450.0}}}}
iex> Benchee.Formatters.JSON.format(suite)
%{
  "Some Input" =>
    "{\"statistics\":{\"My Job\":{\"std_dev_ratio\":0.4,\"std_dev_ips\":800.0,\"std_dev\":200.0,\"median\":450.0,\"ips\":2.0e3,\"average\":500.0}},\"sort_order\":[\"My Job\"],\"run_times\":{\"My Job\":[200,400,400,400,500,500,700,900]}}"
}
format_measurements(statistics, run_times) 
format_measurements(%{optional(Benchee.Suite.key) => Benchee.Statistics.t}, %{optional(Benchee.Suite.key) => [number]}) :: String.t

Transforms the benchmarking results of run times and staistics (for one input) to JSON.

Examples 

iex> run_times  = %{"My Job" => [200, 400, 400, 400, 500, 500, 700, 900]}
iex> statistics =  %{"My Job" => %{average: 500.0, ips: 2000.0,
...>   std_dev: 200.0, std_dev_ratio: 0.4, std_dev_ips: 800.0,
...>   median: 450.0}}
iex> Benchee.Formatters.JSON.format_measurements(statistics, run_times)
"{\"statistics\":{\"My Job\":{\"std_dev_ratio\":0.4,\"std_dev_ips\":800.0,\"std_dev\":200.0,\"median\":450.0,\"ips\":2.0e3,\"average\":500.0}},\"sort_order\":[\"My Job\"],\"run_times\":{\"My Job\":[200,400,400,400,500,500,700,900]}}"
output(map) 
output(Benchee.Suite.t) :: Benchee.Suite.t

Uses Benchee.Formatters.JSON.format/1 to transform the statistics output to a JSON, but also already writes it to a file defined in the initial configuration under formatter_options: %{json: %{file: "my.json"}}