Job is an instance of a benchmark.

Every job has a corresponding temporary Erlang module generated. Use source/1 to get the source code of the generated module. The structure of this code is an implementation detail and may change between releases.

Job controls how many workers are executing runner code in a tight loop. It does not restart a failing worker, user must ensure proper error handing and reporting. If a worker process crashes, standard CRASH REPORT message is printed to the log (console).

Job accepts a code_map() containing at least a runner function definition.

See callable() for accepted function definitions.

      erlang
   erlperf:compare([
      #{runner => fun (V) -> rand:mwc59(V) end, init_runner => {rand, mwc59_seed, []}},
      #{runner => "run(V) -> rand:mwc59(V).", init_runner => {rand, mwc59_seed, []}}
   ], #{}).
   [4371541,131460130]

In the example above, callable defined as fun is 30 times slower than the code compiled from the source. The difference is caused by the Erlang Runtime implementation, where indirect calls via fun are considerably more expensive. As a rule of thumb, source code provides the smallest overhead, followed by MFA tuples.

      erlang
     erlperf_job:start_link(
         #{
             runner => "run(Max) -> rand:uniform(Max).",
             init => {pg, start_link, [scope]},
             init_runner =>
                 fun ({ok, Pid}) ->
                     {total_heap_size, THS} = erlang:process_info(Pid, total_heap_size),
                     THS
                 end,
             done => fun ({ok, Pid}) -> gen_server:stop(Pid) end
         }
     ).

      erlang
   erlperf_job:start_link(
       #{
           runner => "runner(Max) -> rand:uniform(Max).",
           init => "init() -> pg:start_link().",
           init_runner => "init_runner({ok, Pid}) ->
               {total_heap_size, THS} = erlang:process_info(Pid, total_heap_size),
               THS.",
           done => "done({ok, Pid}) -> gen_server:stop(Pid)."
       }
   ).

runner-function

Runner function

Runner function represents code that is run in the tight loop, counting iterations aggregated between all workers. To give an example, benchmarking a function that takes exactly a millisecond to execute, with 2 workers, for 2 seconds, will result in 4000 iterations in total. This would be the value returned by sample/1.

Runner definition can accept zero, one or two arguments.

runner/0 ignores the value returned by init_runner/0,1.

      erlang
   #{
      runner => "run(Init) -> io:format(\"~b~n\", [Init]).",
      init_runner => "0."
   }

      erlang
   #{
      runner => "run(Init, Acc) -> io:format(\"~b~n\", [Init + Acc]), Acc + 1.",
      init_runner => "0."
   }

      bash
   ./erlperf --init_runner 'rand:mwc59_seed().' 'run(_, Cur) -> rand:mwc59(Cur).'
   Code                                    ||        QPS       Time
   run(_, Cur) -> rand:mwc59(Cur).          1     123 Mi       8 ns

common-test-usage

Common Test usage

      erlang
   benchmark_rand(Config) when is_list(Config) ->
       %% run timer:sleep(1000) for 5 second, 4 runners
       {ok, Job} = erlperf_job:start_link(#{runner => {timer, sleep, [1000]}}),
       Handle = erlperf_job:handle(Job),
       ok = erlperf_job:set_concurrency(Job, 4), %% 4 runner instances
       InitialIterations = erlperf_job:sample(Handle),
       timer:sleep(5000),
       IterationsIn5Sec = erlperf_job:sample(Handle) - InitialIterations,
       erlperf_job:request_stop(Job), %% use gen:stop(Job) for synchronous call
       %% expect at least 16 iterations (and up to 20)
       ?assert(IterationsIn5Sec >= 16, {too_slow, IterationsIn5Sec}),
       ?assert(IterationsIn5Sec =< 20, {too_fast, IterationsIn5Sec}).