PropCheck.Instrument behaviour (PropCheck v1.4.1) View Source

Provides functions and macros for instrument byte code with additional yields and other constructs to ease testing of concurrent programs and state machines.

Why is instrumentation important?

The Erlang scheduler is relatively predictable and stable with regard to pre-emptive scheduling. This means that every run has more or the less the same amount of virtual machine instructions before a switch to another process happens. These process switches are required to reveal any concurrency bugs. A simple way to provoke more process switches are calls to :erlang.yield() which gives the scheduler the possibility to switch early on to another process. It is not defined if the scheduler reacts on this hint, but it often does and allows for more unpredictable schedules revealing more concurrency bugs.

The usual advice is to sprinkle the code under test with manually added calls to :erlang.yield(), but this is a daunting task. Additionally, you need to remove this additional code before production use.

The instrumentation

The functions in this module automate the instrumentation immediately before running the tests. We instrument call to "interesting" functions of the Erlang and Elixir ecosystem, e.g. calls to GenServer or ets tables. We do this by examining the byte code, checking each function call, and if we found some interesting call target, we add a call to :erlang.yield() immediately before. This is what the PropCheck.YieldInstrumenter module provides. It implements the behaviour Instrument, which requires the implementation of two callbacks handle_function_call/1 and is_instrumentable_function/2. After instrumentation, the code reloading mechanism of the Erlang VM enables the new code and the tests can run.

Typical usage

To ensure instrumentation before running the tests, you implement the setup_all macro of ExUnit:

setup_all do
  Instrument.instrument_module(Cache, YieldInstrumenter)
  :ok # no update of a context
end

In this example, we instrument only a specific module. You can also instrument all modules of an application by calling Instrument.instrument_app(:my_app_under_test, YieldInstrumenter).

Implementing your own instrumenter

For implementing your own instrumenter, you need to get acquainted with the Erlang Abstract Form (EAF), which is the internal abstract syntax tree available to the Erlang VM at runtime. This format is quite different from the Elixir AST, in particular it has not the regular form but consists of many different structures. This requires a lot of cases to be handled for analyizing the AST. Little helpers for encoding the instrumented code is provided by encode_call/1 and encode_value/1 as well as by prepend_call/2. For debugging and revealing the structure of a specific EAF, you can use print_fun/1.

Link to this section Summary

Types

The type for a node in the Erlang Abstract Form encoding an atom value

Type type for a block of expression in Erlang Abstract Form

The type for a remote call in Erlang Abstract Form

Functions

Encodes a call to :erlang.yield() as Erlang Astract Form.

Compiles the abstract code of a module and loads it immediately into the VM.

Enocdes a call given as tuple {m, f, a} as Erlang Abstract Form.

Encodes a call to m.f.(a) as Erlang Abstract Form.

Encodes a value as Erlang Astract Form.

Retrieves the abstract code, i.e. the list of forms, of the given module as found in the code server.

Instruments all modules of an entire OTP application.

Takes the object code of the module, instruments it and update the module in the code server with instrumented byte code.

Checks if the given function is a candidate for instrumentation, i.e. does something interesting with respect to concurrency. Examples are process handling, handling of shared state or sending and receiving messages.

Checks if the code is already instrumented. If not, returns false otherwise returns true

Prepends the call to to_be_wrapped_call by a call to new_call. The result of new_call is ignored.

Debugging aid for analyzing code generations. Prints the restructered Erlang code of function fun in module mod. We use Erlang code here, because Elixir source code cannot generated from the byte code format due to macros, which change the compilation process too heavily.

Callbacks

Handle the instrumentation of a (remote) function call. Must return a valid expression in Erlang Abstract Form.

A callback to decide if the function mod:fun with any arity is a candidate for instrumentation. The default implementation is simply calling instrumentable_function/2.

Link to this section Types

Specs

erl_ast_atom_type() :: {:atom, any(), atom()}

The type for a node in the Erlang Abstract Form encoding an atom value

Specs

erl_ast_block() :: {:block, any(), [any()]}

Type type for a block of expression in Erlang Abstract Form

Specs

erl_ast_remote_call() ::
  {:call, any(), {:remote, any(), erl_ast_atom_type(), erl_ast_atom_type()},
   [any()]}

The type for a remote call in Erlang Abstract Form

Link to this section Functions

Specs

call_yield() :: erl_ast_remote_call()

Encodes a call to :erlang.yield() as Erlang Astract Form.

Link to this function

compile_module(mod, filename, code)

View Source

Compiles the abstract code of a module and loads it immediately into the VM.

Specs

encode_call({m :: module(), f :: atom(), a :: list()}) :: erl_ast_remote_call()

Enocdes a call given as tuple {m, f, a} as Erlang Abstract Form.

Specs

encode_call(m :: module(), f :: atom(), a :: list()) :: erl_ast_remote_call()

Encodes a call to m.f.(a) as Erlang Abstract Form.

Specs

encode_value(val :: any()) :: :erl_parse.abstract_expr()

Encodes a value as Erlang Astract Form.

Link to this function

get_forms_of_module(mod)

View Source

Retrieves the abstract code, i.e. the list of forms, of the given module as found in the code server.

Link to this function

instrument_app(app, instrumenter)

View Source

Specs

instrument_app(app :: atom(), instrumenter :: module()) :: :ok

Instruments all modules of an entire OTP application.

Link to this function

instrument_module(mod, instrumenter)

View Source

Specs

instrument_module(mod :: module(), instrumenter :: module()) :: :ok

Takes the object code of the module, instruments it and update the module in the code server with instrumented byte code.

Link to this function

instrumentable_function(mod, fun)

View Source

Specs

instrumentable_function(
  {:atom, any(), mod :: module()},
  {:atom, any(), fun :: atom()}
) :: boolean()

Checks if the given function is a candidate for instrumentation, i.e. does something interesting with respect to concurrency. Examples are process handling, handling of shared state or sending and receiving messages.

Link to this function

is_instrumented?(module_form)

View Source

Checks if the code is already instrumented. If not, returns false otherwise returns true

Link to this function

prepend_call(to_be_wrapped_call, new_call)

View Source

Specs

prepend_call(
  to_be_wrapped_call :: erl_ast_remote_call(),
  new_call :: erl_ast_remote_call()
) :: erl_ast_block()

Prepends the call to to_be_wrapped_call by a call to new_call. The result of new_call is ignored.

All arugments and return values are in Erlang Astract Form.

Link to this section Callbacks

Link to this callback

handle_function_call(call)

View Source

Specs

handle_function_call(call :: erl_ast_remote_call()) ::
  :erl_parse.abstract_expr()

Handle the instrumentation of a (remote) function call. Must return a valid expression in Erlang Abstract Form.

Link to this callback

is_instrumentable_function(mod, fun)

View Source

Specs

is_instrumentable_function(
  mod :: erl_ast_atom_type(),
  fun :: erl_ast_atom_type()
) :: boolean()

A callback to decide if the function mod:fun with any arity is a candidate for instrumentation. The default implementation is simply calling instrumentable_function/2.