PropCheck.Instrument behaviour (PropCheck v1.4.1) View Source
Provides functions and macros for instrument byte code with additional yields and other constructs to ease testing of concurrent programs and state machines.
Why is instrumentation important?
The Erlang scheduler is relatively predictable and stable with regard to pre-emptive
scheduling. This means that every run has more or the less the same amount of
virtual machine instructions before a switch to another process happens. These
process switches are required to reveal any concurrency bugs. A simple way to
provoke more process switches are calls to :erlang.yield()
which gives the scheduler
the possibility to switch early on to another process. It is not defined if
the scheduler reacts on this hint, but it often does and allows for more
unpredictable schedules revealing more concurrency bugs.
The usual advice is to sprinkle the code under test with manually added
calls to :erlang.yield()
, but this is a daunting task. Additionally, you
need to remove this additional code before production use.
The instrumentation
The functions in this module automate the instrumentation immediately before
running the tests. We instrument call to "interesting" functions of the Erlang
and Elixir ecosystem, e.g. calls to GenServer
or ets
tables. We do this by examining
the byte code, checking each function call, and if we found some interesting call target,
we add a call to :erlang.yield()
immediately before. This is what the
PropCheck.YieldInstrumenter
module provides. It implements the behaviour Instrument
,
which requires the implementation of two callbacks handle_function_call/1
and
is_instrumentable_function/2
. After instrumentation, the code reloading mechanism of
the Erlang VM enables the new code and the tests can run.
Typical usage
To ensure instrumentation before running the tests, you implement the setup_all
macro
of ExUnit
:
setup_all do
Instrument.instrument_module(Cache, YieldInstrumenter)
:ok # no update of a context
end
In this example, we instrument only a specific module. You can also instrument
all modules of an application by calling Instrument.instrument_app(:my_app_under_test, YieldInstrumenter)
.
Implementing your own instrumenter
For implementing your own instrumenter, you need to get acquainted with the Erlang
Abstract Form (EAF), which is the internal abstract syntax tree available to the Erlang VM at runtime.
This format is quite different from the Elixir AST, in particular it has not the regular form but
consists of many different structures. This requires a lot of cases to be handled for analyizing
the AST. Little helpers for encoding the instrumented code is provided by encode_call/1
and
encode_value/1
as well as by prepend_call/2
. For debugging and revealing the structure of
a specific EAF, you can use print_fun/1
.
Link to this section Summary
Types
The type for a node in the Erlang Abstract Form encoding an atom value
Type type for a block of expression in Erlang Abstract Form
The type for a remote call in Erlang Abstract Form
Functions
Encodes a call to :erlang.yield()
as Erlang Astract Form.
Compiles the abstract code of a module and loads it immediately into the VM.
Enocdes a call given as tuple {m, f, a}
as Erlang Abstract Form.
Encodes a call to m.f.(a)
as Erlang Abstract Form.
Encodes a value as Erlang Astract Form.
Retrieves the abstract code, i.e. the list of forms, of the given module as found in the code server.
Instruments all modules of an entire OTP application.
Takes the object code of the module, instruments it and update the module in the code server with instrumented byte code.
Checks if the given function is a candidate for instrumentation, i.e. does something interesting with respect to concurrency. Examples are process handling, handling of shared state or sending and receiving messages.
Checks if the code is already instrumented. If not, returns false
otherwise returns true
Prepends the call to to_be_wrapped_call
by a call to new_call
.
The result of new_call
is ignored.
Debugging aid for analyzing code generations. Prints the restructered Erlang code of function
fun
in module mod
. We use Erlang code here, because Elixir source code cannot generated from
the byte code format due to macros, which change the compilation process too heavily.
Callbacks
Handle the instrumentation of a (remote) function call. Must return a valid expression in Erlang Abstract Form.
A callback to decide if the function mod:fun
with any arity is a candidate
for instrumentation. The default implementation is simply calling
instrumentable_function/2
.
Link to this section Types
Specs
The type for a node in the Erlang Abstract Form encoding an atom value
Specs
Type type for a block of expression in Erlang Abstract Form
Specs
erl_ast_remote_call() :: {:call, any(), {:remote, any(), erl_ast_atom_type(), erl_ast_atom_type()}, [any()]}
The type for a remote call in Erlang Abstract Form
Link to this section Functions
Specs
call_yield() :: erl_ast_remote_call()
Encodes a call to :erlang.yield()
as Erlang Astract Form.
Compiles the abstract code of a module and loads it immediately into the VM.
Specs
encode_call({m :: module(), f :: atom(), a :: list()}) :: erl_ast_remote_call()
Enocdes a call given as tuple {m, f, a}
as Erlang Abstract Form.
Specs
encode_call(m :: module(), f :: atom(), a :: list()) :: erl_ast_remote_call()
Encodes a call to m.f.(a)
as Erlang Abstract Form.
Specs
encode_value(val :: any()) :: :erl_parse.abstract_expr()
Encodes a value as Erlang Astract Form.
Retrieves the abstract code, i.e. the list of forms, of the given module as found in the code server.
Specs
Instruments all modules of an entire OTP application.
Specs
Takes the object code of the module, instruments it and update the module in the code server with instrumented byte code.
Specs
instrumentable_function( {:atom, any(), mod :: module()}, {:atom, any(), fun :: atom()} ) :: boolean()
Checks if the given function is a candidate for instrumentation, i.e. does something interesting with respect to concurrency. Examples are process handling, handling of shared state or sending and receiving messages.
Checks if the code is already instrumented. If not, returns false
otherwise returns true
Specs
prepend_call( to_be_wrapped_call :: erl_ast_remote_call(), new_call :: erl_ast_remote_call() ) :: erl_ast_block()
Prepends the call to to_be_wrapped_call
by a call to new_call
.
The result of new_call
is ignored.
All arugments and return values are in Erlang Astract Form.
Specs
Debugging aid for analyzing code generations. Prints the restructered Erlang code of function
fun
in module mod
. We use Erlang code here, because Elixir source code cannot generated from
the byte code format due to macros, which change the compilation process too heavily.
Link to this section Callbacks
Specs
handle_function_call(call :: erl_ast_remote_call()) :: :erl_parse.abstract_expr()
Handle the instrumentation of a (remote) function call. Must return a valid expression in Erlang Abstract Form.
Specs
is_instrumentable_function( mod :: erl_ast_atom_type(), fun :: erl_ast_atom_type() ) :: boolean()
A callback to decide if the function mod:fun
with any arity is a candidate
for instrumentation. The default implementation is simply calling
instrumentable_function/2
.