Sets up an ExUnit.Case module for property-based testing.

Runs tests for a property.

This macro provides ad hoc syntax to write properties. Let's see a quick example to get a feel of how it works:

check all int1 <- integer(),
          int2 <- integer(),
          int1 > 0 and int2 > 0,
          sum = int1 + int2 do
  assert sum > int1
  assert sum > int2
end

Everything between check all and do is referred to as clauses. Clauses are used to specify the values to generate in order to test the properties. The actual tests that the properties hold live in the do block.

Clauses work exactly like they work in the gen/1 macro.

The body passed in the do block is where you test that the property holds for the generated values. The body is just like the body of a test: use ExUnit.Assertions.assert/2 (and friends) to assert whatever you want.

Options

• :initial_size - (non-negative integer) the initial generation size used to start generating values. The generation size is then incremented by 1 on each iteration. See the "Generation size" section of the StreamData documentation for more information on generation size. Defaults to 1.

• :max_runs - (non-negative integer) the total number of generations to run. Defaults to 100.

• :max_run_time - (non-negative integer) the total number of time (in milliseconds) to run a given check for. This is not used by default, so unless a value is given then the length of the test will be determined by :max_runs. If both :max_runs and :max_run_time are given, then the check will finish at whichever comes first, :max_runs or :max_run_time.

• :max_shrinking_steps - (non-negative integer) the maximum numbers of shrinking steps to perform in case a failing case is found. Defaults to 100.

• :max_generation_size - (non-negative integer) the maximum generation size to reach. Note that the size is increased by one on each run. By default, the generation size is unbounded.

• :initial_seed - (integer) the initial seed used to drive the random generation. When check all is run with the same initial seed more than once, then every time the terms generated by the generators will be the same as all other runs. This is useful when you want to deterministically reproduce a result. However, it's usually better to leave :initial_seed to its default value, which is taken from ExUnit's seed: this way, the random generation will follow options like --seed used in ExUnit to deterministically reproduce tests.

It is also possible to set the values for :initial_size, :max_runs, :max_run_time, and :max_shrinking_steps through your project's config files. This is especially helpful in combination with :max_runs when you want to run more iterations on your continuous integration platform, but keep your local tests fast:

# config/test.exs
import Config

config :stream_data,
  max_runs: if System.get_env("CI"), do: 1_000, else: 50

Examples

Check that all values generated by the StreamData.integer/0 generator are integers:

check all int <- integer() do
  assert is_integer(int)
end

Check that String.starts_with?/2 and String.ends_with?/2 always hold for concatenated strings:

check all start <- binary(),
          finish <- binary(),
          concat = start <> finish do
  assert String.starts_with?(concat, start)
  assert String.ends_with?(concat, finish)
end

Check that Kernel.in/2 returns true when checking if an element taken out of a list is in that same list (changing the number of runs):

check all list <- list_of(integer()),
          member <- member_of(list),
          max_runs: 50 do
  assert member in list
end

Using check all in doctests

check all can be used in doctests. Make sure that the module where you call doctest(MyModule) calls use ExUnitProperties. Then, you can call check all in your doctests:

@doc """
Tells if a term is an integer.

    iex> check all i <- integer() do
    ...>   assert int?(i)
    ...> end
    :ok

"""
def int?(i), do: is_integer(i)

check all always returns :ok, so you can use that as the return value of the whole expression.

Syntactic sugar to create generators.

This macro provides ad-hoc syntax to write complex generators. Let's see a quick example to get a feel of how it works. Say we have a User struct:

defmodule User do
  defstruct [:name, :email]
end

We can create a generator of users like this:

email_generator = map({binary(), binary()}, fn {left, right} -> left <> "@" <> right end)

user_generator =
  gen all name <- binary(),
          email <- email_generator do
    %User{name: name, email: email}
  end

Everything between gen all and do is referred to as clauses. You can write clauses to specify the values to generate. You can then use those values in the do body. The newly-created generator will generate values that are the return value of the do body using the generated values in the clauses.

Clauses

As seen in the example above, clauses can be of the following types:

• value generation - they have the form pattern <- generator where generator must be a generator. These clauses take a value out of generator on each run and match it against pattern. Variables bound in pattern can be then used throughout subsequent clauses and in the do body. If pattern doesn't match a generated value, it's treated like a filter (see the "filtering" clauses described below).

• filtering and binding - they have the form expression. If a filtering clause returns a truthy value, then the set of generated values that appear before the filtering clause is considered valid and generation continues. If the filtering clause returns a falsey value, then the current value is considered invalid and a new value is generated. Note that filtering clauses should not filter out too many times; in case they do, a StreamData.FilterTooNarrowError error is raised (same as StreamData.filter/3). Filtering clauses can be used also to assign variables: for example, a = :foo is a valid clause.

The behaviour of the clauses above is similar to the behaviour of clauses in Kernel.SpecialForms.for/1.

Body

The return value of the body passed in the do block is what is ultimately generated by the generator return by this macro.

Shrinking

See the module documentation for more information on shrinking. Clauses affect shrinking in the following way:

• filtering clauses affect shrinking like StreamData.filter/3
• value generation clauses affect shrinking similarly to StreamData.bind/2

@spec pick(StreamData.t(a)) :: a when a: term()

Picks a random element generated by the StreamData generator data.

This function uses the current ExUnit seed to generate a random term from data. The generation size (see Generation size) is chosen at random between in 1..100. If you want finer control over the generation size, you can use functions like StreamData.resize/2 to resize data or StreamData.scale/2 to scale the generation size.

Examples

ExUnitProperties.pick(StreamData.integer())
#=> -21

Defines a not-implemented property test with a string.

Provides a convenient macro that allows a property test to be defined with a string, but not yet implemented. The resulting property test will always fail and print a "Not implemented" error message. The resulting test case is also tagged with :not_implemented.

This behavior is similar to ExUnit.Case.test/1.

Examples

property "this will be a property test in the future"

Defines a property and imports property-testing facilities in the body.

This macro is similar to ExUnit.Case.test/3, except that it denotes a property. In the given body, all the functions exposed by StreamData are imported, as well as check/2.

When defining a test whose body only consists of one or more check/2 calls, it's advised to use property/3 so as to clearly denote and scope properties. Doing so will also improve reporting.

Examples

use ExUnitProperties

property "reversing a list doesn't change its length" do
  check all list <- list_of(integer()) do
    assert length(list) == length(:lists.reverse(list))
  end
end