qcheck
QuickCheck-inspired property-based testing with integrated shrinking
Overview
Rather than specifying test cases manually, you describe the invariants that values of a given type must satisfy (“properties”). Then, generators generate lots of values (test cases) on which the properties are checked. Finally, if a value is found for which a given property does not hold, that value is “shrunk” in order to find an nice, informative counter-example that is presented to you.
This module has functions for running and configuring property tests as well as generating random values (with shrinking) to drive those tests.
For usage examples, see the project README.
Running tests
Configuring test runs
Generators
- The Generator type
Here is a list of generator functions grouped by category.
Combinators
- return
- map
- bind
- apply
- map2
- map3
- map4
- map5
- map6
- tuple2
- tuple3
- tuple4
- tuple5
- tuple6
- from_generators
- from_weighted_generators
Ints
Floats
Characters
- char
- char_uniform_inclusive
- char_uppercase
- char_lowercase
- char_digit
- char_print_uniform
- char_uniform
- char_alpha
- char_alpha_numeric
- char_from_list
- char_whitespace
- char_print
Strings
- string
- string_from
- string_non_empty
- string_with_length
- string_with_length_from
- string_non_empty_from
- string_generic
Lists
Dicts
Sets
Other
Trees
There are functions for dealing with the Tree type directly, but they are low-level and you should not need to use them much.
- The Tree type
- make_primitive_tree
- return_tree
- map_tree
- map2_tree
- bind_tree
- apply_tree
- sequence_list
- option_tree
- tree_to_string
- tree_to_string_
Shrinking
There are some public functions for dealing with shrinks and shrinking. Similar to the Tree functions, you often won’t need to use these directly.
- shrink_atomic
- shrink_int_towards
- shrink_int_towards_zero
- shrink_float_towards
- shrink_float_towards_zero
Notes
- If something is marked as being “unspecified”, do not depend on it, as it
may change at any time without a major version bump. This mainly applies
to the various
*_to_string
functions. TestError
,TestErrorMessage
, and association functions will likely become private as they are mainly internal machinery for displaying errors.failwith
,try
, andrescue
will also likely become private as they deal with internal property test running machinery.
Types
Configuration for the property-based testing.
test_count
: The number of tests to run for each property.max_retries
: The number of times to retry the tested property while shrinking.random_seed
: The seed for the random generator.
pub type Config {
Config(test_count: Int, max_retries: Int, random_seed: Seed)
}
Constructors
-
Config(test_count: Int, max_retries: Int, random_seed: Seed)
Generator(a)
is a random generator for values of type a
.
Note: Because it exposes the prng Seed
type, it is likely that this type
will become opaque in the future.
pub type Generator(a) {
Generator(fn(Seed) -> #(Tree(a), Seed))
}
Constructors
-
Generator(fn(Seed) -> #(Tree(a), Seed))
pub opaque type TestErrorMessage
Functions
pub fn apply(
f f: Generator(fn(a) -> b),
generator x: Generator(a),
) -> Generator(b)
apply(f, x)
applies a function generator, f
, and an argument generator,
x
, into a result generator.
pub fn apply_tree(f: Tree(fn(a) -> b), x: Tree(a)) -> Tree(b)
pub fn bind(
generator generator: Generator(a),
f f: fn(a) -> Generator(b),
) -> Generator(b)
bind(generator, f)
generates a value of type a
with generator
, then
passes that value to f
, which uses it to generate values of type b
.
pub fn char() -> Generator(String)
char()
generates characters with a bias towards printable ASCII
characters, while still hitting some edge cases.
pub fn char_alpha() -> Generator(String)
char_alpha()
generates alphabetic (ASCII) characters.
pub fn char_alpha_numeric() -> Generator(String)
char_alpha_numeric()
generates alphanumeric (ASCII) characters.
pub fn char_digit() -> Generator(String)
char_digit()
generates digits from 0
to 9
, inclusive.
pub fn char_from_list(chars: List(String)) -> Generator(String)
char_from_list(chars)
generates characters from the given list of
characters.
pub fn char_lowercase() -> Generator(String)
char_lowercase()
generates lowercase (ASCII) letters.
pub fn char_print() -> Generator(String)
char_print()
generates printable ASCII characters, with a bias towards
alphanumeric characters.
pub fn char_print_uniform() -> Generator(String)
char_print_uniform()
generates printable ASCII characters.
pub fn char_uniform() -> Generator(String)
char_uniform()
generates characters uniformly distributed across the
default range.
pub fn char_uniform_inclusive(
low low: Int,
high high: Int,
) -> Generator(String)
char_uniform_inclusive(low, high)
generates “characters” uniformly
distributed between low
and high
, inclusive. Here, “characters” are
strings of a single codepoint.
Note: this function is slightly weird in that it takes the integer
representation of the range of codepoints, not the strings themselves.
This behavior will likely change.
These char_*
functions are mainly used for setting up the string
generators.
Shrinks towards a
when possible, but won’t go outside of the range.
pub fn char_uppercase() -> Generator(String)
char_uppercase()
generates uppercase (ASCII) letters.
pub fn char_whitespace() -> Generator(String)
char_whitespace()
generates whitespace (ASCII) characters.
pub fn default_config() -> Config
default()
returns the default configuration for the property-based testing.
pub fn dict_generic(
key_generator key_generator: Generator(a),
value_generator value_generator: Generator(b),
max_length max_length: Int,
) -> Generator(Dict(a, b))
dict_generic(key_generator, value_generator, max_len)
generates dictionaries with keys
from key_generator
and values from value_generator
with lengths up to max_len
.
Shrinks on size then on elements.
pub fn failwith(
original_value original_value: a,
shrunk_value shrunk_value: a,
shrink_steps shrink_steps: Int,
error_msg error_msg: String,
) -> b
pub fn float() -> Generator(Float)
float()
generates floats with a bias towards smaller values and shrinks
towards 0.0
.
pub fn float_uniform_inclusive(
low: Float,
high: Float,
) -> Generator(Float)
pub fn from_generators(
generators: List(Generator(a)),
) -> Generator(a)
from_generators(generators)
chooses a generator from a list of generators
weighted uniformly, then chooses a value from that generator.
pub fn from_weighted_generators(
generators: List(#(Float, Generator(a))),
) -> Generator(a)
from_generators(generators)
chooses a generator from a list of generators
weighted by the given weigths, then chooses a value from that generator.
pub fn generate_tree(
generator: Generator(a),
seed: Seed,
) -> #(Tree(a), Seed)
generate(gen, seed)
generates a value of type a
and its shrinks using the generator gen
.
You should not use this function directly. It is for internal use only.
pub fn given(
generator generator: Generator(a),
property property: fn(a) -> Bool,
) -> Nil
A specialized version of run
that uses the default configuration.
pub fn given_result(
generator generator: Generator(a),
property property: fn(a) -> Result(b, c),
) -> Nil
A specialized version of run_result
that uses the default configuration.
pub fn int_uniform() -> Generator(Int)
int_uniform()
generates uniformly distributed integers across a large
range and shrinks towards 0
.
Note: this generator does not hit interesting or corner cases very often.
pub fn int_uniform_inclusive(
low low: Int,
high high: Int,
) -> Generator(Int)
int_uniform_inclusive(low, high)
generates integers uniformly distributed
between low
and high
, inclusive.
Shrinks towards 0
, but won’t shrink outside of the range [low, high]
.
pub fn list_generic(
element_generator: Generator(a),
min_length min_len: Int,
max_length max_len: Int,
) -> Generator(List(a))
list_generic(element_generator, min_len, max_len)
generates lists of
elements from element_generator
with lengths between min_len
and
max_len
, inclusive.
Shrinks first on the number of elements, then on the elements themselves.
pub fn make_primitive_tree(
root x: a,
shrink shrink: fn(a) -> Iterator(a),
) -> Tree(a)
pub fn map(
generator generator: Generator(a),
f f: fn(a) -> b,
) -> Generator(b)
map(generator, f)
transforms the generator generator
by applying f
to
each generated value. Shrinks as generator
shrinks, but with f
applied.
pub fn map2(
f f: fn(a, b) -> c,
g1 g1: Generator(a),
g2 g2: Generator(b),
) -> Generator(c)
map2(f, g1, g2)
transforms two generators, g1
and g2
, by applying f
to each pair of generated values.
pub fn map3(
f f: fn(a, b, c) -> d,
g1 g1: Generator(a),
g2 g2: Generator(b),
g3 g3: Generator(c),
) -> Generator(d)
map3(f, g1, g2, g3)
transforms three generators, g1
, g2
, and g3
, by
applying f
to each triple of generated values.
pub fn map4(
f f: fn(a, b, c, d) -> e,
g1 g1: Generator(a),
g2 g2: Generator(b),
g3 g3: Generator(c),
g4 g4: Generator(d),
) -> Generator(e)
map4(f, g1, g2, g3, g4)
transforms four generators, g1
, g2
, g3
, and
g4
, by applying f
to each quadruple of generated values.
pub fn map5(
f f: fn(a, b, c, d, e) -> f,
g1 g1: Generator(a),
g2 g2: Generator(b),
g3 g3: Generator(c),
g4 g4: Generator(d),
g5 g5: Generator(e),
) -> Generator(f)
map5(f, g1, g2, g3, g4, g5)
transforms five generators, g1
, g2
, g3
,
g4
, and g5
, by applying f
to each quintuple of generated values.
pub fn map6(
f f: fn(a, b, c, d, e, f) -> g,
g1 g1: Generator(a),
g2 g2: Generator(b),
g3 g3: Generator(c),
g4 g4: Generator(d),
g5 g5: Generator(e),
g6 g6: Generator(f),
) -> Generator(g)
map6(f, g1, g2, g3, g4, g5, g6)
transforms six generators, g1
, g2
, g3
,
g4
, g5
, and g6
, by applying f
to each sextuple of generated values.
pub fn new_test_error(
original_value orig: a,
shrunk_value shrunk: a,
shrink_steps steps: Int,
error_msg error_msg: String,
) -> TestError(a)
pub fn nil() -> Generator(Nil)
nil()
is the Nil
generator. It always returns Nil
and does not shrink.
pub fn option(generator: Generator(a)) -> Generator(Option(a))
option(gen)
is an Option
generator that uses gen
to generate Some
values. Shrinks towards None
then towards shrinks of gen
.
pub fn option_tree(tree: Tree(a)) -> Tree(Option(a))
pub fn rescue(thunk: fn() -> a) -> Result(a, TestErrorMessage)
This function should only be called to rescue a function that my call
failwith
at some point to raise an exception. It will likely
raise otherwise.
pub fn rescue_error(f: fn() -> a) -> Result(a, String)
pub fn return(a: a) -> Generator(a)
return(a)
creates a generator that always returns a
and does not shrink.
pub fn return_tree(x: a) -> Tree(a)
pub fn run(
config config: Config,
generator generator: Generator(a),
property property: fn(a) -> Bool,
) -> Nil
run(config, generator, property)
runs the property
function against some
test cases generated by the generator
function according to the specified
config
The run
function returns Nil
if the property holds (i.e., return
True
for all test cases), or panics if the property does not
hold for some test case a
(i.e., returns False
or panic
s).
pub fn run_result(
config config: Config,
generator generator: Generator(a),
property property: fn(a) -> Result(b, c),
) -> Nil
run_result(config, generator, property)
is like run
but the property
function returns a Result
instead of a Bool
.
pub fn sequence_list(l: List(Tree(a))) -> Tree(List(a))
sequence_list(list_of_trees)
sequsences a list of trees into a tree of lists.
pub fn set_generic(
element_generator: Generator(a),
max_length max_len: Int,
) -> Generator(Set(a))
set_generic(element_generator, max_len)
generates sets of elements from
element_generator
.
Shrinks first on the number of elements, then on the elements themselves.
pub fn shrink_atomic() -> fn(a) -> Iterator(a)
The atomic
shrinker treats types as atomic, and never attempts to produce
smaller values.
pub fn shrink_float_towards(
destination destination: Float,
) -> fn(Float) -> Iterator(Float)
pub fn shrink_float_towards_zero() -> fn(Float) ->
Iterator(Float)
pub fn shrink_int_towards(
destination destination: Int,
) -> fn(Int) -> Iterator(Int)
pub fn shrink_int_towards_zero() -> fn(Int) -> Iterator(Int)
pub fn small_positive_or_zero_int() -> Generator(Int)
small_positive_or_zero_int()
generates small integers well suited for
modeling the sizes of sized elements like lists or strings.
Smaller numbers are more likely than larger numbers.
Shrinks towards 0
.
pub fn small_strictly_positive_int() -> Generator(Int)
small_strictly_positive_int()
generates small integers strictly greater
than 0
.
pub fn string() -> Generator(String)
`string() generates strings with the default character generator and the default length generator.
pub fn string_from(
char_generator: Generator(String),
) -> Generator(String)
string_from(char_generator)
generates strings from the given character generator
using the default length generator.
pub fn string_generic(
char_generator: Generator(String),
length_generator: Generator(Int),
) -> Generator(String)
string_generic(char_generator, length_generator)
generates strings with
characters from char_generator
and lengths from length_generator
.
pub fn string_non_empty() -> Generator(String)
string_non_empty()
generates non-empty strings with the default character
generator and the default length generator.
pub fn string_non_empty_from(
char_generator: Generator(String),
) -> Generator(String)
string_non_empty_from(char_generator)
generates non-empty strings from the given
character generator using the default length generator.
pub fn string_with_length(length: Int) -> Generator(String)
string_with_length(length)
generates strings of the given length
with the
default character generator.
pub fn string_with_length_from(
generator: Generator(String),
length: Int,
) -> Generator(String)
string_with_length_from(gen, length)
generates strings of the given
length
from the given generator.
pub fn test_error_message_original_value(
msg: TestErrorMessage,
) -> String
pub fn test_error_message_shrink_steps(
msg: TestErrorMessage,
) -> String
pub fn test_error_message_shrunk_value(
msg: TestErrorMessage,
) -> String
pub fn tree_to_string(
tree: Tree(a),
a_to_string: fn(a) -> String,
) -> String
tree_to_string(tree, element_to_string)
converts a tree into an unspecified string representation.
element_to_string
: a function that converts individual elements of the tree to strings.
pub fn tree_to_string_(
tree: Tree(a),
a_to_string: fn(a) -> String,
max_depth max_depth: Int,
) -> String
Like tree_to_string
but with a configurable max_depth
.
pub fn tuple2(
g1: Generator(a),
g2: Generator(b),
) -> Generator(#(a, b))
tuple2(g1, g2)
generates a tuple of two values, one each from generators
g1
and g2
.
pub fn tuple3(
g1: Generator(a),
g2: Generator(b),
g3: Generator(c),
) -> Generator(#(a, b, c))
tuple3(g1, g2, g3)
generates a tuple of three values, one each from
generators g1
, g2
, and g3
.
pub fn tuple4(
g1: Generator(a),
g2: Generator(b),
g3: Generator(c),
g4: Generator(d),
) -> Generator(#(a, b, c, d))
tuple4(g1, g2, g3, g4)
generates a tuple of four values, one each from
generators g1
, g2
, g3
, and g4
.
pub fn tuple5(
g1: Generator(a),
g2: Generator(b),
g3: Generator(c),
g4: Generator(d),
g5: Generator(e),
) -> Generator(#(a, b, c, d, e))
tuple5(g1, g2, g3, g4, g5)
generates a tuple of five values, one each from
generators g1
, g2
, g3
, g4
, and g5
.
pub fn tuple6(
g1: Generator(a),
g2: Generator(b),
g3: Generator(c),
g4: Generator(d),
g5: Generator(e),
g6: Generator(f),
) -> Generator(#(a, b, c, d, e, f))
tuple6(g1, g2, g3, g4, g5, g6)
generates a tuple of six values, one each
from generators g1
, g2
, g3
, g4
, g5
, and g6
.
pub fn with_max_retries(
config: Config,
max_retries: Int,
) -> Config
with_max_retries()
returns a new configuration with the given max retries.
pub fn with_random_seed(
config: Config,
random_seed: Seed,
) -> Config
with_random_seed()
returns a new configuration with the given random seed.
pub fn with_test_count(config: Config, test_count: Int) -> Config
with_test_count()
returns a new configuration with the given test count.