AB (AB v0.2.1)

View Source

A macro-based property testing generator that analyzes function typespecs to automatically generate appropriate test data and output validators.

Usage

defmodule MyModuleTest do
  use ExUnit.Case
  use ExUnitProperties
  import AB

  # Generate property test for a function with typespec
  property_test MyModule, :my_function
end

Summary

Functions

Macro that benchmarks two functions with identical typespecs using Benchee.

Macro that compares two functions with identical typespecs using the same generated test data.

Creates input generators from type specifications. Optionally accepts a module to resolve user-defined type aliases.

Creates invalid input generators from type specifications.

Creates output validators from type specifications.

Creates a struct generator from @type definition.

Formats a typespec AST into a human-readable string using Elixir's standard functions where possible.

Extracts the typespec for a given function.

Parses a spec into input and output types.

Macro that generates property testing utilities for a function based on its typespec.

Macro that generates robustness tests for a function based on its typespec.

Runs a property test and returns detailed failure information including actual values.

Analyzes a failed test using runtime values and suggests a corrected typespec.

Compares two type specifications for equivalence.

Macro that generates property tests for all public functions in a module.

Macro that validates struct type definitions against function typespecs.

Validates type consistency between @type definitions and @spec definitions.

Functions

benchmark_test(arg1, arg2, opts \\ [])

(macro)
@spec benchmark_test({module(), atom()}, {module(), atom()}, keyword()) :: Macro.t()

Macro that benchmarks two functions with identical typespecs using Benchee.

Takes two module/function pairs, verifies their typespecs are identical, and if so, runs a benchmark comparison using generated test data.

Example

# Benchmark two sorting implementations
benchmark_test {AB, :a}, {AB, :b}

# With custom options
benchmark_test {AB, :a}, {AB, :b}, time: 5, memory_time: 2

This will generate a benchmark test that compares performance of both functions.

compare_test(arg1, arg2, opts \\ [])

(macro)
@spec compare_test({module(), atom()}, {module(), atom()}, keyword()) :: Macro.t()

Macro that compares two functions with identical typespecs using the same generated test data.

Takes two module/function pairs, verifies their typespecs are identical, and if so, runs both functions on the same generated inputs to ensure they produce identical outputs.

Example

# Compare two sorting implementations
compare_test {AB, :a}, {AB, :b}

# With verbose output
compare_test {AB, :a}, {AB, :b}, verbose: true

This will generate a property test that validates both functions behave identically.

create_input_generator_runtime(input_types, module \\ nil)

@spec create_input_generator_runtime([any()], module() | nil) :: Enumerable.t()

Creates input generators from type specifications. Optionally accepts a module to resolve user-defined type aliases.

create_invalid_input_generator_runtime(input_types, module \\ nil)

@spec create_invalid_input_generator_runtime([any()], module() | nil) ::
  Enumerable.t()

Creates invalid input generators from type specifications.

create_output_validator_runtime(output_type)

@spec create_output_validator_runtime(any()) :: (any() -> boolean())

Creates output validators from type specifications.

create_struct_from_type_definition(module)

Creates a struct generator from @type definition.

format_type_for_display(type_ast)

@spec format_type_for_display(any()) :: String.t()

Formats a typespec AST into a human-readable string using Elixir's standard functions where possible.

get_function_spec(module, function_name)

Extracts the typespec for a given function.

parse_spec(spec)

Parses a spec into input and output types.

property_test(module, function_name, opts \\ [])

(macro)
@spec property_test(module(), atom(), keyword()) :: Macro.t()

Macro that generates property testing utilities for a function based on its typespec.

Takes a module and function name, analyzes the typespec, and generates:

  1. Input generators based on the function's parameter types
  2. Output validators based on the function's return type
  3. A complete property test

Example

# For a function with spec: @spec sort_list([integer()]) :: [integer()]
property_test MyModule, :sort_list

# With verbose output
property_test MyModule, :sort_list, verbose: true

This will generate a property test that validates the function behavior.

robust_test(module, function_name, opts \\ [])

(macro)
@spec robust_test(module(), atom(), keyword()) :: Macro.t()

Macro that generates robustness tests for a function based on its typespec.

Takes a module and function name, analyzes the typespec, and generates:

  1. Invalid input generators that create data NOT matching the function's parameter types
  2. Tests that verify functions either raise errors or return invalid output when given invalid input
  3. Ensures functions fail gracefully rather than silently accepting wrong input types

Example

# For a function with spec: @spec process(integer()) :: string()
robust_test MyModule, :process

# With verbose output
robust_test MyModule, :process, verbose: true

This will generate tests that verify the function properly rejects invalid input.

run_property_test_with_details(module, function_name, opts)

@spec run_property_test_with_details(module(), atom(), keyword()) ::
  {:ok, integer()} | {:error, map()}

Runs a property test and returns detailed failure information including actual values.

This is used by the mix task to capture failure details for typespec suggestions. Returns {:ok, count} on success or {:error, failure_details} on failure.

suggest_typespec_correction(module, function_name, actual_inputs, actual_result, mismatch_type)

@spec suggest_typespec_correction(module(), atom(), list(), any(), atom()) ::
  %{
    old_spec: String.t(),
    new_spec: String.t(),
    old_spec_ast: {[any()], any()},
    new_spec_ast: {[any()], any()},
    reason: String.t()
  }
  | {:error, String.t()}

Analyzes a failed test using runtime values and suggests a corrected typespec.

Takes actual runtime values (inputs and result) along with the current typespec, infers the correct types from the values, and suggests a corrected typespec.

Parameters

  • module: The module containing the function
  • function_name: The name of the function that failed
  • actual_inputs: List of actual input values used in the test
  • actual_result: The actual result value returned by the function
  • mismatch_type: Either :return or :argument to indicate which type mismatched

Returns

A map containing:

  • :old_spec - The current typespec as a string
  • :new_spec - The suggested corrected typespec as a string
  • :old_spec_ast - The current typespec as AST {input_types, output_type}
  • :new_spec_ast - The suggested typespec as AST {input_types, output_type}
  • :reason - Why the typespec needs correction

Example

iex> AB.suggest_typespec_correction(NumberFunctions, :double, [1.0], 2.0, :return)
%{
  old_spec: "@spec double(number()) :: binary()",
  new_spec: "@spec double(number()) :: float()",
  old_spec_ast: {[{:type, 0, :number, []}], {:type, 0, :binary, []}},
  new_spec_ast: {[{:type, 0, :number, []}], {:type, 0, :float, []}},
  reason: "Return type mismatch: function returns float() but typespec declares binary()"
}

types_equivalent?(type1, type2)

Compares two type specifications for equivalence.

validate_module(module, opts \\ [])

(macro)
@spec validate_module(
  module(),
  keyword()
) :: Macro.t()

Macro that generates property tests for all public functions in a module.

Takes a module and automatically generates property tests for all exported functions that have typespecs defined. This is useful for validating an entire module's API.

Example

# Validate all public functions in a module
validate_module MyModule

# With verbose output
validate_module MyModule, verbose: true

This will generate property tests for each public function with a typespec.

validate_struct_consistency(module)

(macro)
@spec validate_struct_consistency(module()) :: Macro.t()

Macro that validates struct type definitions against function typespecs.

This catches inconsistencies where @type definitions don't match @spec definitions.

Example

# This will fail if @type t :: %AB{a: atom()} but @spec expects integer()
validate_struct_consistency AB

validate_type_consistency(module, function_name)

@spec validate_type_consistency(module(), atom()) :: :ok

Validates type consistency between @type definitions and @spec definitions.