PropertyDamage.SeedLibrary (PropertyDamage v0.1.0)

Manage a collection of interesting seeds for regression testing and sharing.

The Seed Library tracks seeds that have found bugs, allowing you to:

Run known-interesting seeds first before random exploration
Share discovered seeds across team members
Build a regression suite that catches known bug patterns
Track which seeds have been fixed vs still failing

Usage

# Add a seed when you find a bug
{:error, failure} = PropertyDamage.run(model: M, adapter: A)
SeedLibrary.add(failure, tags: [:currency, :capture])

# Run all library seeds first, then continue with random
PropertyDamage.run(model: M, adapter: A, seed_library: "seeds.json")

# Export for CI/sharing
SeedLibrary.export("seeds.json")

Seed Entry Structure

Each entry contains:

seed - The random seed value
model - Model module name (for filtering)
failure_type - What kind of failure it found
check_name - Which check failed (if applicable)
tags - User-provided categorization tags
description - Human-readable description
discovered_at - When the seed was added
last_run - When the seed was last tested
status - :failing, :fixed, :flaky
run_count - How many times this seed has been run
fail_count - How many times it has failed

Integration with PropertyDamage.run

When a seed library is provided, PropertyDamage.run will:

Run all :failing seeds from the library first
Update seed status based on results
Continue with random seed exploration

Summary

Types

seed_entry()

t()

Functions

add(library, failure, opts \\ [])

Add a seed from a failure report to the library.

add_seed(library, seed, opts \\ [])

Add a seed directly (without a failure report).

export(library, path)

Export library to a portable format (for sharing).

format(library)

Format library for display.

get_seeds(library, opts \\ [])

Get all seeds matching certain criteria.

import(library, path)

Import seeds from an exported file.

load(path \\ "property_damage_seeds.json")

Load a seed library from a JSON file.

new()

Create a new empty seed library.

record_run(library, seed, opts \\ [])

Update a seed's status after a test run.

remove(library, seed)

Remove a seed from the library.

save(library, path \\ "property_damage_seeds.json")

Save a seed library to a JSON file.

seed_values(library, opts \\ [])

Get just the seed values (for passing to PropertyDamage.run).

stats(library)

Get statistics about the library.

Types

seed_entry()

@type seed_entry() :: %{
  seed: integer(),
  model: String.t(),
  failure_type: atom(),
  check_name: atom() | nil,
  tags: [atom()],
  description: String.t() | nil,
  discovered_at: String.t(),
  last_run: String.t() | nil,
  status: :failing | :fixed | :flaky | :unknown,
  run_count: non_neg_integer(),
  fail_count: non_neg_integer()
}

t()

@type t() :: %{version: integer(), entries: [seed_entry()]}

Functions

add(library, failure, opts \\ [])

@spec add(t(), PropertyDamage.FailureReport.t(), keyword()) ::
  {:ok, t()} | {:error, term()}

Add a seed from a failure report to the library.

Options

:tags - List of categorization tags (e.g., [:currency, :race_condition])
:description - Human-readable description of what this seed tests

Example

{:error, failure} = PropertyDamage.run(model: M, adapter: A)
{:ok, library} = SeedLibrary.add(library, failure, tags: [:currency])

add_seed(library, seed, opts \\ [])

@spec add_seed(t(), integer(), keyword()) :: {:ok, t()} | {:error, term()}

Add a seed directly (without a failure report).

Useful for importing seeds from external sources or manual entry.

Example

{:ok, library} = SeedLibrary.add_seed(library, 512902757,
  model: "ToyBankTest.Model",
  tags: [:currency_mismatch],
  description: "Captures with mismatched currencies"
)

export(library, path)

@spec export(t(), Path.t()) :: :ok | {:error, term()}

Export library to a portable format (for sharing).

Unlike save/2, this includes only essential fields and uses strings for module names to avoid atom table issues across systems.

format(library)

@spec format(t()) :: String.t()

Format library for display.

get_seeds(library, opts \\ [])

@spec get_seeds(
  t(),
  keyword()
) :: [seed_entry()]

Get all seeds matching certain criteria.

Options

:status - Filter by status (:failing, :fixed, :flaky)
:tags - Filter by tags (entries must have ALL specified tags)
:model - Filter by model name (string match)

Example

# Get all failing seeds
failing = SeedLibrary.get_seeds(library, status: :failing)

# Get currency-related seeds
currency_seeds = SeedLibrary.get_seeds(library, tags: [:currency])

import(library, path)

@spec import(t(), Path.t()) :: {:ok, t(), non_neg_integer()} | {:error, term()}

Import seeds from an exported file.

Merges with existing library, skipping duplicates.

load(path \\ "property_damage_seeds.json")

@spec load(Path.t()) :: {:ok, t()} | {:error, term()}

Load a seed library from a JSON file.

new()

@spec new() :: t()

Create a new empty seed library.

record_run(library, seed, opts \\ [])

@spec record_run(t(), integer(), keyword()) :: t()

Update a seed's status after a test run.

Example

# After running a seed
library = SeedLibrary.record_run(library, seed, failed: true)

remove(library, seed)

@spec remove(t(), integer()) :: {:ok, t()} | {:error, :not_found}

Remove a seed from the library.

save(library, path \\ "property_damage_seeds.json")

@spec save(t(), Path.t()) :: :ok | {:error, term()}

Save a seed library to a JSON file.

seed_values(library, opts \\ [])

@spec seed_values(
  t(),
  keyword()
) :: [integer()]

Get just the seed values (for passing to PropertyDamage.run).

Example

seeds = SeedLibrary.seed_values(library, status: :failing)
# => [512902757, 123456789, ...]

stats(library)

@spec stats(t()) :: map()

Get statistics about the library.