PropertyDamage.Analysis (PropertyDamage v0.1.0)

View Source

Advanced failure analysis tools for understanding and debugging test failures.

This module provides tools that go beyond basic shrinking to help users understand why a failure occurs and what specifically triggers it.

Features

  • Causal Explanation: Understand why each command in the shrunk sequence is needed
  • Trigger Isolation: Find the minimal change that eliminates the failure
  • Test Generation: Generate reproducible test code from failures

Usage

{:error, failure} = PropertyDamage.run(model: M, adapter: A)

# Understand why each command is needed
PropertyDamage.Analysis.explain(failure)

# Find what triggers the bug
PropertyDamage.Analysis.isolate_trigger(failure)

# Generate a test case
PropertyDamage.Analysis.generate_test(failure, format: :exunit)

Summary

Functions

explain(report)

Explain why each command in the shrunk sequence is needed for the failure.

format_explanation(explanation)

Format an explanation as a human-readable string.

generate_test(report, opts \\ [])

Generate a reproducible test case from a failure.

isolate_trigger(report, opts \\ [])

Find the minimal change that eliminates the failure.

Functions

explain(report)

@spec explain(PropertyDamage.FailureReport.t()) :: map()

Explain why each command in the shrunk sequence is needed for the failure.

Returns a structured explanation showing:

  • The dependency graph between commands
  • Which command triggers the failure
  • Why each other command must be present

Example

PropertyDamage.Analysis.explain(failure)
# Returns:
# %{
#   commands: [
#     %{index: 0, command: "CreateAccount", role: :dependency,
#       reason: "Creates account used by command 2"},
#     %{index: 1, command: "CreateAuthorization", role: :dependency,
#       reason: "Creates pending hold affecting available balance"},
#     %{index: 2, command: "CreditAccount", role: :trigger,
#       reason: "Causes currency_consistency check to fail"}
#   ],
#   failure: %{type: :check_failed, check: :currency_consistency, ...}
# }

format_explanation(explanation)

@spec format_explanation(map()) :: String.t()

Format an explanation as a human-readable string.

generate_test(report, opts \\ [])

@spec generate_test(
  PropertyDamage.FailureReport.t(),
  keyword()
) :: String.t()

Generate a reproducible test case from a failure.

Creates runnable code that reproduces the failure, suitable for adding to a test suite or sharing with colleagues.

Options

  • :format - Output format (:exunit, :script, :markdown). Default: :exunit
  • :module_name - Module name for ExUnit tests. Default: "ReproductionTest"
  • :include_setup - Include model/adapter setup code. Default: true

Example

PropertyDamage.Analysis.generate_test(failure, format: :exunit)
# Generates ExUnit test code

isolate_trigger(report, opts \\ [])

@spec isolate_trigger(
  PropertyDamage.FailureReport.t(),
  keyword()
) :: {:ok, map()} | {:error, term()}

Find the minimal change that eliminates the failure.

This performs differential analysis to identify exactly what triggers the bug. It tries variations of the failing command to find the smallest change that makes the failure disappear.

Returns

A map containing:

  • :trigger_command - The command that triggers the failure
  • :changes - List of changes that eliminate the failure
  • :likely_cause - Inferred cause based on the changes

Example

PropertyDamage.Analysis.isolate_trigger(failure)
# %{
#   trigger_command: %CreditAccount{...},
#   changes: [
#     %{field: :currency, original: "EUR", fixed: "USD",
#       description: "Changing currency from EUR to USD eliminates failure"}
#   ],
#   likely_cause: "Currency mismatch: account currency (USD) differs from operation currency (EUR)"
# }