mix test.baseline (sc v1.0.2)

Update the baseline of passing tests.

This task helps maintain test/passing_tests.json by analyzing current test status and providing guidance on adding new passing tests to the regression suite.

Usage

# Analyze current test status and get guidance
mix test.baseline

# Add specific test files to the baseline (verifies they pass first)
mix test.baseline add test/scion_tests/basic/basic3_test.exs
mix test.baseline add test/scion_tests/path/test1.exs test/scion_tests/path/test2.exs

The analysis mode will show discrepancies between current passing tests and the baseline, providing suggestions for manual review and update.

Summary

Functions

extract_passing_test_files_from_output(output, test_type)

Finds test files that are currently passing by testing each file individually.

extract_test_summary(output)

Extracts a test summary from ExUnit output.

load_passing_tests()

Loads the current passing tests configuration from test/passing_tests.json.

Functions

extract_passing_test_files_from_output(output, test_type)

@spec extract_passing_test_files_from_output(String.t(), String.t()) :: [String.t()]

Finds test files that are currently passing by testing each file individually.

This approach is slower but more reliable than parsing trace output.

extract_test_summary(output)

@spec extract_test_summary(String.t()) :: String.t()

Extracts a test summary from ExUnit output.

Parses ExUnit output to extract test counts, handling both formats:

With excluded count: "290 tests, 97 failures, 163 excluded"
Without excluded count: "8 tests, 0 failures"

Returns a formatted string like "30/127 passing" showing passing tests out of total tests run (excluding excluded tests).

Examples

iex> output = "290 tests, 97 failures, 163 excluded"
iex> Mix.Tasks.Test.Baseline.extract_test_summary(output)
"30/127 passing"

iex> output = "8 tests, 0 failures"
iex> Mix.Tasks.Test.Baseline.extract_test_summary(output)
"8/8 passing"

load_passing_tests()

@spec load_passing_tests() :: {:ok, map()} | {:error, String.t()}

Loads the current passing tests configuration from test/passing_tests.json.