ExFairness.Report (ExFairness v0.5.1)

Fairness report generation and export.

Provides comprehensive fairness assessment across multiple metrics with multiple export formats (Markdown, JSON).

Examples

iex> predictions = Nx.tensor([1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1])
iex> labels = Nx.tensor([1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1])
iex> sensitive = Nx.tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
iex> report = ExFairness.Report.generate(predictions, labels, sensitive)
iex> Map.has_key?(report, :overall_assessment)
true

Summary

Types

report()

Functions

generate(predictions, labels, sensitive_attr, opts \\ [])

Generates a comprehensive fairness report across multiple metrics.

to_json(report)

Exports a fairness report to JSON format.

to_markdown(report)

Exports a fairness report to Markdown format.

Types

report()

@type report() :: %{
  optional(:demographic_parity) =>
    ExFairness.Metrics.DemographicParity.result(),
  optional(:equalized_odds) => ExFairness.Metrics.EqualizedOdds.result(),
  optional(:equal_opportunity) => ExFairness.Metrics.EqualOpportunity.result(),
  optional(:predictive_parity) => ExFairness.Metrics.PredictiveParity.result(),
  optional(:calibration) => ExFairness.Metrics.Calibration.result(),
  overall_assessment: String.t(),
  passed_count: non_neg_integer(),
  failed_count: non_neg_integer(),
  total_count: non_neg_integer()
}

Functions

generate(predictions, labels, sensitive_attr, opts \\ [])

@spec generate(Nx.Tensor.t(), Nx.Tensor.t(), Nx.Tensor.t(), keyword()) :: report()

Generates a comprehensive fairness report across multiple metrics.

Parameters

predictions - Binary predictions tensor (0 or 1)
labels - Binary labels tensor (0 or 1)
sensitive_attr - Binary sensitive attribute tensor (0 or 1)
opts - Options:
- :metrics - List of metrics to include (default: all available, calibration only when :probabilities is provided)
- :threshold - Fairness threshold to pass to all metrics
- :min_per_group - Minimum samples per group
- :probabilities - Predicted probabilities (required for :calibration)
- CI/testing options forwarded to metrics:
  - :include_ci - Enable bootstrap confidence intervals
  - :bootstrap_samples, :confidence_level, :stratified
  - :statistical_test - e.g., :z_test | :chi_square | :permutation
  - :alpha, :n_permutations

Returns

A map containing:

Metric results (one key per requested metric)
:overall_assessment - Summary of fairness across all metrics
:passed_count - Number of metrics that passed
:failed_count - Number of metrics that failed
:total_count - Total number of metrics evaluated

Examples

iex> predictions = Nx.tensor([1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1])
iex> labels = Nx.tensor([1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1])
iex> sensitive = Nx.tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
iex> report = ExFairness.Report.generate(predictions, labels, sensitive, metrics: [:demographic_parity])
iex> report.total_count
1

to_json(report)

@spec to_json(report()) :: String.t()

Exports a fairness report to JSON format.

Parameters

report - A fairness report generated by generate/4

Returns

A JSON-formatted string containing the report.

Examples

iex> predictions = Nx.tensor([1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1])
iex> labels = Nx.tensor([1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1])
iex> sensitive = Nx.tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
iex> report = ExFairness.Report.generate(predictions, labels, sensitive, metrics: [:demographic_parity])
iex> json = ExFairness.Report.to_json(report)
iex> String.starts_with?(json, "{")
true

to_markdown(report)

@spec to_markdown(report()) :: String.t()

Exports a fairness report to Markdown format.

Parameters

report - A fairness report generated by generate/4

Returns

A Markdown-formatted string containing the report.

Examples

iex> predictions = Nx.tensor([1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1])
iex> labels = Nx.tensor([1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1])
iex> sensitive = Nx.tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
iex> report = ExFairness.Report.generate(predictions, labels, sensitive, metrics: [:demographic_parity])
iex> markdown = ExFairness.Report.to_markdown(report)
iex> String.contains?(markdown, "# Fairness Report")
true