ExFairness.Metrics.EqualizedOdds (ExFairness v0.5.1)

Equalized Odds fairness metric.

Equalized odds requires that both the true positive rate (TPR) and false positive rate (FPR) are equal across groups defined by the sensitive attribute.

Mathematical Definition

P(Ŷ = 1 | Y = 1, A = 0) = P(Ŷ = 1 | Y = 1, A = 1)  # Equal TPR
P(Ŷ = 1 | Y = 0, A = 0) = P(Ŷ = 1 | Y = 0, A = 1)  # Equal FPR

The disparities are measured as:

Δ_TPR = |TPR_{A=0} - TPR_{A=1}|
Δ_FPR = |FPR_{A=0} - FPR_{A=1}|

When to Use

When both false positives and false negatives matter
Criminal justice (wrongful conviction and wrongful acquittal both harmful)
Medical diagnosis (both missed diseases and false alarms matter)
When accuracy across all outcomes is important

Limitations

May conflict with demographic parity when base rates differ
Requires sufficient samples of both positive and negative labels
More restrictive than equal opportunity (which only checks TPR)

References

Hardt, M., Price, E., & Srebro, N. (2016). "Equality of Opportunity in Supervised Learning." NeurIPS.

Examples

iex> predictions = Nx.tensor([1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1])
iex> labels = Nx.tensor([1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1])
iex> sensitive = Nx.tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
iex> result = ExFairness.Metrics.EqualizedOdds.compute(predictions, labels, sensitive)
iex> result.passes
true

Summary

Types

result()

Functions

compute(predictions, labels, sensitive_attr, opts \\ [])

Computes equalized odds disparity between groups.

Types

result()

@type result() :: %{
  group_a_tpr: float(),
  group_b_tpr: float(),
  group_a_fpr: float(),
  group_b_fpr: float(),
  tpr_disparity: float(),
  fpr_disparity: float(),
  passes: boolean(),
  threshold: float(),
  interpretation: String.t()
}

Functions

compute(predictions, labels, sensitive_attr, opts \\ [])

@spec compute(Nx.Tensor.t(), Nx.Tensor.t(), Nx.Tensor.t(), keyword()) :: result()

Computes equalized odds disparity between groups.

Parameters

predictions - Binary predictions tensor (0 or 1)
labels - Binary labels tensor (0 or 1)
sensitive_attr - Binary sensitive attribute tensor (0 or 1)
opts - Options:
- :threshold - Maximum acceptable disparity for both TPR and FPR (default: 0.1)
- :min_per_group - Minimum samples per group for validation (default: 10)

Returns

A map containing:

:group_a_tpr - True positive rate for group A
:group_b_tpr - True positive rate for group B
:group_a_fpr - False positive rate for group A
:group_b_fpr - False positive rate for group B
:tpr_disparity - Absolute difference in TPR
:fpr_disparity - Absolute difference in FPR
:passes - Whether both disparities are within threshold
:threshold - Threshold used
:interpretation - Plain language explanation of the result

Examples

iex> predictions = Nx.tensor([1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0])
iex> labels = Nx.tensor([1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0])
iex> sensitive = Nx.tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
iex> result = ExFairness.Metrics.EqualizedOdds.compute(predictions, labels, sensitive)
iex> result.passes
false