Computes the accuracy of the given predictions for binary and multi-class classification problems.

Examples

iex> Scholar.Metrics.Classification.accuracy(Nx.tensor([1, 0, 0]), Nx.tensor([1, 0, 1]))
#Nx.Tensor<
  f32
  0.6666666865348816
>

iex> y_true = Nx.tensor([0, 1, 1, 1, 1, 0, 2, 1, 0, 1], type: :u32)
iex> y_pred = Nx.tensor([0, 2, 1, 1, 2, 2, 2, 0, 0, 1], type: :u32)
iex> Scholar.Metrics.Classification.accuracy(y_true, y_pred)
#Nx.Tensor<
  f32
  0.6000000238418579
>

iex> y_true = Nx.tensor([0, 1, 1, 1, 1, 0, 2, 1, 0, 1], type: :u32)
iex> y_pred = Nx.tensor([0, 2, 1, 1, 2, 2, 2, 0, 0, 1], type: :u32)
iex> Scholar.Metrics.Classification.accuracy(y_true, y_pred, normalize: false)
#Nx.Tensor<
  u64
  6
>

Computes area under the curve (AUC) using the trapezoidal rule.

This is a general function, given points on a curve.

Examples

iex> y = Nx.tensor([0, 0, 1, 1])
iex> pred = Nx.tensor([0.1, 0.4, 0.35, 0.8])
iex> distinct_value_indices = Scholar.Metrics.Classification.distinct_value_indices(pred)
iex> {fpr, tpr, _thresholds} = Scholar.Metrics.Classification.roc_curve(y, pred, distinct_value_indices)
iex> Scholar.Metrics.Classification.auc(fpr, tpr)
#Nx.Tensor<
  f32
  0.75
>

Compute average precision (AP) from prediction scores.

AP summarizes a precision-recall curve as the weighted mean of precisions achieved at each threshold, with the increase in recall from the previous threshold used as the weight: $$ AP = sum_n (R_n - R_{n-1}) P_n $$

where $ P_n $ and $ R_n $ are the precision and recall at the nth threshold.

Examples

iex> y_true = Nx.tensor([0, 0, 1, 1])
iex> scores = Nx.tensor([0.1, 0.4, 0.35, 0.8])
iex> distinct_value_indices = Scholar.Metrics.Classification.distinct_value_indices(scores)
iex> weights = Nx.tensor([1, 1, 2, 2])
iex> ap = Scholar.Metrics.Classification.average_precision_score(y_true, scores, distinct_value_indices, weights)
iex> ap
#Nx.Tensor<
  f32
  0.8999999761581421
>

Computes the balanced accuracy score for multi-class classification

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

• :sample_weights - Sample weights of the observations.

• :adjusted (boolean/0) - If true, the balanced accuracy is adjusted for chance (depends on the number of classes). The default value is false.

Examples

iex> y_true = Nx.tensor([0, 1, 2, 0, 1, 2], type: {:u, 32})
iex> y_pred = Nx.tensor([0, 2, 1, 0, 0, 1], type: {:u, 32})
iex> Scholar.Metrics.Classification.balanced_accuracy_score(y_true, y_pred, num_classes: 3)
#Nx.Tensor<
  f32
  0.3333333432674408
>
iex> y_true = Nx.tensor([0, 1, 2, 0, 1, 2], type: {:u, 32})
iex> y_pred = Nx.tensor([0, 2, 1, 0, 0, 1], type: {:u, 32})
iex> sample_weights = [1, 1, 1, 2, 2, 2]
iex> Scholar.Metrics.Classification.balanced_accuracy_score(y_true, y_pred, num_classes: 3, sample_weights: sample_weights, adjusted: true)
#Nx.Tensor<
  f32
  0.0
>

Computes the precision of the given predictions with respect to the given targets for binary classification problems.

If the sum of true positives and false positives is 0, then the result is 0 to avoid zero division.

Examples

iex> Scholar.Metrics.Classification.binary_precision(Nx.tensor([0, 1, 1, 1]), Nx.tensor([1, 0, 1, 1]))
#Nx.Tensor<
  f32
  0.6666666865348816
>

Computes the recall of the given predictions with respect to the given targets for binary classification problems.

If the sum of true positives and false negatives is 0, then the result is 0 to avoid zero division.

Examples

iex> Scholar.Metrics.Classification.binary_recall(Nx.tensor([0, 1, 1, 1]), Nx.tensor([1, 0, 1, 1]))
#Nx.Tensor<
  f32
  0.6666666865348816
>

Computes the sensitivity of the given predictions with respect to the given targets for binary classification problems.

Examples

iex> Scholar.Metrics.Classification.binary_sensitivity(Nx.tensor([0, 1, 1, 1]), Nx.tensor([1, 0, 1, 1]))
#Nx.Tensor<
  f32
  0.6666666865348816
>

Computes the specificity of the given predictions with respect to the given targets for binary classification problems.

If the sum of true negatives and false positives is 0, then the result is 0 to avoid zero division.

Examples

iex> Scholar.Metrics.Classification.binary_specificity(Nx.tensor([0, 1, 1, 1]), Nx.tensor([1, 0, 1, 1]))
#Nx.Tensor<
  f32
  0.0
>

Compute the Brier score loss.

The smaller the Brier score loss, the better, hence the naming with "loss". The Brier score measures the mean squared difference between the predicted probability and the actual outcome. The Brier score always takes on a value between zero and one, since this is the largest possible difference between a predicted probability (which must be between zero and one) and the actual outcome (which can take on values of only 0 and 1). It can be decomposed as the sum of refinement loss and calibration loss. If predicted probabilities are not in the interval [0, 1], they will be clipped.

The Brier score is appropriate only for binary outcomes.

Options

• :sample_weights - Sample weights of the observations. The default value is 1.0.

• :pos_label (integer/0) - Label of the positive class. The default value is 1.

Examples

iex> y_true = Nx.tensor([0, 1, 1, 0])
iex> y_prob = Nx.tensor([0.1, 0.9, 0.8, 0.3])
iex> Scholar.Metrics.Classification.brier_score_loss(y_true, y_prob)
#Nx.Tensor<
  f32
  0.03750000149011612
>

Compute Cohen's kappa: a statistic that measures inter-annotator agreement.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

• :weights - Weighting to calculate the score.

• :weighting_type - Weighting type to calculate the score.

Examples

iex> y1 = Nx.tensor([0, 1, 1, 0, 1, 2])
iex> y2 = Nx.tensor([0, 2, 1, 0, 0, 1])
iex> Scholar.Metrics.Classification.cohen_kappa_score(y1, y2, num_classes: 3)
#Nx.Tensor<
  f32
  0.21739131212234497
>

iex> y1 = Nx.tensor([0, 1, 1, 0, 1, 2])
iex> y2 = Nx.tensor([0, 2, 1, 0, 0, 1])
iex> Scholar.Metrics.Classification.cohen_kappa_score(y1, y2, num_classes: 3, weighting_type: :linear)
#Nx.Tensor<
  f32
  0.3571428060531616
>

Calculates the confusion matrix given rank-1 tensors which represent the expected (y_true) and predicted (y_pred) classes.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

Examples

iex> y_true = Nx.tensor([0, 0, 1, 1, 2, 2], type: :u32)
iex> y_pred = Nx.tensor([0, 1, 0, 2, 2, 2], type: :u32)
iex> Scholar.Metrics.Classification.confusion_matrix(y_true, y_pred, num_classes: 3)
#Nx.Tensor<
  u64[3][3]
  [
    [1, 1, 0],
    [1, 0, 1],
    [0, 0, 2]
  ]
>

iex> y_true = Nx.tensor([0, 0, 1, 1, 2, 2], type: {:u, 32})
iex> y_pred = Nx.tensor([0, 1, 0, 2, 2, 2], type: {:u, 32})
iex> sample_weights = [2, 5, 1, 1.5, 2, 8]
iex> Scholar.Metrics.Classification.confusion_matrix(y_true, y_pred, num_classes: 3, sample_weights: sample_weights, normalize: :predicted)
#Nx.Tensor<
  f32[3][3]
  [
    [0.6666666865348816, 1.0, 0.0],
    [0.3333333432674408, 0.0, 0.1304347813129425],
    [0.0, 0.0, 0.8695651888847351]
  ]
>

Compute error rates for different probability thresholds (DET).

Note: This metric is used for evaluation of ranking and error tradeoffs of a binary classification task.

Examples

iex> y_true = Nx.tensor([0, 0, 1, 1])
iex> y_score = Nx.tensor([0.1, 0.4, 0.35, 0.8])
iex> distinct_value_indices = Scholar.Metrics.Classification.distinct_value_indices(y_score)
iex> {fpr, fnr, thresholds} = Scholar.Metrics.Classification.det_curve(y_true, y_score, distinct_value_indices)
iex> fpr
#Nx.Tensor<
  f32[4]
  [1.0, 0.5, 0.5, 0.0]
>
iex> fnr
#Nx.Tensor<
  f32[4]
  [0.0, 0.0, 0.5, 0.5]
>
iex> thresholds
#Nx.Tensor<
  f32[4]
  [0.10000000149011612, 0.3499999940395355, 0.4000000059604645, 0.800000011920929]
>

iex> y_true = Nx.tensor([0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1])
iex> y_score = Nx.tensor([0.1, 0.1, 0.2, 0.2, 0.3, 0.3, 0.4, 0.4, 0.5, 0.5, 0.6, 0.7, 0.7, 0.8, 0.9])
iex> distinct_value_indices = Scholar.Metrics.Classification.distinct_value_indices(y_score)
iex> {fpr, fnr, thresholds} = Scholar.Metrics.Classification.det_curve(y_true, y_score, distinct_value_indices)
iex> fpr
#Nx.Tensor<
  f32[9]
  [1.0, 0.6666666865348816, 0.6666666865348816, 0.6666666865348816, 0.3333333432674408, 0.3333333432674408, 0.1666666716337204, 0.1666666716337204, 0.0]
>
iex> fnr
#Nx.Tensor<
  f32[9]
  [0.0, 0.0, 0.2222222238779068, 0.4444444477558136, 0.4444444477558136, 0.6666666865348816, 0.6666666865348816, 0.8888888955116272, 0.8888888955116272]
>
iex> thresholds
#Nx.Tensor<
  f32[9]
  [0.10000000149011612, 0.20000000298023224, 0.30000001192092896, 0.4000000059604645, 0.5, 0.6000000238418579, 0.699999988079071, 0.800000011920929, 0.8999999761581421]
>

It's a helper function for Scholar.Metrics.Classification.roc_curve and Scholar.Metrics.Classification.roc_auc_score functions. You should call it and use as follows:

distinct_value_indices = Scholar.Metrics.Classification.distinct_value_indices(scores)
{fpr, tpr, thresholds} = Scholar.Metrics.Classification.roc_curve(y_true, scores, distinct_value_indices, weights)

Calculates F1 score given rank-1 tensors which represent the expected (y_true) and predicted (y_pred) classes.

If all examples are true negatives, then the result is 0 to avoid zero division.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

• :average - This determines the type of averaging performed on the data.

  • :macro - Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.

  • :weighted - Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label).

  • :micro - Calculate metrics globally by counting the total true positives, false negatives and false positives.

  • :none - The F-score values for each class are returned.

  The default value is :none.

Examples

iex> y_true = Nx.tensor([0, 1, 1, 1, 1, 0, 2, 1, 0, 1], type: :u32)
iex> y_pred = Nx.tensor([0, 2, 1, 1, 2, 2, 2, 0, 0, 1], type: :u32)
iex> Scholar.Metrics.Classification.f1_score(y_true, y_pred, num_classes: 3)
#Nx.Tensor<
  f32[3]
  [0.6666666865348816, 0.6666666865348816, 0.4000000059604645]
>
iex> Scholar.Metrics.Classification.f1_score(y_true, y_pred, num_classes: 3, average: :macro)
#Nx.Tensor<
  f32
  0.5777778029441833
>
iex> Scholar.Metrics.Classification.f1_score(y_true, y_pred, num_classes: 3, average: :weighted)
#Nx.Tensor<
  f32
  0.6399999856948853
>
iex> Scholar.Metrics.Classification.f1_score(y_true, y_pred, num_classes: 3, average: :micro)
#Nx.Tensor<
  f32
  0.6000000238418579
>
iex> Scholar.Metrics.Classification.f1_score(Nx.tensor([1, 0, 1, 0]), Nx.tensor([0, 1, 0, 1]), num_classes: 2, average: :none)
#Nx.Tensor<
  f32[2]
  [0.0, 0.0]
>

Calculates F-beta score given rank-1 tensors which represent the expected (y_true) and predicted (y_pred) classes.

If all examples are true negatives, then the result is 0 to avoid zero division.

$$ F_\beta = (1 + \beta^2) \cdot \frac{\mathrm{precision} \cdot \mathrm{recall}}{(\beta^2 \cdot \mathrm{precision}) + \mathrm{recall}} $$

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

• :average - This determines the type of averaging performed on the data.

  • :macro - Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.

  • :weighted - Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label).

  • :micro - Calculate metrics globally by counting the total true positives, false negatives and false positives.

  • :none - The F-score values for each class are returned.

  The default value is :none.

Examples

iex> y_true = Nx.tensor([0, 1, 1, 1, 1, 0, 2, 1, 0, 1], type: :u32)
iex> y_pred = Nx.tensor([0, 2, 1, 1, 2, 2, 2, 0, 0, 1], type: :u32)
iex> Scholar.Metrics.Classification.fbeta_score(y_true, y_pred, Nx.u32(1), num_classes: 3)
#Nx.Tensor<
  f32[3]
  [0.6666666865348816, 0.6666666865348816, 0.4000000059604645]
>
iex> Scholar.Metrics.Classification.fbeta_score(y_true, y_pred, Nx.u32(2), num_classes: 3)
#Nx.Tensor<
  f32[3]
  [0.6666666865348816, 0.5555555820465088, 0.625]
>
iex> Scholar.Metrics.Classification.fbeta_score(y_true, y_pred, Nx.f32(0.5), num_classes: 3)
#Nx.Tensor<
  f32[3]
  [0.6666666865348816, 0.8333333134651184, 0.29411765933036804]
>
iex> Scholar.Metrics.Classification.fbeta_score(y_true, y_pred, Nx.u32(2), num_classes: 3, average: :macro)
#Nx.Tensor<
  f32
  0.6157407760620117
>
iex> Scholar.Metrics.Classification.fbeta_score(y_true, y_pred, Nx.u32(2), num_classes: 3, average: :weighted)
#Nx.Tensor<
  f32
  0.5958333611488342
>
iex> Scholar.Metrics.Classification.fbeta_score(y_true, y_pred, Nx.f32(0.5), num_classes: 3, average: :micro)
#Nx.Tensor<
  f32
  0.6000000238418579
>
iex> Scholar.Metrics.Classification.fbeta_score(Nx.tensor([1, 0, 1, 0]), Nx.tensor([0, 1, 0, 1]), Nx.tensor(0.5), num_classes: 2, average: :none)
#Nx.Tensor<
  f32[2]
  [0.0, 0.0]
>
iex> Scholar.Metrics.Classification.fbeta_score(Nx.tensor([1, 0, 1, 0]), Nx.tensor([0, 1, 0, 1]), 0.5, num_classes: 2, average: :none)
#Nx.Tensor<
  f32[2]
  [0.0, 0.0]
>

Computes the log loss, aka logistic loss or cross-entropy loss.

The log-loss is a measure of how well a forecaster performs, with smaller values being better. For each sample, a forecaster outputs a probability for each class, from which the log loss is computed by averaging the negative log of the probability forecasted for the true class over a number of samples.

y_true should contain num_classes unique values, and the sum of y_prob along axis 1 should be 1 to respect the law of total probability.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

• :normalize (boolean/0) - If true, return the mean loss over the samples. Otherwise, return the sum of losses over the samples. The default value is true.

• :sample_weights - Sample weights of the observations. The default value is 1.0.

Examples

iex> y_true = Nx.tensor([0, 0, 1, 1])
iex> y_prob = Nx.tensor([[0.9, 0.1], [0.8, 0.2], [0.3, 0.7], [0.01, 0.99]])
iex> Scholar.Metrics.Classification.log_loss(y_true, y_prob, num_classes: 2)
#Nx.Tensor<
  f32
  0.17380733788013458
>
iex> Scholar.Metrics.Classification.log_loss(y_true, y_prob, num_classes: 2, normalize: false)
#Nx.Tensor<
  f32
  0.6952293515205383
>
iex> weights = Nx.tensor([0.7, 2.3, 1.3, 0.34])
iex(361)> Scholar.Metrics.Classification.log_loss(y_true, y_prob, num_classes: 2, sample_weights: weights)
#Nx.Tensor<
  f32
  0.22717177867889404
>

Matthews Correlation Coefficient (MCC) provides a measure of the quality of binary classifications.

It returns a value between -1 and 1 where 1 represents a perfect prediction, 0 represents no better than random prediction, and -1 indicates total disagreement between prediction and observation.

Computes the precision of the given predictions with respect to the given targets for multi-class classification problems.

If the sum of true positives and false positives is 0, then the result is 0 to avoid zero division.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

Examples

iex> y_true = Nx.tensor([0, 1, 1, 1, 1, 0, 2, 1, 0, 1], type: :u32)
iex> y_pred = Nx.tensor([0, 2, 1, 1, 2, 2, 2, 0, 0, 1], type: :u32)
iex> Scholar.Metrics.Classification.precision(y_true, y_pred, num_classes: 3)
#Nx.Tensor<
  f32[3]
  [0.6666666865348816, 1.0, 0.25]
>

Compute precision-recall pairs for different probability thresholds.

Note: this implementation is restricted to the binary classification task.

Examples

iex> y_true = Nx.tensor([0, 0, 1, 1])
iex> scores = Nx.tensor([0.1, 0.4, 0.35, 0.8])
iex> distinct_value_indices = Scholar.Metrics.Classification.distinct_value_indices(scores)
iex> weights = Nx.tensor([1, 1, 2, 2])
iex> {precision, recall, thresholds} = Scholar.Metrics.Classification.precision_recall_curve(y_true, scores, distinct_value_indices, weights)
iex> precision
#Nx.Tensor<
  f32[5]
  [0.6666666865348816, 0.800000011920929, 0.6666666865348816, 1.0, 1.0]
>
iex> recall
#Nx.Tensor<
  f32[5]
  [1.0, 1.0, 0.5, 0.5, 0.0]
>
iex> thresholds
#Nx.Tensor<
  f32[4]
  [0.10000000149011612, 0.3499999940395355, 0.4000000059604645, 0.800000011920929]
>

Calculates precision, recall, F-score and support for each class. It also supports a beta argument which weights recall more than precision by it's value.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

• :average - This determines the type of averaging performed on the data.

  • :macro - Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.

  • :weighted - Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label).

  • :micro - Calculate metrics globally by counting the total true positives, false negatives and false positives.

  • :none - The F-score values for each class are returned.

  The default value is :none.

• :beta - Determines the weight of recall in the combined score. For values of beta > 1 it gives more weight to recall, while beta < 1 favors precision. The default value is 1.

Examples

iex> y_true = Nx.tensor([0, 1, 1, 1, 1, 0, 2, 1, 0, 1], type: :u32)
iex> y_pred = Nx.tensor([0, 2, 1, 1, 2, 2, 2, 0, 0, 1], type: :u32)
iex> Scholar.Metrics.Classification.precision_recall_fscore_support(y_true, y_pred, num_classes: 3)
{Nx.f32([0.6666666865348816, 1.0, 0.25]),
 Nx.f32([0.6666666865348816, 0.5, 1.0]),
 Nx.f32([0.6666666865348816, 0.6666666865348816, 0.4000000059604645]),
 Nx.u64([3, 6, 1])}
iex> Scholar.Metrics.Classification.precision_recall_fscore_support(y_true, y_pred, num_classes: 3, average: :macro)
{Nx.f32([0.6666666865348816, 1.0, 0.25]),
 Nx.f32([0.6666666865348816, 0.5, 1.0]),
 Nx.f32(0.5777778029441833),
 Nx.Constants.nan()}
iex> Scholar.Metrics.Classification.precision_recall_fscore_support(y_true, y_pred, num_classes: 3, average: :weighted)
{Nx.f32([0.6666666865348816, 1.0, 0.25]),
 Nx.f32([0.6666666865348816, 0.5, 1.0]),
 Nx.f32(0.6399999856948853),
 Nx.Constants.nan()}
iex> Scholar.Metrics.Classification.precision_recall_fscore_support(y_true, y_pred, num_classes: 3, average: :micro)
{Nx.f32(0.6000000238418579),
 Nx.f32(0.6000000238418579),
 Nx.f32(0.6000000238418579),
 Nx.Constants.nan()}

iex> y_true = Nx.tensor([1, 0, 1, 0], type: :u32)
iex> y_pred = Nx.tensor([0, 1, 0, 1], type: :u32)
iex> opts = [beta: 2, num_classes: 2, average: :none]
iex> Scholar.Metrics.Classification.precision_recall_fscore_support(y_true, y_pred, opts)
{Nx.f32([0.0, 0.0]),
 Nx.f32([0.0, 0.0]),
 Nx.f32([0.0, 0.0]),
 Nx.u64([2, 2])}

Computes the recall of the given predictions with respect to the given targets for multi-class classification problems.

If the sum of true positives and false negatives is 0, then the result is 0 to avoid zero division.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

Examples

iex> y_true = Nx.tensor([0, 1, 1, 1, 1, 0, 2, 1, 0, 1], type: :u32)
iex> y_pred = Nx.tensor([0, 2, 1, 1, 2, 2, 2, 0, 0, 1], type: :u32)
iex> Scholar.Metrics.Classification.recall(y_true, y_pred, num_classes: 3)
#Nx.Tensor<
  f32[3]
  [0.6666666865348816, 0.5, 1.0]
>

Compute Area Under the Receiver Operating Characteristic Curve (ROC AUC) from prediction scores.

Note: this implementation is restricted to the binary classification task.

Examples

iex> y_true = Nx.tensor([0, 0, 1, 1])
iex> scores = Nx.tensor([0.1, 0.4, 0.35, 0.8])
iex> distinct_value_indices = Scholar.Metrics.Classification.distinct_value_indices(scores)
iex> weights = Nx.tensor([1, 1, 2, 2])
iex> Scholar.Metrics.Classification.roc_auc_score(y_true, scores, distinct_value_indices, weights)
#Nx.Tensor<
  f32
  0.75
>

Compute Receiver operating characteristic (ROC).

Note: this implementation is restricted to the binary classification task.

Examples

iex> y_true = Nx.tensor([0, 0, 1, 1])
iex> scores = Nx.tensor([0.1, 0.4, 0.35, 0.8])
iex> distinct_value_indices = Scholar.Metrics.Classification.distinct_value_indices(scores)
iex> weights = Nx.tensor([1, 1, 2, 2])
iex> {fpr, tpr, thresholds} = Scholar.Metrics.Classification.roc_curve(y_true, scores, distinct_value_indices, weights)
iex> fpr
#Nx.Tensor<
  f32[5]
  [0.0, 0.0, 0.5, 0.5, 1.0]
>
iex> tpr
#Nx.Tensor<
  f32[5]
  [0.0, 0.5, 0.5, 1.0, 1.0]
>
iex> thresholds
#Nx.Tensor<
  f32[5]
  [1.7999999523162842, 0.800000011920929, 0.4000000059604645, 0.3499999940395355, 0.10000000149011612]
>

Computes the sensitivity of the given predictions with respect to the given targets for multi-class classification problems.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

Examples

iex> y_true = Nx.tensor([0, 1, 1, 1, 1, 0, 2, 1, 0, 1], type: :u32)
iex> y_pred = Nx.tensor([0, 2, 1, 1, 2, 2, 2, 0, 0, 1], type: :u32)
iex> Scholar.Metrics.Classification.sensitivity(y_true, y_pred, num_classes: 3)
#Nx.Tensor<
  f32[3]
  [0.6666666865348816, 0.5, 1.0]
>

Computes the specificity of the given predictions with respect to the given targets for multi-class classification problems.

If the sum of true negatives and false positives is 0, then the result is 0 to avoid zero division.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

Examples

iex> y_true = Nx.tensor([0, 1, 1, 1, 1, 0, 2, 1, 0, 1], type: :u32)
iex> y_pred = Nx.tensor([0, 2, 1, 1, 2, 2, 2, 0, 0, 1], type: :u32)
iex> Scholar.Metrics.Classification.specificity(y_true, y_pred, num_classes: 3)
#Nx.Tensor<
  f32[3]
  [0.8571428656578064, 1.0, 0.6666666865348816]
>

Top-k Accuracy classification score.

This metric computes the number of times where the correct label is among the top k labels predicted (ranked by predicted scores).

For binary task assumed that y_score have values from 0 to 1.

Options

• :num_classes (pos_integer/0) - Required. Number of classes contained in the input tensors

• :k (integer/0) - Number of top elements to look at for computing accuracy. The default value is 5.

• :normalize (boolean/0) - If true, return the fraction of correctly classified samples. Otherwise, return the number of correctly classified samples. The default value is true.

Examples

iex> y_true = Nx.tensor([0, 1, 2, 2, 0])
iex> y_score = Nx.tensor([[0.5, 0.2, 0.1], [0.3, 0.4, 0.5], [0.4, 0.3, 0.2], [0.1, 0.3, 0.6], [0.9, 0.1, 0.0]])
iex> Scholar.Metrics.Classification.top_k_accuracy_score(y_true, y_score, k: 2, num_classes: 3)
#Nx.Tensor<
  f32
  0.800000011920929
>

iex> y_true = Nx.tensor([0, 1, 2, 2, 0])
iex> y_score = Nx.tensor([[0.5, 0.2, 0.1], [0.3, 0.4, 0.5], [0.4, 0.3, 0.2], [0.1, 0.3, 0.6], [0.9, 0.1, 0.0]])
iex> Scholar.Metrics.Classification.top_k_accuracy_score(y_true, y_score, k: 2, num_classes: 3, normalize: false)
#Nx.Tensor<
  u64
  4
>

iex> y_true = Nx.tensor([0, 1, 0, 1, 0])
iex> y_score = Nx.tensor([0.55, 0.3, 0.1, -0.2, 0.99])
iex> Scholar.Metrics.Classification.top_k_accuracy_score(y_true, y_score, k: 1, num_classes: 2)
#Nx.Tensor<
  f32
  0.20000000298023224
>

Zero-one classification loss.

Options

• :normalize (boolean/0) - If true, return the fraction of incorrectly classified samples. Otherwise, return the number of incorrectly classified samples. The default value is true.

Examples

iex> y_pred = Nx.tensor([1, 2, 3, 4])
iex> y_true = Nx.tensor([2, 2, 3, 4])
iex> Scholar.Metrics.Classification.zero_one_loss(y_true, y_pred)
#Nx.Tensor<
  f32
  0.25
>

iex> y_pred = Nx.tensor([1, 2, 3, 4])
iex> y_true = Nx.tensor([2, 2, 3, 4])
iex> Scholar.Metrics.Classification.zero_one_loss(y_true, y_pred, normalize: false)
#Nx.Tensor<
  u64
  1
>