Anvil.Agreement.Accumulator (Anvil v0.1.1)

Incrementally accumulates agreement statistics for online computation.

Maintains running statistics that can be updated as new labels arrive, avoiding the need to recompute from scratch on every label submission.

Summary

Types

t()

Functions

add_label(acc, label)

Adds a label to the accumulator, updating statistics.

compute_kappa(acc)

Computes Cohen's kappa from the accumulated statistics.

merge(acc1, acc2)

Merges two accumulators, combining their statistics.

new()

Creates a new empty accumulator.

Types

t()

@type t() :: %Anvil.Agreement.Accumulator{
  confusion_matrix: map(),
  label_counts: map(),
  labeler_counts: map(),
  last_updated: DateTime.t() | nil
}

Functions

add_label(acc, label)

@spec add_label(t(), map()) :: t()

Adds a label to the accumulator, updating statistics.

The label should have the structure:

labeler_id: identifier for the labeler
values: map of field names to values

Examples

iex> acc = Accumulator.new()
iex> label = %{labeler_id: "l1", values: %{"coherence" => 4}}
iex> acc = Accumulator.add_label(acc, label)
iex> acc.labeler_counts
%{"l1" => 1}

compute_kappa(acc)

@spec compute_kappa(t()) :: {:ok, float()} | {:error, term()}

Computes Cohen's kappa from the accumulated statistics.

Returns {:ok, kappa} if sufficient data exists, {:error, reason} otherwise.

Examples

iex> acc = Accumulator.new()
iex> acc = Accumulator.add_label(acc, %{labeler_id: "l1", values: %{"field" => "a"}})
iex> acc = Accumulator.add_label(acc, %{labeler_id: "l2", values: %{"field" => "a"}})
iex> Accumulator.compute_kappa(acc)
{:ok, 1.0}  # Perfect agreement

merge(acc1, acc2)

@spec merge(t(), t()) :: t()

Merges two accumulators, combining their statistics.

Useful for parallel computation where multiple accumulators process different subsets of data.

Examples

iex> acc1 = Accumulator.new() |> Accumulator.add_label(%{labeler_id: "l1", values: %{}})
iex> acc2 = Accumulator.new() |> Accumulator.add_label(%{labeler_id: "l2", values: %{}})
iex> merged = Accumulator.merge(acc1, acc2)
iex> map_size(merged.labeler_counts)
2

new()

@spec new() :: t()

Creates a new empty accumulator.

Examples

iex> Accumulator.new()
%Accumulator{
  confusion_matrix: %{},
  label_counts: %{},
  labeler_counts: %{},
  last_updated: nil
}