LLMDB.History.Backfill (LLM DB v2026.3.0)

Backfills model history by diffing committed provider snapshots across git history.

This module is intentionally git-driven and infrastructure-free: it reads priv/llm_db/providers/*.json from commit history and writes append-only NDJSON event files under a local history directory.

Summary

Types

check_result()

summary()

Functions

check(opts \\ [])

Checks whether generated history is current with git history.

diff_models(previous_models, current_models)

Diffs two model maps and returns deterministic model events.

run(opts \\ [])

Runs a full history backfill from git.

sync(opts \\ [])

Incrementally syncs history output from the last generated commit to :to (default HEAD).

Types

check_result()

@type check_result() ::
  :history_unavailable
  | :up_to_date
  | {:outdated, %{new_commits: non_neg_integer(), latest_commit: String.t()}}

summary()

@type summary() :: %{
  commits_scanned: non_neg_integer(),
  commits_processed: non_neg_integer(),
  snapshots_written: non_neg_integer(),
  events_written: non_neg_integer(),
  output_dir: String.t(),
  from_commit: String.t() | nil,
  to_commit: String.t() | nil
}

Functions

check(opts \\ [])

@spec check(keyword()) :: {:ok, check_result()} | {:error, term()}

Checks whether generated history is current with git history.

Returns :history_unavailable when meta.json is missing, :up_to_date when no metadata commits are pending, or {:outdated, ...} when new commits exist.

diff_models(previous_models, current_models)

@spec diff_models(%{optional(String.t()) => map()}, %{optional(String.t()) => map()}) ::
  [map()]

Diffs two model maps and returns deterministic model events.

Expects maps keyed by "provider:model_id" with normalized model payload values.

run(opts \\ [])

@spec run(keyword()) :: {:ok, summary()} | {:error, term()}

Runs a full history backfill from git.

Options

:from - Optional start commit (inclusive)
:to - Optional end commit/reference (default: "HEAD")
:output_dir - Output directory (default: "priv/llm_db/history")
:force - Remove previously generated history files first (default: false)

sync(opts \\ [])

@spec sync(keyword()) :: {:ok, summary()} | {:error, term()}

Incrementally syncs history output from the last generated commit to :to (default HEAD).

If no history output exists yet, this performs a full backfill into the output directory.

Options

:to - Optional end commit/reference (default: "HEAD")
:output_dir - Output directory (default: "priv/llm_db/history")