Anvil.Export.Manifest (Anvil v0.1.1)

Export manifest for tracking dataset lineage and reproducibility.

Manifests include:

Export metadata (queue, schema version, format)
File hash for integrity verification
Export parameters for reproducibility
Anvil version for compatibility tracking

Summary

Types

t()

Functions

compute_file_hash(path)

Computes the SHA256 hash of a file.

from_json(json)

Parses a JSON string into a manifest struct.

load(path)

Loads a manifest from a file.

new(params)

Creates a new manifest with the given parameters.

save(manifest, custom_path \\ nil)

Saves the manifest to a file.

to_json(manifest)

Converts the manifest to a JSON string.

Types

t()

@type t() :: %Anvil.Export.Manifest{
  anvil_version: String.t(),
  export_id: String.t(),
  exported_at: DateTime.t(),
  format: :csv | :jsonl | :parquet | :huggingface,
  output_path: String.t(),
  parameters: map(),
  queue_id: binary(),
  row_count: non_neg_integer(),
  sample_version: String.t() | nil,
  schema_definition_hash: String.t() | nil,
  schema_version_id: binary(),
  sha256_hash: String.t()
}

Functions

compute_file_hash(path)

@spec compute_file_hash(String.t()) :: {:ok, String.t()} | {:error, term()}

Computes the SHA256 hash of a file.

Uses streaming to handle large files without loading them into memory.

Examples

iex> Anvil.Export.Manifest.compute_file_hash("/tmp/export.csv")
{:ok, "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"}

from_json(json)

@spec from_json(String.t()) :: {:ok, t()} | {:error, term()}

Parses a JSON string into a manifest struct.

Examples

iex> json = ~s({"export_id": "exp_123", ...})
iex> Anvil.Export.Manifest.from_json(json)
{:ok, %Anvil.Export.Manifest{...}}

load(path)

@spec load(String.t()) :: {:ok, t()} | {:error, term()}

Loads a manifest from a file.

Examples

iex> Anvil.Export.Manifest.load("/tmp/export.csv.manifest.json")
{:ok, %Anvil.Export.Manifest{...}}

new(params)

@spec new(map()) :: t()

Creates a new manifest with the given parameters.

Examples

iex> Anvil.Export.Manifest.new(%{
...>   queue_id: "queue_123",
...>   schema_version_id: "schema_v1",
...>   format: :csv,
...>   output_path: "/tmp/export.csv",
...>   row_count: 100,
...>   sha256_hash: "abc123",
...>   exported_at: ~U[2025-12-01 10:00:00Z],
...>   parameters: %{}
...> })
%Anvil.Export.Manifest{...}

save(manifest, custom_path \\ nil)

@spec save(t(), String.t() | nil) :: :ok | {:error, term()}

Saves the manifest to a file.

By default, saves to <output_path>.manifest.json. A custom path can be provided as the second argument.

Examples

iex> manifest = %Anvil.Export.Manifest{output_path: "/tmp/export.csv", ...}
iex> Anvil.Export.Manifest.save(manifest)
:ok
# Creates /tmp/export.csv.manifest.json

iex> Anvil.Export.Manifest.save(manifest, "/tmp/custom.json")
:ok
# Creates /tmp/custom.json

to_json(manifest)

@spec to_json(t()) :: String.t()

Converts the manifest to a JSON string.

Examples

iex> manifest = %Anvil.Export.Manifest{...}
iex> Anvil.Export.Manifest.to_json(manifest)
"{\"export_id\": \"exp_123\", ...}"