# `Aludel.Evals`
[🔗](https://github.com/ccarvalho-eng/aludel/blob/main/lib/aludel/evals.ex#L1)

Context for managing evaluation suites, test cases, and runs.

# `change_suite`

```elixir
@spec change_suite(Aludel.Evals.Suite.t(), map()) :: Ecto.Changeset.t()
```

Returns a changeset for tracking suite changes.

# `change_test_case`

```elixir
@spec change_test_case(Aludel.Evals.TestCase.t(), map()) :: Ecto.Changeset.t()
```

Returns a changeset for tracking test case changes.

# `create_suite`

```elixir
@spec create_suite(map()) ::
  {:ok, Aludel.Evals.Suite.t()} | {:error, Ecto.Changeset.t()}
```

Creates a new suite.

# `create_suite_run`

```elixir
@spec create_suite_run(map()) ::
  {:ok, Aludel.Evals.SuiteRun.t()} | {:error, Ecto.Changeset.t()}
```

Creates a new suite run.

# `create_test_case`

```elixir
@spec create_test_case(map()) ::
  {:ok, Aludel.Evals.TestCase.t()} | {:error, Ecto.Changeset.t()}
```

Creates a new test case.

# `create_test_case_document`

```elixir
@spec create_test_case_document(map()) ::
  {:ok, Aludel.Evals.TestCaseDocument.t()} | {:error, Ecto.Changeset.t()}
```

Creates a test case document.

# `delete_suite`

```elixir
@spec delete_suite(Aludel.Evals.Suite.t()) ::
  {:ok, Aludel.Evals.Suite.t()} | {:error, Ecto.Changeset.t()}
```

Deletes a suite.

# `delete_suite_run`

```elixir
@spec delete_suite_run(Aludel.Evals.SuiteRun.t()) ::
  {:ok, Aludel.Evals.SuiteRun.t()} | {:error, Ecto.Changeset.t()}
```

Deletes a suite run.

# `delete_test_case`

```elixir
@spec delete_test_case(Aludel.Evals.TestCase.t()) ::
  {:ok, Aludel.Evals.TestCase.t()} | {:error, Ecto.Changeset.t()}
```

Deletes a test case.

# `delete_test_case_document`

```elixir
@spec delete_test_case_document(Aludel.Evals.TestCaseDocument.t()) ::
  {:ok, Aludel.Evals.TestCaseDocument.t()} | {:error, Ecto.Changeset.t()}
```

Deletes a test case document.

# `execute_suite`

```elixir
@spec execute_suite(
  Aludel.Evals.Suite.t(),
  Aludel.Prompts.PromptVersion.t(),
  Aludel.Providers.Provider.t()
) :: {:ok, Aludel.Evals.SuiteRun.t()} | {:error, term()}
```

Executes a test suite against a prompt version and provider.

Runs all test cases for the suite, evaluating their assertions
against the LLM output and creating a suite_run with results.

## Parameters
  - suite: The test suite to execute
  - prompt_version: The prompt version to use
  - provider: The LLM provider to call

## Returns
  - `{:ok, suite_run}` with execution results
  - `{:error, reason}` if execution fails

# `get_suite!`

```elixir
@spec get_suite!(binary()) :: Aludel.Evals.Suite.t()
```

Gets a suite by ID, raising if not found.

# `get_suite_run!`

```elixir
@spec get_suite_run!(binary()) :: Aludel.Evals.SuiteRun.t()
```

Gets a suite run by ID, raising if not found.

# `get_suite_with_prompt!`

```elixir
@spec get_suite_with_prompt!(binary()) :: Aludel.Evals.Suite.t()
```

Gets a suite by ID with prompt preloaded, raising if not found.

# `get_suite_with_test_cases!`

```elixir
@spec get_suite_with_test_cases!(binary()) :: Aludel.Evals.Suite.t()
```

Gets a suite with all test cases preloaded.

# `get_suite_with_test_cases_and_prompt!`

```elixir
@spec get_suite_with_test_cases_and_prompt!(binary()) :: Aludel.Evals.Suite.t()
```

Gets a suite with test cases and prompt preloaded.

# `get_test_case!`

```elixir
@spec get_test_case!(binary()) :: Aludel.Evals.TestCase.t()
```

Gets a test case by ID, raising if not found.

# `get_test_case_document!`

```elixir
@spec get_test_case_document!(binary()) :: Aludel.Evals.TestCaseDocument.t()
```

Gets a single test case document.

Raises `Ecto.NoResultsError` if the document does not exist.

# `get_test_case_with_documents!`

```elixir
@spec get_test_case_with_documents!(binary()) :: Aludel.Evals.TestCase.t()
```

Gets a test case with documents preloaded.

# `launch_suite_execution`

```elixir
@spec launch_suite_execution(pid(), binary(), binary(), binary()) ::
  {:ok, reference()} | {:error, term()}
```

Launches suite execution in a supervised task and reports completion
back to the given recipient process.

# `list_suite_runs`

```elixir
@spec list_suite_runs() :: [Aludel.Evals.SuiteRun.t()]
```

Lists all suite runs in the system.

# `list_suite_runs_for_suite`

Gets suite runs for a specific suite.

# `list_suite_runs_for_suite_with_associations`

Gets suite runs for a specific suite with prompt_version and provider
preloaded.

# `list_suites`

```elixir
@spec list_suites() :: [Aludel.Evals.Suite.t()]
```

Lists all suites in the system.

# `list_suites_with_prompt`

```elixir
@spec list_suites_with_prompt() :: [Aludel.Evals.Suite.t()]
```

Lists all suites with their associated prompt preloaded.

# `list_test_cases`

```elixir
@spec list_test_cases() :: [Aludel.Evals.TestCase.t()]
```

Lists all test cases in the system.

# `pass_rates_by_prompt`

```elixir
@spec pass_rates_by_prompt() :: [map()]
```

Calculates pass rates grouped by prompt.

Returns a list of maps with prompt info and pass rate statistics.

# `reload_suite_run_with_associations`

```elixir
@spec reload_suite_run_with_associations(Aludel.Evals.SuiteRun.t()) ::
  Aludel.Evals.SuiteRun.t()
```

Reloads a suite run with associations preloaded.

# `retry_suite_run_test_case`

```elixir
@spec retry_suite_run_test_case(Aludel.Evals.SuiteRun.t(), binary()) ::
  {:ok, Aludel.Evals.SuiteRun.t()} | {:error, term()}
```

Retries a single test case result within an existing suite run.

The existing embedded result is replaced in-place and the suite run
aggregates are recalculated from the updated result set.

# `update_suite`

```elixir
@spec update_suite(Aludel.Evals.Suite.t(), map()) ::
  {:ok, Aludel.Evals.Suite.t()} | {:error, Ecto.Changeset.t()}
```

Updates an existing suite.

# `update_test_case`

```elixir
@spec update_test_case(Aludel.Evals.TestCase.t(), map()) ::
  {:ok, Aludel.Evals.TestCase.t()} | {:error, Ecto.Changeset.t()}
```

Updates an existing test case.

---

*Consult [api-reference.md](api-reference.md) for complete listing*