Bardo.ExperimentManager.ExperimentManager (Bardo v0.1.0)

View Source

The ExperimentManager is responsible for orchestrating neuroevolution experiments.

It handles the complete lifecycle of experiments:

  1. Creation and configuration of experiments with parameters
  2. Starting and coordinating evolutionary runs across populations
  3. Tracking experiment progress and collecting results
  4. Providing status updates and access to results
  5. Managing experiment persistence and reporting

Specifically, it has three main functionalities:

  1. Run the population_manager N number of times, waiting for the population_manager's trace after every run.
  2. Create the experiment entry in the database, and keep updating its trace_acc as it itself accumulates the traces from spawned population_managers. This enables persistence across restarts.
  3. When the experiment_manager has finished performing N number of evolutionary runs, it calculates statistics and produces reports of the results for analysis.

Summary

Functions

Returns a specification to start this module under a supervisor.

Configure an existing experiment with the given parameters.

Export experiment results to a file.

Get the best solution from an experiment.

Get all active experiments.

Get a list of all experiments.

Create a new experiment with the given name.

Get the configured population manager module. This is used primarily for testing to allow mocks.

Start a new experiment run with default parameters. This creates a default experiment and starts it immediately.

Set the population manager module. This is used primarily for testing to allow mocks.

Start an experiment with the given ID.

Set the fitness function for evaluating solutions in an experiment.

Start the ExperimentManager as a linked process.

Get the status of an experiment.

Stop an experiment.

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

configure(experiment_id, config)

@spec configure(String.t(), map()) :: :ok | {:error, term()}

Configure an existing experiment with the given parameters.

Parameters

  • experiment_id - ID of the experiment to configure
  • config - Configuration parameters for the experiment

Returns

  • :ok - If the experiment was configured successfully
  • {:error, reason} - If there was an error configuring the experiment

Examples

iex> config = %{
...>   runs: 5,
...>   generations: 50,
...>   population_size: 100,
...>   morphology: :xor,
...>   selection_method: :tournament,
...>   backup_flag: true
...> }
iex> ExperimentManager.configure("experiment_1621234567890", config)
:ok

export_results(experiment_id, file_path, format \\ :json)

@spec export_results(String.t(), String.t(), atom()) :: :ok | {:error, term()}

Export experiment results to a file.

Parameters

  • experiment_id - ID of the experiment to export
  • file_path - Path to save the results to
  • format - Format to export in (:csv, :json, or :binary)

Returns

  • :ok - If the results were exported successfully
  • {:error, reason} - If there was an error exporting the results

Examples

iex> ExperimentManager.export_results("experiment_1621234567890", "results.json", :json)
:ok

get_best_solution(experiment_id)

@spec get_best_solution(String.t()) :: {:ok, map()} | {:error, term()}

Get the best solution from an experiment.

Parameters

  • experiment_id - ID of the experiment to get the best solution from

Returns

  • {:ok, solution} - Best solution found in the experiment
  • {:error, reason} - If there was an error getting the best solution

Examples

iex> ExperimentManager.get_best_solution("experiment_1621234567890")
{:ok, %{
  fitness: 0.98,
  genotype: %{...},
  phenotype: %{...}
}}

list_active()

@spec list_active() :: {:ok, [String.t()]} | {:error, term()}

Get all active experiments.

Returns

  • {:ok, [experiment_id]} - List of active experiment IDs
  • {:error, reason} - If there was an error getting the active experiments

Examples

iex> ExperimentManager.list_active()
{:ok, ["experiment_1621234567890", "experiment_1621234567891"]}

list_all()

@spec list_all() :: {:ok, [map()]} | {:error, term()}

Get a list of all experiments.

Returns

  • {:ok, [experiment]} - List of all experiments with their basic information
  • {:error, reason} - If there was an error getting the experiments

Examples

iex> ExperimentManager.list_all()
{:ok, [
  %{id: "experiment_1621234567890", name: "XOR Experiment", status: :completed},
  %{id: "experiment_1621234567891", name: "FX Experiment", status: :in_progress}
]}

new_experiment(name)

@spec new_experiment(String.t()) :: {:ok, String.t()} | {:error, term()}

Create a new experiment with the given name.

Parameters

  • name - Name of the experiment

Returns

  • {:ok, experiment_id} - Experiment ID of the created experiment
  • {:error, reason} - If there was an error creating the experiment

Examples

iex> ExperimentManager.new_experiment("XOR Experiment")
{:ok, "experiment_1621234567890"}

population_manager_module()

Get the configured population manager module. This is used primarily for testing to allow mocks.

run()

@spec run() :: :ok | {:error, term()}

Start a new experiment run with default parameters. This creates a default experiment and starts it immediately.

Returns

  • :ok - If the run was started successfully
  • {:error, reason} - If there was an error starting the run

set_population_manager_module(module)

Set the population manager module. This is used primarily for testing to allow mocks.

start(experiment_id)

@spec start(String.t()) :: :ok | {:error, term()}

Start an experiment with the given ID.

Parameters

  • experiment_id - ID of the experiment to start

Returns

  • :ok - If the experiment was started successfully
  • {:error, reason} - If there was an error starting the experiment

Examples

iex> ExperimentManager.start("experiment_1621234567890")
:ok

start_evaluation(experiment_id, fitness_function)

@spec start_evaluation(String.t(), function() | atom()) :: :ok | {:error, term()}

Set the fitness function for evaluating solutions in an experiment.

Parameters

  • experiment_id - ID of the experiment
  • fitness_function - Function to evaluate fitness of solutions

Returns

  • :ok - If the fitness function was set successfully
  • {:error, reason} - If there was an error setting the fitness function

Examples

iex> fitness_fn = fn solution -> solution.output == [0, 1, 1, 0] end
iex> ExperimentManager.start_evaluation("experiment_1621234567890", fitness_fn)
:ok

start_link(args \\ [])

@spec start_link(keyword()) :: GenServer.on_start()

Start the ExperimentManager as a linked process.

status(experiment_id)

@spec status(String.t()) ::
  {:not_started, map()}
  | {:in_progress, map()}
  | {:completed, map()}
  | {:error, term()}

Get the status of an experiment.

Parameters

  • experiment_id - ID of the experiment to get status for

Returns

  • {:in_progress, status} - If the experiment is in progress, with status details
  • {:completed, results} - If the experiment is completed, with results
  • {:error, reason} - If there was an error getting the status

Examples

iex> ExperimentManager.status("experiment_1621234567890")
{:in_progress, %{
  run: 2,
  total_runs: 5,
  generation: 45,
  generations: 50,
  best_fitness: 0.95,
  avg_fitness: 0.72
}}

stop(experiment_id)

@spec stop(String.t()) :: :ok | {:error, term()}

Stop an experiment.

Parameters

  • experiment_id - ID of the experiment to stop

Returns

  • :ok - If the experiment was stopped successfully
  • {:error, reason} - If there was an error stopping the experiment

Examples

iex> ExperimentManager.stop("experiment_1621234567890")
:ok