Bardo.ExperimentManager.ExperimentManager (Bardo v0.1.0)
View SourceThe ExperimentManager is responsible for orchestrating neuroevolution experiments.
It handles the complete lifecycle of experiments:
- Creation and configuration of experiments with parameters
- Starting and coordinating evolutionary runs across populations
- Tracking experiment progress and collecting results
- Providing status updates and access to results
- Managing experiment persistence and reporting
Specifically, it has three main functionalities:
- Run the population_manager N number of times, waiting for the population_manager's trace after every run.
- Create the experiment entry in the database, and keep updating its trace_acc as it itself accumulates the traces from spawned population_managers. This enables persistence across restarts.
- When the experiment_manager has finished performing N number of evolutionary runs, it calculates statistics and produces reports of the results for analysis.
Summary
Functions
Returns a specification to start this module under a supervisor.
Configure an existing experiment with the given parameters.
Export experiment results to a file.
Get the best solution from an experiment.
Get all active experiments.
Get a list of all experiments.
Create a new experiment with the given name.
Get the configured population manager module. This is used primarily for testing to allow mocks.
Start a new experiment run with default parameters. This creates a default experiment and starts it immediately.
Set the population manager module. This is used primarily for testing to allow mocks.
Start an experiment with the given ID.
Set the fitness function for evaluating solutions in an experiment.
Start the ExperimentManager as a linked process.
Get the status of an experiment.
Stop an experiment.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor
.
Configure an existing experiment with the given parameters.
Parameters
experiment_id
- ID of the experiment to configureconfig
- Configuration parameters for the experiment
Returns
:ok
- If the experiment was configured successfully{:error, reason}
- If there was an error configuring the experiment
Examples
iex> config = %{
...> runs: 5,
...> generations: 50,
...> population_size: 100,
...> morphology: :xor,
...> selection_method: :tournament,
...> backup_flag: true
...> }
iex> ExperimentManager.configure("experiment_1621234567890", config)
:ok
Export experiment results to a file.
Parameters
experiment_id
- ID of the experiment to exportfile_path
- Path to save the results toformat
- Format to export in (:csv, :json, or :binary)
Returns
:ok
- If the results were exported successfully{:error, reason}
- If there was an error exporting the results
Examples
iex> ExperimentManager.export_results("experiment_1621234567890", "results.json", :json)
:ok
Get the best solution from an experiment.
Parameters
experiment_id
- ID of the experiment to get the best solution from
Returns
{:ok, solution}
- Best solution found in the experiment{:error, reason}
- If there was an error getting the best solution
Examples
iex> ExperimentManager.get_best_solution("experiment_1621234567890")
{:ok, %{
fitness: 0.98,
genotype: %{...},
phenotype: %{...}
}}
Get all active experiments.
Returns
{:ok, [experiment_id]}
- List of active experiment IDs{:error, reason}
- If there was an error getting the active experiments
Examples
iex> ExperimentManager.list_active()
{:ok, ["experiment_1621234567890", "experiment_1621234567891"]}
Get a list of all experiments.
Returns
{:ok, [experiment]}
- List of all experiments with their basic information{:error, reason}
- If there was an error getting the experiments
Examples
iex> ExperimentManager.list_all()
{:ok, [
%{id: "experiment_1621234567890", name: "XOR Experiment", status: :completed},
%{id: "experiment_1621234567891", name: "FX Experiment", status: :in_progress}
]}
Create a new experiment with the given name.
Parameters
name
- Name of the experiment
Returns
{:ok, experiment_id}
- Experiment ID of the created experiment{:error, reason}
- If there was an error creating the experiment
Examples
iex> ExperimentManager.new_experiment("XOR Experiment")
{:ok, "experiment_1621234567890"}
Get the configured population manager module. This is used primarily for testing to allow mocks.
@spec run() :: :ok | {:error, term()}
Start a new experiment run with default parameters. This creates a default experiment and starts it immediately.
Returns
:ok
- If the run was started successfully{:error, reason}
- If there was an error starting the run
Set the population manager module. This is used primarily for testing to allow mocks.
Start an experiment with the given ID.
Parameters
experiment_id
- ID of the experiment to start
Returns
:ok
- If the experiment was started successfully{:error, reason}
- If there was an error starting the experiment
Examples
iex> ExperimentManager.start("experiment_1621234567890")
:ok
Set the fitness function for evaluating solutions in an experiment.
Parameters
experiment_id
- ID of the experimentfitness_function
- Function to evaluate fitness of solutions
Returns
:ok
- If the fitness function was set successfully{:error, reason}
- If there was an error setting the fitness function
Examples
iex> fitness_fn = fn solution -> solution.output == [0, 1, 1, 0] end
iex> ExperimentManager.start_evaluation("experiment_1621234567890", fitness_fn)
:ok
@spec start_link(keyword()) :: GenServer.on_start()
Start the ExperimentManager as a linked process.
@spec status(String.t()) :: {:not_started, map()} | {:in_progress, map()} | {:completed, map()} | {:error, term()}
Get the status of an experiment.
Parameters
experiment_id
- ID of the experiment to get status for
Returns
{:in_progress, status}
- If the experiment is in progress, with status details{:completed, results}
- If the experiment is completed, with results{:error, reason}
- If there was an error getting the status
Examples
iex> ExperimentManager.status("experiment_1621234567890")
{:in_progress, %{
run: 2,
total_runs: 5,
generation: 45,
generations: 50,
best_fitness: 0.95,
avg_fitness: 0.72
}}
Stop an experiment.
Parameters
experiment_id
- ID of the experiment to stop
Returns
:ok
- If the experiment was stopped successfully{:error, reason}
- If there was an error stopping the experiment
Examples
iex> ExperimentManager.stop("experiment_1621234567890")
:ok