Hyperparameter Optimization: Evolve ML model hyperparameters for best performance.

Problem Description

Optimize hyperparameters (learning rate, layer sizes, dropout, activation) to maximize model validation accuracy. Uses a surrogate fitness function that simulates training without running actual neural networks.

This demonstrates:

• Mixed-type parameters: Floats, integers, lists, and enums
• Schema-driven evolution: Type-aware mutations
• Log-scale optimization: Learning rate in log space
• Caching: Avoid re-evaluating identical configurations

Genome Representation

Hyperparameter map:

%{
  learning_rate: 0.001,
  hidden_layers: [128, 64],
  dropout_rate: 0.2,
  activation: :relu,
  batch_size: 32
}

Fitness Evaluation

Surrogate function rewards:

• Moderate learning rates (0.0001-0.01)
• 2-3 hidden layers
• Medium layer sizes (64-128)
• Low dropout
• Specific activations (:relu, :gelu)

Usage

iex> Jido.Evolve.Examples.HyperparameterTuning.run()
# Evolution progress shown...
# Converges to good hyperparameter configuration

Expected Results

Converges to validation accuracy >0.85 in 50-100 generations, demonstrating schema-aware mutation and crossover.