Temporal Silo Guide

What is the Temporal Silo?

The Temporal Silo is the time management controller in the Liquid Conglomerate architecture. It manages how long to evaluate individuals, episode durations, reaction time budgets, and early termination decisions. Poor time management wastes 30-70% of evaluation compute on hopeless individuals or cuts off promising ones too early.

Think of the Temporal Silo as a project manager with a stopwatch. It decides how long each individual gets to prove itself, terminates hopeless cases early to save resources, and extends evaluation time when task complexity demands it. Without it, every individual - good or bad - receives the same fixed evaluation time.

The Temporal Silo solves two fundamental problems in neuroevolution:

1. Wasted Compute: Stopping hopeless individuals early saves 2-5x compute
2. Adaptive Difficulty: Matching episode length to task complexity improves results

Architecture Overview

The Temporal Silo operates as a three-level hierarchical controller:

Key Principle: Time is Money

The Temporal Silo operates on the principle that evaluation time is the primary cost in neuroevolution:

• Bad individuals waste time if evaluated fully
• Good individuals may need more time to show potential
• Different tasks require different episode lengths
• Real-time applications have strict timing constraints

How It Works

Sensors (Inputs)

The Temporal Silo observes 12 sensors describing time-related dynamics:

Actuators (Outputs)

The Temporal Silo controls 10 parameters governing time management:

Early Termination

Early termination is the primary mechanism for saving compute:

Decision Logic:

1. Wait for min_steps_before_termination to gather data
2. Check if fitness < early_termination_fitness threshold
3. Also consider fitness trend (is it improving?)
4. If hopeless, terminate and emit early_termination event

Benefits:

• 30-70% compute savings by stopping bad individuals
• False termination rate can be tuned via threshold
• Frees resources for promising individuals

The Control Loop

1. Per Step: Check reaction time limits (for real-time applications)
2. Per Evaluation: Track elapsed time, check timeout, check early termination
3. Per Episode: Record episode length, update efficiency metrics
4. Per Generation: Adapt episode targets based on performance
5. Emit Events: Publish timeout/termination/convergence events

%% Typical usage in evaluation
evaluate_individual(Individual, State) ->
    %% Get temporal parameters
    Params = temporal_silo:get_temporal_params(TempPid),
    #{evaluation_timeout_ms := Timeout,
      early_termination_fitness := TermThreshold} = Params,

    StartTime = erlang:monotonic_time(millisecond),
    evaluate_loop(Individual, StartTime, Timeout, TermThreshold, State).

evaluate_loop(Individual, StartTime, Timeout, TermThreshold, State) ->
    Elapsed = erlang:monotonic_time(millisecond) - StartTime,

    %% Check timeout
    case temporal_silo:check_timeout(TempPid, Elapsed) of
        true ->
            {timeout, current_fitness(Individual)};
        false ->
            %% Check early termination
            Fitness = current_fitness(Individual),
            case temporal_silo:should_terminate_early(TempPid, Fitness) of
                true ->
                    {early_termination, Fitness};
                false ->
                    %% Continue evaluation
                    step_evaluation(Individual, State)
            end
    end.

Integration with the Neuroevolution Engine

Wiring Diagram

The Temporal Silo integrates with evaluation and other silos:

Data Sources:

• evaluation_engine - Episode lengths, reaction times, fitness history
• task_silo - Stagnation severity (triggers longer episodes)
• economic_silo - Computation budget (constrains timeouts)

Data Consumers:

• evaluation_engine - Timeout limits, episode targets
• individuals - Reaction time limits, termination thresholds
• neuroevolution_events - Event bus for monitoring

Cross-Silo Interactions

The Temporal Silo exchanges signals with other silos:

Signals Sent: | Signal | To | Description | |--------|-----|-------------| | time_pressure | Task | High pressure = need simpler solutions | | convergence_status | Resource | Near convergence = can reduce compute | | episode_efficiency | Economic | Compute spent per fitness gained | | eval_time_constraint | Competitive | Available time for evaluation matches |

Signals Received: | Signal | From | Effect | |--------|------|--------| | computation_budget | Economic | Constrains evaluation timeout | | stagnation_severity | Task | Stagnation = try longer episodes | | population_diversity | Distribution | Low diversity = earlier termination | | resource_pressure | Resource | Pressure reduces available time |

Engine Integration Points

%% Start Temporal Silo
{ok, _} = temporal_silo:start_link(#temporal_config{
    enabled = true,
    min_episode_length = 100,
    max_episode_length = 10000,
    min_timeout_ms = 500,
    max_timeout_ms = 30000,
    enable_early_termination = true,
    min_steps_before_termination = 20,
    enforce_reaction_time = false,
    emit_events = true
}),

%% Query for temporal parameters
get_evaluation_params() ->
    Params = temporal_silo:get_temporal_params(?MODULE),
    #{
        timeout => maps:get(evaluation_timeout_ms, Params),
        episode_target => maps:get(episode_length_target, Params),
        early_term_threshold => maps:get(early_termination_fitness, Params)
    }.

%% Record completed episode
on_episode_complete(IndividualId, Length, Fitness) ->
    temporal_silo:record_episode(TempPid, IndividualId, Length),
    %% Temporal silo tracks for efficiency analysis.

Training Velocity Impact

The Temporal Silo provides the largest training velocity improvement of all silos by eliminating wasted compute on hopeless individuals.

Practical Examples

Example 1: Early Termination Saves Compute

%% Scenario: Individual showing poor performance early
%% After 50 steps, fitness = 0.05 (very low)

%% Temporal Silo parameters:
%% - early_termination_fitness = 0.1
%% - min_steps_before_termination = 20

%% Decision: Terminate early
%% Without early termination: 1000 steps wasted
%% With early termination: Only 50 steps used
%% Savings: 950 steps = 95% compute saved for this individual

Example 2: Stagnation Triggers Longer Episodes

%% Scenario: Task Silo reports stagnation
%% Cross-silo signal: stagnation_severity = 0.8

%% Temporal Silo responds:
%% - episode_length_target increased from 1000 to 1500
%% - evaluation_timeout increased proportionally

%% Rationale: Complex problems may need more time
%% The stuck population might benefit from longer evaluation

{episode_extended, #{
    old_length => 1000,
    new_length => 1500,
    reason => stagnation
}}

Example 3: Budget Constrains Timeouts

%% Scenario: Economic Silo signals budget pressure
%% Cross-silo signal: computation_budget = 0.3 (low)

%% Temporal Silo responds:
%% - evaluation_timeout_ms reduced from 5000 to 2000
%% - early_termination_fitness raised to 0.15 (more aggressive)

%% Rationale: Limited budget = faster decisions
%% Accept lower accuracy for speed

{temporal_params_updated, #{
    old_timeout => 5000,
    new_timeout => 2000,
    trigger => budget_pressure
}}

Tuning Guide

Key Parameters

Common Pitfalls

1. Termination threshold too aggressive: Good individuals killed early

   • Symptom: Best fitness drops unexpectedly
   • Fix: Lower early_termination_fitness to 0.05

2. Episodes too short: Not enough time to show potential

   • Symptom: All individuals seem equally bad
   • Fix: Increase episode_length_target 2-4x

3. No early termination: Wasting compute on hopeless cases

   • Symptom: Evaluation takes forever, most individuals are bad
   • Fix: Enable early termination, start with threshold 0.1

4. Patience too high: Stuck on local optima

   • Symptom: No improvement but keeps running
   • Fix: Lower patience_threshold to 10-15 generations

Debugging Tips

%% Get current sensor values
Context = build_temporal_context(State),
Sensors = temporal_silo_sensors:collect_sensors(Context),
io:format("Convergence rate: ~.2f~n", [lists:nth(9, Sensors)]),
io:format("Oscillation freq: ~.2f~n", [lists:nth(10, Sensors)]),
io:format("Eval efficiency: ~.2f~n", [lists:nth(12, Sensors)]),

%% Get current actuator values
Params = temporal_silo:get_temporal_params(TempPid),
io:format("Timeout: ~p ms~n", [maps:get(evaluation_timeout_ms, Params)]),
io:format("Episode target: ~p~n", [maps:get(episode_length_target, Params)]),
io:format("Term threshold: ~.2f~n", [maps:get(early_termination_fitness, Params)]).

Events Reference

The Temporal Silo emits events on significant actions:

Example Event Payload:

{early_termination, #{
    silo => temporal,
    timestamp => 1703318400000,
    generation => 42,
    payload => #{
        individual_id => <<"agent_123">>,
        reason => low_fitness,
        fitness => 0.03,
        steps_completed => 75
    }
}}

L0 Hyperparameters (L1-Tuned)

L1 Hyperparameters (L2-Tuned)

Configuration Examples

Real-time Application Config

#temporal_config{
    enabled = true,
    min_episode_length = 100,
    max_episode_length = 1000,
    min_timeout_ms = 500,
    max_timeout_ms = 5000,
    enable_early_termination = true,
    min_steps_before_termination = 50,
    enforce_reaction_time = true,  % Critical for real-time
    emit_events = true
}.

Exploratory Research Config

#temporal_config{
    enabled = true,
    min_episode_length = 1000,
    max_episode_length = 100000,
    min_timeout_ms = 10000,
    max_timeout_ms = 600000,  % 10 minutes
    enable_early_termination = false,  % Let everything run
    enforce_reaction_time = false,
    emit_events = true
}.

Cost-Optimized Config

#temporal_config{
    enabled = true,
    min_episode_length = 50,
    max_episode_length = 500,
    min_timeout_ms = 100,
    max_timeout_ms = 2000,
    enable_early_termination = true,
    min_steps_before_termination = 10,  % Aggressive
    emit_events = true
}.

Source Code Reference

Further Reading

• Liquid Conglomerate Overview - Full LC architecture
• Economic Silo - Budget and cost management
• Task Silo - Hyperparameter adaptation
• Resource Silo - Computational resource management
• Meta-Controller Guide - L2 strategic layer

References

Early Termination and Adaptive Evaluation

• Jaderberg, M., et al. (2017). "Population Based Training of Neural Networks." arXiv preprint arXiv:1711.09846.
• Li, L., et al. (2018). "Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization." JMLR.

Credit Assignment and Temporal Difference Learning

• Sutton, R.S. (1988). "Learning to Predict by the Methods of Temporal Differences." Machine Learning.
• Williams, R.J. (1992). "Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning." Machine Learning.

Convergence Detection

• Henderson, P., et al. (2018). "Deep Reinforcement Learning that Matters." AAAI.