macula-neuroevolution-esdb

View Source

Bridge library between macula-neuroevolution lineage tracking and erl-esdb-gater.

Hex.pm Documentation

Overview

This library provides an implementation of the neuroevolution_lineage_events behaviour using erl-esdb-gater as the event store backend. It enables persistent genealogy tracking for evolved neural networks.

Performance Design

Lineage tracking NEVER blocks the evolution loop:

  • persist_event/persist_batch return immediately (fire-and-forget)
  • I/O happens asynchronously in spawned processes
  • Errors are logged, not propagated
  • Under extreme load, events may be lost (acceptable trade-off)

Architecture

macula-neuroevolution          macula-neuroevolution-esdb         erl-esdb-gater
+---------------------+       +------------------------+       +-----------------+
| neuroevolution_     |       | esdb_lineage_backend   |       | esdb_gater_api  |
| lineage_events      |<------| (fire-and-forget)      |------>| append_events   |
| (behaviour)         |       |     spawn -> write     |       | (async)         |
+---------------------+       +------------------------+       +-----------------+

Installation

Add to your rebar.config:

{deps, [
    {macula_neuroevolution_esdb, "~> 0.1.0"}
]}.

Usage

Initialize the Backend

Config = #{store_id => my_lineage_store},
{ok, State} = esdb_lineage_backend:init(Config).

Persist Lineage Events (Fire-and-Forget)

%% Single event - returns immediately
Event = #{
    event_type => offspring_born,
    individual_id => <<"ind-001">>,
    parent_ids => [<<"ind-000">>],
    generation => 1
},
ok = esdb_lineage_backend:persist_event(Event, State).

%% Batch of events - returns immediately
Events = [
    #{event_type => fitness_evaluated, individual_id => <<"ind-001">>, fitness => 0.85},
    #{event_type => mutation_applied, individual_id => <<"ind-001">>, mutation_type => weight_perturb}
],
ok = esdb_lineage_backend:persist_batch(Events, State).

Read Events (For Recovery Only)

%% WARNING: This blocks! Only use for recovery/replay, not during evolution.
StreamId = <<"individual-ind-001">>,
Opts = #{from => 0, limit => 100, direction => forward},
{ok, Events} = esdb_lineage_backend:read_stream(StreamId, Opts, State).

Subscribe to Events

%% Subscribe to individual's events (for projections)
StreamId = <<"individual-ind-001">>,
ok = esdb_lineage_backend:subscribe(StreamId, self(), State).

%% Receive events
receive
    {lineage_event, StreamId, Event} ->
        handle_event(Event)
end.

Stream Routing

Events are automatically routed to streams based on entity type:

Event TypesStream Pattern
Birth, death, fitness, mutationsindividual-{id}
Speciation, lineage divergencespecies-{id}
Generation, capacity eventspopulation-{id}
Coalition lifecyclecoalition-{id}

Supported Events

Individual Events

  • offspring_born, pioneer_spawned, clone_produced, immigrant_arrived
  • individual_culled, lifespan_expired, individual_perished
  • individual_matured, fertility_waned
  • fitness_evaluated, fitness_improved, fitness_declined, champion_crowned
  • mutation_applied, neuron_added, neuron_removed
  • connection_added, connection_removed, weight_perturbed
  • knowledge_transferred, skill_imitated, behavior_cloned
  • weights_grafted, structure_seeded, mentor_assigned, mentorship_concluded
  • mark_acquired, mark_inherited, mark_decayed

Species Events

  • lineage_diverged, species_emerged, lineage_ended, lineage_merged

Population Events

  • generation_completed, population_initialized, population_terminated
  • stagnation_detected, breakthrough_achieved
  • carrying_capacity_reached, catastrophe_occurred

Coalition Events

  • coalition_formed, coalition_dissolved, coalition_joined

Dependencies

License

Apache License 2.0