Overview

This release refactors macula-tweann into a true library by extracting domain-specific morphologies into optional examples. The focus is on making the library generic, extensible, and suitable for embedding in applications like macula-arcade.

Phase: Refactoring Duration: 1-2 weeks Prerequisites: v0.8.8 (documentation and diagram restructuring complete)

Objectives

1. Extract morphologies from library core - Move domain-specific morphologies to examples
2. Create morphology behavior/callback interface - Generic API for custom morphologies
3. Add morphology registration system - Runtime registration of external morphologies
4. Reorganize project structure - Library code vs examples vs tests
5. Update documentation - Guide for implementing custom morphologies
6. Prepare for macula-arcade integration - Demonstrate snake game morphology

Architecture Changes

Current Problem

macula-tweann/src/morphology.erl contains domain-specific code:

% These are APPLICATION code, not LIBRARY code!
get_sensors(xor_mimic) -> ...
get_sensors(pole_balancing) -> ...
get_sensors(forex_trader) -> ...
get_sensors(flatland) -> ...

This couples the library to specific problem domains. Applications must modify library code to add new morphologies.

New Architecture

1. Generic Morphology Behavior

% src/morphology_behaviour.erl
-module(morphology_behaviour).

-callback get_sensors(morphology_name()) -> [sensor_spec()].
-callback get_actuators(morphology_name()) -> [actuator_spec()].
-callback get_substrate_cpps(morphology_name()) -> substrate_cpps().
-callback get_substrate_cep_ranges(morphology_name()) -> cep_ranges().

-type morphology_name() :: atom().
-type sensor_spec() :: #sensor{}.
-type actuator_spec() :: #actuator{}.
-type substrate_cpps() :: list().
-type cep_ranges() :: list().

2. Morphology Registry

% src/morphology_registry.erl
-module(morphology_registry).
-export([register/2, unregister/1, get/1, list_all/0]).

%% Register external morphology module
-spec register(atom(), module()) -> ok | {error, term()}.
register(MorphologyName, Module) ->
    case code:ensure_loaded(Module) of
        {module, Module} ->
            % Verify module implements morphology_behaviour
            case lists:member(morphology_behaviour, Module:module_info(attributes)) of
                true ->
                    ets:insert(morphology_registry, {MorphologyName, Module}),
                    ok;
                false ->
                    {error, not_a_morphology_module}
            end;
        {error, Reason} ->
            {error, Reason}
    end.

%% Get morphology module
-spec get(atom()) -> {ok, module()} | {error, not_found}.
get(MorphologyName) ->
    case ets:lookup(morphology_registry, MorphologyName) of
        [{MorphologyName, Module}] -> {ok, Module};
        [] -> {error, not_found}
    end.

%% List all registered morphologies
-spec list_all() -> [atom()].
list_all() ->
    [Name || {Name, _Module} <- ets:tab2list(morphology_registry)].

3. Updated morphology.erl (Library Core)

% src/morphology.erl (REFACTORED - now generic)
-module(morphology).
-export([get_sensors/1, get_actuators/1, ...]).

%% Delegate to registered morphology module
get_sensors(MorphologyName) ->
    case morphology_registry:get(MorphologyName) of
        {ok, Module} ->
            Module:get_sensors(MorphologyName);
        {error, not_found} ->
            error({morphology_not_found, MorphologyName})
    end.

get_actuators(MorphologyName) ->
    case morphology_registry:get(MorphologyName) of
        {ok, Module} ->
            Module:get_actuators(MorphologyName);
        {error, not_found} ->
            error({morphology_not_found, MorphologyName})
    end.

New Project Structure

macula-tweann/
  src/                          # Core library (PUBLISHED)
    genotype.erl
    neuron.erl
    morphology_behaviour.erl    # NEW: Behavior definition
    morphology_registry.erl     # NEW: Registry for external morphologies
    morphology.erl              # REFACTORED: Generic delegator
    ...

  include/
    records.hrl
    morphology_behaviour.hrl    # NEW: Behavior types and specs

  examples/                     # Example morphologies (NOT compiled by default)
    xor/
      README.md                 # Problem description
      src/
        morphology_xor.erl      # MOVED from src/morphology.erl
      test/
        xor_test.erl

    pole_balancing/
      README.md
      src/
        morphology_pole_balancing.erl
      test/
        pole_balancing_test.erl

    forex/
      README.md
      src/
        morphology_forex.erl
      test/
        forex_test.erl

    flatland/
      README.md
      src/
        morphology_flatland.erl
      test/
        flatland_test.erl

  test/                         # Core library tests
    genotype_test.erl
    neuron_test.erl
    morphology_registry_test.erl  # NEW: Test registration

  guides/
    custom-morphology.md        # NEW: How to implement morphology
    examples-guide.md           # NEW: How to use examples

  design_docs/
  rebar.config                  # Does NOT compile examples/ by default

rebar.config Changes

% Default profile: library only
{erl_opts, [
    debug_info,
    {i, "include"}
]}.

% Examples profile: compile examples
{profiles, [
    {examples, [
        {erl_opts, [{i, "include"}]},
        {extra_src_dirs, ["examples/*/src"]},
        {eunit_compile_opts, [{i, "examples"}]}
    ]},

    {test, [
        {erl_opts, [debug_info, {i, "include"}]},
        {extra_src_dirs, ["test", "examples/*/test"]}
    ]}
]}.

% Hex package: include examples but don't compile
{hex, [
    {doc, #{provider => ex_doc}},
    {files, [
        "src",
        "include",
        "examples",        % Include source
        "design_docs",
        "guides",
        "rebar.config",
        "README.md",
        "LICENSE"
    ]}
]}.

Implementation Tasks

1. Create Morphology Behavior (2 days)

Files to create:

• src/morphology_behaviour.erl - Behavior definition
• src/morphology_registry.erl - Registry implementation
• include/morphology_behaviour.hrl - Types and specs
• test/morphology_registry_test.erl - Registry tests

Acceptance Criteria:

• [ ] Behavior defined with callbacks
• [ ] Registry supports register/unregister/get/list_all
• [ ] Registry validates modules implement behavior
• [ ] Tests verify registration and lookup

2. Refactor morphology.erl (1 day)

Changes:

• Extract domain-specific functions to examples/
• Keep only generic delegation logic
• Update to use morphology_registry

Before:

get_sensors(xor_mimic) -> [#sensor{...}];
get_sensors(pole_balancing) -> [#sensor{...}];
% ... 200+ lines of domain code

After:

get_sensors(MorphologyName) ->
    case morphology_registry:get(MorphologyName) of
        {ok, Module} -> Module:get_sensors(MorphologyName);
        {error, not_found} -> error({morphology_not_found, MorphologyName})
    end.

Acceptance Criteria:

• [ ] morphology.erl delegates to registry
• [ ] No domain-specific code in src/morphology.erl
• [ ] All tests pass with refactored code

3. Extract Morphologies to Examples (3 days)

For each morphology (xor, pole_balancing, forex, flatland):

1. Create examples/<name>/ directory structure
2. Create examples/<name>/README.md with:
   • Problem description
   • How to run
   • Expected results
3. Move morphology code to examples/<name>/src/morphology_<name>.erl
4. Implement -behaviour(morphology_behaviour)
5. Create examples/<name>/test/<name>_test.erl

Example: examples/xor/src/morphology_xor.erl

-module(morphology_xor).
-behaviour(morphology_behaviour).

-export([get_sensors/1, get_actuators/1]).
-include("records.hrl").

get_sensors(xor_mimic) ->
    [#sensor{
        id = {{-1, generate_id()}, sensor},
        name = xor_input,
        type = standard,
        cortex_id = undefined,
        scape = {private, xor_sim},
        vector_length = 2,
        fanout_ids = [],
        generation = 0,
        format = no_geo,
        parameters = [xor]
    }];
get_sensors(_) ->
    error(invalid_morphology).

get_actuators(xor_mimic) ->
    [#actuator{
        id = {{1, generate_id()}, actuator},
        name = xor_output,
        type = standard,
        cortex_id = undefined,
        scape = {private, xor_sim},
        vector_length = 1,
        fanin_ids = [],
        generation = 0,
        format = no_geo,
        parameters = [xor]
    }];
get_actuators(_) ->
    error(invalid_morphology).

Example: examples/xor/README.md

# XOR Morphology Example

## Problem Description

The XOR (exclusive or) problem is a classic neural network benchmark. The goal is to evolve a network that can correctly classify all four XOR inputs:

| Input 1 | Input 2 | Output |
|---------|---------|--------|
| 0       | 0       | 0      |
| 0       | 1       | 1      |
| 1       | 0       | 1      |
| 1       | 1       | 0      |

## Running the Example

% Compile with examples profile rebar3 as examples compile

% Start shell rebar3 as examples shell

% Register morphology morphology_registry:register(xor_mimic, morphology_xor).

% Create and evolve agent genotype:init_db(), Constraint = #constraint{morphology = xor_mimic}, {ok, AgentId} = genotype:construct_agent(Constraint), genome_mutator:mutate(AgentId).

## Expected Results

After 50-100 generations, the best agent should achieve >95% accuracy on XOR classification.

Acceptance Criteria:

• [ ] All 4 morphologies extracted
• [ ] Each has README with problem description
• [ ] Each implements morphology_behaviour
• [ ] Examples compile with rebar3 as examples compile
• [ ] Examples DO NOT compile by default

4. Update Documentation (2 days)

New guides to create:

guides/custom-morphology.md

# Implementing a Custom Morphology

This guide shows how to create a custom morphology for your problem domain.

## Step 1: Define Your Problem

Identify:
- Sensor inputs (what does the network observe?)
- Actuator outputs (what actions can it take?)
- Fitness function (how do you measure success?)

## Step 2: Create Morphology Module

-module(my_morphology). -behaviour(morphology_behaviour). -export([get_sensors/1, get_actuators/1, ...]).

get_sensors(my_problem) ->

% Define sensors
[#sensor{...}].

get_actuators(my_problem) ->

% Define actuators
[#actuator{...}].

## Step 3: Register at Runtime

% In your application startup morphology_registry:register(my_problem, my_morphology).

## Step 4: Use in Agent Construction

Constraint = #constraint{morphology = my_problem}, {ok, AgentId} = genotype:construct_agent(Constraint).

See `examples/` directory for reference implementations.

guides/examples-guide.md

# Using Example Morphologies

The `examples/` directory contains reference implementations of morphologies for various problem domains.

## Available Examples

- **xor/** - XOR classification (classic benchmark)
- **pole_balancing/** - Cart-pole balancing (control task)
- **forex/** - Forex trading (time series prediction)
- **flatland/** - Prey/predator simulation (multi-agent)

## Running Examples

### Option 1: Compile All Examples

rebar3 as examples compile rebar3 as examples shell

### Option 2: Copy to Your Application

cp examples/xor/src/morphology_xor.erl my_app/src/

Add to your application's morphology registration

## Example Structure

Each example includes:
- `README.md` - Problem description and usage
- `src/morphology_<name>.erl` - Morphology implementation
- `test/<name>_test.erl` - Tests

Update existing docs:

• README.md - Add "Library Philosophy" section
• guides/overview.md - Link to custom morphology guide
• guides/architecture.md - Document morphology system

Acceptance Criteria:

• [ ] custom-morphology.md complete with examples
• [ ] examples-guide.md explains how to use examples
• [ ] README.md updated with library philosophy
• [ ] All guides link to morphology documentation

5. Update Tests (1 day)

Changes:

• Update tests to use registry
• Add tests for morphology_behaviour validation
• Add integration tests with example morphologies

Example test:

-module(morphology_integration_test).
-include_lib("eunit/include/eunit.hrl").

xor_morphology_registration_test() ->
    % Register example morphology
    ok = morphology_registry:register(xor_mimic, morphology_xor),

    % Verify registration
    {ok, morphology_xor} = morphology_registry:get(xor_mimic),

    % Test delegation
    Sensors = morphology:get_sensors(xor_mimic),
    ?assertEqual(1, length(Sensors)),

    % Cleanup
    ok = morphology_registry:unregister(xor_mimic).

invalid_morphology_test() ->
    % Unregistered morphology should error
    ?assertError(
        {morphology_not_found, nonexistent},
        morphology:get_sensors(nonexistent)
    ).

Acceptance Criteria:

• [ ] All core tests pass without examples
• [ ] Example tests pass with examples profile
• [ ] Integration tests verify registration workflow

Quality Gates

v0.9.0 Acceptance Criteria

1. Library Core

   • [ ] morphology_behaviour.erl defined
   • [ ] morphology_registry.erl implemented
   • [ ] morphology.erl refactored to delegate
   • [ ] No domain-specific code in src/

2. Examples

   • [ ] 4 morphologies extracted (xor, pole, forex, flatland)
   • [ ] Each has README and tests
   • [ ] Examples compile with rebar3 as examples compile
   • [ ] Examples DO NOT compile by default

3. Documentation

   • [ ] custom-morphology.md guide complete
   • [ ] examples-guide.md explains usage
   • [ ] README.md explains library philosophy
   • [ ] All examples have README

4. Tests

   • [ ] All core tests pass
   • [ ] Example tests pass
   • [ ] Integration tests verify registry
   • [ ] Zero dialyzer warnings

5. Hex Package

   • [ ] Examples included in tarball (source only)
   • [ ] Library compiles without examples
   • [ ] Documentation references examples

Benefits

For Library Users

✅ Clean separation - Library vs application code ✅ Extensibility - Add custom morphologies without forking ✅ Smaller footprint - No unused domain code ✅ Better examples - Reference implementations with docs

For Library Maintainers

✅ Focused scope - Library does one thing well ✅ Easier testing - Core tests independent of examples ✅ Lower maintenance - Examples can evolve separately ✅ Better architecture - Generic, reusable components

For Applications (e.g., macula-arcade)

✅ Self-contained - Snake morphology lives in arcade repo ✅ No library changes - Just register morphology at runtime ✅ Full control - Customize without affecting library ✅ Clean dependencies - Library has no game logic

macula-arcade Integration Example

After v0.9.0, integrating TWEANN with Snake:

1. Add dependency (macula-arcade/system/apps/macula_arcade/mix.exs):

defp deps do
  [
    {:macula_tweann, "~> 0.9.0"}
  ]
end

2. Implement morphology (macula-arcade/system/apps/macula_arcade/lib/tweann/snake_morphology.ex):

defmodule MaculaArcade.TWEANN.SnakeMorphology do
  @behaviour :morphology_behaviour

  def get_sensors(:snake_player) do
    # 7 inputs: food direction (2D), wall distance (4D), score (1D)
    [sensor(...)]
  end

  def get_actuators(:snake_player) do
    # 4 outputs: up, down, left, right
    [actuator(...)]
  end
end

3. Register at startup (macula_arcade/lib/application.ex):

def start(_type, _args) do
  # Register snake morphology
  :morphology_registry.register(:snake_player, MaculaArcade.TWEANN.SnakeMorphology)

  children = [...]
  Supervisor.start_link(children, strategy: :one_for_one)
end

4. Use in game (macula_arcade/lib/games/snake/ai_controller.ex):

# Evolve snake AI
:genotype.init_db()
constraint = %{morphology: :snake_player}
{:ok, agent_id} = :genotype.construct_agent(constraint)
{:ok, phenotype} = :constructor.construct_phenotype(agent_id)

No library modifications needed!

Migration Path

For Current Users

Before (v0.8.x):

% Morphology is hardcoded in library
Constraint = #constraint{morphology = xor_mimic},
{ok, AgentId} = genotype:construct_agent(Constraint).

After (v0.9.0):

% Register morphology first (if using examples)
morphology_registry:register(xor_mimic, morphology_xor),

% Then use as before
Constraint = #constraint{morphology = xor_mimic},
{ok, AgentId} = genotype:construct_agent(Constraint).

Breaking Changes:

• Applications using built-in morphologies must register them first
• Migration script provided in examples/migrate.erl

Migration Script

examples/migrate.erl:

-module(migrate).
-export([register_all_examples/0]).

%% Helper to register all example morphologies
register_all_examples() ->
    ok = morphology_registry:register(xor_mimic, morphology_xor),
    ok = morphology_registry:register(pole_balancing, morphology_pole_balancing),
    ok = morphology_registry:register(forex_trader, morphology_forex),
    ok = morphology_registry:register(flatland, morphology_flatland),
    ok.

Usage:

% Add to your application startup
application:ensure_all_started(macula_tweann),
migrate:register_all_examples().

Effort Estimate

Next Steps

After v0.9.0 completion:

1. Integrate with macula-arcade - Snake morphology demo
2. v1.0.0 - Performance optimization, production readiness
3. Mesh integration - Distributed evolution (requires macula core)

References

• Elixir behaviours: https://elixir-lang.org/getting-started/typespecs-and-behaviours.html
• Hex package best practices: https://hex.pm/docs/publish
• Library design patterns: "Designing for Scalability with Erlang/OTP"

Version: 0.9.0 Phase: Refactoring Status: Planned