Multi-Layer Perceptron (feedforward neural network).
The simplest deep learning architecture - stacked dense layers with nonlinear activations. Despite their simplicity, MLPs remain effective for many tasks and serve as building blocks in more complex architectures.
Architecture
Input [batch, input_size]
|
v
+-------------------+
| Dense + Act + Drop| Layer 1
+-------------------+
|
v
+-------------------+
| Dense + Act + Drop| Layer 2
+-------------------+
|
v
Output [batch, last_hidden_size]Options
:input_size- Input feature dimension (required for standalone build):hidden_sizes- List of hidden layer sizes (default: [512, 512]):activation- Activation function (default: :relu):dropout- Dropout rate (default: 0.1):layer_norm- Apply layer normalization (default: false):residual- Add residual/skip connections (default: false)
Usage
# Standalone model
model = MLP.build(input_size: 256, hidden_sizes: [512, 256])
# As a backbone from an existing input
backbone = MLP.build_backbone(input, [512, 256], :relu, 0.1)Summary
Functions
Build a standalone MLP model.
Build an MLP backbone from an existing Axon input layer.
Build a temporal MLP that processes sequences by taking the last frame.
Get the output size of an MLP with the given hidden sizes.
Types
@type build_opt() :: {:input_size, pos_integer()} | {:hidden_sizes, [pos_integer()]} | {:activation, atom()} | {:dropout, float()} | {:layer_norm, boolean()} | {:residual, boolean()}
Options for build/1.
Functions
Build a standalone MLP model.
Options
:input_size- Input dimension (required):hidden_sizes- List of hidden layer sizes (default: [512, 512]):activation- Activation function atom (default: :relu):dropout- Dropout rate (default: 0.1):layer_norm- Apply layer normalization after each dense layer (default: false):residual- Add residual connections between layers (default: false)
Returns
An Axon model outputting [batch, last_hidden_size].
Build an MLP backbone from an existing Axon input layer.
This is the core builder used by both build/1 and build_temporal/1.
Options
:layer_norm- Apply layer normalization (default: false):residual- Add residual connections (default: false)
Build a temporal MLP that processes sequences by taking the last frame.
Useful for single-frame processing in temporal pipelines.
Options
:embed_dim- Input embedding size per frame (required):hidden_sizes- List of hidden layer sizes (default: [512, 512]):activation- Activation function (default: :relu):dropout- Dropout rate (default: 0.1):window_size- Expected sequence length (default: 60)
@spec output_size(keyword()) :: non_neg_integer()
Get the output size of an MLP with the given hidden sizes.