EfficientNet - Compound Scaling of Neural Networks.
EfficientNet uses a compound scaling method that uniformly scales depth, width, and resolution with a set of fixed scaling coefficients. The core building block is the MBConv (Mobile Inverted Bottleneck Convolution) with squeeze-and-excitation attention.
Since this library targets 1D feature vectors (not images), we implement a simplified dense-layer version of EfficientNet's architecture:
- MBConv blocks use inverted residual structure with dense layers
- Squeeze-Excitation attention operates on feature channels
- Compound scaling adjusts depth and width
Architecture
Input [batch, input_dim]
|
v
+--------------------------------------+
| Stem Dense |
+--------------------------------------+
|
v
+--------------------------------------+
| MBConv Block 1: |
| Expand -> Transform -> SE -> Proj |
| + Residual |
+--------------------------------------+
| (repeat, scaled by depth_mult)
v
+--------------------------------------+
| Head Dense + Classifier |
+--------------------------------------+
|
v
Output [batch, num_classes or last_dim]Usage
model = EfficientNet.build(
input_dim: 256,
base_dim: 32,
depth_multiplier: 1.0,
width_multiplier: 1.0,
num_classes: 10
)References
- Tan & Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks" (ICML 2019)
- https://arxiv.org/abs/1905.11946
Summary
Functions
Build an EfficientNet-style model.
MBConv (Mobile Inverted Bottleneck) block with Squeeze-Excitation.
Get the output size of an EfficientNet model.
Squeeze-Excitation attention block.
Types
@type build_opt() :: {:base_dim, pos_integer()} | {:depth_multiplier, pos_integer()} | {:dropout, float()} | {:input_dim, pos_integer()} | {:num_classes, pos_integer() | nil} | {:width_multiplier, pos_integer()}
Options for build/1.
Functions
Build an EfficientNet-style model.
Options
:input_dim- Input feature dimension (required):base_dim- Base stem dimension (default: 32):depth_multiplier- Depth scaling factor (default: 1.0):width_multiplier- Width scaling factor (default: 1.0):num_classes- If provided, adds classifier head (default: nil):dropout- Dropout rate before classifier (default: 0.0)
Returns
An Axon model: [batch, input_dim] -> [batch, last_dim or num_classes]
@spec mbconv_block(Axon.t(), pos_integer(), keyword()) :: Axon.t()
MBConv (Mobile Inverted Bottleneck) block with Squeeze-Excitation.
Options
:expand_ratio- Expansion ratio for inverted bottleneck (default: 6):se_ratio- Squeeze-Excitation reduction ratio (default: 0.25):name- Layer name prefix
@spec output_size(keyword()) :: pos_integer()
Get the output size of an EfficientNet model.
@spec squeeze_excitation(Axon.t(), pos_integer(), keyword()) :: Axon.t()
Squeeze-Excitation attention block.
Computes channel attention weights by squeezing spatial information and exciting (re-weighting) channels based on their importance.