MobileNet - Depthwise Separable Convolutions for Efficient Inference.
MobileNet uses depthwise separable convolutions to build lightweight models suitable for mobile and edge deployment. A depthwise separable convolution factorizes a standard convolution into a depthwise convolution (per-channel) and a pointwise (1x1) convolution, reducing computation by a factor of approximately 1/output_channels + 1/kernel_size^2.
Since Axon's convolution support focuses on sequence/image data and this library targets 1D feature vectors, we approximate the MobileNet architecture using dense layers with a depthwise-separable structure:
- "Depthwise": per-group dense transform (group-wise linear)
- "Pointwise": standard dense layer (channel mixing)
Architecture
Input [batch, input_dim]
|
v
+--------------------------------------+
| Stem: Dense + BN + ReLU6 |
+--------------------------------------+
|
v
+--------------------------------------+
| Depthwise Separable Block 1: |
| Depthwise: group-wise dense |
| Pointwise: 1x1 dense |
| + BatchNorm + ReLU6 |
+--------------------------------------+
| (repeat for each hidden_dim)
v
+--------------------------------------+
| Global Average Pool + Classifier |
+--------------------------------------+
|
v
Output [batch, num_classes or last_dim]Usage
model = MobileNet.build(
input_dim: 256,
hidden_dims: [64, 128, 256],
width_multiplier: 1.0,
num_classes: 10
)References
- Howard et al., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications" (2017)
- https://arxiv.org/abs/1704.04861
Summary
Functions
Build a MobileNet-style model with depthwise separable dense layers.
Depthwise separable block: group-wise dense + pointwise dense.
Get the output size of a MobileNet model.
Types
@type build_opt() :: {:activation, atom()} | {:dropout, float()} | {:hidden_dims, pos_integer()} | {:input_dim, pos_integer()} | {:num_classes, pos_integer() | nil} | {:width_multiplier, pos_integer()}
Options for build/1.
Functions
Build a MobileNet-style model with depthwise separable dense layers.
Options
:input_dim- Input feature dimension (required):hidden_dims- List of channel dimensions (default: [64, 128, 256]):width_multiplier- Channel width scaling factor (default: 1.0):num_classes- If provided, adds classifier head (default: nil):activation- Activation function (default: :relu6):dropout- Dropout rate before classifier (default: 0.0)
Returns
An Axon model: [batch, input_dim] -> [batch, last_dim or num_classes]
@spec depthwise_separable_block(Axon.t(), pos_integer(), keyword()) :: Axon.t()
Depthwise separable block: group-wise dense + pointwise dense.
Options
:activation- Activation function (default: :relu6):name- Layer name prefix
@spec output_size(keyword()) :: pos_integer()
Get the output size of a MobileNet model.