DenseNet (Densely Connected Convolutional Network) implementation.
DenseNet connects each layer to every other layer in a feed-forward fashion. Within a dense block, each layer receives the feature maps of all preceding layers as input (via concatenation), encouraging feature reuse and reducing parameter count compared to traditional CNNs.
Architecture
Input [batch, H, W, C]
|
+-----v----------+
| Stem | 7x7 conv stride 2, BN, ReLU, 3x3 max pool
+--+--------------+
|
+--v--------------+
| Dense Block 1 | Each layer concatenates all previous feature maps
+--+--------------+
|
+--v--------------+
| Transition 1 | 1x1 conv (compress) + 2x2 avg pool (downsample)
+--+--------------+
|
+--v--------------+
| Dense Block 2 |
+--+--------------+
|
... (repeat)
|
+--v--------------+
| Final BN + ReLU |
+--+--------------+
|
+--v--------------+
| Global AvgPool |
+--+--------------+
|
+--v--------------+
| Dense | num_classes outputs
+-----------------+Configurations
| Model | block_config | growth_rate | Params |
|---|---|---|---|
| DenseNet-121 | [6, 12, 24, 16] | 32 | ~8M |
| DenseNet-169 | [6, 12, 32, 32] | 32 | ~14M |
| DenseNet-201 | [6, 12, 48, 32] | 32 | ~20M |
| DenseNet-264 | [6, 12, 64, 48] | 32 | ~34M |
Usage
# DenseNet-121 for CIFAR-10
model = DenseNet.build(
input_shape: {nil, 32, 32, 3},
num_classes: 10,
growth_rate: 32,
block_config: [6, 12, 24, 16]
)
# Compact DenseNet for small datasets
model = DenseNet.build(
input_shape: {nil, 32, 32, 3},
num_classes: 10,
growth_rate: 12,
block_config: [4, 8, 12, 8],
compression: 0.5
)Summary
Functions
Build a DenseNet model.
Build a dense block where each layer receives all previous feature maps.
Get the output size (num_classes) for a DenseNet model.
Build a transition layer between dense blocks.
Types
@type build_opt() :: {:block_config, [pos_integer()]} | {:bn_size, pos_integer()} | {:compression, float()} | {:dropout, float()} | {:growth_rate, pos_integer()} | {:initial_channels, pos_integer()} | {:input_shape, tuple()} | {:num_classes, pos_integer() | nil}
Options for build/1.
Functions
Build a DenseNet model.
Options
:input_shape- Input shape as{nil, height, width, channels}(required):num_classes- Number of output classes (default: 10):growth_rate- Number of new feature maps per dense layer (default: 32):block_config- List of layer counts per dense block (default: [6, 12, 24, 16]):compression- Compression factor for transitions, 0.0-1.0 (default: 0.5):initial_channels- Channels after stem conv (default: growth_rate * 2):dropout- Dropout rate in dense layers (default: 0.0):bn_size- Bottleneck width multiplier for BN-ReLU-1x1 layers (default: 4)
Returns
An Axon model outputting [batch, num_classes].
@spec dense_block(Axon.t(), pos_integer(), keyword()) :: {Axon.t(), pos_integer()}
Build a dense block where each layer receives all previous feature maps.
Within a dense block, layer i receives the concatenation of feature maps
from layers 0, 1, ..., i-1 as input. Each layer produces growth_rate
new feature maps.
Parameters
input- Input Axon node[batch, H, W, C]num_layers- Number of dense layers in this block
Options
:growth_rate- New feature maps per layer (default: 32):num_channels- Current number of input channels (required for tracking):dropout- Dropout rate (default: 0.0):bn_size- Bottleneck width multiplier (default: 4):name- Layer name prefix (default: "dense_block")
Returns
Tuple of {output_node, total_channels} where total_channels is
num_channels + num_layers * growth_rate.
@spec output_size(keyword()) :: pos_integer()
Get the output size (num_classes) for a DenseNet model.
@spec transition_layer(Axon.t(), pos_integer(), keyword()) :: Axon.t()
Build a transition layer between dense blocks.
Transitions reduce spatial dimensions (2x downsampling via average pooling) and optionally compress the number of feature maps with a 1x1 convolution.
Parameters
input- Input Axon node[batch, H, W, C]out_channels- Number of output channels after compression
Options
:name- Layer name prefix (default: "transition")
Returns
An Axon node with shape [batch, H/2, W/2, out_channels].