NeRF: Neural Radiance Fields network (Mildenhall et al., 2020).

Maps 3D coordinates (and optionally viewing directions) to color and density values using Fourier positional encoding followed by an MLP with skip connections. This is the core network architecture used in Neural Radiance Fields for novel view synthesis.

Architecture

Coordinates [batch, 3]
      |
+-----v--------------------+
| Fourier Encoding          |  gamma(p) = [p, sin(2^0*pi*p), cos(2^0*pi*p), ...]
+---------------------------+
      |
      v
[batch, 3 * (2*L + 1)]
      |
+-----v--------------------+
| Dense Layer 1             |  ReLU
| Dense Layer 2             |  ReLU
| ...                       |
| Dense Layer K (skip_layer)|  Concatenate encoded input, ReLU
| ...                       |
| Dense Layer N             |  ReLU
+---------------------------+
      |
      +---> Density sigma [batch, 1]
      |
      +---> Feature -> concat(directions_encoded) -> Dense -> RGB [batch, 3]
      |
      v
Output [batch, 4]  (RGB + density)

Differences from Vision Models

Unlike other vision models in Edifice, NeRF does not take image inputs. Instead, it takes raw 3D coordinates and optional viewing directions, making it fundamentally a coordinate-to-color mapping network.

Usage

# With viewing directions
model = NeRF.build(
  hidden_size: 256,
  num_layers: 8,
  skip_layer: 4,
  num_frequencies: 10,
  use_viewdir: true
)

# Without viewing directions
model = NeRF.build(use_viewdir: false)

References

• Mildenhall et al., "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis" (ECCV 2020)
• https://arxiv.org/abs/2003.08934