DDIM: Denoising Diffusion Implicit Models.

Implements DDIM from "Denoising Diffusion Implicit Models" (Song et al., ICLR 2021). Uses the same training objective as DDPM but enables deterministic sampling with far fewer steps via a non-Markovian reverse process.

Key Innovation: Deterministic Sampling

DDPM requires ~1000 steps because each step adds stochastic noise. DDIM reformulates the reverse process as deterministic:

DDPM (stochastic, ~1000 steps):
  x_{t-1} = mu(x_t, eps_theta) + sigma_t * z,  z ~ N(0, I)

DDIM (deterministic, ~50 steps):
  x_{t-1} = sqrt(alpha_{t-1}) * pred_x0 + sqrt(1-alpha_{t-1}) * pred_dir
  where:
    pred_x0 = (x_t - sqrt(1-alpha_t) * eps_theta) / sqrt(alpha_t)
    pred_dir = sqrt(1-alpha_{t-1}) * eps_theta

The eta parameter interpolates between deterministic (eta=0) and stochastic DDPM (eta=1).

Architecture

Uses the same denoising network as DDPM (noise predictor conditioned on timestep and observations). The difference is in sampling, not training.

Same training as DDPM:
  1. Sample x_0, noise, timestep
  2. Compute noisy x_t
  3. Predict noise: eps_theta(x_t, t, obs)
  4. Loss: MSE(eps, eps_theta)

DDIM sampling (fewer steps):
  1. Choose stride S (e.g., skip every 20 steps)
  2. For t in [1000, 980, 960, ..., 20, 0]:
     x_{t-S} = ddim_step(x_t, t, eps_theta)
  3. Return x_0

Usage

# Build denoising network (same as DDPM)
model = DDIM.build(
  obs_size: 287,
  action_dim: 64,
  action_horizon: 8,
  num_steps: 1000
)

# DDIM sampling with fewer steps
schedule = DDIM.make_schedule(num_steps: 1000)
actions = DDIM.ddim_sample(params, predict_fn, obs, noise,
  schedule: schedule,
  ddim_steps: 50,
  eta: 0.0
)

Reference

• Paper: "Denoising Diffusion Implicit Models"
• arXiv: https://arxiv.org/abs/2010.02502