Diffusion noise-prediction objective

Trains a denoising network to identify the exact Gaussian noise used to corrupt a clean sample at a randomly selected timestep.

The simple diffusion loss is the expectation over time, clean samples, and Gaussian noise of the squared two norm between the true noise epsilon and predicted noise epsilon theta of noisy x t and time t.

Show the model a noisy sample and noise level, then ask it to reconstruct the noise that was added.

Expectation is the outer reduction, while the squared norm defines the per-example regression error.

The DDPM derivation connects this simple denoising regression to a variational objective for the learned reverse diffusion process.