All Models/image/DDPM Diffusion

Denoising Diffusion Probabilistic Models (DDPM)

Diffusion2020 · UC Berkeley (Ho et al.)

Iterative Gaussian score denoising reversing a Markov corruption chain

Execution Paradigm:Iterative Reverse Denoising Sampling (O(K) Forward Passes)

Iterative Denoising Sequence (Full Canvas Parallel per Step)

Interactive Stage (INFERENCE)Canvas is 100% random Gaussian white noise.

DDPM Reverse Denoising SamplingNoise Factor: 100%

Starting from random Gaussian white noise, the neural network predicts and subtracts the noise component over iterative steps.

Simulated 8×8 Latent Patch (x_t)

Signal √ᾱ_t: 0%Noise √(1 - ᾱ_t): 100%

State Equation & Score Function

x_{t-1} = μ_θ(x_t, t) + σ_t · z

Loss: L_simple = ||ε - ε_θ(x_t, t)||^2

Forward Pass:O(K) Iterative Steps

Network Architecture:U-Net or Diffusion Transformer (DiT)

Step 1 of 3

1. Sample Pure Gaussian White Noise x_T

Draw initial state x_T ~ 𝒩(0, I) from standard normal distribution. At this point, the canvas contains zero discernible structural signal.

💡 Takeaway: The generator begins completely unconstrained in the high-dimensional noise space.

Active Tensor Dimensions & Memory Shapes

x_T[B, C, H, W]

Initial white noise canvas

sample_ddpm_reverse.py

def sample_ddpm(model, shape, T):

x = torch.randn(shape) # x_T pure Gaussian white noise

for t in reversed(range(1, T + 1)):

t_tensor = torch.full((shape[0],), t, dtype=torch.long)

eps_pred = model(x, t_tensor)

z = torch.randn_like(x) if t > 1 else 0

x = compute_x_prev(x, eps_pred, t, z) # Denoise 1 step

return x # x_0 clean image

Click code line to jump to corresponding stepPyTorch Reference

Why This Architecture Matters

Overcame the training instability and mode collapse of GANs, serving as the foundational mathematical engine behind Stable Diffusion, Midjourney, and DALL-E 2/3.

Runtime Complexity & Bottlenecks

Training Objective:Simplified L_simple MSE Loss: 𝔼_{t, x_0, ε}[ ||ε - ε_θ(x_t, t)||^2 ]

Inference Bottleneck:O(K) full-image U-Net forward passes (typically 20–1000 steps)

Foundational & Breakthrough Papers for Denoising Diffusion Probabilistic Models (DDPM)

3 Curated References

NeurIPS 2020 (arXiv:2006.11239) · 2020

Denoising Diffusion Probabilistic Models

Jonathan Ho, Ajay Jain, Pieter Abbeel (UC Berkeley)

Core Contribution:Proved equivalence to score matching and introduced the simple ε-prediction MSE objective.

CVPR 2022 (arXiv:2112.10752) · 2022

High-Resolution Image Synthesis with Latent Diffusion Models

Rombach et al. (LMU Munich / Runway)

Core Contribution:Moved diffusion from high-resolution pixel space to compressed VAE latent space (Stable Diffusion).

ICCV 2023 (arXiv:2212.09748) · 2023

Scalable Diffusion Models with Transformers (DiT)

Peebles & Xie (UC Berkeley / NYU)

Core Contribution:Replaced convolutional U-Net with ViT-based Diffusion Transformers, proving compute scaling laws.

PreviousMasked Diffusion LM

Next AlgorithmFlow Matching