How Modern Generative Models Actually Train & Generate
From the sequential token bottlenecks of Autoregressive LLMs to parallel coarse-to-fine Next-Concept Modeling (NCM) and continuous Flow Matching. Learn through synchronized visual animations, mathematical formulations, and PyTorch pseudocode.
Explore by Modality
Each modality poses unique dimensional and computational constraints—from discrete symbolic text tokens to continuous pixel manifolds and acoustic waveforms.
Text & Language
(3 algorithms available)Explore how language models evolve from sequential next-token prediction (LLMs) to parallel coarse-to-fine concept hierarchies (NCM) and discrete masked diffusion.
Autoregressive Transformer (LLM)
Parallel teacher-forced causal training; sequential KV-cached token decoding
Next-Concept Modeling (NCM)
Multiscale Residual VQ hierarchy + level-parallel coarse-to-fine text generation
Masked Diffusion Language Model (MDLM / LLaDA)
Continuous-time discrete token masking + iterative bidirectional unmasking
Image & Vision
(2 algorithms available)See how visual generators transform Gaussian noise into coherent scenes via iterative score denoising, straight ODE flow trajectories, or next-scale VQ pyramids.
Denoising Diffusion Probabilistic Models (DDPM)
Iterative Gaussian score denoising reversing a Markov corruption chain
Flow Matching / Rectified Flow
Straight-line ODE probability transport between Gaussian noise and data
Audio & Speech
(0 algorithms available)Understand how neural audio codecs factorize 24kHz+ waveforms into semantic and coarse-to-fine acoustic Residual VQ layers.
Video & World
(0 algorithms available)Visualize how 3D Variational Autoencoders and Diffusion Transformers denoise spacetime patches jointly across frames.
Multimodal
(0 algorithms available)Inspect how early-fusion models tokenize or project multiple modalities into one shared sequence with modality-aware attention masks.
The Generative Paradigm Race
Why are labs moving away from pure autoregressive left-to-right generation? Compare Autoregressive LLMs, Next-Concept Modeling (NCM), Masked Discrete Diffusion, and Continuous Flow Matching side-by-side to understand trade-offs in inference passes, compute, and parallelizability.