mlgenerate.comVisual Generative ML
Start Learning
All Models/text/Next-Concept (NCM)

Next-Concept Modeling (NCM)

Coarse-to-Fine VQ2026 · Stanford Hazy Research & Together AI

Multiscale Residual VQ hierarchy + level-parallel coarse-to-fine text generation

Execution Paradigm:Coarse-to-Fine Level-Parallel Generation (O(L) Steps)
O(L) Coarse-to-Fine Passes (All Positions in Level Parallel)
Interactive Stage (INFERENCE)All Level-1 macro-concept slots fire in parallel on a short compressed sequence.
NCM Inference: Parallel Coarse-to-Fine HierarchyFixed L = 3 Passes vs O(T) Token Steps

Instead of generating tokens one by one in 8–500 serial passes, NCM generates an entire response in only 3 hierarchy passes. Within each pass, all horizontal positions are predicted in parallel.

Pass 1: Level 1 (Macro Concepts)
✓ Synthesized in Parallel
[Topic: AI]
[Tone: Academic]
2Pass 2: Level 2 (Mid-Scale Semantics)
Awaiting previous level
...
...
...
...
3Pass 3: Level 3 (Fine Word Details)
Awaiting previous level
...
...
...
...
...
...
...
...
Final Surface Text Decoded via VQ Decoder:
Refining hierarchy (Pass 1 of 3)...
Step 1 of 3

1. Level 1 — Parallel Macro-Concept Skeleton

Predict all coarse Level-1 concept codes c_1 = [c_{1,1} .. c_{1,T/4}] in parallel. Because Level 1 is spatially compressed (T/4 length), this initial global planning pass requires minimal compute.

💡 Takeaway: Locks in the high-level semantic trajectory of the entire response before choosing exact surface words.

Active Tensor Dimensions & Memory Shapes

c_1[B, T/4]
Sampled macro-concept indices
z_acc[B, T, D]
Upsampled Level-1 concept canvas
generate_ncm_parallel.py
1
def generate_ncm(prompt_cond, ncm_model, vq_decoder, num_levels=4):
2
z_accum = 0; hierarchy_codes = []
3
for level in range(num_levels): # Only L steps total (L << T)!
4
# Predict ALL positions at this concept scale in parallel
5
logits_l = ncm_model.forward_level(prompt_cond, hierarchy_codes, level)
6
codes_l = sample_parallel(logits_l) # [B, T_l] sampled at once
7
z_accum = z_accum + upsample(codebook_lookup(codes_l, level))
8
hierarchy_codes.append(codes_l)
9
# Decode summed multiscale concepts into final surface tokens
10
return vq_decoder.decode_to_tokens(z_accum) # [B, T]
Click code line to jump to corresponding stepPyTorch Reference
Why This Architecture Matters

Breaks the O(T) sequential latency barrier of Autoregressive LLMs while avoiding the full-sequence compute waste and quality drop of few-step Diffusion Language Models—achieving up to 69% faster inference than AR baselines.

Runtime Complexity & Bottlenecks

Training Objective:Stage 1: RVQ Reconstruction + Commitment Loss | Stage 2: Level-Parallel Cross-Entropy

Inference Bottleneck:O(L) hierarchy passes (L ≪ T, parallel within each level)

Foundational & Breakthrough Papers for Next-Concept Modeling (NCM)

3 Curated References
arXiv:2610.14162 · 2026

Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

Stanford Hazy Research & Together AI

Core Contribution:Introduced two-stage multiscale Residual VQ concept tokenization and level-parallel coarse-to-fine language generation.
NeurIPS 2024 Best Paper (arXiv:2404.02905) · 2024

Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (VAR)

Tian et al. (Peking University / ByteDance)

Core Contribution:Established multiscale residual VQ next-scale prediction in vision, inspiring hierarchical coarse-to-fine generation.
NeurIPS 2017 (arXiv:1711.00937) · 2017

Neural Discrete Representation Learning (VQ-VAE)

van den Oord, Vinyals, Kavukcuoglu (DeepMind)

Core Contribution:Foundational vector quantization bottleneck mapping continuous embeddings to discrete codebooks.
PreviousAutoregressive LLM
Next AlgorithmMasked Diffusion LM