1. Continuous embedding approach (Diffusion-LM; Li et al., 2022): Map tokens to continuous embeddings, run Gaussian diffusion there, then round back to tokens
2. Discrete diffusion (D3PM; Austin et al., 2021): Replace Gaussian noise with discrete corruption — randomly changing tokens to other tokens or to [MASK]

The discrete approach turns out to be simpler, more effective, and connects beautifully to something you already know.

Instead of adding Gaussian noise, the forward process randomly masks tokens:

• At each step, each unmasked token has a small probability of being replaced with [MASK]
• As $t$ increases, more tokens become masked
• At $t = T$: every token is masked — the sequence is pure noise (all [MASK])
• At $t = 0$: the original clean text

Rao-Blackwellization is a statistical technique: when estimating something, if you can compute part of the answer exactly instead of approximating it, your overall estimate becomes more precise. In diffusion training, this means replacing some approximation steps with exact calculations — leading to less noisy gradients and faster learning.

To generate text with MDLM, we reverse the masking:

1. Start with a sequence of all [MASK] tokens (length $N$)
2. At each step, the model predicts a probability distribution over the vocabulary for each masked position
3. Unmask a subset of positions by sampling from these distributions
4. Repeat until all positions are unmasked

D3PM (Discrete Denoising Diffusion Probabilistic Models) provides the theoretical foundation for all discrete diffusion. Instead of Gaussian noise, D3PM uses transition matrices $\mathbf{Q}_t$ that define how tokens corrupt at each step:

$$q(\mathbf{x}_t \mid \mathbf{x}_{t-1}) = \text{Cat}(\mathbf{x}_t;\; \mathbf{x}_{t-1} \mathbf{Q}_t)$$

where $\text{Cat}$ is the categorical distribution and $\mathbf{Q}_t$ specifies the probability of each token transitioning to any other token.

LLaDA asks: can discrete diffusion compete with autoregressive models at scale? They trained an 8 billion parameter masked diffusion model on 2.3 trillion tokens and found:

• On MMLU, LLaDA-8B (66.6) matches LLaMA 3-8B (66.2)
• On GSM8K (math reasoning), LLaDA-8B (74.0) substantially outperforms LLaMA 3-8B (52.8)
• LLaDA handles instruction following, reasoning, and long-form generation
• It naturally solves the reversal curse — since it doesn't generate left-to-right, it can answer "Who is A's mother?" even when trained on "B is the mother of A"

Before discrete diffusion matured, Diffusion-LM tried a different approach: run Gaussian diffusion in continuous embedding space:

1. Map each token to its embedding vector (e.g., 768-dim)
2. Run standard Gaussian diffusion on these embeddings
3. At generation time, round each denoised embedding back to the nearest vocabulary token

1. The editing analogy: A first draft with blanks is easier to improve than starting from scratch. Each unmasking step gives the model more context, making the remaining predictions easier.

2. The jigsaw puzzle analogy: In a jigsaw puzzle, each piece you place constrains where other pieces go. Similarly, each unmasked token constrains the remaining tokens — the problem gets easier as you go.

3. The ensemble effect: At each step, the model makes predictions using bidirectional context from all visible tokens. Early steps leverage global structure; later steps leverage local details. This multi-scale reasoning is hard for autoregressive models.

• RLHF (Lecture 16): Reinforcement Learning from Human Feedback — humans rank model outputs, and the model is trained to prefer higher-ranked responses
• DPO (Direct Preference Optimization): A simpler alternative to RLHF that skips the reward model and directly optimizes the language model on human preference pairs
• KV-caching: During autoregressive generation, previously computed key/value vectors are stored and reused so each new token only requires one forward pass through the new position — not the entire sequence