On stage at I/O last week Sundar Pichai called Gemini Diffusion an “experiment.” That is polite understatement. The new model abandons the token-by-token playbook that has defined every large language model since GPT-2 and instead treats language the way Imagen treats pixels: add noise, then learn to remove it. The result? A research prototype that spits out 1.5K tokens, edits itself mid-generation, and still matches Gemini 2.0 Flash-Lite on code accuracy (while beating it on latency by an order of magnitude).
The model, technically
Gemini Diffusion keeps a vanilla Transformer backbone for representation learning but replaces the autoregressive decoder with a discrete denoising scheduler. During inference the model starts from pure noise in embedding space and performs ~16 reverse-diffusion steps to produce a full block of tokens, then optionally repeats for longer contexts (if none of this makes sense to you, hang in there, we’ll get to that later).
Google’s own benchmark shows 1,479 tokens with <1s overhead. That is five-fold faster than Flash-Lite and roughly 40× GPT-4 Turbo’s public numbers. On HumanEval the model hits 89.6% pass @1, statistically tied with Flash-Lite, and it stays within single-digit deltas on every other listed bench except GPQA (science reasoning) where big-context retrieval still matters.
It’s live as of last week as a gated demo and SDK wait-list. Google calls it a research model, but the keynote ran live coding and Gmail reply demos without a hiccup.
How text diffusion works
Traditional diffusion adds Gaussian noise to continuous vectors, trains a network to invert that corruption, then samples by running the inversion in reverse. In layman’s terms: diffusion is teaching the model to clean up a noisy page.
- Mess it up. Hide or swap random words so the text looks scrambled.
- Fix it. Train the network to guess the original words.
- Repeat. Judge success by how much better the cleaned-up page matches the real one.