DiffusionGemma: Google's open diffusion LLM

What DiffusionGemma is: Google's open diffusion LLM

DiffusionGemma is an experimental open-weight language model from the DiffusionGemma Team at Google DeepMind. It is a diffusion LLM. It starts from a block of 256 random tokens and refines the whole block at once, instead of writing text from left to right.

Most chat models work the other way. An autoregressive (AR) model picks one token, appends it, and runs the network again for the next one. Each token costs one forward pass, which is one full run through the network. DiffusionGemma gets about 20 tokens out of each forward pass. The table sets the two side by side.

Two ways to produce text

	Autoregressive Gemma 4	DiffusionGemma
Unit of work	One token per step	A 256-token canvas per step
Order	Strictly left to right	All positions in the block at once
Tokens per forward pass	1 (3 to 6 with speculative decoding)	About 20 on average
Revision	A token is final once written	Tokens in the current canvas can still change
Best regime	Moderate to large batches	Few users, latency you can feel

Fine-tuned from Gemma 4 26B A4B, not trained from scratch

The team did not pretrain a new model. It started from Gemma 4 26B A4B, a mixture-of-experts model. A mixture-of-experts model holds many small expert sub-networks and sends each token to only a few of them. This one has 25.2B parameters in total, and 3.8B of them are active for any one token.

Training ran in two stages and used under 10% of the starting model's total training token budget. Stage one is supervised fine-tuning (SFT). It teaches the model to denoise a noisy 256-token block while it reads a clean context. Stage two combines sampler distillation with reinforcement learning (SD·RL) in a single online phase. It improves answer quality and cuts the number of steps at the same time.

Open weights under Apache 2.0, with no usage barriers

The weights come under Apache 2.0, with full access to the model parameters. The report says this license removes friction for experimental prototypes and for commercial applications alike. It also says researchers get a white-box baseline for studying discrete diffusion.

The authors explain why they released it. Gemini Diffusion and Mercury sit behind proprietary APIs. Other open-weight text diffusion models either reason poorly and lack multimodal understanding, or fail to deliver the latency gains, or both.

In the authors' words, no earlier text diffusion model was highly intelligent, exceptionally fast, and openly accessible. That is their claim about their own release, not an independent finding. For another recent open-weight release and its license, see GLM 5.2 benchmarks, pricing, and Claude Code setup.

How a text diffusion model like DiffusionGemma works

A text diffusion model generates by removing noise: in training, noise is added to clean text and the network learns to undo it. During generation, the model starts from random tokens and refines them.

A diffusion language model refines 256 tokens in 12 steps

The unit of work is a canvas of 256 token slots. Each canvas starts as uniformly random tokens, and the model refines it for at most 48 denoising steps.

The model usually stops much earlier. Adaptive stopping ends the loop when two things hold at once. The model is confident, meaning the mean predictive entropy (a measure of how unsure it is) across the canvas is below 0.005. And the most likely tokens from two consecutive steps are identical. With this rule, the average is about 12 steps, which the report says cuts overall latency 4x compared with always spending the full 48-step budget. Code needs fewer steps than natural language, and harder code problems such as LiveCodeBench need more than easier ones such as HumanEval.

In each step, the sampler accepts tokens from the least uncertain to the most uncertain until an entropy budget of 0.1 is used up. It then replaces every other position with fresh random tokens, so those slots get another try on the next pass.

Tokens stay editable within a canvas, then freeze

DiffusionGemma uses multinomial (uniform) diffusion, not masked diffusion. Any token in the current canvas can be replaced in a later step, so the model can fix its own mistakes while the canvas is still open. Once a canvas is finished, its tokens are permanently frozen.

Each step also has a weakness. The model updates all positions at the same time, so a token cannot see what the other positions are choosing in that same step. The report says this can occasionally create local inconsistencies. The self-conditioning signal and the entropy-bounded sampler are the report's countermeasures.

Block-autoregressive generation, one canvas after another

An answer longer than 256 tokens is a chain of canvases. The model denoises one canvas, commits it to the history, and starts the next one. The history sits in a KV cache, a stored copy of what earlier tokens contributed to attention, so it does not have to be recomputed each time.

The design splits the work in two. A causal encoder reads the history, and a bidirectional decoder builds each new canvas. This lets the model append only the newest canvas to the cache instead of re-reading the whole context. The report presents this as a blend of diffusion and autoregressive generation.

DiffusionGemma speed: what 1,500 tokens per second means

The headline figure comes from the abstract: about 20 tokens per forward pass and roughly 1,500 output tokens per second on one NVIDIA H100 GPU, averaged over the full evaluation suite. The report's own throughput analysis serves a single request, which is batch size 1, in FP8 precision, with 4,096 input tokens and 1,024 output tokens. Keep that condition in mind for every number below.

About 20 tokens per pass against speculative decoding

Plain AR decoding gives 1 token per forward pass. Speculative decoding drafts a candidate of typically 8 tokens with a small drafter model, and the main model checks the draft in one pass. With a draft of 8, that gives 3 to 6 tokens per forward pass. DiffusionGemma writes a 256-token block in around 12 passes, which averages about 20.

Tokens produced per forward pass

Speculative decoding with a draft length of 8 gives 3 to 6 tokens per pass. DiffusionGemma averages about 20, because it needs no draft-then-check loop. Bars show tokens per forward pass.

05101520

1AR, plain

3Speculative, low

6Speculative, high

20DiffusionGemma

The second training stage did much of this work. SD·RL moved tokens per forward pass from 5 to nearly 20 and added 10 points to the combined GPQA-Diamond and LiveCodeBench-v6 score. It also cut answer length nearly in half compared with the SFT checkpoint. Fewer tokens, each needing fewer passes, is how the evaluation suite ends up with less than 5% of the forward passes the Gemma 4 AR baseline uses. The count includes one extra pass per canvas that stores the finished canvas in the KV cache.

Why a 256-token step costs only 3.2x a single AR step

Each diffusion step handles 256x more tokens than an AR step, yet it costs only 3.2x the GPU kernel time. That was measured on an H100 in FP8, with 4,096 input tokens, 1,024 output tokens, and batch size 1. End-to-end latency is about 1 ms above kernel time for both models.

The reason is where AR serving spends its time. For one request, it is memory-bound: most of the time goes to moving weights and cache out of memory, not to arithmetic. A diffusion step does more arithmetic per token, but it fits into that slack.

The throughput calculation follows from this. Tokens per second equals tokens per forward pass divided by step time. One denoising step takes 13.56 ms end to end on average (12.63 ms of it GPU time). The report assumes 19.74 tokens per forward pass, the average over the 7 benchmarks in its Table 3. That gives 1,456 tokens per second. Gemma 4 AR reaches 204 on the same setup, so DiffusionGemma is 7.1x faster. Gemma 4 with multi-token prediction (MTP), which predicts several tokens per pass, reaches 303, so DiffusionGemma is 4.8x faster. The 1,456 is a calculation from the tokens-per-pass average, not a separately timed run.

The mixture-of-experts penalty: 84 experts instead of 8

The 3.2x comes from three causes, which the report names. The biggest is the mixture-of-experts layer. An AR step sends one token to 8 unique experts per layer. A 256-token canvas touches about 84 unique experts per layer, measured on the PG-19 benchmark. Loading more expert weights from memory makes the MoE kernel 4.3x slower. The authors call this the same kind of penalty that verification adds in speculative decoding, only at larger scale. For a dense model with no experts, they say the feed-forward slowdown would be under 2x.

Where one diffusion step loses time

Part of the step	DiffusionGemma against one AR step
MoE kernel	4.3x slower, from about 84 experts instead of 8
Attention	4x slower, because it covers a 256-token canvas
Sampling	3.06 ms against 0.56 ms
Other operations	At most 2x slower

Sampling costs more because it runs in full precision over a 262k-token vocabulary, using standard PyTorch with torch.compile rather than hand-written GPU kernels. The authors did not write custom GPU kernels for it.

Running DiffusionGemma with vLLM and Hugging Face

Two pieces of released tooling cover the practical side. You can load the model through Hugging Face Transformers or serve it with vLLM, and you set a handful of sampler values. Neither needs a custom denoising loop.

Reference implementations in Transformers and vLLM

The report names Hugging Face Transformers and vLLM as the places with reference inference implementations. On the Transformers side, pull request #46540, titled "GPU go brr [DiffusionGemma]", was merged into the main branch on 10 June 2026 and labeled "New model".

The vLLM work solves a serving problem. Denoising steps and KV-cache updates are two kinds of forward pass, and both can land in the same batch. vLLM extends asynchronous scheduling to diffusion models and adds a per-sequence causal-attention flag, so the batch runs without extra CPU-GPU synchronization.

Sampler settings that decide DiffusionGemma latency

The report recommends one set of sampler values, and most of them you have already met above. The block below lists all five.

Recommended sampler settings (Table 2)

Canvas length

256 tokens

Maximum denoising steps

48

Adaptive stopping threshold

Mean entropy 0.005

Token selection threshold

Entropy budget 0.1

Temperature

Linear schedule from 0.8 down to 0.4

Steps are the main dial for latency. Each denoising step costs a fixed amount of compute, so the total number of steps drives end-to-end latency. In the report's step sweep, run with adaptive stopping and temperature annealing switched off, quality rose steadily up to 48 steps and showed diminishing returns after that. In practice the average is about 12 steps, depending on the task.

The report calls the entropy-bounded sampler its default and recommended one, and says the model is not strictly bound to it. Anyone testing a different sampler should compare against these settings first.

Third-party results: up to 2,000 tokens per second

The report also cites early benchmarking by outside inference providers. It says Unsloth measured speeds of up to 2,000 tokens per second on an NVIDIA RTX 6000. This is a third party's result as the paper reports it. The GPU is different from the H100 behind the 1,500 figure, so the two numbers are not directly comparable.

DiffusionGemma vs Gemma 4: where autoregressive still wins

DiffusionGemma wins on latency for single requests and loses ground on quality and under heavier load. The table lists the limits that the report itself supports.

Where autoregressive Gemma 4 still leads

Limit	What the report says
Absolute quality	Adapting to diffusion carries a performance penalty against the AR baseline
Answer length	Final checkpoint's outputs are about half as long as the SFT checkpoint's
Concurrency	AR with MTP gains a throughput advantage around 32 concurrent requests
Batch tuning	Batch sizes above 1 are not specifically optimized yet
Cross-model charts	The models in the speed chart were measured on different hardware

Quality below the Gemma 4 26B A4B starting point

The report states that adapting the model to diffusion introduces a performance penalty against the original AR baseline. The authors accept that trade. Starting from an existing AR backbone trades some absolute quality for a large saving in training cost, and they present the speed gain as a new operating point for latency-critical applications.

The same weights can also run in AR mode. Their scores there land between DiffusionGemma's diffusion mode and the Gemma 4 baseline it started from.

Throughput flips at about 32 concurrent users

At low batch sizes, DiffusionGemma beats Gemma 4 with MTP on both per-user speed and total throughput. AR models start to gain a throughput advantage only at moderate batch sizes, around 32 concurrent requests. The test used an H100 in FP8 with the PG-19 benchmark, 4,096 input tokens and 1,024 output tokens, and a draft length of 4 for the MTP model.

The 32 is a measurement for today's code, not a fixed limit. The authors say these results come without targeted optimization for batch sizes above 1: kernel selection is suboptimal and sampling is not tuned for larger batches. They expect top-k truncation in the sampling step to bring significant throughput gains at higher batch sizes with negligible effect on output quality.

Answers about half as long as the SFT checkpoint's

SD·RL made the model concise. The final checkpoint writes outputs about half as long as the SFT checkpoint's. This is a second speed multiplier, because fewer tokens also mean fewer canvases.

The report notes the cost: the model gives up some of the gains that usually come from longer reasoning traces. If your task needs long, step-by-step answers, test how DiffusionGemma handles them before committing.

Reading the benchmark chart: hardware differs by model

Figure 1 of the report plots quality against output speed for DiffusionGemma, the Gemma 4 family, and other diffusion models. Quality is the average of GPQA-Diamond and LiveCodeBench-v6. The speeds were not measured on the same machine:

• Gemma 4 and DiffusionGemma: one H100 in FP8, batch size 1
• Nemotron 14B: one H100 in bfloat16, batch size 1
• LLaDA 2.1 Flash 100B: 8 NVIDIA B200 GPUs in bfloat16, batch size 1
• Mercury 2: the public OpenRouter API with high reasoning effort

Because of this, only the Gemma 4 and DiffusionGemma points are a like-for-like comparison. Read the other models as rough placement.

Which workloads suit DiffusionGemma, and which do not

The measured conditions point to one rule. DiffusionGemma pays off when few users share the GPU and each of them waits for the answer. It loses its edge when many requests are batched together.

Where DiffusionGemma fits: low concurrency, latency-bound

Take an internal drafting assistant that a few staff use at once, each waiting for the reply on screen. The batch is small, so the GPU has spare compute that diffusion can use. The result is a reply that arrives in far fewer forward passes. That is the latency-critical, low-concurrency case the report describes. The report's own examples of early users point the same way: multilingual speech recognition and interactive radiology report drafting.

Now take a public API with many simultaneous users, or an overnight batch job. Requests are batched heavily, so AR gets good use of the GPU, and it regains the throughput lead at moderate batch sizes. For that workload, plain Gemma 4 is the better pick, and it is the stronger model on quality anyway.

Fit by deployment shape

Good fit

Low concurrency, latency visible to a user, self-hosted

Poor fit

High-throughput APIs, offline batch generation

Test on your tasks

Answer quality and length against Gemma 4

Watch for

Concurrency near 32 requests

Hybrid diffusion and AR decoding as a next step

The same weights can generate text autoregressively with only minor performance degradation. The report calls this dual mode a possibility: requests could be routed by latency needs and task complexity, and the two decoding styles could be combined. That is a direction the authors raise, not a feature in the release.

Conclusion: when DiffusionGemma is worth adopting

DiffusionGemma is worth trying when each user's wait for an answer is the thing you care about and you run the model yourself. The weights are open under Apache 2.0. The model averages about 20 tokens per forward pass and reaches roughly 1,500 tokens per second at batch size 1 on one H100. The advantage fades at around 32 concurrent users, and its absolute quality is lower than the Gemma 4 AR model it started from.

Before you commit, measure it on your own stack. Check latency at the concurrency you expect, compare answer quality and length against Gemma 4 on your tasks, and start from the recommended sampler settings.