Gemma-4-31B-It-DFlash Drafts Speed Into Your Local LLM
Gemma-4-31B-It-DFlash is a drafter model that works alongside Google’s Gemma 4 31B Instruct to speed up text generation for local deployments. Instead of generating tokens one by one, it uses block diffusion to draft multiple tokens in parallel, then lets the full model verify them in larger chunks. When used with vLLM or SGLang, this approach can deliver over five times faster responses on a single GPU while keeping output quality pinned to the original model.
Z-lab, the independent AI group behind this release, trained the small DFlash drafter specifically for Gemma 4 31B Instruct, optimizing it for speculative decoding. The project was built to help users who run large models on their own hardware escape the slow, token-by-token experience that makes local assistants feel sluggish. Privacy-conscious professionals, serious hobbyists, and small agencies can now handle reasoning-heavy chat, coding, or math tasks with much shorter waits, all without sending data to a cloud service.
Speculative decoding drafter for local speedups
Key features of the DFlash drafter
- Parallel token drafting using block diffusion.
- Up to 5.8× generation speedup on a B300 GPU.
- Seamless integration with vLLM and SGLang servers.
- Supports chat thinking mode for reasoning tasks.
- Average acceptance length of 6–8 tokens per step.
- Single-GPU deployment with no cloud dependency.
- Strong performance across math and coding benchmarks.
This drafter is for people running Gemma 4 31B on high-end consumer GPUs who need snappier responses during interactive coding, math problem-solving, or extended chats. Small agencies can keep client data fully local while handling multiple concurrent users with better throughput than autoregressive generation alone. Because everything stays on-device, the setup helps privacy-minded users avoid API calls and still get responsive, high-quality output.
What you should know before deploying
The DFlash drafter was trained with a block diffusion method that proposes several candidate tokens in a single forward pass, leaving the full model to verify and accept the longest matching sequence. At low concurrency, speedups are striking—for example, a single request on the Math500 benchmark jumped from 77 to 447 tokens per second—while even at higher loads the gain remains meaningful. Keep in mind that DFlash requires a compatible serving engine (vLLM or SGLang) and enough GPU memory to hold both the drafter and the main model at the same time.
"DFlash is a speculative decoding method that uses a lightweight block diffusion model to draft multiple tokens in parallel." — Source: Hugging Face
Project Links
