Unsloth Polishes Gemma-4-26B-A4B-It-Qat-GGUF With Speed Boosts

    
        By vramkickedin    
     | 
    
            June 24, 2026 at 6:54 pm        
    
     | 
    
        2 min read

Unsloth has released gemma-4-26B-A4B-it-qat-GGUF, a new quantized version of Google DeepMind’s Gemma 4 26B Mixture-of-Experts model. It uses Quantization-Aware Training to preserve near-original quality while shrinking the model’s memory footprint. The release also ships a Multi-Token Prediction drafter to speed up text generation through speculative decoding.

The team at Unsloth, who've also quantized Gemma-4-12B-It-GGUF, converted the official QAT checkpoints into GGUF format, making it compatible with llama.cpp and other popular inference engines. This allows developers to run the 26B-parameter model with only 3.8 billion active parameters, dramatically reducing hardware requirements. A built-in MTP drafter uses the target model’s cache to predict multiple tokens at once, verified on the fly for lossless acceleration.

What’s inside the unsloth release

Key Features

Multi-Token Prediction drafter for faster generation.
Near-lossless Q4_K_XL quantization format.
QAT preserves quality comparable to bfloat16.
Only 3.8B active parameters during inference.
256K token context window for long documents.
GGUF format works with llama.cpp and vLLM.
Native function calling for agentic workflows.
Supports image understanding and reasoning.

This release is for developers who need to run a powerful reasoning model on consumer GPUs or workstations. The MoE architecture and quantization let you handle long-context tasks like coding, document analysis, and agentic workflows without expensive hardware. The included MTP drafter further improves throughput, making interactive use smoother.

Developer notes and limitations

Unsloth notes that the MTP drafter shares the target model’s KV cache, so it does not alter the output. All drafted tokens are verified by the main model, ensuring lossless acceleration. The repository also includes other precision variants and explicit usage instructions in the MTP folder.