Bonsai-Image-Ternary-4B-Gemlite-2bit Shrinks A 4B Image Model Into A Tiny 1.21 GB For Fast Local Art
The new Bonsai-Image-Ternary-4B-Gemlite-2bit model compresses a 4-billion-parameter text-to-image diffusion transformer into just 1.21 GB. It uses ternary weights—each limited to -1, 0, or +1 with shared scaling—to shrink the model by 6.4 times compared to a standard 16-bit version. On an RTX 3080, it generates a 1024×1024 image in about 4.5 seconds, putting high-quality local image generation within reach for consumer GPUs.
Prism ML who were behind the Bonsai-Image-Binary-4B-Gemlite-1bit model, also created this release to bring modern diffusion-transformer behavior into memory-constrained settings. They applied ternary quantization to the FLUX.2 Klein 4B architecture, retaining a quality-oriented approach by including the zero state in the weight set. The deployment relies on Gemlite low-bit GEMM kernels for NVIDIA hardware and offloads the text encoder after prompt processing, so the denoising loop keeps only the compact transformer active.
Compact model for local image creation
Key Features
- 1.21 GB transformer, 6.4× smaller than FP16.
- 4.5 seconds per 1024² image on RTX 3080.
- Runs natively on Linux and Windows.
- Ternary weights {-1, 0, +1} for fidelity.
- 4-step sampler, guidance 1.0, no negative prompts.
- Text encoder offloaded after prompt encoding.
- Supports 832×1248 and 512² resolutions.
- Gemlite CUDA low-bit kernels.
The model suits creative professionals and serious hobbyists with NVIDIA consumer GPUs who want fast, private image generation. Small agencies and privacy-conscious users can keep prompts and assets entirely on a local machine. Developers serving images on commodity hardware benefit from lower memory pressure and straightforward setup across Windows and Linux.
Where it fits on the quality spectrum
Benchmarks show the ternary model scores close to the original FLUX.2 Klein 4B on GenEval, HPSv3, and DPG-Bench while using only a fraction of the memory. It is not bit-identical to the full-precision version, and quality can vary with fine details like small text or strict object counts. The current implementation uses Gemlite kernels because native ternary hardware paths are not yet standard, and after compressing the transformer, the VAE decoder becomes a more noticeable memory component.
“Bonsai Image variants move the quality-footprint frontier: they bring modern diffusion-transformer behavior into a memory range previously occupied by much smaller, lower-capability models.” — Source: Hugging Face
Project Links
