Colored-Noise-Sampling Sharpens Diffusion Models Without Retraining
A new open-source release called Colored-Noise-Sampling changes how diffusion models add randomness during image generation, giving better results without extra training. Standard samplers treat all frequency details the same and inject uniform white noise throughout the process. This project, named Colored Noise Sampling (CNS), instead reads a precomputed progress index to route noise energy toward frequency bands that still need detail work.
The tool was created by Hadar Davidson alongside Noam Issachar and Sagie Benaim from the Hebrew University of Jerusalem. They published a paper explaining that diffusion models naturally build broad shapes first and fine textures last, so wasting noise on already-finished parts hurts final quality. Their solution is a plug-and-play sampler that swaps in directly—no model changes, no retraining, just smarter noise routing.
A smarter way to spend the noise budget
What makes CNS different
- Routes noise energy to unresolved frequency bands.
- Plug-and-play sampler requiring no model retraining.
- Uses precomputed gamma matrices for progress measurement.
- Maintains strict variance conservation for convergence.
- Works across SiT, JiT, and FLUX architectures.
- Boosts unguided SiT-XL/2 FID from 8.26 to 6.27.
- Includes ready-to-use gamma matrices for quick setup.
- Open-source under the MIT license.
This tool is built for researchers and serious hobbyists who already run diffusion models locally and want higher quality outputs from existing checkpoints. It shines for privacy-conscious pros who cannot send data to cloud APIs, since it improves generation without touching the model weights. Anyone running SiT-XL/2 or similar architectures on pro-consumer GPUs can drop in CNS and see immediate FID score improvements across both guided and unguided sampling.
What developers should know
The project requires a precomputed gamma matrix that maps how each radial frequency band progresses over the sampling trajectory, with ready-made files provided for the published configurations. You can also generate fresh matrices yourself using the included scripts, which run spectral analysis until the per-frequency confidence interval stabilizes. The sampler only works in SDE mode and includes several tunable flags like energy scaling, frequency tilting, and gamma matrix division for fine control.
"The result is a strictly plug-and-play sampler substitution — same model, same number of steps, only the noise injection changes." — Source: GitHub
Project Links
