Sageattention-Autotune Drops Automatic Kernel Tuning For Faster AI Art

Sageattention-autotune is a fresh open source release that adds automatic block size tuning to the SageAttention 2 kernel. It brings CUDA and Triton kernel ports with quality-of-life improvements to speed up attention calculations. The project offers ready-to-use settings for Nvidia RTX 30 series cards while keeping a separate branch for newer RTX 40 and 50 series GPUs.

Developer woct0rdho who also another speed boost tool for ComfyUI created this as a drop-in replacement for the official SageAttention package. They ported the core kernels and built an autotune system that picks the best block sizes without manual configuration. Because it installs under the same package name, existing users can switch over with minimal code changes.

Autotuned performance across GPU generations

This tool is meant for developers and researchers who rely on SageAttention in diffusion models or transformer pipelines. It saves time by automatically adjusting kernel parameters for different attention operations. Users on consumer RTX 30 series hardware get a ready-made performance boost, while those on newer cards can use the dedicated branch for similar gains.

What the developer wants you to know

The project is labeled experimental and expects users to have some familiarity with Python package installs. PyTorch version 2.12 or higher is required to avoid issues with torch.compile. One current limitation is that the Triton kernel does not yet support mixed fp16+fp32 accumulation, so some models may need manual tuning of the pv_accum_dtype setting to prevent output artifacts.

"This repo also serves as an example of how to do autotune when multiple kernels (like quant kernel and attn kernel) need consistent parameters (like block sizes)." — Source: GitHub