MiniMax-M3 Handles 1M Tokens Across Text Images And Video Natively

MiniMax-M3 is a new native multimodal AI model from MiniMaxAI that processes text, images, and video with a 1 million token context window. The model contains about 428 billion total parameters but activates only 23 billion during inference, keeping it efficient to run. It introduces MiniMax Sparse Attention (MSA) to dramatically lower compute costs at extreme context lengths.

MiniMaxAI who also released MiniMax-M2.7 trained the model from scratch on mixed text, image, and video data, fusing the modalities deeply from the very first step. The team designed MSA to overcome the quadratic cost of standard attention, allowing the model to handle million-token contexts without sacrificing output quality. This approach makes M3 especially useful for long-form coding, multi-step agent workflows, and persistent memory tasks.

Native multimodal design with long-context efficiency

This model is for developers building applications that require understanding long documents, codebases, or multi-turn agent tasks with multiple modalities. You can download and run it locally using popular inference frameworks, and its adaptive reasoning mode helps balance speed and accuracy.

What developers should know

The sparse attention mechanism, MSA, uses a blockwise approach that selects the top-k key-value blocks per query group, achieving speedups without accuracy loss. MiniMaxAI co-designed an optimized GPU kernel that delivers 14.2x prefill and 7.6x decoding wall-clock speedups on H800 GPUs paired with MSA. The team recommends temperature=1.0, top\_p=0.95, and top\_k=40 for best performance, and the model is available for download directly from Hugging Face.

“M3 delivers 9× prefill and 15× decode speedups compared to M2 at 1M context, reducing per-token compute to 1/20.” — Source: Hugging Face model page