XiaomiMiMo Debuts MiMo-Audio-7B-Instruct For Smart Sound Generation
MiMo-Audio-7B-Instruct is a new audio language model designed to understand and generate sound based on simple instructions. It learns from a massive amount of audio data to perform tasks like voice conversion and speech editing without needing specific fine-tuning. This release brings advanced audio processing capabilities directly to local systems.
The LLM-Core-Team at XiaomiMiMo who recently released MiMo-Audio-7B-Base developed this tool to help machines generalize to new audio tasks just like humans do. They scaled the training data to over one hundred million hours of audio to unlock strong learning capabilities. This approach allows the model to handle diverse tasks such as generating realistic talk shows and debates.
Advanced audio generation capabilities
Key Features
- Understands complex spoken language and audio.
- Performs voice conversion and style transfer.
- Generates realistic talk shows and debates.
- Runs locally using a Gradio interface.
This tool is built for developers and hobbyists who want to run advanced audio models on their own hardware. Users can experiment with speech continuation and instruct text-to-speech features offline. It provides a flexible framework for evaluating and expanding audio generation capabilities.
Model architecture and system requirements
The model pairs a patch encoder and decoder with a language model to handle high-rate audio sequences efficiently. It relies on the MiMo-Audio-Tokenizer to process sound at 25 Hz and generate 200 tokens per second. Running the demo requires Linux, Python 3.12, and a CUDA version of 12.0 or higher.
"MiMo-Audio-7B-Instruct achieves open-source SOTA on audio understanding benchmarks, spoken dialogue benchmarks and instruct-TTS evaluations, approaching or surpassing closed-source models." - Source: Hugging Face
Project Links
