XiaomiMiMo Delivers MiMo-Audio-7B-Base For Realistic Voice Generation
The new release called MiMo-Audio-7B-Base is an open-source audio language model designed to learn new tasks from just a few examples. It processes over one hundred million hours of audio data to understand and generate complex spoken content. This model can perform tasks like voice conversion and speech editing without needing specific fine-tuning for each task.
The developer who brought us MiMo-Code and MiMo-V2.5-Pro built this system to bring human-like generalization capabilities to the audio domain. They scaled the training process to help the model understand speech intelligence and audio content naturally. Their work proves that predicting the next piece of data works just as well for audio as it does for text.
Audio understanding and speech generation features
Key Features
- Scales to one hundred million hours.
- Generalizes to new unseen audio tasks.
- Generates highly realistic talk show audio.
- Achieves top open-source audio benchmark performance.
- Performs voice conversion and audio editing.
This tool is built for developers and researchers working on advanced speech intelligence projects. Creators can use it to generate realistic audio content like recitations or debates for various media applications. Anyone needing to edit speech or transfer voice styles will find its few-shot learning capabilities highly useful.
Development details and benchmark results
During development, the team used a specialized tokenizer operating at 25 Hz to convert audio into manageable data. The base model currently achieves state of the art performance among open-source alternatives for speech intelligence benchmarks. Developers can run a local interface using Python 12 and CUDA 12 or higher to test its interactive features.
"By scaling MiMo-Audio's pretraining data to over one hundred million of hours, we observe the emergence of few-shot learning capabilities across a diverse set of audio tasks." Source: Hugging Face
Project Links
