Dasheng-Audiogen By Xiaomi Research Crafts Mixed Audio From Text

The newly released Dasheng-Audiogen is an open source artificial intelligence model that creates full audio scenes from text descriptions. Instead of producing just one type of sound, it can blend speech, music, sound effects, and environmental noise together into a single file. Users provide written prompts with specific tags to control different sound layers independently.

The development team at Xiaomi-research built this framework to solve the problem of separate audio generators that cannot mix sounds properly. They trained the system using a high-dimensional representation that helps the model understand complex overlapping noises. This approach allows a single text prompt to produce realistic mixed audio without requiring multiple specialized tools.

Key features and ideal users

This tool is built for creators who need custom background audio for videos or games without relying on separate specialized software. Developers can integrate it directly into their local pipelines to automate sound design tasks for larger projects. Running locally provides complete control over the generation process and keeps all data private on your own hardware.

Development notes and technical requirements

The repository notes that the model requires Python 3.10 and is currently not compatible with transformers version 5.x. Developers should also ensure they have the correct versions of torch and torchaudio installed to prevent runtime errors. The team provides audio IDs and captions for testing to ensure users can reproduce the exact results shared in their research paper.

"Audio generation has long been fragmented, with speech, music, and sound effects produced by domain-specific models that fail to jointly generate coherent audio scenes from a single description." Source: Arxiv