Elato Music Creates Voice Guided Offline Music For Your Home By Akdeb

The new release called Elato Music is a local-first macOS application that connects to an ESP32-S3 device to stream realtime generative music throughout your house. It uses an open music generation model to create continuous audio steered by voice commands, text prompts, and audio examples. The system processes everything locally on your Mac without requiring an internet connection once the models are downloaded.

A developer named Akdeb who also made Open-toys created this project by combining an ESP32 microcontroller with a MacBook running local AI models. They built the application to use Whisper for speech transcription and a Qwen model for deciding how to alter the music. This setup allows the hardware to process voice commands like adding drums or making the sound lo-fi entirely offline.

Local hardware and software features

This tool is for people who want a private, offline music generation setup in their home. Users can benefit from the ability to change music styles through simple voice commands during a live session. It also serves those interested in building interactive hardware projects using local AI models.

Project notes and limitations

The developer notes that realtime generation requires an Apple Silicon Mac, with the base model needing a stronger Pro or Max-class machine. A known limitation is that the local language model can sometimes misunderstand speech or choose imperfect tool calls. Users must supply their own ESP32-S3 dev board with specific microphone and speaker components to make the system work.

The microcontroller is just a tiny Arduino-based device with a mic and speaker, and the MacBook M4 Pro runs Magenta Realtime 2 locally and streams the audio back to the device. Source: Reddit