Stability AI’s New Model Generates Audio from Text

Stability AI, renowned for its revolutionary AI-powered art generator Stable Diffusion, now unveils a game-changing open AI model, Stable Audio Open. This cutting-edge model is not just another tool, but a gateway to a new era of sound and music creation. It can generate sounds and songs from simple text descriptions, offering a unique and exciting prospect for sound designers and musicians.

Capabilities and Training
Stable Audio Open allows users to create recordings up to 47 seconds long based on text prompts, such as “Rock beat played in a treated studio, session drumming on an acoustic kit.” The model was trained on a diverse dataset of approximately 486,000 samples from Freesound and the Free Music Archive, ensuring the use of royalty-free recordings and a wide range of musical styles and genres.

Stable Audio Open is not just a tool, but a creative companion. It can generate various audio elements, including drum beats, instrument riffs, ambient noises, and production elements for videos, films, and TV shows. But the real magic happens when you fine-tune the model with your own audio data. This feature allows for personalized and unique sound creation, empowering you to unleash your creativity like never before.

Limitations and Commercial Use
Despite its impressive capabilities, Stable Audio Open has some limitations. It must be optimized for producing full songs, melodies, or vocals. Stability AI suggests that users seeking these features should consider their premium Stable Audio service. Additionally, the model is restricted from commercial use under its terms of service and performs unevenly across different musical styles and languages due to biases in the training data.

Controversies and Challenges
Stability AI has recently faced significant challenges, including the resignation of its VP of generative audio, Ed Newton-Rex, over the company’s stance on using copyrighted works to train AI models. This controversy has heightened the focus on copyright issues in AI-generated music. In response, Stability AI has taken steps to ensure its models are trained on royalty-free recordings. In May, Sony Music warned AI companies against unauthorized use of its content, and a new law in Tennessee aims to curb AI abuses in music.

Future Prospects
Stable Audio Open represents a strategic move by Stability AI to shift the narrative and highlight its commitment to responsible, ethical, and open-source development while promoting its premium products. The company’s initiative comes amid increasing scrutiny and legal challenges in the AI and music industries, and we are dedicated to addressing these issues in a transparent and responsible manner.

Stability AI encourages sound designers, musicians, and developers to explore Stable Audio Open, available on Hugging Face. Your feedback is invaluable in our ongoing efforts to improve and refine the model. This release marks a significant step forward in the responsible development of AI audio generation, setting the stage for future innovations.

Source: Stability

---

Like this article?  Keep up to date with AI news, apps, tools and get tips and tricks on how to improve with AI.  Sign up to our Free AI Newsletter

Also, come check out our free AI training portal and community of business owners, entrepreneurs, executives and creators. Level up your business with AI ! New courses added weekly. 

You can also follow us on X