Stability AI reveals how it built Stable Audio Open

Stable Audio Open is Stability AI's open-weights model designed specifically to generate high-fidelity sound effects and ambient field recordings from text prompts.

It runs locally on consumer GPUs and spits out up to 47 seconds of full 44.1kHz stereo audio. Whether you need realistic footsteps, creaking doors, ambient cinematic textures, or drum loops for a music track, it generates them directly from text.

How it was built

The model relies on three key technical components: an autoencoder that compresses raw audio waveforms into manageable sequences, a T5-based text encoder for conditioning, and a diffusion transformer (DiT) operating inside the latent space.

The autoencoder hits a low latent rate of 21.5Hz, which drastically cuts down sequence length without destroying acoustic fidelity. While the architecture shares roots with Stable Audio 2.0, this release swaps CLAP embeddings for T5 conditioning to give users tighter control over detailed prompts.

To keep the model clean from copyright minefields, Stability AI trained it on nearly 500,000 Creative Commons recordings. The dataset pulls 472,618 samples from Freesound and 13,874 tracks from the Free Music Archive, limited strictly to CC-0, CC-BY, and CC-Sampling+ licenses.

Before training began, the team ran audio through the PANNs audio tagger to identify potential music clips, then passed those clips through Audible Magic's content detection system to strip out any copyrighted commercial music.

How to run it and what to watch out for

The weights are live on Hugging Face right now, and the accompanying research paper is up on arXiv.

You do not need a server cluster to use it. Everyday inference works on typical consumer GPUs, and if you want to fine-tune the model on custom sound packs or specific audio durations, you can do it locally on a single A6000 card.

Licensing follows the Stability AI Community License. It is completely free for research and non-commercial projects, as well as commercial use for creators and businesses pulling in under $1M in annual revenue. Enterprises beyond that threshold will need a separate commercial license.

Keep your expectations in check: Stability AI explicitly notes that Stable Audio Open struggles with coherent human speech and full-song musical composition. Its wheelhouse is sound design and foley, not pop vocals.

Why it matters

For indie game devs, video editors, and sound designers, this is an immediate win. Instead of scouring royalty-free libraries or risking copyright strikes with opaque commercial generators, you get a clean, locally hostable engine to forge bespoke sound beds on demand.

Stability AI made a very sharp trade-off here. By foregoing copyrighted music training sets and narrowing the focus to short-form sound design and foley, they delivered an open-weights model that creators can actually deploy commercially without looking over their shoulders.