Meta's Audiobox creates speech and sound effects from text

Meta just unveiled Audiobox, a successor to its Voicebox model that unifies the generation of human speech, discrete sound effects, and ambient soundscapes into a single system controlled by natural language prompts.

While earlier models like Voicebox focused mainly on speech generation tasks, Audiobox takes controllable audio editing a step further by accepting both text and voice prompts.

Prompting speech, foley, and atmosphere

Audiobox allows users to describe exactly what they want to hear using plain text. For example, typing "a running river and birds chirping" produces an environmental soundscape, while "a young woman speaks with a high pitch and fast pace" synthesizes an entirely new voice from scratch.

Even more compelling is its dual-input capability. Users can provide a short voice recording to lock in a specific vocal timbre, and then use text prompts to dictate environment or delivery, such as "in a cathedral" or "speaks sadly and slowly."

Meta claims Audiobox is the first model to allow this kind of freeform voice restyling via combined audio and text inputs. In benchmark tests, Meta reported that Audiobox outperformed Voicebox by more than 30 percent in style similarity, while also surpassing top diffusion models like AudioLDM2, VoiceLDM, and TANGO in text faithfulness and overall audio quality.

Flow-matching and audio infilling

Under the hood, Audiobox inherits Voicebox's flow-matching modeling method and guided audio generation framework. This architecture enables audio infilling, functioning much like an inpainting brush in image generation.

This means creators can edit existing audio tracks selectively, such as dropping distinct thunderclaps directly into a rainy soundscape without re-recording the entire track.

Given the obvious risks around voice impersonation and audio deepfakes, Meta is not releasing model weights to the broad public. Instead, it is granting research-only licenses to selected academic partners and speech researchers, while releasing an interactive web demo to showcase its capabilities.

Why it matters

Audiobox is a major milestone in turning audio production into a promptable workflow. For game developers, indie filmmakers, and podcasters, sourcing specific foley effects or setting up acoustic studio environments has always required expensive hardware, deep sound-engineering expertise, and massive sample libraries.

By treating ambient audio, speech, and sound effects under one unified architecture, Meta has bridged the gap between synthetic speech and environmental sound design. It demonstrates that natural language is now granular enough to serve as a mixing console.

However, the dual-input restyling feature highlights why voice cloning remains a regulatory minefield. Meta’s guarded research-only release makes sense from a brand safety standpoint, but as flow-matching techniques spread across the open-source community, audio provenance and voice verification tools will become an urgent necessity rather than an afterthought.