Nvidia's Fugatto creates sounds you've never heard

Imagine playing a simple piano melody and turning it into a soaring violin solo or a lifelike vocal performance with a single text prompt. Or watching the mechanical rumble of a train gradually morph into a full orchestral crescendo.

Nvidia just pulled back the curtain on Fugatto, short for Foundational Generative Audio Transformer Opus 1. The company is hyping it as the "world's most flexible sound machine," positioning it as an all-in-one Swiss Army knife for audio generation and transformation.

Yet despite the flashy demos, Nvidia is keeping the model locked behind closed doors for now.

A 2.5-billion-parameter audio shape-shifter

Unlike standard audio generators that merely spit out full songs from scratch, Fugatto's real superpower lies in modifying existing audio.

It can listen to an existing recording and perform surgical edits. Users can extract vocals from a mixed track, transform MIDI files into realistic vocal lines, change a singer's accent and emotional delivery, or overlay disparate acoustic elements to create entirely unheard soundscapes.

Under the hood, the full Fugatto model packs 2.5 billion parameters. Nvidia trained it over the course of a year using a bank of DGX systems powered by 32 H100 Tensor Core GPUs.

The training data consisted of millions of audio samples sourced from open datasets. Nvidia also highlighted its globally diverse research team, spanning contributors from India, Brazil, China, Jordan, and South Korea, which helped supercharge the model's multi-accent and multilingual capabilities.

Bryan Catanzaro, Nvidia’s VP of applied deep learning research, told Reuters that generative audio could reshape music production just like synthesizers did over the past fifty years, unlocking brand-new tools for both studios and everyday creators.

Why Nvidia is hitting the brakes

So why can't developers get their hands on it? In short: copyright landmines and voice-cloning risks.

Catanzaro acknowledged that any generative tool carries potential hazards, noting that people might use it to create content the company would rather avoid. He confirmed that Nvidia has no immediate plans to release Fugatto to developers while the team debates how to roll it out safely.

The music industry is already on high alert. Major record labels like Sony, Warner, and Universal sued AI music startups Suno and Uncharted Labs in June over massive copyright infringement. Hollywood is equally tense, underscored by Scarlett Johansson's recent clash with OpenAI over voice likeness.

With Meta also keeping its Movie Gen audio-video model private, Nvidia seems unwilling to risk becoming the next target of label litigation.

Why it matters

Fugatto marks a crucial shift in AI audio from pure generation to modular, fine-grained control.

Most current text-to-music tools act like lottery tickets: you type a prompt and hope for a decent track. Fugatto, by contrast, behaves like an intelligent plugin for modern Digital Audio Workstations. Features like transforming voice timbres, turning MIDI into vocal mockups, and swapping accents target the most tedious production tasks in music and podcast editing.

However, having groundbreaking tech locked in a corporate research lab doesn't help working producers today. We think Nvidia’s cautious stance makes sense from a legal liability standpoint, but it also reveals the dilemma tech giants face when crossing from hardware suppliers into generative content creators.

It looks like creators will have to wait before touching this engine directly. But the demo makes one thing clear: the future of music production will not be about AI replacing human arrangement, but about text prompts replacing dozens of complex audio effect plugins.