LongCat's new model clones any voice in seconds

Cloning someone's voice used to require studio-grade audio and hours of custom fine-tuning. Today, that barrier is effectively down to zero.

Researchers released LongCat Audio Edit (published in research as LongCat-AudioDiT). It is a diffusion-based text-to-speech model capable of cloning anyone's voice using only a few seconds of raw audio. Just feed it a snippet and a transcript, and it will read any new script while mimicking the original tone, accent, and style.

Voice cloning in under 10 seconds

Along with the open-source weights, the team launched a free Hugging Face Space for testing. The workflow is dead simple: drop in a few seconds of reference audio, paste the matching transcript, type in your new prompt, and hit generate.

In early demos, generation took under 10 seconds. Whether fed a thick Australian accent or an expressive Indian female voice, the model managed to capture not just basic pitch, but subtle speech rhythm and cadence.

The release includes two model variants. A 3.5-billion-parameter flagship version weighs in at around 15.3GB and is built for maximum audio fidelity. Alongside it sits an efficient 6GB lightweight variant designed to fit comfortably onto standard consumer GPUs. Both the source code and model weights are freely available.

Bypassing spectrograms entirely

Traditional diffusion TTS models typically rely on intermediate acoustic representations, like mel-spectrograms, before running a separate vocoder. That multi-stage setup often accumulates reconstruction errors.

LongCat takes a different route by operating directly in the waveform latent space. The pipeline is stripped down to just a waveform variational autoencoder (Wav-VAE) and a diffusion backbone. On top of that, the researchers swapped standard classifier-free guidance for adaptive projection guidance and fixed long-standing training-inference mismatches.

On the Seed benchmark, the 3.5B model set fresh state-of-the-art results for zero-shot voice cloning, beating Seed-TTS by pushing speaker similarity from 0.809 to 0.818 on Seed-ZH, and from 0.776 to 0.797 on the challenging Seed-Hard set. The paper also uncovered a counterintuitive insight: higher reconstruction fidelity in the autoencoder does not automatically translate to better downstream speech synthesis.

Why it matters

For creators and developers, high-fidelity voice cloning is finally becoming a commodity you can run privately. Instead of paying per-minute API fees to cloud providers or sending sensitive audio across the wire, a 6GB model lets anyone build localized dubbing pipelines and interactive voice agents right on a consumer GPU.

For the speech synthesis industry, this is proof that architecture simplicity wins. Skipping intermediate representations and eliminating manual dataset cleanups produced a model that beats heavily engineered pipelines. Expect competing labs to quickly abandon mel-spectrograms in favor of direct waveform latent diffusion.

That said, the flip side is obvious. When anyone can clone a voice off a five-second clip taken from a noisy phone call, audio spoofing and scam risks escalate sharply. The technical breakthrough is impressive, but watermarking and provenance tools are now desperately lagging behind.