Scenema Audio: An expressive TTS model pulled from LTX 2.3

Most AI voice generators sound like an overly polite customer service rep reading from a manual. They get the words right, but they completely fail at acting.

ScenemaAI is trying to fix that with Scenema Audio, a new open-source expressive speech engine. Instead of training a voice model from scratch, the team extracted the audio diffusion transformer straight out of LTX 2.3, Lightricks' 22B audiovisual model.

Directing AI with stage directions

What makes Scenema Audio immediately compelling is how you control it. You do not just provide text; you feed it an XML prompt filled with stage directions through action tags.

That means you can tell the model to start calm and casual, shift into explosive rage two sentences later, and finish with a breathless whisper after a long pause. The emotional arc transitions naturally within a single 15-second generation window instead of requiring manual post-production splices.

It also handles scene acoustics. If you specify an open dock during a thunderstorm, the model weaves the howling wind and cracking thunder directly into the audio while forcing the speaker to shout over the storm.

Zero-shot voice cloning is built in. Giving it 10 to 20 seconds of clean reference audio with some natural vocal dynamics lets you graft that identity onto any performance, even if the original speaker never screamed or sobbed on tape.

What it takes to run locally

Under the hood, the pipeline relies on Google's Gemma 3 12B for text encoding, an 8-step distilled latent diffusion model, and SeedVC for voice continuity, outputting 48kHz stereo sound. It supports 13 languages, including English, Spanish, Chinese, Japanese, and French.

Hardware requirements are surprisingly reasonable for what it does. The base pipeline fits into 16GB of VRAM if you stream the text encoder via CPU and system RAM. If you have an RTX 4090 with 24GB VRAM, running INT8 weights hits 1.57x real-time speed, churning out 55 seconds of finished audio in about 35 seconds.

There are real caveats. Complex multi-syllable words and proper nouns occasionally get garbled, generation is capped at roughly 15 seconds per chunk before Kokoro kicks in to split text, and Gemma 3 requires accepting terms on Hugging Face.

Why it matters

For indie game developers, animators, and solo video creators, this is an instant win. High-end voice acting has always been one of the biggest budget bottlenecks in production, and standard commercial TTS APIs sound too robotic for dramatic storytelling. Getting studio-grade acting out of a local GPU changes that equation.

More broadly, it highlights an exciting trend: cannibalizing big audiovisual models for specialized tasks. Lightricks built LTX 2.3 to make synchronized video and audio, but ScenemaAI proved that the audio transformer itself learned rich, human vocal nuances along the way.

We think the prompt design using stage directions is brilliantly intuitive. Treating speech generation like a script reading rather than a text conversion task is how human dialogue actually works. It still has rough edges around pronunciation and long-form consistency, but this is easily one of the most expressive open-source voice models available today.