D-ID's new tool translates videos, clones voices, and syncs lips

Remember when the internet went crazy animating vintage family portraits to make them talk? D-ID, the startup behind that viral wave, is now setting its sights on solving one of media's biggest real-world headaches: video localization.

D-ID just rolled out AI Video Translate, a new tool designed to translate spoken videos across languages while cloning the speaker’s voice and rewriting their lip movements to match the new audio track.

The feature is powered by D-ID’s proprietary Rosetta-1 model and supports 30 languages out of the gate, including Mandarin, Spanish, French, Japanese, Hindi, and Arabic. The tool is available via D-ID Studio and its API, with a one-month trial currently live on the site.

Solo shots only, please

While the concept sounds like a creator's dream, D-ID has laid down some strict guardrails for what the tool can handle.

Videos have to run between 10 seconds and 5 minutes, with files kept under 2GB. Even trickier: the tool only supports a single person in the frame, and they need to stay facing the camera with their face visible the entire time to get acceptable results.

Pricing-wise, the translation feature is currently free for existing D-ID subscribers. Paid plans start at $56 per year for entry-level credit packages and climb up to $1,293 annually before hitting bespoke enterprise rates.

D-ID raised a $25 million Series B back in 2022 to chase U.S. enterprise accounts, pitching localized video as a cheap way for marketing, entertainment, and social teams to scale abroad.

A crowded room of AI translators

Multilingual audio used to be reserved for top-tier creators—like MrBeast, who famously used YouTube’s native tools to dub videos into 11 languages. But AI has kicked the door wide open.

Microsoft recently announced on-the-fly AI translation and dubbing for YouTube videos, and Vimeo dropped its own voice-cloning translation tools in July. Meanwhile, tools from ElevenLabs, Descript, Speechify, Veed, Captions.ai, and avatar-driven platforms like HeyGen are all fighting for the exact same marketing and creator budgets.

With open-source lip-sync libraries like Wav2lip lowering the barrier to entry, proprietary tech needs to stand out fast.

Why it matters

For solo creators and lean marketing teams targeting international markets, a unified tool that handles translation, voice replication, and lip-syncing without a multi-app workflow is genuinely useful. It turns global reach into a few clicks rather than a studio production bill.

However, the current guardrails tell an honest story: it is built almost exclusively for direct-to-camera talking heads. If you produce interviews, multi-host shows, or dynamic social clips where people look away from the lens, this won't work yet.

More broadly, voice-cloned lip sync is rapidly commoditizing. We think D-ID’s move is necessary to protect its enterprise turf, but with specialized rivals like ElevenLabs dominating natural voice quality and HeyGen owning the generative avatar space, D-ID's Rosetta-1 model has to deliver flawless realism to keep paying subscribers from wandering off.