Luma AI's Modify Video lets you swap characters without losing motion

Changing a video's setting without ruining the performance has always been a painful creative dilemma. Swap an actor's environment in traditional VFX, and you are usually looking at expensive green screens, motion tracking, and days of re-rendering. Even with first-generation generative video tools, prompting a scene change almost always scrambled the original actor's movements and facial ticks into a hallucinated mess.

Luma AI rolled out Modify Video, a video-to-video editing tool built straight into its Dream Machine: Ray 2 model. The promise is straightforward: it keeps the camera work, posture, lip sync, and performance intact, while letting you overhaul the environment, lighting, and characters around them.

Locking the take while swapping the world

Instead of treating a video as a blank canvas driven purely by text prompts, Modify Video relies on performance signals extracted directly from the source clip. It captures full-body pose dynamics, facial micro-expressions, and camera trajectory, and uses them as scaffolding for the new render.

That setup opens up a range of fast iterations. A director can film an actor walking through a dusty garage and swap the background into a bustling starship bridge, or map the human take directly onto a CG creature without losing the cadence of the original performance. For targeted fixes, users can also alter isolated elements like swapping wardrobe colors, aging a character, or inserting a UFO into the background without setting up green screens or manual tracking points.

Control is governed by three structured presets. Adhere prioritizes original structure and focuses mainly on retexturing. Flex strikes a middle ground between input fidelity and generative creativity. Reimagine goes fully unrestrained, translating humans into monsters or surreal surrealist entities. The tool supports 16:9 aspect ratio at 720p native resolution and can be driven entirely by visual references or first frames, making text prompts optional.

Luma noted that in blind evaluations with external raters, Modify Video consistently outperformed Runway V2V in motion retention, temporal consistency, and facial animation coherence.

Why it matters

We think the AI video race has officially transitioned from raw generation to pipeline control. Impressive text-to-video clips make for great social media demos, but actual production studios need tools that respect directorial intent and continuity across frames.

For indie creators, game developers, and previsualization artists, Modify Video substantially lowers the floor for high-concept prototyping. Instead of spending days rigging rough 3D models or staging green-screen mockups for a client pitch, a creator can record a rough phone video in their living room and instantly turn it into multiple stylized scene variants within minutes.

Luma made a smart move by packaging deep latent-space guidance into three digestible presets rather than overloading users with dense technical parameter trees. It feels designed for working artists who want fast, predictable results.

The trade-offs, however, are still tangible. The tool is currently capped at 10 seconds per run, and a 720p native resolution means you cannot export final cinema-grade deliverables out of the box. It is an exceptional sandbox for pitching, previz, and social edits, but true high-end finishing will still require upscaling and traditional composite work downstream.