Runway launches Gen-1 to turn boring footage into stylized video

While everyone was still obsessing over text-to-image generators like Stable Diffusion, Runway quietly decided to tackle something much harder: moving pictures.

The startup, which actually helped co-create Stable Diffusion, just unveiled Gen-1. It is a new generative AI model that ditches the dream of conjuring videos from scratch, opting instead for what it calls video to video.

Basically, you take ordinary, everyday footage and feed it to the model alongside a text prompt or a reference image. Gen-1 then redraws the entire clip in whatever aesthetic you want, keeping the original motion and composition intact.

And this goes way beyond slapping on an Instagram filter. In Runway's demos, a guy walking down an ordinary street morphs into a handmade claymation. A clip of someone trudging through snow turns into grainy footage of the moon landing.

Five ways to hack video production

Instead of treating video generation like a single party trick, Runway split Gen-1 into five distinct creative modes aimed squarely at production workflows.

Stylization lets you apply the look of a single still frame or text description directly onto moving footage.

Storyboard turns rough mockups into fully stylized animations. In one clip, a stack of books sitting on a tabletop transforms into a cinematic nighttime skyline of skyscrapers.

Mask lets creators isolate a moving subject and swap its look while leaving the background untouched.

Render takes untextured, bare-bones 3D models and paints them into finished, lit scenes, while Customization offers granular control over all these variables.

Runway CEO Cristóbal Valenzuela told MIT Technology Review that the system incorporates years of insight into how filmmakers and VFX editors actually operate in post-production. The company's earlier tools were already put to work on last year's hit film Everything Everywhere All at Once, and Valenzuela claims we are now very close to seeing full-length AI-generated feature films.

Why it matters

We think Runway made a clever tactical choice here.

Pure text-to-video tools are notoriously hard to steer. Prompting a model to move a camera just right or keep a character consistent across frames usually yields a chaotic mess. By anchoring the generation to source footage, Gen-1 solves the control problem. The human filmmaker handles timing, blocking, and movement, while the AI does the heavy lifting on style and rendering.

For indie filmmakers, commercial animators, and concept artists, this could cut production cycles from months to days. You can literally shoot a prototype scene on an iPhone or block out a scene with cardboard cutouts, then let the model turn it into high-end concept footage.

That said, high-polish demos do not always reflect real-world messiness. Whether Gen-1 can maintain temporal consistency across long takes without visual flickering remains an open question until early research access expands in the coming weeks.