LivePortrait turns any still face into video in 12 milliseconds
Kwai's research team just open-sourced LivePortrait, an efficient portrait animation model that brings still human or animal photos to life using driving videos. Unlike diffusion models, it uses implicit keypoints to clock in at just 12.8ms per frame on an RTX 4090.
LivePortrait is an open-source animation framework that brings still portraits to life using a driving video. You feed it a single reference image of a human or animal along with a driving video, and it transfers the facial expressions and head poses onto the still photo to generate a smooth video.
It generalizes well across realistic portraits, oil paintings, sculptures, and 3D renders. Fine-tuning on animal datasets also allows human driving videos to control still images of cats, dogs, and pandas with impressive precision.
Skipping diffusion for 12.8ms frame times
Most modern facial animation tools rely on diffusion models, which look great but demand heavy compute and run painfully slow. LivePortrait takes a different path by expanding on implicit-keypoint frameworks to balance efficiency and control.
To make it generalize without losing realism, the team scaled training data to around 69 million high-quality frames using a mixed image-video strategy. They found that compact implicit keypoints can effectively represent facial blendshapes.
Building on this insight, they introduced a stitching module and two retargeting modules powered by tiny MLPs. These add virtually zero computational overhead while significantly improving motion controllability.
In terms of raw speed, LivePortrait hits a frame generation time of just 12.8 milliseconds on a single RTX 4090 GPU running PyTorch, outperforming heavy diffusion-based pipelines in practical throughput.
How to run it
Both the inference code and pretrained weights are completely open-sourced under the KwaiVGI organization on GitHub. There is also an interactive demo available on Hugging Face if you want to test your own images without local setup.
The system runs in two stages: training base extractors and decoders from scratch, followed by fine-tuning the lightweight stitching modules. For everyday users, downloading the official weights is enough to get local inference up and running.
Why it matters
Anyone building real-time avatars, digital humans, or lightweight video dubbing tools should test this out immediately. Diffusion-based pipelines are often too slow for practical interactive workflows, but a 12.8ms frame budget brings seamless animation within reach.
We think Kwai's biggest win here was ignoring the diffusion hype train. By doubling down on implicit keypoints and smart MLP stitching, they delivered a genuinely lightweight tool that gives developers high frame rates without sacrificing expressive facial control.
Sources (1)
- liveportrait.github.io(opens in a new tab)Originalliveportrait.github.io