Snap can clone dynamic characters from a single video
Snap Research just introduced Set and Sequence, a framework for personalizing text-to-video models using just one video clip. By splitting dynamic concepts into an appearance frame set and a motion sequence via two-step LoRA encoding, it lets DiT models capture both how a subject looks and moves, making character cloning far more flexible.
Teaching an AI model what your dog looks like is easy enough. Teaching it that very specific, goofy head shake your dog does whenever it gets excited? That has been a massive headache for video generation.
Snap Research just dropped a paper addressing this exact bottleneck. They introduced a framework to capture what they call "Dynamic Concepts"—elements defined not just by their static visual appearance, but by their characteristic movement.
Splitting the look from the motion
Most video personalization techniques stumble because they treat an object and its movement as one tangled mess. Snap's new approach, dubbed Set and Sequence, separates the two completely inside a DiT-based video model.
First, they break down a video into an unordered bag of individual frames, called the Set. They train a LoRA strictly on these shuffled stills so the model learns the visual appearance without worrying about time.
Next comes the Sequence step. The researchers freeze the LoRA basis learned from the static images so the appearance stays locked in. Then, they train temporal LoRA coefficients over the coherent video sequence to absorb the motion dynamics.
By layering the motion weights directly onto the frozen visual foundation, the model learns the signature movement without degrading the subject's identity.
Why it matters
Personalizing video generation has been stuck in the shadow of image generation for a while. In image models, you feed a few photos into LoRA or DreamBooth and you get a consistent character. But video demands consistency across both space and time, and trying to learn both simultaneously usually ends with distorted faces or stiff, generic motions.
We think Snap's two-step decoupling is remarkably clean. By forcing the model to nail down appearance on unordered frames first, it removes temporal confusion before teaching the motion.
If this makes its way into consumer tools, creators won't just be able to prompt custom characters into existence; they will be able to capture unique physical quirks, distinct dance moves, or brand-specific animations from a single reference clip.
That said, this is still research out of arXiv, not a clickable feature inside Snapchat or an open-source WebUI yet. How well it generalizes when a dynamic concept interacts with complex camera movements or extreme lighting shifts remains to be proven in the wild.
Sources (1)
- snap-research.github.io(opens in a new tab)Originalsnap-research.github.io