Viggle swaps video characters using just a single repainted frame

Viggle-Animate is a new open-source video tool that replaces the main character across an entire shot from just a single edited frame.

All you have to do is take a driving clip, grab one frame, and repaint the character into whoever or whatever you want using your favorite image editor. Hand that painted still and the source video back to the model, and it maps the motion, camera tracking, and timing straight onto the newcomer.

No skeletons, no masks, and crisp fast motion

Traditional video replacement pipelines are notorious for running a marathon of preprocessing tasks. They usually string together pose estimators for wireframe skeletons, segmentation models for cutout masks, background plate generators, and face trackers. Every added model is another point where fidelity leaks out.

Viggle-Animate scraps the entire scaffolding. There are no pose skeletons, no segmentation channels, and no user-written text prompts in the video loop. The text encoder is never even loaded during generation.

Because your reference still is literally a repainted frame of the driving shot, the pose, framing, and ambient lighting naturally match the scene. The model doesn't have to guess how light falls on the subject. That makes it exceptionally solid on fast action: rapid head whips, flying kicks, and big jumps hold their shape frame for frame instead of dissolving into mush.

It also generalizes beyond humans effortlessly. Because there is no human pose skeleton enforcing a neck and two arms, you can animate animals, claymation figurines, robots, or even an airliner rigged to a dancer's limbs.

3-step distillation and 26-second renders

Under the hood, Viggle-Animate is a 33.1B full finetune of MiniMax-H3's ref2va transformer, distilled with DMD down to just three forward passes per shot.

Benchmarked on a single B200 GPU rendering 124 frames (480×832 at 24 fps, about 5.2 seconds), it finishes in just 26 seconds. Compared to Wan2.2-Animate-14B taking 160 seconds across 40 passes, Viggle renders 6.1x faster overall and over 10x faster purely on sampling, without even counting Wan's heavy preprocessing phase.

The catch right now is hardware. The repository ships the 33.1B finetuned transformer across 14 shards (roughly 66 GB) and a 2.5 GB LoRA delta. Running it cleanly in bf16 demands at least a 96 GB VRAM card, as an 80 GB card peaks out during rendering. You can use CPU offloading to drop active VRAM to around 12 GB, though it slows things down considerably. The team has tested consumer deployments using NVFP4 quantization on a 32 GB RTX 5090, but those scripts aren't in this initial drop.

As for limitations, heavy multi-character interactions and abrupt camera cuts still struggle. However, lip-sync is surprisingly solvable: by passing `--audio` to pin the driving clip's audio through the audio VAE, the model's mouth movements lock tightly onto spoken lines instead of remaining silent.

The weights and code are live on Hugging Face, ComfyUI community nodes are rolling out, and a hosted demo is available to play with.

Why it matters

This release introduces a remarkably practical design pattern for video generation. Instead of forcing a massive video model to simultaneously imagine what a character looks like and how they move, Viggle splits the workload.

State-of-the-art image tools are already fantastic at handling lighting, anatomy, and styling inside a single frame. Viggle-Animate simply lets the image editor do the heavy visual lifting once, leaving the video model to do what it does best: propagate motion. For animators, VFX artists, and meme creators, turning an entire video into a fresh animation now comes down to painting a single convincing still.