Alibaba's Wan-Dancer generates full dance routines from pure audio

Wan-Dancer-14B is an open-source music-to-dance video model developed by Alibaba's Wan team. Feed it a static character portrait and an audio file, and it automatically generates rhythm-synced, minute-scale dance routines without requiring any human dance reference footage.

Most current AI dance tools rely strictly on motion transfer. You have to feed them a video of an actual human dancer, extract the pose skeleton, and retarget it onto a character. Wan-Dancer bypasses this constraint entirely by planning novel choreographies straight from raw musical context.

What it can do

Wan-Dancer translates auditory rhythms directly into coherent body movements across a variety of genres, including hip-hop, Chinese classical dance, K-Pop, Latin dance, and tap.

Beyond automated end-to-end generation, users can insert custom keyframes into the pipeline to steer specific poses and control how the choreography unfolds over time.

To keep dancers from morphing or drifting over long durations, the architecture uses a two-stage hierarchical framework. First, a global planning module reads the entire track to set keyframe trajectories with long-range continuity. A second refinement stage then takes those frames and renders high-resolution video with temporal consistency.

Getting it running

The project is fully open-sourced under the Apache 2.0 license, with model checkpoints, inference scripts, and the technical paper publicly available.

Running it on local hardware requires serious horsepower. The downloaded weights total around 85GB, meaning you will need a high-end workstation GPU. If you want to bypass local setup, functional demos are already hosted on Hugging Face Space and ModelScope Studio.

Inference relies on running two successive bash scripts: one for global keyframe generation and another for local refinement. The developers also noted that official ComfyUI integration is actively in the works.

Why it matters

For digital creators, VTubers, and music video animators, this removes one of the biggest bottlenecks in character animation. You no longer need motion capture suits or stock dance clips to choreograph an original song. Feeding audio into an image to get a full routine is a huge conceptual leap forward.

That said, the 85GB footprint remains a steep barrier for casual users and indie creators on modest rigs. Until the community rolls out quantized GGUF weights or optimized ComfyUI nodes, Wan-Dancer is best suited for studios and researchers with enterprise-grade compute.