Mocap Anything v2 animates arbitrary 3D skeletons from single videos

Motion capture has always had a creature problem. If you need clean movement for a human, you can rent a suit or book a mocap studio. But if your 3D asset is a four-legged beast, a bird, or an alien with six arms, animators usually have to keyframe every single joint by hand.

Researchers officially open-sourced Mocap Anything v2, an AI framework capable of turning regular monocular video into clean, ready-to-animate 3D skeletons for practically any creature.

Whether it is a real person, an animal, or a fictional character with unusual limbs, the system extracts motion and transfers it cleanly. Both inference code and training scripts are now public on GitHub.

Going fully end-to-end

Single-camera motion capture has tried solving arbitrary skeletons before, but older methods almost always ran a clumsy two-step pipeline.

First, a neural network predicted 3D joint positions from the video. Next, an analytical inverse kinematics (IK) solver tried to mathematically figure out how those bones were rotating. The catch is that joint positions do not fully define joint rotations. Twist along the bone axis remains ambiguous, and because the traditional IK solver is non-differentiable, the model could never optimize for the final animation output. The result was familiar to any 3D artist: jitter, unnatural flips, and broken wrists.

Mocap Anything v2 fixes this by making the entire process learnable from end to end. To clear up the rotation ambiguity, the researchers introduce a reference pose-rotation pair from the target asset alongside its rest pose, defining a clean local coordinate system.

By predicting rotations directly without relying on intermediate 3D meshes, and pairing the model with a skeleton-aware Graph-guided Multi-Head Attention module, the architecture gets both sharper and much leaner. Benchmark tests on Truebones Zoo and Objaverse show rotation errors plummeting from roughly 17 degrees down to around 10 degrees, reaching 6.54 degrees on unseen skeletons, all while running roughly 20 times faster than mesh-based alternatives.

Why it matters

For indie animators and game developers, this lowers the floor on creature animation significantly.

You no longer need a dedicated studio or custom tracking gear to rig quadruped or monster motion. A decent documentary clip of a tiger running or a video of your pet dog can now provide usable, animatable rotation bones that plug straight into Maya, Blender, or Unreal Engine.

For the tech stack, it bridges a frustrating gap in generative 3D. A lot of recent AI tools produce meshes that look great in video previews but collapse into a mess of gimbal lock once imported into a real rigging pipeline. Mocap Anything v2 targets the raw bone rotations that production rigs actually care about.

We think the team made a very smart tradeoff here: ditching heavy mesh reconstruction to focus entirely on clean, differentiated skeletal output. You still need a basic rest pose and reference pair for your target asset, so there is still a slight technical hurdle for complete beginners. But with the weights, code, and training pipeline now open on GitHub, it is one of the most practical animation tools to drop this year.