PhyMotion puts AI video inside a physics engine to fix broken limbs

If you have spent any time with AI video generators, you have almost certainly seen the horrors: people doing yoga who suddenly grow a third arm, or martial artists whose legs twist inside out while sliding across the floor.

Today's video models are great at rendering cinematic lighting and slow pans, but they fall apart the moment human bodies engage in complex motion like figure skating or acrobatics. The core problem is that video rewards typically rely on 2D perceptual signals. If a floating body or snapped joint looks reasonably sharp in 2D, the model often gets rewarded anyway.

Researchers dropped PhyMotion, an open-source reward framework built to fix this exact anatomical nightmare.

Putting 3D humanoids into MuJoCo

Instead of grading pixels on a flat screen, PhyMotion pulls the 3D human body out of the generated video clip.

It recovers SMPL body meshes from the video frames and retargets that motion onto a humanoid inside MuJoCo, the well-known physics simulator often used in robotics.

Once inside the physics engine, the movement gets evaluated across three distinct axes: kinematic plausibility, contact and balance consistency, and dynamic feasibility.

Basically, the system asks very specific questions. Are the joints rotating beyond human limits? Are the feet actually making solid contact with the floor, or is the character ice-skating through thin air? Does the center of mass make sense?

Because the feedback is structured and interpretable, the generator learns exactly which physical laws it broke, rather than trying to guess why a flat 2D discriminator disliked the clip.

A 68-point Elo jump

The team found that PhyMotion's scores correlate far better with human judgment than existing reward signals.

When plugged into reinforcement learning post-training, the reward drove major improvements across both autoregressive and bidirectional video architectures. In blind human evaluations, models tuned with PhyMotion scored a 68-point Elo gain over baseline rewards.

In practical terms, tricky prompts like kung fu kicks and yoga poses stopped producing phantom limbs and rubber joints. It is not completely flawless, as minor glitches still show up in extreme edge cases, but the code is now public with setup instructions for developers to run locally.

Why it matters

We think PhyMotion is a brilliant example of outsourcing hard problems instead of brute-forcing them with more parameters.

AI video models do not need to rediscover classical mechanics from scratch through raw pixels. By piping generated motions through an established physics engine like MuJoCo, the researchers gave video generators an objective ground truth for human biomechanics with modest training overhead.

For creators and game developers, this kind of physics-grounded post-training is the missing link for usable action footage. Right now, generating reliable dance routines or fight scenes is an exercise in endless prompt rerolling because models cannot keep limbs intact. Tools like PhyMotion could slash those failure rates dramatically once commercial video platforms adopt similar pipelines.

For the wider video AI race, the battleground is clearly moving past static visual quality toward physical realism. Expect more teams to borrow tricks from robotics and simulation to keep their models tethered to reality.