HumanX teaches humanoid robots to shoot hoops from human video

HumanX is an open-source full-stack framework designed to turn standard monocular human videos into agile, physically plausible interaction skills for humanoid robots. If you are developing humanoid control, it lets you teach robots complex tasks without touching motion-capture suits.

What it can do

The repository comes with code and pre-processed motion data to reproduce several complex skills directly in simulation.

The system has already trained a Unitree G1 humanoid to perform basketball catch-and-throw motions, basketball pump-fake shots, soccer kicking, and bin pick-and-place tasks.

More importantly, you can feed in your own everyday monocular videos. The pipeline extracts human-object interaction dynamics and converts them into control policies that the robot can practice in simulation.

How the pipeline works

HumanX divides the task between two core components named XGen and XMimic, using standardized motion files as the bridge.

The workflow begins with GVHMR, which recovers world-grounded human motion straight from a single-camera video feed.

Next, UMR retargets the extracted human motion onto the skeleton of the Unitree G1 robot.

Then XGen takes over. It edits and synthesizes physically plausible humanoid-object interaction motions, ultimately exporting robot and interaction data as .npz files.

Finally, XMimic converts those .npz files into .pkl motion files and trains imitation learning policies inside its humanoidverse simulation environment.

How to run it

The full project is available on GitHub under wyhuai/humanx, organized into xgen and xmimic subdirectories with local motion data and asset tools.

The codebase is released under the Apache-2.0 license with a Commons-Clause addition.

Developers can clone the repo to inspect robot motions with the XGen editor, synthesize interaction datasets, or jump straight into XMimic instructions to train, evaluate, and visualize policies in simulation.

Why it matters

Teaching bipeds high-agility sports like basketball usually requires multi-camera motion capture rigs or painstaking physics tuning. HumanX shows that widely available 2D video can serve as a direct source of interactive behavior, significantly lowering data acquisition costs.

However, the current release focuses strictly on simulation. While embodied AI and reinforcement learning researchers should definitely clone and benchmark the repo, bridging these dynamic policies to physical hardware and navigating the Commons-Clause license remain the next hurdles.