Researchers control video world models with VR headsets
Researchers just introduced a human-centric video world model that turns tracked head and hand poses from a VR headset into real-time, egocentric interactive video. Running at around 11 FPS, it lets users physically interact with generated environments instead of relying solely on text or keyboard controls. While fidelity is still early, it marks a significant step toward real-time generated XR.