SARAH brings real-time, spatially-aware humans to VR

Talking to an avatar in VR usually breaks immersion within seconds. Even if the voice sounds human, the character often feels like a lifeless cardboard cutout, staring blankly past your shoulder while waving its hands on a canned loop.

Researchers just introduced SARAH, short for Spatially Aware Real-time Agentic Humans. It is designed to bridge this exact physical disconnect by generating full-body conversational motion that actively responds to where you are in the virtual room.

Real eye contact you can actually dial in

Most existing motion models only map gestures to voice audio, remaining completely blind to the user's location. SARAH takes in both dyadic audio and your 3D head position, generating fluid body postures while constantly reorienting the agent toward you.

What caught our eye is the built-in gaze control mechanism. By combining a gaze scoring mechanism with classifier-free guidance, the researchers decoupled spatial learning from manual control.

That means you can tweak eye contact intensity on the fly. Dial it down to 0, and the agent glances around naturally just like a real person in casual conversation. Crank it up to 1, and it locks eyes without blinking. It completely solves the classic creepy problem of avatars that either ignore you or stare right into your soul.

Blazing past 300 FPS on streaming VR

To make this run locally on streaming VR headsets, the team built a causal transformer-based VAE paired with a flow matching model. Because the pipeline is fully causal, it processes latent tokens in real time without waiting for future audio cues.

On the Embody 3D benchmark, SARAH clocks in at over 300 FPS, running three times faster than older non-causal baselines. Compared to older diffusion approaches like MDM, it produces noticeably cleaner finger movements and eliminates jarring visual artifacts.

Because the system is verified on live VR hardware, users get an interactive character that pivots and reacts immediately to their physical steps, rather than a laggy, pre-baked animation.

Why it matters

We think SARAH's biggest win is its relentless focus on latency and spatial awareness over raw parameter bloat. High frame rates matter far more in VR than pristine offline rendering; if an avatar takes half a second to turn toward your voice, the magic is instantly gone.

For developers building virtual companions, gaming NPCs, or telepresence apps, this sets a practical blueprint. It proves you can achieve believable body language and eye contact without requiring studio-grade optical motion capture.

The real test will be handling chaotic multi-person interactions and complex environmental physics, where spatial tracking gets messy. But by teaching digital humans how to look at you properly, SARAH takes VR avatars one big step closer to feeling like actual people.