Nvidia reveals D-Rex: Expressive full-body avatars you can relight anywhere

Virtual production has a dirty secret: getting the lighting right on an actor is often ten times harder than keying them out of a green screen. Drop a digital human into a sunset scene with flat studio lighting, and the illusion instantly shatters.

Nvidia just introduced D-Rex, short for Diffusion Rendering for Relightable Expressive Avatars. The new framework tackles one of the toughest problems in digital humans: keeping subtle facial expressions intact while allowing free-viewpoint relighting under arbitrary environments.

Decoupling performance from lighting

Traditionally, making a full-body avatar that responds to new lighting requires explicit 3D intrinsic decomposition. The engine has to break the human down into geometry, skin reflectance, materials, and complex light transport physics.

When relighting is tightly coupled to 3D avatar modeling, facial animation usually suffers. Micro-expressions get smoothed out or distorted by noisy geometry registration, leaving characters looking stiff and lifeless in gaming and virtual production setups.

D-Rex takes a completely different path by untangling the two tasks. Instead of baking complex illumination physics into the 3D model, it treats relighting strictly as an image-space post-process.

First, an expressive full-body avatar is trained strictly under neutral white light. This base model only cares about getting human motion, poses, camera perspectives, and nuanced facial details right. It produces clean, flat-lit driving frames without worrying about shadows.

Generative AI as the virtual gaffer

Once the performance is rendered flat, a video diffusion model takes over the lighting rig.

Nvidia took a pre-trained video diffusion relighting model and fine-tuned it via LoRA on paired flat-lit and relit frames captured inside a light stage. The model takes the flat-lit avatar video and translates it to match any target HDR illumination environment.

Because the diffusion model brings strong generative priors, it naturally handles subsurface skin scattering, highlights, and clothing shadows while maintaining view and temporal consistency across frames.

Crucially, because the driving avatar is trained under standard white light, D-Rex does not require specialized modifications to the underlying 3D capture rig. You can plug it on top of almost any existing white-light full-body avatar system.

Why it matters

For years, digital humans have been split into two camps: rigid 3D rigs that are controllable but look artificial, and pure generative video that looks photorealistic but lacks fine-grained steering.

D-Rex bridges this gap by letting 3D handle the structural performance and generative diffusion handle the photorealism. We think this hybrid pipeline—3D for geometry and 2D diffusion for lighting transport—points toward the future of virtual production and gaming telepresence.

If you build games or virtual sets, this approach eliminates the need for heavyweight, finicky physical reflectance modeling just to change a skybox. You get full facial fidelity and dynamic lighting without compromise.

The catch is that D-Rex is currently an academic preview with a paper and project page, not an off-the-shelf software plugin. While the decoupling is brilliant, video diffusion models still run the risk of minor temporal artifacts during extreme, high-speed motions, which production teams will want to stress-test once code drops.