Relit-LiVE relights video without camera tracking

Changing the lighting on an already captured video has always been a painful task in VFX. You either spend hours tracking camera poses to reconstruct the 3D scene, or throw a generative model at it and pray the shadows don't flicker like a broken strobe light.

A research team has tackled that exact failure mode with Relit-LiVE, a video relighting framework accepted to SIGGRAPH 2026. Secondary reports note its implementation alongside Wan2.1, delivering pose-free relighting for both videos and still images.

Jointly predicting the scene and the light

Previous relighting pipelines typically failed in one of two ways. Direct relighting methods broke spatial consistency and left nasty traces of the original illumination behind. Render-based pipelines looked more accurate, but they demanded strict camera pose tracking and fragile intrinsic decomposition.

Relit-LiVE bypasses the camera tracking requirement entirely by combining relighting and lighting prediction into a single diffusion process. It predicts the relit frames alongside viewpoint-aligned, per-frame warped environment maps.

Under the hood, an RGB-intrinsic fusion renderer feeds raw reference frames directly into the pipeline. This preserves subtle material roughness and fine surface details that purely intrinsic methods usually wash out. To keep things stable over long clips, multi-stage training pairs latent interpolation with cycle-consistent self-supervised learning.

More than a lighting toggle

The framework goes beyond swapping day for night. Because it understands the underlying illumination, it can handle multiple scene-editing jobs in a single pass.

One standout capability is video delighting, which strips baked-in shadows and specular highlights out of the footage while keeping temporal coherence intact across frames. It also supports material swapping and inserting 3D objects, allowing new assets to naturally inherit shadows and reflections without manual post-comp work.

Why it matters

We think dropping camera tracking priors is the real game-changer here. Most neural rendering tools collapse the moment they face jerky handheld clips, motion blur, or textureless surfaces that break matchmoving software. Relit-LiVE proves that diffusion models can internally align light and geometry without explicit camera supervision.

For visual effects artists, video editors, and game developers, this turns relighting and delighting into practical tools rather than technical bottlenecks. Being able to take a single video clip, strip the sun out, swap a surface material, and drop in a 3D asset in one unified workflow will save days of tedious prep work.

Generating per-frame warped environment maps alongside full video frames is bound to be compute-heavy, but it points to a smarter direction: fusing physical rendering principles directly into diffusion paths works far better than treating illumination like an afterthought.