Tencent's WorldCrafter keeps generated worlds intact when you turn around

If you have ever explored an open-world video game, you expect the house behind you to still be there when you spin the camera around.

Generative video models, however, have historically been terrible at this. The moment a camera pans away, the scene vanishes from memory. Turn back a few seconds later, and windows have shifted, walls have melted, and the landscape has morphing into something completely unrecognizable.

Tencent researchers rolled out WorldCrafter, an interactive video world model built specifically to fix this spatial amnesia.

Turning around without breaking the illusion

WorldCrafter takes a single text prompt or an initial reference image and turns it into an explorable environment. Users can drive the camera along custom trajectories, wandering through static architecture or dynamic scenes for minutes at a time.

Even when moving subjects walk out of the frame and later reappear, the model tracks their continuity instead of inventing someone new.

To prove this is real 3D consistency rather than lucky 2D hallucination, the team reconstructed dense 3D point clouds and camera paths directly from the output videos. The geometry stayed rock solid across repeated revisits.

Across a benchmark of 145 scenes and 725 camera trajectories, WorldCrafter outpaced existing baselines on long-horizon revisit consistency and camera control accuracy, topping the overall VBench leaderboard.

The implicit memory trick

Most prior efforts tried either brutal temporal stacking or fragile explicit geometric warping and depth maps, which easily produce ugly tearing artifacts.

WorldCrafter bypasses explicit geometry altogether. It uses an implicit 3D-aware memory paired with a pose-conditioned readout module. When you steer the camera toward a specific angle, the generator queries past frames using target camera coordinates and extracts a compact set of memory tokens.

In plain English: the model keeps a mental map of what it already generated, pulling up only the relevant visual history when you look back in that direction.

The project page and code are out, featuring both a Base Model and a few-step distilled Fast Model capable of real-time streaming interaction. The catch? You will need around 147GB of storage space to house all components locally.

Why it matters

We think Tencent took a remarkably smart detour here. Instead of forcing heavy traditional 3D graphics pipelines into diffusion architectures, they let implicit neural representations handle spatial memory.

For interactive media and game developers, this is a massive milestone. It brings generative video one step closer to playable, on-the-fly virtual sandboxes where wandering around feels natural rather than fragile.

That said, the 147GB footprint means independent creators will not be running this casually on standard home rigs anytime soon. This remains heavy-duty infrastructure meant for well-equipped labs and production studios.

Still, WorldCrafter raises the competitive bar for the entire world model space. High visual fidelity is table stakes now. If your model cannot remember what was behind the user three seconds ago, it is not a real world.