AnchorWeave turns single images into walkable 3D video worlds

Turn around in a video game and look back, and the tree behind you is still there. Try that in most AI video generators, and the model instantly forgets what it just drew, hallucinating a totally new room out of thin air.

AnchorWeave, an open-source framework released on February 16, takes aim at this exact goldfish-memory problem. It lets you feed in a single starting picture and generate a walkable 3D video world you can navigate with keyboard controls.

Trading messy global maps for local memories

Previous attempts at camera-controllable video generation tried to build full global 3D reconstructions in the background. The catch? Even tiny camera pose and depth errors snowball across views, turning the underlying geometry into a noisy mess that corrupts the video.

AnchorWeave takes a clever shortcut called retrieved local geometric memories. Instead of forcing a massive, misaligned global map to hold together, it breaks the scene into clean, localized 3D chunks.

When you move through the space, a coverage-driven retrieval system pulls the right chunks for your trajectory. A multi-anchor weaving controller then seamlessly stitches them into the diffusion process, reconciling perspective glitches on the fly.

WASD exploration from a single frame

You only need to provide a single initial image to get started. From there, you can steer the camera with familiar WASD or arrow keys, just like playing an exploration game.

Each iteration generates up to 81 frames, roughly a few seconds of continuous footage. Because the underlying geometric anchors stay fixed, you can daisy-chain multiple generations together to build out longer walkable clips.

It works beyond first-person camera fly-throughs, too. The framework can handle third-person view inputs, keeping both the moving character and the surrounding environment stable.

The project is fully open source on GitHub with local setup instructions, accompanied by a paper on arXiv. This initial release is built on CogVideoX, with the team planning to swap in stronger base video models down the road to boost fidelity.

Why it matters

Interactive world models are supposed to power the future of gaming, spatial computing, and robotics training. But hallucinated layouts have kept them stranded in toy-demo territory. If a virtual room changes every time you look away, you cannot build interactive software on top of it.

AnchorWeave stands out because it solves spatial persistence with a pragmatic engineering compromise. Instead of demanding flawless global 3D geometry from a generative model, it treats spatial memory as a retrieval and weaving problem.

For indie game devs and concept artists, turning a static concept art plate into a navigable 3D scene without opening Blender is huge for rapid prototyping.

That said, you should keep your expectations grounded. Running on CogVideoX means visual quality and texturing are still rough around the edges, and generating in 81-frame chunks requires patience. But as a proof of concept, it shows that persistent virtual worlds do not require heavy, fragile 3D pipelines to feel real.