A city-scale world model lets you walk real Seoul streets

Most generative world models love to daydream. They conjure up plausible-looking cities purely out of thin air, but push the camera a few hundred meters forward and the world quickly turns into a melted mess of broken physics.

Researchers introduced Seoul World Model (SWM), flipping that entire approach on its head. Instead of asking a neural net to imagine an entire metropolis from scratch, SWM grounds its generation directly in the real streets of Seoul, producing stable, multi-kilometer video tours that don't fall apart.

Turning street views into video RAG

The secret sauce here is essentially retrieval-augmented generation (RAG) applied to video. Given camera coordinates, trajectory actions, and a text prompt, SWM queries a geo-indexed database of real street-view imagery to anchor each generated chunk.

To make those flat photos useful, the pipeline splits conditioning into two paths. A geometric route projects the closest street view into the target viewpoint using depth data to establish spatial layout, while a semantic route feeds the raw images into the transformer so the model can latch onto actual architectural details.

Working with real-world street view data is notoriously messy. Street views are taken at random dates with traffic everywhere, and relying on them directly usually leads to ghostly cars bleeding into new frames.

The team solved this with cross-temporal pairing. By deliberately pairing reference photos with target videos captured at completely different times, the model is forced to ignore passing traffic and focus strictly on permanent buildings and sidewalks. To patch blind spots where street-view cars never drive, they also trained on 10,000 synthetic videos generated in an Unreal Engine-based CARLA simulator across pedestrian paths and free-camera angles.

Dropping anchors into the future

Long-horizon autoregressive video generation almost always dies from compounding error. Most models cling to an attention anchor placed at the very first frame, but that reference becomes useless once you travel down the block.

SWM gets around this with what the authors call a Virtual Lookahead (VL) Sink. Before generating the next chunk, the model pulls a clean street-view image of where the camera is heading in the future and uses it as a forward anchor. It effectively keeps the camera from drifting off course by constantly re-grounding it to ground truth down the road.

In benchmarks across Seoul, Busan, and Ann Arbor, this setup let cameras roam for hundreds of meters and even multiple kilometers without turning into digital sludge. And because it remains a generative model at heart, you can still type in prompts to flood the streets or summon Godzilla right between real Gangnam high-rises.

Why it matters

We think SWM's biggest breakthrough is philosophical. Instead of waiting for massive foundational models to magically understand real-world geography through sheer compute, it shows that borrowing RAG from language models gives you photorealistic, long-range simulations right now.

For robotics and autonomous vehicle teams, this is a massive deal. Hand-crafting 3D synthetic cities in game engines takes months of manual labor, while SWM proves that publicly available street-level imagery can serve as a scaffold for scalable, physics-aware training grounds.

There are obvious catches, of course. The model cannot take you inside buildings, and its fidelity drops sharply wherever street-view coverage is sparse or out of date.

Still, grounding world generation in real-world geospatial data feels like the first genuinely practical path toward an interactive digital twin of our planet. It turns out video models don't need to dream up everything from scratch when the real world is already sitting right there on disk.