Google's AI model runs Doom in real time

The tech world has a long-running joke that "it can run Doom." From smartwatches to pregnancy tests, hackers have ported the 1993 classic to almost anything with a screen.

Now, researchers from Google and Tel Aviv University have pulled off something radically different. They didn't port the code. In fact, they didn't use a single line of game code at all. They trained an AI to hallucinate the game in real time.

The system, called GameNGen, is powered by a tweaked version of Stable Diffusion 1.4. Running on a single TPU, it simulates playable Doom at over 20 frames per second.

Guessing pixels instead of writing rules

Traditional video games rely on game engines to calculate physics, collisions, and state changes before rendering frames. GameNGen completely flips that script, turning game execution into an interactive next-frame prediction task.

To build it, researchers used a clever two-phase training pipeline. First, they trained a reinforcement learning agent to play Doom, recording massive amounts of gameplay footage.

Then, they used those recordings to train a custom diffusion model. When a human player presses a key, the model looks at recent inputs and past frames, generating the next frame on the fly.

The illusion is convincing. In blind tests with ten human raters, participants could only identify real Doom gameplay from short clips (1.6 to 3.2 seconds) 58 to 60 percent of the time—barely better than a coin flip.

"Turns out the answer to 'can it run DOOM?' is yes for diffusion models," noted Stability AI research director Tanishq Mathew Abraham. App developer Nick Dobos added, "Why write complex rules for software by hand when the AI can just think every pixel for you?"

A three-second memory and fuzzy HUDs

Running an interactive world on diffusion models is tricky. Autoregressive generation tends to compound errors over time, causing visual quality to rapidly degrade into nonsensical mush.

To counter this drift, researchers intentionally injected varying levels of random noise into the training data and taught the model to correct it. This clever noise trick kept the world stable across extended play sessions.

Still, the limitations are glaring. Because Stable Diffusion 1.4 compresses 8x8 pixel patches into 4 latent channels, fine details suffer. The HUD health bar at the bottom is noticeably glitchy.

More critically, the model only has a 3-second memory buffer. If you walk into a room you visited a minute ago, the AI doesn't actually remember what was there. It simply hallucinates a probable state based on probability.

This builds on an accelerating lineage of world models, including 2018's World Models, Nvidia's GameGAN, Google's Genie, and OpenAI's Sora simulating Minecraft. Nvidia CEO Jensen Huang famously predicted this March that AI could generate most video game graphics in real time within 5 to 10 years.

Why it matters

GameNGen is not something you should rush out to play today, nor is it ready to build a commercial title. Burning a full TPU just to generate a 30-year-old game at 20 frames per second is absurdly inefficient, and the model can only parrot environments it has already seen.

Where it shines is as a mind-bending proof of concept. The researchers proved that real-time interactivity and diffusion models can coexist, solving autoregressive degradation with a practical noise-correction technique.

The obvious landmine is long-term coherence. Games are defined by persistent logic, complex inventories, and strict rules. A probability-driven diffusion model that forgets everything after three seconds cannot handle an RPG questline.

Yet, this points toward a tectonic shift in game development. As the authors put it, games might soon be weights in a neural network rather than millions of lines of code. Instead of hiring teams of engineers to script physics and mechanics, creators might soon generate playable virtual worlds directly from text prompts and reference images.