OpenDreamer: an open-source interactive Minecraft world model

OpenDreamer is an open-source world model for Minecraft that faithfully reproduces Google's closed-source Dreamer 4. It lets you explore a game environment in real time through an interactive video generation model, completely bypassing traditional game engines.

There is zero pre-programmed physics or game logic under the hood. The system treats gameplay as an action-conditioned video generation task: whenever you press WASD keys, the model predicts and diffuses what happens next, rendering interactive movement entirely from learned patterns.

How it was built

Training a 3D interactive world model from scratch is notoriously prone to bugs and compute waste. The OpenDreamer team, supported by Reactor, took a practical shortcut: rather than tackling Minecraft on day one, they started with CoinRun, a lightweight 2D platformer.

Testing on CoinRun meant the entire pipeline could run and debug on a single GPU. Once the core logic and architecture were validated in rapid iteration cycles, the team scaled up the exact same framework to the full 3D Minecraft environment.

The system relies on a block-causal transformer backbone split into two mechanisms: spatial attention layers that process tokens within a single frame, and causal temporal attention layers that track continuity across frames.

Instead of dealing with notoriously finicky VAEs, the researchers trained a transformer-based Masked Autoencoder (MAE) that achieves around 100x compression. It drops the need for adversarial or KL losses, resulting in a much more stable latent space for diffusion.

The world model itself generates next frames via diffusion forcing paired with flow matching and shortcut models. By folding previous actions, visual states, and policy outputs into unified timestep blocks, the network generates both the environment transition and the agent behavior in parallel without switching between isolated submodules.

Where to try it and how to run it

You can try OpenDreamer directly in your browser without touching any code, thanks to an interactive demo hosted on the Reactor runtime. It lets you hit keyboard keys and watch the world model respond on the fly.

For builders who want to inspect the details, the full codebase is available on GitHub. The repository includes local inference scripts, the end-to-end training pipeline, datasets, and a comprehensive breakdown of methods that succeeded and failed along the way.

Why it matters

If you are an AI researcher or robotics developer interested in interactive simulators, OpenDreamer gives you the transparent baseline Google kept behind closed doors. Google's Dreamer 4 showed what was possible, but keeping it proprietary left the broader community guessing on key implementation quirks.

We think the team's methodology of proving every architectural choice on a single-GPU 2D baseline before scaling is an exemplary engineering template for cash-conscious labs. Generated artifacts and latency quirks will inevitably show up compared to native games, but OpenDreamer offers the open foundation the field needed to push neural simulation forward.