DeepMind trains an army of AlphaZeros to beat itself

Even the strongest chess engines can get stumped by wicked puzzles designed specifically to exploit machine blind spots. AlphaZero conquered the board through self-play, but a lone brain can still develop tunnel vision.

Google DeepMind decided to solve this by ditching the lone genius model. Researchers built AZdb, an ensemble league of AlphaZero agents that play in radically different styles—and it just took down the original legend.

Building a grandmaster prep team

Instead of training one model to do everything, DeepMind used behavioral diversity and response diversity to force agents into different habits.

Behavioral diversity pushes agents to maximize the difference in average piece positions, while response diversity pits them against vastly different opponents. Over time, each agent developed distinct personalities, favoring specific openings, unique pawn structures, and varying piece survival tendencies.

DeepMind drew inspiration from human history, citing events like Kasparov versus The World, where Garry Kasparov admitted he had never expended as much effort in any game in his life. Top grandmasters never prep alone; they hire teams of players with contrasting styles to poke holes in their prep.

Two heads really are better than one

When tested on deceptive chess puzzles crafted to fool traditional chess engines, AZdb solved twice as many puzzles as a single AlphaZero when given ample thinking time.

For head-to-head play, DeepMind introduced what it calls sub-additive planning. Essentially, AZdb evaluates the board and deploys whichever agent specializes in that specific opening.

That dynamic tag-team approach gave AZdb a massive 50 ELO rating boost over standard AlphaZero.

Why it matters

Reinforcement learning often hits a wall when a single agent over-optimizes within its own narrow perspective. We think DeepMind’s shift toward a collaborative ensemble is a brilliant acknowledgment that raw compute cannot always brute-force past blind spots; diversity of perspective is required.

This matters well beyond chess. As AI labs race toward complex autonomous agents in software engineering and scientific discovery, the winner might not be a single monolithic model, but a coordinated league of models arguing from different angles.

That said, the overhead is real. Running multiple agents ballooned compute requirements, and picking the right agent via sub-additive planning works well in a turn-based board game, but it looks much harder to pull off in messy real-world workflows.