Google DeepMind drops Gemma Scope 2 to peer inside Gemma 3's brain

Gemma Scope 2 is an open-source interpretability suite designed to act like a high-resolution microscope for language models, letting researchers dissect how Gemma 3 actually processes information under the hood.

It covers the entire Gemma 3 family, scaling from 270M all the way up to 27B parameters. DeepMind calls this the largest-ever open-source release of interpretability tools from an AI lab, requiring over 1 trillion trained parameters and roughly 110 Petabytes of storage to produce.

What you can do with it

LLMs can pull off impressive reasoning feats, but their internal gears remain notoriously opaque. When a model misbehaves, pinpointing why it went off the rails is usually guesswork.

With Gemma Scope 2, researchers can trace potential failure modes across the model's entire architecture. You can inspect how jailbreaks slip past defenses, see why hallucinations trigger, catch sycophantic behavior in the act, and verify whether a model's stated chain-of-thought actually matches its internal calculations.

The release also provides dedicated tools tuned specifically for Gemma 3's chat variants, allowing for deeper audits into refusal mechanisms and complex multi-step interactions.

How it works under the hood

The suite attaches sparse autoencoders and transcoders to every single layer of Gemma 3. These components unpack dense internal activations into distinct, human-interpretable concepts.

To make sense of algorithms spread across multiple network layers, DeepMind implemented skip-transcoders and cross-layer transcoders, making it much easier to follow complex multi-step computations.

They also leveraged advanced training setups like the Matryoshka training technique. This helps the autoencoders isolate richer features while fixing structural blind spots found in the first generation of Gemma Scope.

If you want to poke around without setting up local infrastructure, an interactive demo is already live courtesy of Neuronpedia.

Why it matters

For AI safety researchers and red teamers, this is an enormous gift. Mechanistic interpretability on modern frontier architectures has historically been gatekept by massive compute budgets, forcing outside researchers to play with toy models that never show the emergent behaviors seen at scale.

By open-sourcing production-grade probes for models up to 27B, DeepMind is effectively democratizing a 110-petabyte research pipeline. The catch is that interpreting billions of exposed features is still an uphill battle that requires significant engineering discipline, but it gives the open-source ecosystem a rare, genuine shot at understanding model safety from the inside out.