Google opens up Gemma Scope to peek inside language models

Gemma Scope is a comprehensive open-source interpretability suite designed to show you exactly what is happening inside Google's Gemma 2 language models. Basically, it acts like a high-powered microscope, breaking down dense neural activations into millions of understandable individual concepts.

Instead of inspecting just a single layer or a tiny toy model, Google trained sparse autoencoders across every layer and sublayer of Gemma 2 2B and Gemma 2 9B. The suite includes more than 400 sparse autoencoders with over 30 million learned features, scaling to hundreds of billions of autoencoder parameters.

How it actually works

When a model processes text, individual neurons fire for many completely unrelated ideas at the same time. A single neuron might light up for Michael Jordan and also for cooking an omelette. This superposition makes standard neural networks nearly impossible to read directly.

Sparse autoencoders fix this by leveraging sparsity. Even though a model knows millions of concepts, it only needs a tiny handful of them to respond to any specific sentence. Gemma Scope uses specialized neural networks to pull those mixed signals apart into distinct, readable features without human researchers ever telling the system what concepts to look for.

Google also built this using a new JumpReLU architecture. Standard autoencoders historically struggled to simultaneously detect whether a feature was present and estimate its exact strength. JumpReLU balances both sides properly, cutting down reconstruction error significantly.

Alongside Gemma Scope, Google also open-sourced Mishax, the internal interpretability tool that powered this entire research workflow.

Specs, compute, and how to test it

Pulling this off required massive engineering muscle. Google dumped around 20 PiB of raw activations to disk during training—roughly equal to one million copies of the English Wikipedia—and burned about 15% of the compute required to train Gemma 2 9B itself.

All SAE weights are completely open. If you want to explore the features without writing code or provisioning heavy hardware, an interactive demo is live on Neuronpedia. You can click through text prompts and watch features fire across different layers in real time.

If you prefer running experiments locally, the weights plug directly into modern mechanistic interpretability pipelines alongside open weights for Gemma 2 2B and 9B.

Why it matters

For AI safety researchers and ML tinkerers, Gemma Scope eliminates months of expensive GPU training and activation dumping. It turns Gemma 2 into the definitive testbed for cracking open reasoning chains, catching hallucinations before they surface, and debugging agent manipulation or jailbreaks.

We think Google made a smart, unusually generous move here. Interpretability has long been bottlenecked by the prohibitive cost of training large autoencoders across entire networks. The main hurdle now is interpretation itself: 30 million features are out in the wild, but figuring out what every single one does—and turning that insight into automated safety guardrails—is still an enormous open challenge.