Google opens up Gemma Scope to peek inside language models
Google just released Gemma Scope, a massive open-source suite of over 400 sparse autoencoders to help researchers decipher Gemma 2's inner workings. By isolating more than 30 million learned features across layers, the tool functions like a microscope for mechanistic interpretability. Why it matters: Cracking open neural network black boxes is key to catching hallucinations, deception, and safety vulnerabilities before they cause harm.