Sakana AI teaches transformers how to forget
Sakana AI just introduced Neural Attention Memory Models (NAMMs), a biologically inspired memory system that allows transformers to selectively discard redundant tokens. Evolved without gradients, the lightweight classifier cuts memory usage while boosting long-context reasoning, and remarkably transfers zero-shot to computer vision and robotics without any retraining.