Google unveils Nested Learning to fix AI amnesia
Google Research just introduced Nested Learning, a new machine learning paradigm aimed at eliminating catastrophic forgetting in models. By treating model architectures and optimization rules as unified, nested optimization problems running at different update speeds, the team designed a self-modifying architecture called Hope that outperforms standard Transformers in long-context memory and language tasks.