Google unveils Nested Learning to fix AI amnesia

If you learn how to play tennis tomorrow, your brain doesn't instantly delete your ability to speak English. Current large language models, however, basically do.

Today's AI is stuck between two extremes: whatever fits into its immediate context window, and the frozen, static weights baked in during pre-training. Try to update an LLM continuously with new knowledge, and it usually suffers from catastrophic forgetting, trashing old skills to make room for new ones.

Google Research unveiled Nested Learning, a new machine learning paradigm published at NeurIPS 2025 that aims to rethink how neural networks learn from the ground up.

Merging architecture with optimization

For decades, AI research has treated neural network architecture and optimization rules as two separate disciplines. You build the model, and then you pick an optimizer to train it.

Google's researchers argue this separation is an illusion. Nested Learning treats an entire machine learning model as a system of interconnected, multi-level optimization problems running in parallel. Each level processes its own context flow, just operating at a different update frequency.

Under this hood, optimizers are simply viewed as associative memory modules. Traditional optimizers often lean on basic dot-product similarity, making them fragile when handling messy real-world data. By swapping the underlying objective to standard loss metrics like L2 regression, the researchers derived new formulations for momentum that make learning far more resilient.

Enter Hope and continuum memory

In standard Transformers, memory is essentially black and white. The sequence attention handles short-term context, while feedforward layers store static long-term knowledge.

Nested Learning introduces what Google calls a Continuum Memory System (CMS), turning memory into a full spectrum of modules that update at varying speeds.

To prove it works, Google built Hope, a self-modifying recurrent architecture inspired by the earlier Titans model. While Titans capped parameter updates at two levels, Hope achieves unbounded in-context learning levels by continuously optimizing its own memory via a self-referential loop. In public benchmarks, Hope posted lower perplexity, higher common-sense reasoning scores, and notably better retention in long-context Needle-In-Haystack evaluations.

Why it matters

For developers and engineers, this paper attacks AI's biggest structural pain point: models that cannot continuously learn in production without expensive, full-scale retraining. If this paradigm pans out, building agents that remember you over months without breaking their core reasoning becomes viable. Right now, though, Hope is still a proof-of-concept paper from NeurIPS, so don't expect drop-in Hugging Face weights by tomorrow morning.

For the AI industry, Nested Learning poses a direct challenge to the brute-force consensus. Instead of endlessly expanding context windows or layering fragile external RAG pipelines on top of frozen models, Google is asking whether the model's internal memory architecture should have been dynamic from day one.

We think unifying architecture design and optimization algorithms is an exceptionally clever theoretical leap. The big open question, however, is training stability at frontier scale. Letting a model modify its own memory loops across multiple update frequencies is notoriously difficult to tame when scaled to hundreds of billions of parameters, and proving this holds up in real-world clusters is the next massive hurdle.