Sakana AI just unveiled Transformer², an adaptive system that adjusts LLM weights at test time to tackle different tasks like a living brain. Using Singular Value Finetuning and RL, it beats LoRA across math and coding benchmarks while requiring far fewer parameters.
Sakana AI just introduced Neural Attention Memory Models (NAMMs), a biologically inspired memory system that allows transformers to selectively discard redundant tokens. Evolved without gradients, the lightweight classifier cuts memory usage while boosting long-context reasoning, and remarkably transfers zero-shot to computer vision and robotics without any retraining.
Sakana AI researchers just developed CycleQD, an evolutionary framework that merges specialized small models into multi-skilled AI swarms without expensive fine-tuning. By using model merging and SVD for crossover and mutation, it outpaced traditional fine-tuning on Llama 3-8B tests, offering a cheaper path to balanced AI agents.
Sakana AI just unveiled The AI Scientist, a fully automated pipeline that handles the entire research lifecycle independently. Starting from a code template, it brainstorms hypotheses, runs experiments, plots figures, writes full LaTeX manuscripts, and even peer-reviews them for about $15 per paper with open-source code on GitHub.
Sakana AI just launched Evo-Ukiyoe and Evo-Nishikie, two image models dedicated to Japan’s historic ukiyo-e woodblock art. Fine-tuned on over 24,000 archival pieces from Ritsumeikan University, they can create traditional prints from Japanese text prompts or colorize antique monochrome illustrations. Both models are openly available on Hugging Face for research purposes.
