Sakana AI's Transformer² rewires its weights in real time
Sakana AI just unveiled Transformer², an adaptive system that adjusts LLM weights at test time to tackle different tasks like a living brain. Using Singular Value Finetuning and RL, it beats LoRA across math and coding benchmarks while requiring far fewer parameters.