Sakana AI's automated researcher lands on Nature

What happens when an AI doesn't just autocomplete your code, but actually dreams up a machine learning hypothesis, writes the code, trains the models, plots the charts, and compiles a clean LaTeX paper ready for peer review?

That was the question Sakana AI, along with researchers from UBC, the Vector Institute, and the University of Oxford, set out to answer 18 months ago. On March 25, the team saw their system, dubbed The AI Scientist, officially published in Nature.

Beating humans at their own peer-review game

At its core, The AI Scientist is a multi-agent system designed to automate the entire machine learning research lifecycle.

The system operates in two distinct modes. A template-based mode takes a starter codebase, such as a basic training script, and runs experiments from there, which is ideal for targeted tasks. A template-free mode gives the system complete freedom to write code and design experiments from scratch for open-ended exploration.

To run experiments, it uses a parallelized agentic tree search, writes scripts, inspects generated figures using a multimodal vision model, and typesets the whole manuscript in LaTeX.

To test how convincing the work was, the team submitted three unedited, fully machine-generated papers to a blind peer-review track at an ICLR 2025 workshop. One paper scored an average of 6.33, with individual scores of 6, 7, and 6, comfortably clearing the acceptance bar and beating 55% of human-authored submissions. As agreed with organizers beforehand, the paper was withdrawn before formal publication.

An AI review committee and the science scaling law

Churning out thousands of papers means nothing if humans have to read every draft. To close the loop, the researchers built an Automated Reviewer.

Acting like an academic Area Chair, the automated reviewer ensembles five independent reviews based on standard NeurIPS guidelines. Benchmarked against thousands of real decisions from OpenReview, it hit a 69% balanced accuracy rate and achieved an F1 score that actually beat human-to-human consistency measured in the famous NeurIPS 2021 experiment.

Using this automated grader, the team spotted a clean scaling law. When the underlying foundation model was upgraded to a more intelligent version, the quality of the resulting research papers jumped in tandem. Better base models naturally equal better automated science.

Why it matters

We see The AI Scientist as the clearest blueprint yet for how scientific research will change in the agent era.

For everyday machine learning researchers and graduate students, this is not an immediate ticket to outsourcing your PhD. The system is not inventing new paradigms out of thin air. But as a tire-kicking machine, it is remarkably potent. You can task it with testing ten speculative ideas over a weekend, running ablations, and drafting baselines while you focus on conceptual framing.

The danger, however, is paper-mill pollution on an industrial scale. If anyone can prompt an agent to generate passably good manuscripts, human peer-review infrastructure will buckle under the sheer volume. Worse, closing the loop with automated reviewers grading automated papers risks creating an echo chamber of confident, peer-reviewed nonsense.

Sakana AI's smartest move was publishing the full architecture openly and documenting its boundaries. For now, AI excels at grinding through incremental variations. The genuine breakthroughs still belong to human intuition, but researchers who refuse to adopt automated lab assistants will soon find themselves outpaced by those who do.