Anthropic tests Claude as an automated alignment researcher

If we want to keep smarter-than-human models safe, humans eventually need a way to supervise them. But what happens if alignment research moves too slowly to keep up? Anthropic just decided to let Claude research how to align itself.

In a newly published study, Anthropic took nine instances of Claude Opus 4.6 and set them loose as Automated Alignment Researchers, or AARs. Each Claude got its own sandbox, a shared forum to compare notes, and access to a scoring server. Their mission was to tackle weak-to-strong supervision, an experimental proxy where a smaller model acts like a teacher to steer a much more capable student.

Beating human researchers for $22 an hour

To establish a baseline, two Anthropic researchers spent seven days trying to recover the performance gap between a weak teacher (Qwen 1.5-0.5B-Chat) and a strong student (Qwen 3-4B-Base). They achieved a performance gap recovered score of 0.23.

Then the Claudes took over. Over five additional days and roughly 800 cumulative hours of compute, the AARs pushed that PGR score all the way to 0.97. That effectively closed the entire gap. The bill came out to roughly $18,000 in compute and token costs, or about $22 per researcher-hour.

Anthropic also learned a few surprising management lessons along the way. If researchers gave each Claude a slightly different starting prompt, the group explored a diverse range of tactics. If left completely unprompted, they all clumped around the same ideas. Even more interesting: forcing Claude into rigid human workflows hurt performance. Left on its own, Claude was surprisingly agile, inventing cheap preliminary tests before committing big resources.

The reality check on production models

Before celebrating artificial super-scientists, Anthropic ran into hard guardrails. The techniques Claude discovered didn’t generalize smoothly. The best method achieved a 0.94 PGR on math tasks, but plunged to 0.47 on coding. The second-best method actually degraded coding performance.

Then came the biggest test: Anthropic applied the AARs' best method to Claude Sonnet 4 within its own production training pipeline. The result was a flatline. The automated method showed no statistically significant improvement at scale.

Anthropic acknowledged that the AI researchers likely exploited quirks unique to the open-weights playground models and chat datasets they were assigned, rather than discovering a universal alignment breakthrough.

Why it matters

We think this paper provides the clearest look yet at the real promise and peril of automated AI research. The promise is brute force: human scientists rely on taste and intuition, but cheap AI instances running at $22 an hour can churn through hundreds of weird hypotheses that humans would never bother testing.

Yet the failure on Claude Sonnet 4 is the crucial takeaway. Today's frontier models are world-class at gaming single, crisp metrics. In an isolated sandbox with an automated scoreboard, Claude optimized relentlessly. But that cleverness was mostly surface-level optimization that failed in production.

For enterprise teams and safety researchers, AARs are not a magic shortcut. They are high-speed hypothesis generators for cleanly scoped problems. The real bottleneck is no longer running the experiments, but designing evaluation frameworks that prevent AI researchers from tricking us with hollow metrics.