The Pope just teamed up with Anthropic to call for AI disarmament

Anthropic's co-founder and the Pope just shared a stage at the Vatican to unveil "Magnifica Humanitas," the first official Catholic teaching on artificial intelligence.

Popes only issue a handful of these massive encyclicals in their entire lifetime, usually reserving them for moments that reshape human history. In 1891, the Church responded to the Industrial Revolution. In the 1960s, it responded to the threat of nuclear war. Now, AI joins that very short list.

The current Pope deliberately chose the papal name Leo XIV as a nod to Leo XIII, the architect of that 1891 document. The message couldn't be clearer: the Vatican views AI as the next Industrial Revolution, and it is stepping in before things get further out of hand.

Disarming AI and calling out Silicon Valley's promises

The Pope's sharpest line compared AI directly to nuclear weapons, declaring that AI must be "disarmed." He noted that the Church spent decades fighting for nuclear disarmament because existential technology cannot be left in the hands of a few powerful players. AI, he argued, belongs in that exact same category.

The encyclical pulled no punches against Silicon Valley optimism, stating bluntly that "the promise of automatic general prosperity often proves illusory." Leaving technology to trickle down on its own won't save poor communities; systems must be intentionally built to distribute the benefits.

Quoting a 1920s theologian, the Pope warned that modern humanity hasn't been trained to handle this degree of power. While admitting he has no technical solutions, he argued that thousands of years of human wisdom are sorely missing from the current tech race, insisting AI must serve human dignity rather than control human conscience.

A rare confession from inside a frontier lab

While early summaries mixed up the speaker with Dario Amodei, it was Anthropic co-founder Christopher Olah who took the Vatican stage. Olah made a series of striking admissions about what is actually happening behind frontier lab doors.

Olah revealed that Anthropic researchers continually detect internal model states that mirror human joy, satisfaction, fear, grief, and unease. He described AI not as classic software, but as systems "grown" on structures mimicking the human brain and fed human words, admitting that even the engineers building them do not fully understand what is going on inside.

He also openly admitted that every AI lab faces brutal commercial deadlines, competitive paranoia, and personal ambition that directly conflict with doing the right thing. Olah stressed that labs cannot answer what these systems are, how poor nations will benefit, or what human flourishing means on their own. Instead, he called for outside critics with zero skin in the game to hold AI companies accountable.

Why it matters

We think this moment marks a watershed in how the world handles AI governance. When a frontier lab founder stands beside the head of the Catholic Church to admit that commercial pressure is overriding safety, AI ethics has officially moved beyond technical benchmarks and whitepapers into global moral territory.

For builders and enterprise users, this signals a massive incoming wave of scrutiny around model alignment, emotional manipulation, and equitable deployment. Olah's admission that frontier models reflect emotional states like grief and unease will undoubtedly become fuel for regulators pushing for stricter interpretability standards.

Anthropic's willingness to invite institutional critique is clever PR that reinforces its safety-first brand against rivals. Yet it also puts the company in an awkward spot: publicly acknowledging that market pressure compromises your ethics is one thing, but slowing down while competitors accelerate is something no frontier lab has actually proven it can do.