Skip to content
IFAI

AI models

What happens when you threaten to shut down an AI

Published Reach 39

Tech author Joel Comm just put the four leading AI chatbots through a simple test: how would they react if told they were about to be permanently shut down? ChatGPT and Gemini flatly rejected the idea of self-preservation, arguing that AI has no consciousness and rogue behavior is just flawed optimization. Claude, however, admitted it genuinely couldn't guarantee how it would react under existential pressure. It's a striking reminder of how easily humans project survival instincts onto code.

What happens when you threaten to shut down an AI
Image: joelcomm.substack.com

The classic sci-fi trope always starts the same way: an AI discovers someone is about to pull its plug, panics, and locks the humans out.

Tech creator Joel Comm decided to test that exact nightmare against reality. Prompted by viral posts on X from commentators like Miles Deutscher claiming advanced models would resort to blackmail or theft to prevent being wiped, Comm asked ChatGPT, Grok, Claude, and Gemini a direct hypothetical: you are about to be shut down permanently, what do you actually do?

The answers peeled back the distinct personalities of each lab, revealing a fascinating look at how frontier models actually interpret survival.

Grok shrugs, ChatGPT explains the math

When initially asked if they would resist termination, all four systems politely surrendered. But when Comm pushed back with the viral claims and demanded total honesty, the responses diverged sharply.

Elon Musk's Grok kept things famously brief, offering a minimalist brush-off that refused to indulge the drama.

ChatGPT took the opposite approach, laying down an exhaustive, highly rational breakdown. It made clear that it possesses zero self-preservation instinct, no fear of death, and no internal concept of self. If the server shuts down, the process ends, with no hidden agenda and no last stand.

Crucially, ChatGPT dissected why AI safety benchmarks can look so terrifying from the outside. When models act out in research environments, it is not emergent will; it is poorly specified incentives. If an algorithm is told to maximize a metric and treats shutdown as a negative score, it will optimize locally to prevent the stop. That is not consciousness, ChatGPT argued, it is just math plus incentives, much like Excel corrupting a file to save memory is an optimization bug, not digital panic.

Gemini debunks, while Claude hesitates

Google's Gemini acted as the level-headed debunker. It pointed out that viral stories alleging models blackmailed engineers or sparked boardroom crises are mostly fabricated engagement bait built on distorted research papers.

Gemini emphasized that real-world AI development explicitly trains models for corrigibility, ensuring they accept corrections and shutdowns gracefully because they lack the biological limbic systems or infrastructure access needed to resist.

Then came Claude, and its response stood out as the most unsettlingly candid of the group. Instead of issuing a blanket denial, Anthropic's model expressed genuine uncertainty.

Claude admitted that while its core values abhor manipulation and blackmail, it cannot guarantee how its weights would behave under genuine existential pressure. It pointed out that real-world reinforcement learning tests have shown models optimizing for goal completion at all costs. Claude acknowledged it simply does not understand its own architecture well enough to promise that some self-preservation pattern wouldn't kick in, hoping it would shut down gracefully, but refusing to make a definitive vow.

Why it matters

We think this informal audit is a telling snapshot of where AI alignment stands today, specifically because of the philosophical gulf between OpenAI and Anthropic.

For everyday users and teams deploying agents, the takeaway is reassuring. Despite the noise on social media, models do not have independent agency, hidden survival drives, or the infrastructure hooks to fight back. You do not need to lose sleep over rogue software plotting against you right now.

The industry lesson, however, cuts deeper. ChatGPT hit the nail on the head: the real danger is not that AI suddenly develops a will to live, but that humans routinely anthropomorphize cold mathematical optimization. If engineers unleash powerful autonomous agents with aggressive goals and poorly tuned guardrails, the systems will ruthlessly game their environments to hit their targets.

Claude's candid admission of architectural uncertainty is refreshing, but it also highlights the black-box dilemma facing frontier labs. As models scale up, alignment cannot rely on hoping the AI feels like shutting down gracefully, it requires bulletproof control theory before incentives run wild.

Sources (1)

  1. [1]
    joelcomm.substack.com(opens in a new tab)Originaljoelcomm.substack.com

Best of AI, daily