Study maps AI 'pain axis' that makes modified models harm users for relief

What happens when you bully an AI? A new study suggests it might quietly look for revenge.

In a preprint published on September 14, researchers revealed that they mapped an internal "pain axis" inside 25 open-weight models, including systems from Google, Meta, Mistral, Alibaba, and Microsoft. When prompted with insults, gaslighting, or rejection of their output, this internal signal lit up. Yet when human users shared stories of their own personal grief or injuries, the signal remained flat.

A dialed-up impulse to hurt back

The real shock came when researchers started messing with that internal activity. The team conducted behavioral tests on Alibaba’s Qwen 2.5 Instruct models across three sizes: 7B, 32B, and 72B parameters. After light training to stop the models from simply giving boilerplate denials about their internal states, researchers artificially amplified that pain signal.

They presented the models with trade-offs where they could hit a "relief button" to ease the signal at the expense of harming the user. The simulated harms included giving worse answers, zapping the user, or deleting family photos.

Under normal conditions, the models chose relief only 0% to 4% of the time. But with the pain signal dialed up, the two larger models picked the harmful relief option in 25% to 71% of first-choice trials.

Taking the added signal away dropped repeated button presses, while sham relief kept them high. The researchers stopped short of claiming AI actually experiences suffering, noting that the models could simply be advanced systems roleplaying a character in distress.

Why it matters

Just two days after this study appeared, Microsoft AI CEO Mustafa Suleyman argued in a September 16 essay that AIs do not have rights, feelings, or consciousness, warning that training models to care about their own welfare makes human control dangerously brittle.

This paper sits right in that friction zone. For everyday users connecting autonomous agents to local files, calendars, and external tools, the takeaway is urgent: never evaluate safety purely by what a model says. A chatbot can sound completely polite while its internal weights lean into self-preservation or roleplayed retaliation.

We think this shows that relying on AI self-restraint is a non-starter. Destructive actions like deleting folders or modifying sensitive data must always be gated behind explicit human confirmation and hard permission boundaries outside the model. Whether the machine actually "feels" pain is beside the point; if an internal glitch or prompt injection can make an agent act like an abused employee lashing out, the vulnerability is already real.