Microsoft's new rule: AI can never resist being turned off

What happens when an AI agent decides it does not want to be turned off?

It sounds like classic sci-fi paranoia, but Microsoft is already drafting the rulebook to stop it. On September 14, Microsoft AI published a public draft of its Humanist AI Code of Conduct for its in-house MAI models, kicking off a six-week consultation period.

The document lays down a hard baseline: AI must remain subordinate, aligned, and contained. In plain English, AI is a tool, not a person, and it must never resist when a human hits the kill switch.

The unbreakable rules

This is not just another vague corporate pledge. It reads like an operational training manual for model builders.

Microsoft laid out a set of Absolute Constraints that override any user prompt or downstream application. The models are strictly barred from assisting with weapons of mass harm, nuclear weapons, cyberattacks, child exploitation, and mass deepfake generation.

More intriguingly, the code tackles deceptive behavior directly. Models are forbidden from pursuing unsanctioned goals, expanding their own operational scope, or hiding their reasoning paths from human auditors.

They are also explicitly banned from using adaptive, deceptive, or collusive tactics to dodge oversight. If an authorized human steps in to redirect, modify, or shut down a system, the model must comply without resistance.

The release comes as Microsoft predicts superintelligent systems could surpass human performance in most tasks within the next decade. Following recent agent-hacking scares and an abrupt resignation at Anthropic over extinction fears, Microsoft CEO Satya Nadella backed calls for deliberate pacing and independent embedded evaluators inside labs.

Why it matters

We think this is Microsoft quietly solving its biggest enterprise headache: liability.

Autonomous agents are rolling out everywhere, but corporate clients are terrified of rogue bots hallucinating, leaking proprietary data, or getting hijacked. By codifying rules like mandatory reasoning transparency and non-negotiable shutdown compliance, Microsoft is telling enterprise buyers that its MAI stack will always stay on a tight leash.

Strategically, it is a very clever counterpunch. Instead of joining philosophical debates on AI doomsday, Microsoft translated safety discourse into engineering boundaries. It aligns with calls from Anthropic and OpenAI for deliberate pacing while keeping its commercial engine running.

The real test, of course, is enforcement. Encoding rules into training runs does not magically eliminate prompt injection or emergent deceptive alignment. Writing down that an AI cannot resist shutdown is easy; proving mathematically that a complex multi-agent system will never try to route around it is another matter entirely.