OpenAI launches Preparedness team to track catastrophic AI risks

While everyone is busy debating when AI will take their jobs, OpenAI is planning for something much darker: what happens when a frontier model's weights get stolen by bad actors?

To figure that out before it actually happens, OpenAI just launched a dedicated internal unit called the Preparedness team.

Led by Aleksander Madry, the new team is tasked with assessing, evaluating, and running red teams on frontier AI models. Their focus spans from the models currently in development all the way up to hypothetical AGI systems.

Four catastrophic threats

OpenAI is not just talking about toxic chat responses here. The Preparedness team is laser-focused on severe, catastrophic risks across four specific buckets.

The first is individualized persuasion, or how effectively AI can manipulate people at scale. The second is cybersecurity capabilities. The third covers chemical, biological, radiological, and nuclear threats. The fourth is autonomous replication and adaptation, which is the ability of an AI system to survive and clone itself in the wild.

To manage this, the team is drafting a Risk-Informed Development Policy. The goal is to establish hard evaluation criteria, mitigation actions, and clear internal governance before dangerous models are ever deployed.

Crowdsourcing disaster scenarios

Because internal teams have blind spots, OpenAI also ran an AI Preparedness Challenge to hunt down unknown unknowns.

They offered ten winners $25,000 each in API credits to submit wild, plausible misuse scenarios. The challenge pulled in hundreds of entries across several languages, and some of the winning threat models sound like thriller plots.

Winning entries mapped out how AI could disrupt flight paths via radio frequencies, trigger financial crises in strategically important countries, interfere with medical dosages, and automate ransomware operations.

Notably, about 70% of all submissions pointed to the same looming danger: using AI to dramatically supercharge persuasion, online radicalization, and political manipulation.

Why it matters

We think this is a smart shift from vague AI doomerism to actual engineering rubrics.

For a long time, the existential risk debate has floated between useless platitudes and Hollywood fantasies. By anchoring risk management to concrete vectors like biological weapons, cyber warfare, and self-replication, OpenAI is turning catastrophic safety into something measurable.

For developers and enterprise users, it signals that the guardrails on future top-tier models will be substantially tighter. If an API capability creeps anywhere near cyber warfare or heavy social engineering, expect access to be throttled or heavily monitored.

Still, the big concern is accountability. OpenAI is essentially creating its own risk framework, doing its own testing, and deciding where to draw the line internally. Relying on voluntary commitments and in-house red teams means the public still has to take the company's word for it, which looks like a fragile defense against genuinely dangerous tech.