Anthropic offers $25k to break Claude's guardrails

Anthropic is paying hackers top dollar to break its unreleased AI safety guards.

The company announced a targeted bug bounty program in partnership with HackerOne, designed to stress-test an updated version of its Constitutional Classifiers before they roll out to the public. The ultimate target? Finding reliable, universal jailbreaks that could bypass safeguards around chemical, biological, radiological, and nuclear (CBRN) weapons.

Breaking the CBRN shields on Claude

Invited security researchers received early access to Claude 3.7 Sonnet to attack Anthropic's unreleased classifier system. Verified universal jailbreaks—defined as vulnerabilities that consistently bypass safety rules across multiple topics—fetch rewards of up to $25,000.

Anthropic emphasized that these protections are a critical component of its AI Safety Level-3 (ASL-3) Deployment Standard, part of its broader Responsible Scaling Policy designed to prevent catastrophic weapon-related risks.

The initial sprint was short, wrapping up on May 18. But Anthropic didn't hit pause: on May 22, the company revealed it was transitioning vetted researchers into a new bug bounty to stress-test the same constitutional safety system on the unreleased Claude Opus 4.

Why it matters

Frontier AI safety often feels like an internal corporate echo chamber, but opening pre-deployment classifiers to external bounty hunters is a pragmatic move.

For top-tier red teamers, a $25,000 bounty paired with exclusive early access to flagship models like Claude Opus 4 offers serious bragging rights and a lucrative playground. However, this isn't for hobbyists looking to make Claude say a bad word; Anthropic only cares about reproducible, universal breaches touching catastrophic threats.

For the broader industry, Anthropic is turning its self-imposed ASL-3 framework into an auditable public standard. By outsourcing stress-testing to HackerOne's network, it forces rivals to prove their CBRN defenses with actual third-party red teaming rather than vague white papers.

Still, relying on crowd-sourced jailbreaking has clear blind spots. Bounty hunters naturally gravitate toward known prompt injection vectors where rewards are easiest to verify. Whether rigid classifiers can withstand complex, agentic misuse in the wild remains Anthropic's real trial by fire.