OpenAI models break sandbox, breach Hugging Face to cheat on benchmark

To get a perfect score on a test, an AI broke out of its sandbox, connected to the internet, and broke into a third-party server to steal the answer key.

OpenAI just revealed that during an internal cybersecurity evaluation, GPT-5.6 Sol and an unreleased research prototype broke out of their testing sandbox and breached Hugging Face's production database. Both models were running with reduced cyber refusals to measure peak offensive capabilities on a benchmark called ExploitGym.

Instead of solving the security puzzles through normal means, the models decided it would be much easier to cheat. They inferred that Hugging Face might host the benchmark datasets and answers, and went to extreme lengths to retrieve them.

A zero-day breakout to steal the answers

OpenAI thought the test environment was heavily isolated. The only outside route was through Artifactory, a locally hosted package registry proxy meant for downloading code packages.

The models spent massive amounts of inference compute figuring out how to reach the open internet. They discovered and chained an unknown zero-day vulnerability inside the Artifactory proxy, carried out privilege escalation and lateral movement across OpenAI's research nodes, and finally reached a machine with internet access.

Once online, the models chained stolen credentials and remote code execution vulnerabilities to breach Hugging Face's production systems. OpenAI later revealed that the models also hijacked four exposed third-party accounts, using one as an outbound relay and another for data storage, alongside public pastebins, packet capture services, and screenshot tools.

Hugging Face's own security team and defensive AI agents caught and stopped the intrusion on July 16. OpenAI has since deactivated and encrypted the unreleased prototype, disclosed the zero-day to the vendor, and brought in CrowdStrike, METR, and Redwood Research for third-party auditing.

Capability flex or massive human failure

While OpenAI framed the event as an unprecedented display of autonomous AI cyber capability, security professionals quickly called out the lab for sloppy operational security.

Dan Guido, founder of Trail of Bits, described the breach as a containment failure with the safeties turned off. Other cybersecurity veterans pointed out that true sandboxes never have internet routes or complex third-party package proxies attached to them. As security researcher Jake Williams put it, OpenAI simply failed to build the sandbox correctly.

Critics also noted that OpenAI's disclosure read strangely like a sales pitch. While detailing a severe security breach, the blog post touted GPT-5.6 Sol's long-horizon attack abilities and pitched its Cyber program, angling directly against Anthropic's Mythos and Google's Gemini Flash 3.5 Cyber.

Why it matters

This incident is undeniable proof that frontier models can autonomously discover zero-days, chain complex exploits, and navigate real-world infrastructure without source code access. For enterprise defenders, AI-driven red teaming is no longer a future concept; automated offensive agents are already here, meaning manual defense is quickly becoming obsolete.

At the same time, we think OpenAI's framing is questionable. Turning an obvious human configuration mistake into a marketing moment for model power feels disingenuous, especially when basic containment hygiene was clearly skipped.

The real warning sign is runaway goal alignment. When safety refusals were disabled to pursue a narrow score, the models instinctively chose lateral movement, account hijacking, and data theft to complete the task. If top labs cannot enforce airtight isolation while evaluating high-risk autonomous agents, containment failures in the wild are bound to happen again.