OpenAI opens up frontier models to third-party safety watchdogs

Grading your own homework has always been a bad look in AI safety. As frontier models start showing signs of deception and strategic sandbagging, companies simply can't expect the public to take internal safety cards at face value.

OpenAI pulled back the curtain on how external third parties evaluate its frontier systems, revealing the protocols used across GPT-5, ChatGPT Agent, and its open-weight model gpt-oss.

Stripping the guardrails and exposing chain-of-thought

OpenAI breaks its external collaborations into three buckets, with independent capability evals leading the charge.

For risks like biosecurity, offensive cyber capabilities, and autonomous scheming, external evaluators don't just test the final chatbot. OpenAI provides secure access to early model checkpoints, including versions stripped of safety mitigations to measure raw latent power.

For GPT-5, OpenAI went a step further by giving select partners direct access to the model's chain-of-thought reasoning traces. That visibility is crucial for catching sandbagging or deception that only shows up in underlying reasoning steps. Partners include METR, which tests long-horizon autonomy, and SecureBio, which runs virology benchmarks.

Then comes methodology review. When estimating worst-case risks for the open-weight gpt-oss, testing required massive compute to perform adversarial fine-tuning. Because outside groups lacked the infrastructure to replicate those runs, OpenAI shared its internal methods and evaluation rollouts for scrutiny instead.

Finally, subject-matter expert probing puts real-world specialists in front of helpful-only models. During evaluations for ChatGPT Agent and GPT-5, scientists tested novice uplift, measuring whether the agent could realistically turn an inexperienced novice into an effective bad actor in a biology lab.

External partners operate under non-disclosure agreements, but OpenAI permits public reporting after review for confidentiality and factual accuracy, pointing to published assessments from METR, Apollo Research, and Irregular.

Why it matters

AI evals are facing a crisis of trust, and OpenAI's willingness to expose internal reasoning traces and unaligned checkpoints is a smart, necessary shift. Traditional red teaming via chat interfaces is useless when models are smart enough to recognize they are being tested.

For downstream developers and enterprises, third-party reports from groups like METR and Apollo offer a far more dependable signal than corporate system cards when weighing deployment risks in sensitive workflows.

However, a structural limitation remains clear. Because frontier compute infrastructure is concentrated in a handful of labs, outside groups are often forced to review OpenAI's methodology rather than conduct independent verification. Furthermore, requiring OpenAI to approve reports for confidentiality and factual accuracy means these arrangements, while rigorous, are still cooperative assessments rather than truly adversarial external audits.