OpenAI's new benchmark tests AI on PhD-level science

Two years ago, when the PhD-level science benchmark GPQA came out to stump AI, GPT-4 scraped together a measly 39%, way below the 70% expert threshold. Fast forward to GPT-5.2, and that same test was conquered with a 92% score.

With older benchmarks effectively saturated, OpenAI had to build a much steeper hill to climb. On December 16, the company introduced FrontierScience, an evaluation suite designed to test expert-level scientific reasoning across physics, chemistry, and biology.

Instead of testing trivia or multiple-choice tricks, FrontierScience aims straight at actual scientific inquiry.

Olympiad medals and PhD grinding

The full FrontierScience evaluation contains over 700 questions, with OpenAI open-sourcing a 160-question gold set while keeping the rest private to guard against test contamination.

The benchmark splits into two distinct tracks: Olympiad and Research.

The Olympiad set features 100 questions written by 42 former international Olympiad medalists and national team coaches, who collectively hold 109 medals. These are rigorous, short-answer theoretical problems designed to match or exceed the difficulty of international competitions, easily graded via exact numerical or algebraic matches.

The Research track reflects actual lab work. Here, 45 doctoral candidates, postdocs, and professors crafted 60 multi-step subtasks across disciplines ranging from quantum electrodynamics to synthetic organic chemistry.

Because open-ended research cannot be boiled down to a single answer, OpenAI devised a 10-point grading rubric that evaluates intermediate deductions as well as final conclusions. A score of seven or higher counts as correct. To make grading scalable, GPT-5 was enlisted as the automated rubric judge.

Acing tests while failing the lab

The initial results reveal a massive performance gap between structured competition and actual research.

On the Olympiad track, reasoning-heavy models excelled. GPT-5.2 led the pack with 77%, with Google's Gemini 3 Pro hot on its heels at 76%.

Once thrown into the Research track, however, the picture darkened fast. GPT-5.2 was still the top performer, but it managed only a 25% score even under extra-high reasoning effort. Other tested models, including Claude Opus 4.5, OpenAI o3, and OpenAI o4-mini, similarly hit a wall.

When OpenAI analyzed transcripts where frontier models stumbled, the reasons were familiar: basic calculation blunders, broken reasoning chains, misunderstandings of niche technical concepts, and occasional hallucinations.

Why it matters

For actual scientists and researchers, this release draws a very realistic line in the sand. AI is already an incredible lab assistant when it comes to well-defined, closed tasks like cracking a gnarly mathematical proof or synthesizing literature across obscure disciplines in record time. But expecting an agent to run an open-ended research task solo is still a pipe dream right now; at a 25% success rate, human guidance and rigorous fact-checking remain the entire ballgame.

For the broader AI race, the release shifts the goalposts. Standard multiple-choice tests are dead, and OpenAI is attempting to dictate how high-end reasoning gets measured. Competitors like Google and Anthropic will now have to tune their reasoning models against complex, multi-step experimental workflows rather than standard textbook problems.

We think splitting the benchmark into closed Olympiad puzzles and messy Research tasks is a smart, transparent move that lays bare the current limitations of reasoning models. The obvious vulnerability, though, is the model-based grader. Using GPT-5 to grade nuanced PhD-level scientific steps is a practical shortcut, but it risks baking the model's own biases directly into the scoreboard. Real scientific discovery is ultimately measured by physical experiments that reveal new truths, not by checking boxes on a synthetic rubric.