OpenAI launches LifeSciBench for real-world biology

Standard benchmarks for LLMs in biology have a glaring flaw: they mostly test whether an AI can ace multiple-choice exams. But in actual biopharma research, nobody asks you for the definition of an enzyme—they hand you conflicting Western blots, raw sequence files, and expect a viable experimental design.

OpenAI just dropped LifeSciBench, a new evaluation benchmark built to test whether AI systems can actually function as research-grade scientific collaborators rather than overqualified search engines.

173 Ph.D. scientists, 19,000 rubric criteria

Real-world life science research rarely produces neat answers. Scientists routinely have to balance incomplete assays, troubleshoot protocols, and weigh translational risks before deciding on the next step.

To build something that mirrors this reality, OpenAI recruited 173 scientists with Ph.D.-level training and direct drug discovery experience across biopharma. Together, they developed 750 tasks across seven applied workflows: evidence handling, analysis, design and optimization, scientific reasoning, validation, translation, and communication.

The dataset is deliberately messy. It features 1,062 attached artifacts, ranging from complex figures and tables to raw sequence data and chemical structure files. Overall, 79% of tasks demand multi-step reasoning (averaging four distinct steps), and more than half require synthesizing data directly from an attached artifact.

Grading is equally rigorous. The benchmark uses 19,020 expert-written rubric criteria—roughly 25 per task. If an AI delivers a seemingly correct top-line conclusion but overlooks a known assay constraint or fails to flag an essential biological caveat, it loses points. OpenAI also ran the entire benchmark past 453 independent expert reviewers, 97% of whom hold a doctorate and have an average of 12 years of industry experience.

Great at pitching theories, terrible at raw calculations

OpenAI tested both GPT-5.5 and GPT-Rosalind, a specialized research variant, and the results draw a very clear line between what current models can and cannot do.

GPT-Rosalind showed solid gains, pushing the overall exact pass rate from 25.7% on GPT-5.5 to 36.1%. The biggest leaps happened in high-level scientific communication (71.1% pass rate) and translation (57.7%), showing that models are getting significantly better at connecting preclinical laboratory findings to clinical trial implications.

However, when forced to do the granular, precise math that wet labs rely on, the systems collapsed. GPT-Rosalind's pass rate tanked to just 14.8% on numeric tasks and 24.0% on sequence or molecular structure outputs.

Handling multi-modal artifacts also proved to be a massive bottleneck. While GPT-Rosalind scored a 45.1% pass rate on pure-text prompts, that figure plunged to 28.1% as soon as a data table, gel image, or chemical structure was attached. For missions like CRISPR donor design or siRNA constructs, the model remains too brittle to trust with physical reagents.

Why it matters

LifeSciBench gives the biotech industry a desperately needed reality check.

For researchers and biotech startups, it clarifies how to deploy AI right now. LLMs are fantastic scientific sounding boards for organizing hypotheses, drafting review memos, and parsing messy literature. But using them to generate exact construct sequences or rely on their quantitative calculations is asking for blown wet-lab budgets. It is worth trying today for scientific communication, but keep human experts firmly in the loop for actual experimental design.

For the AI landscape, OpenAI's move shifts the competitive goalposts. Saturating generic biology QA tests no longer matters; frontier models will now be judged by their ability to execute multi-step scientific workflows with messy, non-text inputs. Anthropic and Google will inevitably face pressure to prove their models can handle real scientific artifacts without hallucinating sequence details.

We think OpenAI's approach is smart because it grades the reasoning process rather than just binary answers. In 14% of the tasks, models built sound reasoning chains but failed the pass threshold due to a minor arithmetic error or an overlooked biological nuance. Until frontier models can bridge that gap between conceptual understanding and exact computational precision, AI will remain a capable scientific advisor rather than an autonomous digital researcher.