OpenAI's GDPval tests AI across 44 real-world jobs

For years, AI benchmarks felt like watching models ace high school exams. They could solve impossible math equations and crush trivia, but ask them to clean up a messy corporate spreadsheet or draft a usable nursing care plan, and things would fall apart fast.

OpenAI just introduced GDPval, a new benchmark built specifically to measure how frontier models perform on real, economically valuable tasks across 44 occupations.

Real work, not academic trivia

Instead of making up synthetic exam questions like MMLU, OpenAI anchored this benchmark in real-world economics.

They started by looking at the top 9 industries contributing more than 5% to U.S. GDP, based on St. Louis Fed data. Using May 2024 Bureau of Labor Statistics numbers, they identified the top occupations by wage bill within those sectors. Then, using the Department of Labor’s O*NET database, they filtered for predominantly knowledge work—defined as jobs where at least 60% of tasks don't require physical labor.

The final list covers 44 jobs, ranging from lawyers, software engineers, and accountants to registered nurses, mechanical engineers, real estate brokers, and even private investigators.

The dataset packs 1,320 specialized tasks, with 220 released openly as a gold set. Crucially, these aren't simple text prompts. They include reference files, spreadsheets, and diagrams. Models must produce actual deliverables like legal briefs, engineering blueprints, care plans, and slide decks.

OpenAI worked with domain experts boasting an average of 14 years of professional experience to design the tasks. Each assignment underwent an average of five rounds of rigorous expert review before making the cut.

Head-to-head against human pros

Grading knowledge work is notoriously subjective, so OpenAI used blind evaluations. Expert graders compared model deliverables against human-created benchmarks without knowing which was AI-generated, rating each output as better, equal to, or worse than human work.

OpenAI tested its own stable—including GPT-4o, o4-mini, o3, and GPT-5—alongside rival flagships like Claude Opus 4.1, Gemini 2.5 Pro, and Grok 4.

They also rolled out an experimental automated AI grader on evals.openai.com designed to mimic human expert evaluations. However, OpenAI noted the automated grader isn't yet reliable enough to fully replace human professionals.

There is one big caveat: GDPval is currently strictly one-shot. Real-world office work requires back-and-forth drafts, clarification, and progressive context building, which this version does not capture. OpenAI plans to add multi-turn interactive workflows in future updates.

Why it matters

We think GDPval marks an overdue shift in how the AI industry measures intelligence. Academic benchmarks have largely saturated, and enterprise buyers don't care about a model’s SAT score—they care about whether it can replace or assist an actual worker on a Tuesday afternoon.

For professionals in those 44 jobs, GDPval is an eye-opening mirror of what current models can produce. That said, it is not an immediate green light for full automation. Because the benchmark tests one-shot outputs, it overlooks the messiest 80% of any job: context gathering, political nuances, and iterative edits. Relying on single-turn generation for high-stakes blueprints or legal briefs remains risky.

Competitively, OpenAI is trying to own the enterprise definition of AGI. By linking its evaluation directly to GDP contribution, it effectively forces competitors like Anthropic and Google to justify their models on economic utility rather than synthetic reasoning scores. The approach is clever, but reducing complex professional roles to deliverable grading risks overlooking how much human work relies on process rather than just the final slide deck.