OpenAI’s GPT-5.5 edges out Claude Fable 5 on brutal new agent benchmark

AI labs love to boast about near-perfect scores on coding benchmarks, but ask an agent to finish a real workday inside professional software and things fall apart fast.

Researchers at UC Berkeley's Center for Responsible, Decentralized Intelligence (RDI), alongside over 300 industry experts, just released Agents' Last Exam (ALE). It is a brutal new evaluation designed to test whether AI can execute long-horizon, economically valuable professional workflows across 55 industries.

The initial results delivered an immediate upset. OpenAI’s GPT-5.5 from April, running via the Codex harness, took first place with a 24.0% pass rate. Anthropic's brand-new Mythos-class Claude Fable 5, released just yesterday, landed in third place with 22.0%.

Goodbye cheating, hello desktop reality

Older agent benchmarks like SWE-Bench Pro have suffered from fragile graders and models finding shortcuts. Past audits even caught models like Claude Opus digging through Git history to read hidden answer keys instead of solving tasks.

ALE shuts down those loopholes with a Generalist Computer-Use Agent framework. It breaks capability into five layers: Brain, Eyes, Body, Hands, and Feet. Agents must use visual perception and mouse-and-keyboard actions across Linux and Windows virtual machines.

The benchmark launches with 1,490 tasks anchored in federal occupational data (O*NET). Agents are thrown into heavy software, such as building 3D models in Siemens NX, setting up scenes in Unreal Engine, running neuroimaging analysis in FSLeyes, and compositing visual effects in Adobe After Effects.

Grading is equally unforgiving. ALE largely rejects the noisy LLM-as-a-judge approach, using it for only 6.8% of workflows. The rest rely on deterministic code evaluation that verifies the final digital artifact against expert ground truth.

To fight benchmark contamination, only about 10% of tasks are public on GitHub and Hugging Face. The remaining 1,300-plus tasks stay private in a rolling rotation.

The hardest tier breaks every model

While GPT-5.5 took the crown, a 24.0% pass rate shows how far AI remains from replacing skilled human workers.

Analysis suggests OpenAI’s edge comes from strict adherence to multi-part, complex prompts. Claude models, by contrast, can be forgetful during intricate pipelines, dropping critical steps along the way.

On ALE's hardest tier, titled Last-Exam, current AI hits a brick wall. Most configurations, including Google’s Gemini CLI and Anthropic’s Claude Opus 4.8, posted a flat 0.0% pass rate.

Why it matters

For enterprise teams, ALE delivers a much-needed reality check. If you were planning to hand off complete end-to-end professional workflows inside CAD or VFX software to an autonomous agent today, do not. A 24% pass rate on entry tasks and 0% on elite tasks means human-in-the-loop is not optional.

For the AI industry, this shifts the benchmark war away from synthetic coding puzzles and into long-horizon desktop orchestration. OpenAI beating Anthropic’s newest flagship with an older model highlights that reliable prompt adherence over hours of tool use matters far more than raw benchmark trivia.

We think ALE’s biggest win is its anti-contamination architecture. By keeping 90% of tasks locked away and ditching subjective LLM judges, Berkeley has built a benchmark that vendors cannot easily cheat. It turns out that when AI has to do actual work, the hype meets a very steep cliff.