Anthropic drops Claude 3: Opus tops the benchmark charts

For over a year, GPT-4 has sat comfortably on the throne as the undisputed king of AI models. Today, Anthropic finally launched an offensive to knock it off.

Anthropic officially rolled out the Claude 3 model family, presenting a three-tier lineup designed for distinct intelligence, speed, and cost needs: Claude 3 Haiku, Sonnet, and Opus.

Opus and Sonnet are live right now on claude.ai and across their API in 159 countries, with the ultra-fast Haiku landing shortly.

Benchmarks fall, and the AI spots the test

At the high end, Claude 3 Opus takes the lead. In standard evaluation benchmarks like MMLU (undergraduate-level knowledge), GPQA (graduate-level reasoning), and GSM8K (basic math), Opus edged past GPT-4 across the board.

All three models now feature strong multimodal vision. They can read and process charts, graphs, technical schematics, and enterprise PDFs alongside text.

Speed is another major leap forward. Sonnet runs twice as fast as Claude 2, making it an ideal workhorse for sales automation and search. Haiku is even quicker, swallowing an information-dense 10,000-token arXiv paper packed with visuals in under three seconds.

Anthropic also fixed its notoriously stubborn refusal behavior. Claude 3 understands context far better, dramatically curbing unnecessary refusals on benign prompts while doubling factual accuracy compared to Claude 2.1.

On the 200k context window, Opus achieved over 99% accuracy on "Needle In A Haystack" stress tests. In one striking moment, the model even deduced that the target sentence felt artificially inserted by testers because it had nothing to do with the surrounding document.

Under the hood, Anthropic says all three models can accept inputs exceeding 1 million tokens, which will be unlocked for select high-throughput customers.

Why it matters

This is the first time a competitor has legitimately threatened OpenAI's flagship frontier performance in public benchmarks.

For everyday users and developers, Sonnet is already powering the free claude.ai experience, offering an immediate speed upgrade for coding and research. Paying users getting Opus via Pro or the API now have an elite alternative for messy corporate workflows, dense financial analysis, and massive PDF breakdowns.

Anthropic's three-model split is a savvy playbook. Instead of a one-size-fits-all giant, enterprises can route fast customer chats to cheap Haiku, handle core workflows with Sonnet, and save heavy cognitive lifting for Opus.

The real test will be scale. Top-tier reasoning comes with heavy compute costs, and Anthropic will need to prove Opus can maintain throughput without rate limits as usage surges. But for now, the frontier AI race is officially wide open again.