OpenAI reveals why language models can't stop hallucinating

If you take a multiple-choice test and have no clue what the answer is, do you leave it blank or take a wild guess?

You guess, of course. Leaving it blank guarantees a zero, while guessing at least gives you a fighting chance. As it turns out, our best AI models have been trained to play by the exact same bad incentives.

OpenAI published a research paper explaining why language models hallucinate. Their core argument is simple: current training and evaluation setups systematically reward models for taking blind guesses over honestly acknowledging uncertainty.

Leaderboards are turning AI into gamblers

Hallucinations are notoriously frustrating because models deliver wrong answers with absolute confidence. When paper co-author Adam Tauman Kalai asked a popular chatbot for his PhD dissertation title, it confidently generated three completely wrong titles. Asked for his birthday, it gave three different wrong dates.

Why does this happen? Think about the math behind public scoreboards. If an AI is asked someone's birthday and doesn't know it, guessing September 10 gives it a 1-in-365 shot at being right. Saying "I don't know" guarantees zero points.

Over thousands of benchmark questions, an aggressive guessing model ends up ranking higher than a careful model that abstains when uncertain. OpenAI pointed to SimpleQA results from its GPT-5 System Card: the older o4-mini model scored slightly higher on raw accuracy, but had a significantly worse hallucination rate because it gambled on uncertain answers.

Pretraining makes this worse. Large models learn grammar, spelling, and code syntax quickly because those follow consistent structural rules. But arbitrary, low-frequency trivia like a stranger's birthday has no underlying pattern, making guessing inevitable during next-token prediction.

Busting common hallucination myths

OpenAI also tackled several industry misconceptions head-on.

First, the idea that a 100% accurate model will eliminate hallucinations is a fantasy. Many real-world questions are inherently unanswerable due to ambiguity or missing information, meaning accuracy will never hit 100%.

Second, humility does not require frontier-scale compute. A tiny model that knows zero Maori can easily say "I don't know" when asked a question in Maori. A massive model that understands some Maori has a much harder time calibrating its confidence. Being well-calibrated requires far less compute than being universally accurate.

Finally, OpenAI argued that introducing standalone hallucination benchmarks won't solve the problem. As long as the primary leaderboards rank models purely on accuracy, developers will keep building models tuned to guess.

Why it matters

OpenAI is pulling a clever move here by reframing hallucinations from an elusive technical glitch into a flawed incentive problem.

For enterprise users and developers, this perspective is overdue. In critical domains like law, healthcare, and enterprise software, a confident lie is disastrous, whereas a model that admits ignorance and asks clarifying questions is genuinely useful. Teams evaluating models need to stop looking at raw accuracy benchmarks and start penalizing models that refuse to say "I don't know."

We think the hard part will be execution across the broader AI ecosystem. Negative marking and partial credit for abstaining are well-understood concepts, but getting competing AI labs to abandon simple leaderboard accuracy will be tough. In a marketing race where higher numbers win headlines, models that play it safe risk looking dumber on paper, even if they are vastly more reliable in the real world.