OpenAI's SimpleQA puts AI honesty to the test
OpenAI just open-sourced SimpleQA, a factual accuracy benchmark packed with 4,326 short, single-answer questions built to expose hallucinations. Frontier models like GPT-4o struggle hard, scoring under 40% correct. Why it matters: Measuring how often models fabricate answers—and how persistently they overstate their confidence—is the first concrete step toward fixing AI's chronic guessing problem.