Why your favorite AI benchmarks are basically meaningless
A deep dive by The Markup found that widely cited benchmarks like MMLU and HellaSwag—heavily touted by Google and Meta—are fundamentally broken. Many tests rely on outdated datasets or amateur trivia that fail to test actual reasoning or safety. Why it matters: Bragging rights on leaderboards don't equal real-world reliability, especially in sensitive areas like medicine.