Researchers are using Super Mario to benchmark AI
UC San Diego’s Hao AI Lab just pitted top AI models against Super Mario Bros. using a custom framework. Anthropic’s Claude 3.7 took the top spot, while reasoning heavyweights like OpenAI’s o1 flunked out because taking several seconds to "think" in a platformer means instant death.