LMArena just open-sourced Arena-Rank, the standalone Python package behind its widely watched AI model leaderboards. Rewritten with a JAX backend, the toolkit calculates rankings over 30x faster than before and introduces reweighting to fairly evaluate models with fewer match records. Beyond LLM evaluations, it can also rank head-to-head match data in sports and gaming.
Scale AI just launched SEAL Showdown, an LLM benchmark designed to challenge LMArena with real-world user preferences. Sourced from contributors across 100+ countries on its Outlier platform, it lets users slice rankings by profession, age, and language. Currently, GPT-5 sweeps every category.
A new joint study from Cohere, Stanford, MIT, and Ai2 accuses popular benchmark LM Arena of favoring tech giants like Meta, OpenAI, and Google. The researchers found that big labs were granted extensive private testing to cherry-pick winning variants and received higher battle sampling rates. LM Arena pushed back against the claims as inaccurate, but promised to update its sampling algorithm.
