Skip to content
IFAI

Chatbot Arena

Product2 stories

  1. IndustryReach 42

    Chatbot Arena under fire for helping AI giants game scores

    A new joint study from Cohere, Stanford, MIT, and Ai2 accuses popular benchmark LM Arena of favoring tech giants like Meta, OpenAI, and Google. The researchers found that big labs were granted extensive private testing to cherry-pick winning variants and received higher battle sampling rates. LM Arena pushed back against the claims as inaccurate, but promised to update its sampling algorithm.

  2. AI modelsReach 51

    OpenAI's updated GPT-4o reclaims the crown on Chatbot Arena

    OpenAI just took back the #1 spot on the LMSYS Chatbot Arena leaderboard with its updated ChatGPT-4o model, edging past Google's Gemini-1.5-Pro-Exp with a 1314 score. Tested anonymously across over 11,000 community votes, the new checkpoint shows noticeable gains in coding (up 30+ points) and hard prompts, and developers can already call it in the API.