Google's mystery Gemini 4 Pro model leaks on Arena with SOTA scores
Google just started testing what looks like Gemini 4 Pro under a "gemini-3.8-flash" alias on the LMSYS Arena. The model generated an intricate mechanical butterfly SVG animation in just ten minutes and reportedly swept state-of-the-art scores in agent coding and OS benchmarks. Google might be back in the lead.
A mystery model labeled "gemini-3.8-flash" just quietly popped up on LMSYS Chatbot Arena, and the community is convinced it is Google's unreleased Gemini 4 Pro in disguise.
Leaked benchmarks and a 10-minute mechanical butterfly
The model quickly turned heads after generating an intricate mechanical butterfly in SVG in just 10 minutes, compared to roughly 30 minutes for Fable.
A leaked benchmark sheet paints an even more aggressive picture across agentic tasks.
On DeepSWE v1.1 for agentic coding, the model reportedly scored 88.7%, topping Astra's 86.9%. On GDPval-AA v2 for real-world knowledge work, it became the only model to break the 2000 Elo threshold, reaching 2064.
It also posted 95.3% on Terminal-bench 2.1 for terminal coding and 86.8% on OSWorld-2.0 for computer use, leading the field on both.
Why it matters
We think this shows Google is gearing up for a serious counteroffensive, zeroing in on hands-on execution rather than just conversational flair. Clearing 86% on OSWorld suggests the model is genuinely capable of navigating desktop environments and terminal pipelines.
For developers, if these numbers hold up upon release, Google could directly challenge frontrunners in software engineering workflows. Still, these impressions rely on blind tests and leaked slides; until an official API arrives, how it handles messy enterprise environments remains to be seen.
Sources (1)
- ithome.com(opens in a new tab)Originalithome.com