xAI debuts Grok-1.5 with 128K context window

Just two weeks after open-sourcing the older Grok-1 weights, Elon Musk’s xAI has pulled the curtain back on its next flagship model: Grok-1.5.

The update targets two of the original model's biggest limitations: mathematical reasoning and memory. Grok-1.5 expands its context window to 128,000 tokens, which is a 16-fold leap over its predecessor, and introduces major performance leaps across coding and math benchmarks.

According to xAI, the new model will roll out on 𝕏 to early testers and existing Grok subscribers within the coming days, with new features slated to follow gradually.

Serious math gains and a massive memory upgrade

The benchmark numbers tell a story of rapid catching up. On the MATH benchmark, covering tough competition problems, Grok-1.5 scored 50.6%, more than doubling the 23.9% achieved by Grok-1.

On the grade-school level GSM8K test, it jumped from 62.9% to 90%. For coding, Grok-1.5 hit 74.1% on HumanEval, outpacing Claude 3 Sonnet at 73%, Gemini Pro 1.5 at 71.9%, and the original March 2023 release of GPT-4 at 67%.

On MMLU, which measures broad knowledge, it rose to 81.3% from 73%, sitting roughly on par with Mistral Large, though still trailing frontier titans like Claude 3 Opus at 86.8%.

The 128K context window means the model can now digest substantially longer documents without losing the plot. In Needle In A Haystack retrieval evaluations, xAI says the model achieved perfect retrieval results across the entire 128K span.

Behind the curtain, xAI also revealed details about its infrastructure. The model was trained using a custom distributed framework built on JAX, Rust, and Kubernetes. The system is designed to automatically detect and eject failing compute nodes while streamlining checkpoints and restarts to keep multi-GPU clusters running without costly downtime.

Why it matters

For paying 𝕏 subscribers, Grok is finally graduating from an edgy conversational bot into a legitimate daily driver for writing code and digesting hefty documents.

For the wider AI race, xAI’s operational velocity is the real headline. Shipping an open model one week and a dramatically upgraded reasoning engine the next shows that Musk’s startup has solved its foundational compute bottlenecks and can iterate at hyperscale speed.

We think betting heavily on custom infrastructure built with Rust and JAX was a shrewd move. At cluster scale, training stability is often the single biggest drag on progress; tackling node failure detection head-on sets them up well for training even larger systems.

That said, xAI still has plenty of ground to cover. Benchmarking against the launch-day version of GPT-4 from early 2023 flatters the results, and Grok-1.5 still lags behind Claude 3 Opus across several primary metrics. The real test begins once early users start pushing its limits in the wild.