Meta delays its next big Llama model

Meta is finding out that spending $65 billion on AI hardware does not automatically buy you a working frontier model.

According to The Wall Street Journal, Meta has run into severe training roadblocks with its next flagship Llama model, codenamed Behemoth. Engineers are struggling to produce meaningful improvements over earlier Llama models, triggering heated internal debates over whether the upgrade is even good enough to justify a public release.

Meta shares dropped over 2% following the report, and the delays are piling up fast.

Pushed back to fall with an empty roster

Behemoth was originally scheduled to drop in April at Meta's first AI developer conference. That got bumped to June, and sources now say the launch has slipped to at least the fall.

While Meta won praise in April for shipping two smaller Llama models at a blistering pace, scaling up to the massive tier has proven far messier. Publicly, Meta claimed Behemoth outpaced rival models from OpenAI, Google, and Anthropic on select benchmarks. Internally, sources told the Journal that the model is wrestling with deep training hurdles, and real-world results fail to match the hype.

Meta's brain drain is not helping. The original early-2023 Llama breakthrough came out of the FAIR team, but 11 of the 14 authors on that seminal paper have since walked out the door. The team tasked with building Behemoth is entirely different.

On top of that, Meta had to swallow an embarrassing controversy in April after admitting it submitted an optimized, benchmark-specific model to an AI leaderboard that differed from the version shipped to the public.

The entire frontier is hitting the wall

Meta is definitely not suffering alone here. The entire industry appears to be slamming into the limits of standard pre-training scaling.

OpenAI has also pushed back its next flagship, GPT-5, rolling out an intermediate GPT-4.5 model instead to keep the hype cycle alive. Anthropic followed the same playbook by postponing Claude 3.5 Opus, although the company claims its release remains imminent.

Throwing tens of thousands of GPUs and petabytes of web text at transformer models is no longer yielding the effortless leaps we saw two years ago.

Why it matters

For developers and the open-source community, Llama is the backbone of independent AI development. Countless startups rely on Meta foot the training bill so they can self-host high-end weights. If Behemoth stalls out, the open-weight world will not have a competitive tier-one model to lean on, forcing builders to optimize existing weights rather than waiting on a savior.

For the tech industry, this is a glaring reality check on spending. Meta is on track for up to $65 billion in capital expenditures this year. Trading at a price-to-earnings ratio of 25.05 as of mid-May, Wall Street wants tangible returns, not benchmark gymnastics.

We think Meta's decision to benchmark-tune its public leaderboard submission was already a symptom of mounting executive anxiety. When you lose 11 out of 14 key researchers and your multi-billion-dollar runs produce marginal gains, throwing more compute at the problem ceases to be a viable strategy. If Meta cannot crack the scaling curve, everyone's AI roadmap will need a rewrite.