Meta rumored to prep Llama 3 to take on GPT-4 — still for free

Watercooler gossip at tech mixers can sometimes reveal more than official keynotes.

Jason Wei, an OpenAI engineer formerly with Google Brain, recently attended a generative AI social event hosted by Meta. While there, he overheard a conversation suggesting Meta now possesses enough computing firepower to train both Llama 3 and Llama 4. More importantly, the rumor claims Llama 3 is being built to match GPT-4's performance, and Meta plans to keep it freely accessible under its open license.

While Wei is a credible figure, this remains unconfirmed gossip. Internal roadmaps change, and Meta has shared zero public timelines. Still, the company moved fast this year, taking just five months between launching Llama 1 in late February and dropping Llama 2 in late July.

Closing the gap with GPT-4, however, will require far more than just burning extra GPU hours. GPT-4 is widely believed to rely on a sophisticated mixture-of-experts architecture, reportedly linking 16 expert models of roughly 111 billion parameters each. Meta's standard dense approach will likely hit walls, meaning Llama 3 might demand architectural overhauls that take much longer to stabilize.

Llama 2 currently touches GPT-3.5 territory in select workflows, and targeted variants like the new Code Llama have shown impressive benchmark spikes. Yet Meta itself acknowledged in its own technical report that a substantial gulf separates Llama 2 from closed heavyweights like GPT-4 and Google's PaLM-2.

Why it matters

If Meta pulls this off, it changes the unit economics of the entire software industry.

For builders and enterprise teams, a free, local model matching GPT-4 would eliminate the painful trade-off between frontier reasoning and data sovereignty. Paying OpenAI hefty API fees to process sensitive proprietary data becomes unnecessary if you can host an equivalent brain on your own clusters. It would instantly trigger a massive wave of private AI deployments.

Strategically, Meta is executing the Android playbook to perfection. As reports in mid-July indicated, Mark Zuckerberg's primary motive is dismantling OpenAI's early monopoly. Meta does not need to sell API access to survive; by commodity-level open-sourcing the base models, the entire global research community fine-tunes, patches, and accelerates Meta's ecosystem for free.

That said, matching a moving target is tricky. While Sam Altman noted earlier this summer that GPT-5 training hadn't kicked off, Google is already prepping its multimodal Gemini model for late this year or early next. If Llama 3 takes a year to emerge, matching the GPT-4 of today might simply leave Meta one generation behind whatever frontier labs ship tomorrow.