Groq scores $640M to take on Nvidia

If you have ever tried a text generation demo running on Groq, you know the feeling. The words don't trickle out like on standard cloud endpoints; they hit the screen in an instantaneous blur. That sheer inference speed just earned the startup a massive war chest.

Groq announced on Monday that it raised $640 million in a new funding round led by Blackrock. Neuberger Berman, Type One Ventures, Cisco, KDDI, and the Samsung Catalyst Fund also chipped in. The round values the Mountain View startup at $2.8 billion, more than doubling its $1 billion valuation from April 2021 and blowing past its original target of raising $300 million at $2.5 billion. Groq has now raised over $1 billion in total.

Heavyweights on board

The funding comes with high-profile personnel moves. Meta chief AI scientist Yann LeCun is joining Groq as a technical advisor. That is an intriguing alignment, especially since Meta invests heavily in its own custom AI silicon. Meanwhile, former head of Intel Foundry and ex-HP CIO Stuart Pann has joined as chief operating officer.

Groq was founded nearly a decade ago by Jonathan Ross, who helped invent Google's custom TPU accelerator, alongside former Google X engineer Douglas Wightman. Groq builds what it calls Language Processing Units, or LPUs. The pitch is simple: running transformer-based generative models up to 10 times faster than traditional processors while consuming one-tenth the power.

Developers have flocked to test the claims. GroqCloud, which serves models like Meta's Llama 3.1, Google's Gemma, and Mistral's Mixtral, surpassed 356,000 registered developers as of July. According to Pann, more than 75% of Fortune 100 companies are already represented among those developers.

The race to deploy 108,000 chips

Showing off blazingly fast API demos is one thing; selling physical silicon into enterprise data centers is another. In March, Groq acquired Definitive Intelligence to launch Groq Systems, a unit dedicated to helping government bodies and sovereign entities deploy LPU clusters in private infrastructure.

Commercial deals are piling up. Groq partnered with public sector contractor Carahsoft, signed an intent to place tens of thousands of LPUs in Earth Wind & Power's Norway facility, and is teaming up with Saudi Arabia's Aramco Digital for Middle East deployments. On the manufacturing side, Groq is shifting from legacy 13nm silicon to Samsung's 4nm foundry line, targeting over 108,000 LPUs deployed by the end of Q1 2025.

Why it matters

For builders, Groq changes the latency math. Real-time voice agents, customer support bots, and live code autocomplete all fall flat when models take seconds to stream tokens. Groq makes open-source weights feel like local software. If low latency is your core bottleneck, GroqCloud is well worth a pilot today, though users should remember that Groq's SRAM-heavy architecture can get expensive when handling massive context windows.

For the industry, this is proof that the AI chip battlefield is decisively tilting toward inference. Nvidia controls between 70% and 95% of the AI accelerator market, but tech giants like Amazon, Google, and Microsoft are building their own silicon, while well-funded upstarts like Etched and D-Matrix are nipping at the edges.

We think Groq played its cards brilliantly by building consumer hype through GroqChat and open-source models before going after sovereign data centers. But the hard part starts now: scaling to 100,000 custom chips in production while Nvidia compresses its roadmap down to annual release cycles is a brutal operational test.