OpenAI explores making its own AI chips

Sam Altman has plenty of cash, but he doesn't have enough GPUs.

OpenAI is now exploring making its own AI chips, according to Reuters. Internal discussions around an in-house silicon strategy date back to at least last year, and Altman has reportedly made snagging more AI chips a top priority. To make it happen, the company is weighing options ranging from acquiring an existing chipmaker to building out a custom design effort from scratch.

A mountain of debt and sold-out GPUs

Right now, OpenAI relies almost entirely on GPU clusters to train and serve heavyweights like ChatGPT, GPT-4, and DALL-E 3. But the explosion of generative AI has pushed hardware supply chains to the brink.

Nvidia's top-tier chips are reportedly sold out through 2024. Even Microsoft, OpenAI's biggest backer, cautioned in a summer earnings report that server shortages are severe enough to threaten service disruptions.

Then there is the sheer cost of keeping the lights on.

According to an estimate from Bernstein analyst Stacy Rasgon, if ChatGPT queries ever reached just 10% of Google Search's volume, it would take roughly $48.1 billion worth of GPUs upfront and around $16 billion annually in new chips just to keep it running.

Custom silicon is a brutal business

OpenAI would hardly be the first giant to try this path.

Google relies heavily on its custom TPU processors to train models like PaLM-2. Amazon offers Trainium and Inferentia chips to AWS customers. Microsoft is reportedly working with AMD on an internal AI accelerator code-named Athena, which OpenAI has supposedly been testing.

Financially, OpenAI looks sturdy enough to dabble. It has raised over $11 billion, is nearing $1 billion in annual revenue, and is reportedly mulling a share sale that could value the startup at $90 billion.

Yet history is littered with cautionary tales. AI chipmaker Graphcore reportedly saw its valuation slashed by $1 billion after a Microsoft deal crumbled and is now slashing jobs amid deepening losses. Intel-owned Habana Labs laid off roughly 10% of its staff, and Meta previously had to scrap experimental custom silicon after running into technical roadblocks.

Even if OpenAI goes all in today, spinning up custom hardware takes years and hundreds of millions of dollars each year. Whether investors like Microsoft will stomach the risk remains an open question.

Why it matters

For everyday developers and businesses building on top of OpenAI, this move highlights just how vulnerable the entire ecosystem is to compute scarcity. Capacity constraints dictate everything from rate limits and API pricing to whether ChatGPT goes down on a busy afternoon. That said, developing bespoke silicon is a multi-year marathon, so don't expect cheaper API credits anytime soon.

For the broader industry, this is an unmistakable shot across Nvidia's bow. When the world's most prominent AI startup feels compelled to design hardware, it proves that GPU pricing power and supply bottlenecks have become intolerable. It accelerates an industry-wide pivot toward vertical integration, where the biggest players will control everything from the transformer architecture down to the silicon die.

We think floating this plan is a clever tactical maneuver. Even if OpenAI ends up building only specialized inference chips or drops the effort entirely, just having the capability in development gives Altman serious leverage when negotiating allocations with Nvidia and infrastructure terms with Microsoft. But the danger is just as real: custom hardware is notoriously unforgiving, and sinking billions into silicon could distract OpenAI from the fierce frontier model race it is currently leading.