OpenAI weighs building its own AI chips

When your product costs a small fortune every time someone types a prompt, you eventually start looking at the hardware bill and wondering if you can just make the parts yourself.

OpenAI is now exploring making its own custom AI chips, according to a report from Reuters. Internal talks around chip strategy have been underway since at least last year. CEO Sam Altman has reportedly made bagging more AI processors a top priority for the company, and options on the table include building an in-house design team or outright acquiring an existing chipmaker.

The GPU crunch has a price tag

Right now, OpenAI relies almost entirely on clusters of GPUs in the cloud to train and serve heavy hitters like GPT-4 and DALL-E 3. GPUs excel at running computations in parallel, which makes them the default engine for modern generative AI.

That dynamic has created a gold rush for Nvidia, but it has completely jammed up the supply chain. Nvidia’s top-tier chips are reportedly sold out through 2024. Even Microsoft sounded the alarm in a summer earnings report, warning that a severe shortage of server hardware could lead to real service disruptions.

Then comes the eye-watering price tag. Bernstein analyst Stacy Rasgon estimated that if ChatGPT query volume ever reached just 10% of Google Search, OpenAI would need roughly $48.1 billion worth of GPUs upfront, plus another $16 billion every year just to keep the lights on.

While OpenAI has banked over $11 billion in venture funding, is closing in on $1 billion in annualized revenue, and is reportedly discussing a share sale at a $90 billion valuation, Rasgon's math makes it obvious that relying purely on retail GPU pricing is not sustainable.

Everyone wants custom silicon, but it is brutal

OpenAI would hardly be the first tech heavyweight to walk this path. Google leans heavily on its in-house TPUs to power massive models like PaLM-2 and Imagen. Amazon offers AWS customers its proprietary Trainium and Inferentia chips. Microsoft is also reportedly teaming up with AMD on an internal chip dubbed Athena, which OpenAI is already testing.

Designing hardware, however, is a notoriously unforgiving game. The graveyard of AI chip efforts is already crowded.

British startup Graphcore had its valuation slashed by $1 billion after a deal with Microsoft fell apart, and it recently announced job cuts amid sliding revenue. Intel-owned Habana Labs laid off roughly 10% of its staff. Even Meta struggled for years with its internal chip designs, ultimately scrapping some experimental hardware entirely.

Building a custom chip from scratch takes years of runway and hundreds of millions of dollars annually. It is not yet clear whether backers like Microsoft want to underwrite that kind of risky detour.

Why it matters

For regular developers and ChatGPT subscribers, OpenAI making chips will not change anything tomorrow. Custom silicon takes years to tape out, produce, and deploy at data center scale. If you are frustrated by token limits or peak-hour slowdowns right now, custom silicon is not the immediate fix.

In our view, this looks less like an immediate pivot to hardware and more like a long-term defensive hedge. Relying solely on Nvidia leaves OpenAI with virtually zero pricing power and caps its growth directly at Nvidia's delivery dock. Even if OpenAI only pursues early designs or a modest acquisition, signaling serious intent gives Altman much stronger leverage when negotiating with Nvidia or trying to secure priority allocations of Microsoft's upcoming Athena chips.

The real risk is distraction. OpenAI's undisputed advantage is its concentration of elite software researchers and model trainers. Hardware development requires an entirely different operational muscle and carries a punishing margin of error. If OpenAI sinks capital and attention into a silicon money pit, it could easily dilute the focus that made it the frontier lab in the first place.