OpenAI turns to Google TPUs to ease its Nvidia habit

There are no permanent enemies in tech, especially when GPU bills start hitting the stratosphere.

OpenAI has reportedly started leasing Google's custom Tensor Processing Units to help power ChatGPT and its expanding suite of products. According to sources cited by Reuters, this marks OpenAI's first major move away from Nvidia hardware, which has served as the backbone of its compute infrastructure since day one.

Slashing costs for 100 million daily users

OpenAI is not dumping Nvidia, but serving everyday inference is getting outrageously expensive.

ChatGPT now serves over 100 million active users every single day. Handling that mountain of queries eats up a massive portion of OpenAI's estimated $40 billion annual compute budget.

To manage costs, OpenAI is renting Google Cloud v6e Trillium TPUs. These chips are not Google's highest-end training processors, but they are built specifically for steady-state inference. They deliver high throughput at substantially lower operating costs than top-tier GPUs.

With this deal, Google Cloud adds another high-profile feather to its cap. Its growing roster of TPU customers now includes Apple, Anthropic, and Safe Superintelligence, alongside OpenAI.

Breaking free from the single-vendor trap

Microsoft Azure remains OpenAI's primary infrastructure partner by a wide margin, but relying solely on Nvidia silicon inside Azure carries clear risks.

GPU supply shortages and brutal pricing pressures have shown every frontier lab the danger of single-vendor lock-in. Bringing Google into the compute mix instantly gives OpenAI more capacity and real pricing leverage.

The real challenge will be software integration. The AI ecosystem has spent years optimizing almost entirely for Nvidia's CUDA platform. How deeply and quickly OpenAI can adapt its production code to run smoothly on Google TPUs remains an open engineering question.

Google declined to comment on the report, while OpenAI did not immediately respond to requests from Reuters.

Why it matters

We think this deal signals a massive shift in how the frontier AI race is run.

The industry is moving rapidly from the training era to the inference era. Training takes a fixed mountain of cash, but running chat interfaces for hundreds of millions of people is a relentless daily operational tax. Driving down the cost of every single token is now the difference between building a sustainable business and bleeding billions.

It is also a clear sign that Nvidia's software monopoly is not entirely bulletproof. When the industry's poster child is willing to port workloads over to TPU to cut inference bills, every other scale-up will follow. Google gets a massive strategic win by positioning its cloud hardware as the premier cost-effective alternative to Nvidia.

OpenAI is playing a smart game by hedging its bets across Microsoft, Nvidia, and Google. But running a hybrid compute stack across competing architectures is an engineering headache, and balancing relationships between your biggest backer and your fiercest rival is a high-wire act with zero room for error.