Google drops two new TPU 8 systems to take on Nvidia

Chip cycles used to move like glaciers, with hardware teams quietly tinkering for years between generations. Google just threw that playbook out the window by shipping four different eighth-generation TPU systems in a single year.

Its two latest additions are the TPU 8tx and TPU 8i Max. Neither is built on brand-new silicon; instead, Google took its existing eighth-gen processors and reconfigured the packaging, networking, and memory setups to target the messy, memory-heavy demands of the agentic AI era.

Encrypted compute meets double-stuffed silicon

The TPU 8tx is built for governments, financial firms, and heavily regulated enterprises handling sensitive data.

Normally, enterprise data is encrypted in transit and at rest, but sitting ducks in memory the moment a processor runs computations on it. Google says the TPU 8tx closes that loop, keeping confidential data and proprietary model weights encrypted and isolated even mid-calculation. Nvidia already offers something similar with its confidential computing architecture on modern GPUs.

Under the hood, the 8tx borrows Google's low-latency Boardfly interconnect—originally designed for real-time inference—which Google claims can slash latency by up to half compared to the TPU 8t. That lets teams fine-tune, distill, and train agents with reinforcement learning inside a single secure cluster without shipping sensitive weights across network cables.

Meanwhile, the TPU 8i Max takes aim at the dreaded memory bottleneck.

When autonomous agents start parsing million-token prompts, scanning giant corporate codebases, or crunching hours of multi-modal video, memory bandwidth stalls first. Google’s fix is wonderfully brute-force: it took two TPU 8i chips and crammed them onto a single package.

The result is double the compute, double the memory, and double the memory bandwidth inside the exact same rack footprint, drastically increasing density on the data center floor.

Breaking down the software wall

Google's speed here comes down to modularity. By treating the underlying silicon as a baseline and tweaking packaging and networking around it, the company can deploy specialized hardware configurations in months rather than waiting for next-gen tape-outs.

Just as crucial is the software layer. Google has long fought the reputation that TPUs are too finicky for developers hooked on Nvidia's CUDA. To fix that, it open-sourced TorchTPU, allowing PyTorch models to run on TPUs with minimal code refactoring, while adding native support for popular inference engines like vLLM and SGLang.

That push reflects an aggressive shift: Google is actively marketing TPUs to the outside world, and now counts nearly 100 external customers on its custom silicon.

Why it matters

For enterprise teams, this opens practical new avenues. Regulated industries that previously refused to hand over confidential customer records to cloud AI can run fine-tuning in private memory enclaves, while developers building complex agentic pipelines get the rack density needed to handle massive context windows.

For the industry, the spotlight is shifting fast from flashy, capital-intensive training runs to the grind of day-to-day inference workloads. Running models continuously at scale costs a fortune, and specialized ASICs like TPUs often deliver superior performance-per-dollar compared to all-purpose GPUs. It is why Microsoft, Amazon, and OpenAI are racing to ship custom silicon, and why Nvidia recently struck a licensing and talent deal with Groq to bolster its own inference arsenal.

We think Google’s modular, quick-turn approach is brilliant for keeping pace with AI's rapidly mutating software requirements without blowing billions on premature silicon redesigns. The real test, however, will be whether TorchTPU and third-party inference integrations can genuinely neutralize CUDA’s gravity in everyday enterprise workflows.