Google rolls out Trillium TPU for all cloud customers

While the entire AI industry is still fighting over Nvidia allocations and waiting months for delivery, Google just quietly took its most powerful internal weapon to market.

Google announced the general availability of Trillium, its sixth-generation Tensor Processing Unit, on Google Cloud. This is the exact same hardware infrastructure Google used to train its newly unveiled Gemini 2.0 model, and now any startup or enterprise can spin it up on demand.

Scaling up without the painful drop-off

On paper, the specs over the previous-generation Cloud TPU v5e are hefty. Trillium packs a 4.7x increase in peak compute performance per chip, doubles both High Bandwidth Memory capacity and inter-chip interconnect bandwidth, and pushes energy efficiency up by 67%.

More importantly, Google engineered this generation to solve the brutal communication bottlenecks that usually happen when clustering thousands of chips together.

A single Jupiter network fabric can now house over 100,000 Trillium chips with 13 Petabits per second of bisectional bandwidth. In real-world benchmarks pre-training a 175-billion parameter model, a cluster of 3,072 chips hit a 99% scaling efficiency. Even scaling that out to 6,144 chips across 24 pods, efficiency barely dipped to 94%.

To tackle memory-bound giants, Google gave Trillium hosts three times more DRAM than v5e. By offloading selective memory tasks to the host, model FLOPs utilization surged by over 50% when training Llama 3.1 405B.

Covering everything from embeddings to image generation

Beyond traditional dense transformers, modern architectures rely heavily on Mixture of Experts networks. Trillium handles MoE training roughly 3.8 times faster than v5e.

On the serving side, where inference bills usually bleed product margins, Trillium is showing strong numbers. In Stable Diffusion XL testing, image generation throughput jumped over 3x compared to v5e, while token throughput on large language models nearly doubled.

Google also slipped in its third-generation SparseCore, custom dataflow silicon built for embedding-heavy workloads. For ad-ranking and recommendation architectures like DLRM DCNv2, throughput spiked by 5x. AI21 Labs has already jumped on the hardware to train its next-generation Jamba and Mamba models.

Why it matters

For developers and AI companies running into GPU supply ceilings or burning cash on cloud credits, Trillium is a massive release. Google claims up to a 2.5x jump in training performance per dollar and a 1.4x boost in inference efficiency. If you are training MoE models or scaling high-volume image and recommendation pipelines, testing a slice on Google Kubernetes Engine today is an easy call.

For the wider landscape, this marks Google’s most aggressive move yet to prove that real frontier-scale AI does not strictly require Nvidia's hardware monopoly. By deeply optimizing the XLA compiler alongside PyTorch and JAX, Google is steadily lowering the historical software friction that kept teams glued to CUDA.

We think Google played this smartly by treating Trillium not as an isolated niche chip, but as a full-stack system covering networking, host memory offloading, and dynamic scheduling. The real catch, however, remains platform lock-in. Building around TPU clusters and proprietary network topologies makes multi-cloud portability painful, meaning engineering teams will need to weigh the raw price-performance gains against being tied directly into Google Cloud.