Tesla fires up massive 10,000 H100 supercomputer

Tesla just flipped the switch on a massive $300 million supercomputer packed with 10,000 Nvidia H100 GPUs, aiming to brute-force its way to autonomous driving.

The new cluster went live on Monday, immediately propelling Tesla into the ranks of the world's most powerful computing operators. The rig delivers 340 FP64 PFLOPS of peak technical compute and 39.58 INT8 ExaFLOPS for AI tasks. To put that in perspective, Leonardo, currently ranked as the world's fourth most powerful supercomputer, puts out 304 FP64 PFLOPS.

Chewing through 200PB of video

Tesla isn't building this just to flex on tech leaderboards. The hardware has one primary mission: training Full Self-Driving (FSD) models at unprecedented speed.

According to Tim Zaman, Tesla's AI infrastructure engineering manager, training autonomous vehicles on raw real-world video demands vastly more infrastructure than training large language models. Tesla's hot-tier cache capacity now exceeds 200PB, an order of magnitude larger than what most generative text AI labs deal with.

Musk recently noted that Tesla plans to spend over $2 billion on AI compute training in 2023 alone, with another $2 billion earmarked for 2024.

The Nvidia squeeze and Dojo

Even with deep pockets, buying enough Nvidia silicon is nearly impossible right now due to global shortages.

That bottleneck explains why Tesla is concurrently sinking more than $1 billion into Dojo, its custom-designed supercomputer powered by proprietary D1 chips. Dojo is coming online alongside the 10,000 H100 cluster, intended to handle fleet data processing and shoulder future FSD workloads.

Running both architectures side by side gives Tesla a dual-engine compute setup that traditional automakers simply cannot match.

Why it matters

We think this launch makes one thing brutally clear: the autonomous vehicle race is no longer just an automotive engineering challenge; it is a hyperscale infrastructure war.

For Tesla owners, pouring 10,000 H100s into 200PB of real driving footage should directly accelerate FSD's end-to-end neural network updates. Complex edge cases that previously took months to retrain can now be iterated in days.

For the broader auto industry, the barrier to entry just spiked dramatically. Legacy automakers still struggle with modular supplier architectures and decentralized ECUs, while Tesla is casually dropping billions annually on compute clusters that rival national scientific laboratories.

There is, however, an obvious risk in this dual-track strategy. Relying on Nvidia leaves Tesla vulnerable to supply chains, while Dojo's custom architecture still has to prove it can deliver promised efficiencies at production scale. But by deploying both at once, Tesla looks determined to buy self-driving victory through sheer computational force.