Nvidia’s H100 GPUs crush new MLPerf benchmarks

Training an AI model the size of GPT-3 used to mean parking a massive cluster of servers for days, if not weeks. Now, that timeline has officially shrunk to coffee-break territory.

NVIDIA just announced that its flagship H100 Tensor Core GPU swept all eight benchmarks in the latest MLPerf training round. Most notably, in the debut generative AI test, a massive commercial cluster of 3,584 H100 GPUs trained the GPT-3 workload in less than eleven minutes.

That record run wasn't staged inside a secret NVIDIA laboratory. It was conducted by GPU cloud provider CoreWeave on infrastructure co-developed with Inflection AI, the startup founded by Mustafa Suleyman and Reid Hoffman to power their personal AI assistant, Pi.

Near-linear scaling across thousands of chips

Beyond sheer raw speed, the latest MLPerf test showed what happens when thousands of H100s work together. Submissions demonstrated near-linear performance scaling as setups expanded from hundreds to thousands of chips.

CoreWeave also showed that running over the cloud delivered performance practically on par with an on-premises supercomputer directly managed by NVIDIA. The linchpin here was NVIDIA's Quantum-2 InfiniBand networking, which kept network latency from choking the cluster.

NVIDIA was also the only hardware maker to submit results across all eight MLPerf benchmarks, spanning computer vision, speech recognition, and medical imaging. In an updated, heavier recommendation system benchmark, NVIDIA was literally the sole company to submit data at all.

Hardware vendors clearly got the memo. Major server builders including ASUS, Dell Technologies, GIGABYTE, Lenovo, and QCT submitted more than 30 configurations powered by H100 chips.

Why it matters

We think this benchmark is less about NVIDIA bragging about silicon horsepower and more about proving its moat in large-scale cluster orchestration. For AI labs and enterprises building proprietary foundation models, shaving training times from weeks to minutes is what makes rapid experimentation viable.

It also highlights how hard it is for competing silicon makers to break in right now. When rivals fail to even submit validated benchmark scores for generative AI workloads, enterprise buyers and venture-backed startups default straight to NVIDIA hardware.

NVIDIA's playbook here is brilliant: by controlling the GPU, the InfiniBand networking fabric, and the containerized CUDA software stack, it turns raw compute into a plug-and-play cloud utility. The risk, however, is that foundational AI infrastructure is increasingly turning into an absolute single-vendor dependency with zero pricing leverage for anyone else.