Nvidia sweeps MLPerf training benchmarks with Blackwell Ultra

NVIDIA just turned training massive AI models into a coffee break.

In the latest MLPerf Training v5.1 benchmark suite, NVIDIA swept all seven industry-standard tests. It delivered the fastest time-to-train across every workload and stood as the only platform to submit results across the entire board.

Blackwell Ultra and the FP4 leap

The star of this round was the GB300 NVL72 rack-scale system, powered by the new Blackwell Ultra architecture, making its training debut after dominating recent inference rounds.

With an identical GPU count, the Blackwell Ultra-powered GB300 NVL72 delivered over 4x the pretraining performance on Llama 3.1 405B and nearly 5x the LoRA fine-tuning speed on Llama 2 70B compared to prior-generation Hopper systems.

Under the hood, Blackwell Ultra packs 279GB of HBM3e memory, doubles attention-layer compute, and dishes out 15 petaflops of NVFP4 compute via new Tensor Cores. Paired with the new Quantum-X800 InfiniBand networking, scale-out bandwidth doubled to 800 Gb/s.

The secret sauce, however, was training with NVFP4 precision. Low-precision math runs fast, but models risk losing critical accuracy. While standard Blackwell runs FP4 at double the rate of FP8, Blackwell Ultra pushes that to 3x. NVIDIA was the only vendor to successfully submit MLPerf training runs using FP4 precision while satisfying strict accuracy thresholds.

10 minutes to train a 405B giant

Scale was on full display. Pooling over 5,000 Blackwell GPUs, NVIDIA set a record by training Llama 3.1 405B in just 10 minutes. That is 2.7x faster than the best Blackwell result from the previous round.

Per-GPU gains showed up clearly as well. A cluster of 2,560 Blackwell GPUs finished the job in 18.79 minutes, running 45% faster than last round's 2,496-GPU setup.

The benchmark itself got a modern refresh. Llama 3.1 8B replaced BERT-large, where 512 Blackwell Ultra GPUs wrapped up training in 5.2 minutes. The state-of-the-art FLUX.1 replaced Stable Diffusion v2, and NVIDIA stood as the sole platform submitting results, clocking 12.5 minutes using 1,152 GPUs.

A massive ecosystem followed along, with 15 partners including Dell, HPE, Lenovo, Lambda, Nebius, and Supermicro submitting results on NVIDIA hardware.

Why it matters

For enterprise teams and researchers training frontier systems, these numbers fundamentally change iteration cycles. Slashing training runs from days to minutes means testing novel architectures can happen during normal working hours rather than across agonizing weekend batches.

Strategically, NVIDIA's biggest triumph here is validating FP4 for training. Competitors are still racing to stabilize FP8 training pipelines; by proving that NVFP4 works within MLPerf's strict accuracy confines, NVIDIA just moved the goalposts once again.

The catch? These massive generational jumps are no longer about raw silicon alone. They rely entirely on full-stack lock-in: NVL72 liquid-cooled racks, proprietary NVFP4 formatting, and Quantum-X800 networking. In our view, while the performance leap is undeniable, it tightens NVIDIA's grip on the entire AI data center stack, making multi-vendor alternatives even harder to justify.