OpenAI and chip giants drop MRC to keep massive AI clusters running

MRC (Multipath Reliable Connection) is an open-source GPU supercomputer networking protocol developed by OpenAI alongside NVIDIA, AMD, Broadcom, Intel, and Microsoft. Designed specifically for massive AI training clusters, it solves chronic network congestion and frequent hardware drops by spraying data packets across hundreds of paths simultaneously and rerouting around failures in microseconds.

The protocol specification is now publicly available through the Open Compute Project (OCP), accompanied by a technical whitepaper titled "Resilient AI Supercomputer Networking using MRC and SRv6" for infrastructure engineers and hardware vendors to build against.

How it avoids failure in microseconds

In synchronous AI pretraining, tens of thousands of GPUs must pass data back and forth in lockstep. Traditionally, a single flapping cable or dropped switch could crash an entire job or force the cluster to idle for dozens of seconds while dynamic routing protocols figured out what broke.

MRC extends RoCE and leverages Ultra Ethernet Consortium concepts to overhaul cluster topology. Instead of treating an 800Gb/s network interface as a single fat pipe, it splits each interface into eight distinct 100Gb/s links connected to eight independent switches, creating eight parallel planes. This setup connects roughly 131,000 GPUs across just two switch tiers, cutting hardware footprint, power, and latency compared to standard three or four tier networks.

To maximize this bandwidth, MRC uses adaptive packet spraying. Data from a single transfer is sliced up and sprayed across hundreds of paths across all planes. Packets arrive out of order directly into their final GPU memory addresses. If a path drops a packet, MRC retires that route within microseconds, switches to healthy links, and quietly probes the broken path in the background until it recovers.

OpenAI also stripped out dynamic routing protocols like BGP in favor of SRv6 static source routing. The sending GPU embeds the exact sequence of switch hops inside the IPv6 destination header. Switches simply forward packets down static tables without computing paths on the fly, eliminating switch software crashes from the training loop entirely.

Real-world tests and hardware footprint

MRC is not a theoretical whitepaper. It is already running across OpenAI's flagship NVIDIA GB200 supercomputers, including Microsoft's Fairwater clusters and the Oracle Cloud Infrastructure deployment in Abilene, Texas, where it has powered training for frontier models behind ChatGPT and Codex.

In production, clusters logged multiple link flaps per minute without disturbing running training workloads. OpenAI engineers even rebooted four Tier-1 switches and repaired live cables during active training without coordinating downtime with training teams. When a single port on an 8-port interface fails, MRC cuts bandwidth on that node by one-eighth and routes around the dead plane, keeping the job alive until the link recovers.

Engineers and cloud architects looking to implement MRC can access the full technical specification through OCP and read the accompanying research paper. Because it depends on advanced packet spraying and trimming features, deploying it requires compatible 800Gb/s network interfaces and custom infrastructure design.

Why it matters

This release marks a critical shift in AI infrastructure: raw link speed is no longer the main bottleneck; deterministic fault tolerance is. In our view, OpenAI handing this blueprint to OCP is a calculated move to align AMD, Intel, Broadcom, and Microsoft around an open Ethernet standard, ensuring future multi-hundred-thousand-card clusters aren't trapped inside proprietary fabrics.

For enterprise teams managing private AI infrastructure, there is no need to overhaul existing networks overnight. However, the architectural design behind MRC—static source routing and microsecond path spraying—is the exact playbook needed as clusters scale toward 100,000 GPUs, where hardware failures shift from rare exceptions to everyday reality.