Meta introduces MetaRoCE protocol for million-GPU AI clusters

Meta has introduced MetaRoCE, a new RDMA transport protocol designed to connect AI clusters at a scale of up to one million GPUs using commodity Ethernet infrastructure. The protocol aims to address the networking bottlenecks encountered during the training and inference of frontier AI models, where synchronization delays across thousands of accelerators can lead to significant wasted compute capacity.

Unlike traditional RoCE, which relies on in-order delivery and discourages packet spraying, MetaRoCE shifts intelligence from network switches to the Network Interface Card (NIC). The NIC maintains separate telemetry for various paths, including congestion and round-trip time, allowing for packet-by-packet spraying. This approach enables packets to be written directly to their final memory locations without requiring reordering buffers, even if they arrive out of order.

Meta is releasing the MetaRoCE specification, a reference software implementation, and a compliance test suite through the Open Compute Project (OCP) to encourage industry-wide adoption.