MetaRoCE: Meta's RDMA over Ethernet for AI, No PFC

What MetaRoCE is: an RDMA over Ethernet transport built for AI

MetaRoCE is a clean-sheet RDMA transport protocol that Meta built for AI workloads on commodity Ethernet, and it places its decisions in the endpoint NIC instead of the switches. RDMA (remote direct memory access) lets one machine read or write another machine's memory directly. The NIC (network interface card) is the network adapter inside each server.

Meta summed up the design in one line: "The fabric sees packets, but the NIC sees intent." Put plainly, a switch handles frames, while the NIC knows what the application is trying to move. Existing RDMA Verbs APIs and software stacks run unmodified, so the change sits in the transport and applications stay as they are. Meta announced MetaRoCE on August 24, 2026, together with the plan to release the specification, a reference software implementation and a compliance test suite through the Open Compute Project.

MetaRoCE vs RoCEv2: no priority flow control, packet spraying instead

RoCEv2 expects the network to deliver every frame in order and enforces that with priority flow control (PFC), while MetaRoCE treats the Ethernet fabric as lossy and uses neither PFC nor pause frames. PFC is a switch feature that pauses senders so that frames are not dropped. Meta writes that this reliance also discourages packet spraying, the technique that gives multiplane, large-scale networks their performance.

MetaRoCE takes the opposite route. It sees the network as many fine-grained logical paths, and each path keeps its own round-trip estimate, ECN state and utilization. Packets are sprayed across those paths and arrive out of order by design.

The fabric requirements Meta names are ECN marking (a congestion signal set by switches) and ECMP (equal-cost multipath routing, which spreads traffic over several routes). "RoCEv2 and MetaRoCE side by side" lines up the two designs.

RoCEv2 and MetaRoCE side by side

Aspect	RoCEv2	MetaRoCE
Where decisions live	Switches	Endpoint NICs
Packet ordering	Required in order	Out of order by design
Loss handling	Prevented via PFC	Assumed, no PFC or pause frames
Paths	Spraying discouraged	Per-path RTT, ECN state and utilization; packets sprayed
Fabric requirements	Losslessness	ECN marking and ECMP

How the NIC detects loss on an RDMA network without lossless Ethernet

Each path keeps a 256-bit selective acknowledgment bitvector, and a gap in that bitvector marks a loss, so exactly the missing packet is resent on the path that dropped it. Each bit in it records whether one packet has been acknowledged. The NIC reads a gap as lost data rather than as a late arrival, because every path carries its own ordered sequence.

Packets from different paths may interleave in any order at the receiver without being read as loss. Every packet also carries its own destination, so data lands directly in its final memory location even when it arrives out of order. Meta's design never asks the fabric to be lossless, which is the requirement PFC exists to enforce on a RoCEv2 network.

86% throughput at 1% loss: RoCEv2 tested, InfiniBand not

On a 64-node AMD GPU cluster, Meta measured about 86% throughput at 1% packet loss and says MetaRoCE kept delivering useful bandwidth even at 10% loss. Meta ran RCCL collectives on that cluster and compared MetaRoCE directly with RoCEv2 across all-reduce and all-to-all operations. It worked with AMD to implement the protocol on Pensando programmable NICs.

On that cluster, Meta reports throughput consistently higher and flow completion times consistently lower than with RoCEv2. The quoted sentence does not say what baseline the 86% is measured against, and it gives no RoCEv2 figure for the same loss rate.

Meta's multiplane tests covered 4-plane and 8-plane topologies with up to 4,000 concurrent connections, and throughput scaled linearly with the number of planes. In simulated plane failures, traffic redistributed on its own, with no action from applications or operators.

These are Meta's own measurements of its own protocol. The comparison runs against RoCEv2, and InfiniBand is not part of it, so the post gives no answer to the RoCEv2 versus InfiniBand question.

MetaRoCE at the Open Compute Project: specification due in October

Meta is contributing the full MetaRoCE specification to the Open Compute Project (OCP) for any vendor to implement, and the specification and reference implementation are due in October 2026. A compliance test suite comes with them. As of the August 24 post, the specification is not public yet.

The reference implementation is a library named libsoftmetaroce. It is a complete transport stack that runs on commodity Linux over standard UDP sockets, without specialized hardware. Meta also calls it the authoritative behavioral model for silicon development and the foundation of its unified compliance framework.

If you are tracking the protocol, October 2026 is the one announced date to watch. Until the specification appears, details beyond Meta's post cannot be checked.

Open problems Meta names for RDMA over Ethernet in AI networking

Meta names three areas still in development: scale-up inside racks, scale-across over thousands of kilometers, and storage and KV-cache traffic. Inside a rack, accelerators exchange small messages where every nanosecond matters, and Meta says it is now optimizing the fast signaling path for short memory operations.

Across buildings thousands of kilometers apart, round trips stretch into milliseconds, and Meta says the work ahead is sharing contended long-haul links fairly. In storage and KV-cache traffic, a single read can fan out to many servers that reply at once, and Meta says keeping its receiver-driven rate hints accurate is still a challenge.

The results have limits of their own. They come from a 64-node AMD GPU cluster with Pensando NICs and were measured against RoCEv2 only, so the post gives no figure for InfiniBand or for other hardware.

The specification itself stays unpublished until the October 2026 release Meta announced. What is on record today is a design that handles loss in the NIC instead of relying on PFC, plus results and a release date that come from Meta alone.