RDMA
A technology that enables one computer to directly read from or write to the memory of another computer without involving the operating system of either host. By bypassing the OS kernel, RDMA significantly reduces latency and CPU overhead, making it widely used in high-performance computing and distributed systems requiring high-throughput, low-latency data transfers.
Core Facts
RDMA allows the NIC to transfer data directly between the memory of two machines, bypassing the CPU and OS, reducing latency to the microsecond level while consuming almost no CPU resources.
Cube Studio supports RDMA high-speed networking for communication optimization in large-scale distributed training.
Timeline (last 90 days)
GDS relies on local NVMe devices that support direct I/O, or on certified networked storage solutions (such as RDMA-based parallel file systems).
DeepEP is a library designed for MoE all-to-all communication, using NVLink within a single machine and RDMA across machines, supporting FP8 transmission, and the latest second version merges the two usage modes of high-throughput training/prefill and low-latency decoding into a single interface.
The HTTP protocol is easier to debug and easier for various language stacks to integrate than proprietary RPC, at the cost of possibly being inferior to shared-memory or RDMA solutions in ultra-low-latency scenarios; the specific trade-off depends on whether the protocol implementation supports batch and streaming transfer.
RDMA bypasses the operating system and CPU, having the network card hardware directly perform memory transfers, achieving microsecond-level latency and extremely low CPU overhead; Apple introduced a Thunderbolt 5 RDMA driver in macOS 26.2.
跨节点LLM推理的可行性根本上受网络互连延迟约束：在标准以太网上运行跨节点张量并行会引入巨大同步延迟，只有具备低延迟RDMA或高速专用链路才具备实用价值。
Modal 集群最多支持 32 个节点和 256 块 GPU，配备 3,200 Gbps RDMA 网络（B300 上为 6,400 Gbps），采用帮派调度并物理同地部署，每个节点必须使用其所有 GPU。
GPU发起的网络通信依赖于GPUDirect RDMA和RDMA两项关键底层技术
RDMA允许一台机器直接读写另一台机器的内存，绕过操作系统内核和CPU的介入，降低延迟
RoCE is an implementation of RDMA over standard Ethernet, with lower deployment costs compared to traditional RDMA deployments that require dedicated InfiniBand networks.
One of Homa's goals is to achieve near-RDMA low-latency performance on ordinary Ethernet while reducing dependence on lossless network characteristics.
5 more timeline events
All Facts (19)
RDMA allows the NIC to transfer data directly between the memory of two machines, bypassing the CPU and OS, reducing latency to the microsecond level while consuming almost no CPU resources.
75%VerifiedCube Studio supports RDMA high-speed networking for communication optimization in large-scale distributed training.
75%UnverifiedMooncake通过RDMA网络实现近乎零拷贝的跨节点KV Cache共享
85%UnverifiedRDMA技术可以将网络通信延迟降低到亚微秒级，在大规模分布式训练中可将训练效率提升20%-50%
75%UnverifiedIn the training of hundred-billion-parameter large models, inter-node communication overhead can account for 30%–50% of total training time.
60%UnverifiedGDS relies on local NVMe devices that support direct I/O, or on certified networked storage solutions (such as RDMA-based parallel file systems).
50%UnverifiedDeepEP is a library designed for MoE all-to-all communication, using NVLink within a single machine and RDMA across machines, supporting FP8 transmission, and the latest second version merges the two usage modes of high-throughput training/prefill and low-latency decoding into a single interface.
50%UnverifiedThe HTTP protocol is easier to debug and easier for various language stacks to integrate than proprietary RPC, at the cost of possibly being inferior to shared-memory or RDMA solutions in ultra-low-latency scenarios; the specific trade-off depends on whether the protocol implementation supports batch and streaming transfer.
50%UnverifiedRDMA bypasses the operating system and CPU, having the network card hardware directly perform memory transfers, achieving microsecond-level latency and extremely low CPU overhead; Apple introduced a Thunderbolt 5 RDMA driver in macOS 26.2.
50%Unverified跨节点LLM推理的可行性根本上受网络互连延迟约束：在标准以太网上运行跨节点张量并行会引入巨大同步延迟，只有具备低延迟RDMA或高速专用链路才具备实用价值。
50%UnverifiedModal 集群最多支持 32 个节点和 256 块 GPU，配备 3,200 Gbps RDMA 网络（B300 上为 6,400 Gbps），采用帮派调度并物理同地部署，每个节点必须使用其所有 GPU。
50%UnverifiedGPU发起的网络通信依赖于GPUDirect RDMA和RDMA两项关键底层技术
50%UnverifiedRDMA允许一台机器直接读写另一台机器的内存，绕过操作系统内核和CPU的介入，降低延迟
50%UnverifiedRoCE is an implementation of RDMA over standard Ethernet, with lower deployment costs compared to traditional RDMA deployments that require dedicated InfiniBand networks.
50%UnverifiedRDMA has extremely high requirements for a lossless (zero packet-loss) network environment, necessitating mechanisms such as Priority Flow Control (PFC), resulting in higher operational complexity than TCP.
50%UnverifiedOne of Homa's goals is to achieve near-RDMA low-latency performance on ordinary Ethernet while reducing dependence on lossless network characteristics.
50%UnverifiedGIN eliminates the fixed overhead of CPU round-trip scheduling by directly invoking RDMA primitives within GPU kernels.
50%UnverifiedThe CX7 NIC on the DGX Spark claims a latency of 1.5 microseconds, but because RDMA must route around the CPU to the GPU, the actual latency is approximately 20 microseconds.
50%UnverifiedHigh-speed networks such as InfiniBand or RoCE directly access remote node memory via RDMA technology, bypassing the security mechanisms of the traditional TCP/IP protocol stack.
50%