NVLink
NVLink is a high-speed chip-to-chip interconnect technology developed by NVIDIA, enabling high-bandwidth, low-latency direct communication between multiple GPUs or between GPUs and CPUs. Compared to conventional PCIe buses, NVLink provides higher data transfer bandwidth and lower latency, supporting efficient data sharing and communication in multi-GPU configurations. It is widely used in high-performance computing, deep learning training, and data center environments.
Core Facts
NVLink 4.0 has a bidirectional bandwidth of up to 900 GB/s, while the bandwidth of a traditional PCIe bus is approximately 64 GB/s.
In the H100 NVLink architecture, 8 GPUs within a single server can be fully interconnected via NVSwitch, achieving a total bandwidth of 900 GB/s, an order-of-magnitude improvement over PCIe 5.0's 64 GB/s.
NVLink 4.0 offers a bidirectional bandwidth of up to 900 GB/s, while cross-node InfiniBand HDR offers only approximately 50 GB/s, a gap of nearly 20 times.
Timeline (last 90 days)
Within a Grace Blackwell node, the internal resource topology exhibits stronger heterogeneous affinity: some tasks have lower communication overhead on tightly coupled CPU-GPU paths, while cross-GPU communication relies on the NVLink switching network.
In actual deployments, tensor parallelism and pipeline parallelism are often combined, along with cross-node unified memory addressing or high-speed interconnects (such as NVLink, InfiniBand or high-speed Ethernet).
MIG instances have limited compute capability, and some instance specifications do not support full NVLink bandwidth, so they are better suited to multi-tenant or task-isolation scenarios rather than pursuing extreme performance for a single task.
DeepEP is a library designed for MoE all-to-all communication, using NVLink within a single machine and RDMA across machines, supporting FP8 transmission, and the latest second version merges the two usage modes of high-throughput training/prefill and low-latency decoding into a single interface.
NVIDIA's Xid numbering system covers GPU hardware and firmware errors, with each number corresponding to a specific class of failure root cause, for example Xid 79 indicates the GPU has fallen off the PCIe bus, Xid 48 corresponds to an uncorrectable double-bit ECC memory error, Xid 74 involves an NVLink link failure, and Xid 119/120 relate to GSP firmware timeouts.
In heterogeneous backends, PCIe bandwidth is typically far lower than NVLink and will become the bottleneck for cross-device scheduling; placing the draft model on a device under the same PCIe root complex as the target model, or directly on the CPU, is often more worthwhile than moving data across NUMA nodes.
Rack-scale systems keep cross-card communication at low latency through unified high-speed interconnects (such as NVLink) and shared memory pools, thereby supporting real-time inference of large models.
GPU clusters rely on NVLink and InfiniBand for high-bandwidth, low-latency communication, with the NCCL library deeply optimized for this; the interconnect topologies of NPU devices are often different, such as Huawei Ascend's HCCL or the TPU's ICI.
Communication optimization methods include compressing gradients to low precision (such as FP16 or INT8), splitting All-Reduce into Reduce-Scatter and All-Gather and overlapping them with backward computation, and adopting hierarchical communication (intra-node NVLink aggregation, cross-node InfiniBand switching).
NVLink connects 72 GPUs into a high-bandwidth scale-up domain, letting a multi-trillion-parameter MoE model run like one coordinated machine.
35 more timeline events
All Facts (20)
NVLink 4.0 has a bidirectional bandwidth of up to 900 GB/s, while the bandwidth of a traditional PCIe bus is approximately 64 GB/s.
90%VerifiedIn the H100 NVLink architecture, 8 GPUs within a single server can be fully interconnected via NVSwitch, achieving a total bandwidth of 900 GB/s, an order-of-magnitude improvement over PCIe 5.0's 64 GB/s.
90%VerifiedNVLink 4.0 offers a bidirectional bandwidth of up to 900 GB/s, while cross-node InfiniBand HDR offers only approximately 50 GB/s, a gap of nearly 20 times.
80%VerifiedTraining requires GPU clusters with high-bandwidth interconnects (such as A100/H100 connected via NVLink), while inference places greater emphasis on single-card memory bandwidth, batch processing throughput, and low latency.
75%VerifiedNVIDIA's latest NVLink 5.0 can provide 1.8 TB/s bidirectional bandwidth per GPU.
75%VerifiedNVLink 4.0 (used in the H100) can achieve a unidirectional bandwidth of 900 GB/s.
70%VerifiedIn benchmark testing, Qwen3 27B achieved a generation speed of approximately 40-something tokens per second on 4 modified 2080 Ti cards without NVLink enabled.
65%VerifiedHigh-end training clusters dominated by NVIDIA rely on InfiniBand high-speed networks (with bandwidth up to 800Gb/s) and NVLink/NVSwitch to achieve direct GPU interconnection
65%UnverifiedNCCL通信库已经针对NVLink拓扑做了深度优化
90%Unverified传统DGX集群中，8块GPU组成一个节点，节点内走NVLink，节点间走InfiniBand
90%Unverified大语言模型训练背后需要精密的数据中心电源分配单元、液冷或风冷散热系统、高速互联网络（NVLink、InfiniBand）以及UPS不间断电源保障
90%Unverified块调度建议将张量并行（TP）限制在NVLink带宽最高的GPU子集内（同一个最小块），数据并行（DP）跨块分布，流水线并行（PP）沿拓扑层次展开
85%UnverifiedGB200 NVL72的72块GPU可以按照NVLink拓扑结构划分为不同粒度的块，包括9个8-GPU块、4个18-GPU块、2个36-GPU块或1个72-GPU块
85%Unverified在大规模训练集群中，节点内通信（NVLink）与节点间通信（InfiniBand或RoCE）的性能差异可达一个数量级
80%UnverifiedThe core goal of NVLink is to break through the bandwidth limitations of the traditional PCIe bus and allow multiple GPUs to communicate directly at extremely high speeds.
70%UnverifiedTensor parallelism splits the computation of a single layer (such as matrix multiplication) across multiple GPUs for parallel execution; it has a high communication frequency and typically requires high-bandwidth interconnects such as NVLink between GPUs.
70%UnverifiedThe fourth-generation NVLink in the Hopper architecture provides 900 GB/s of bandwidth.
70%UnverifiedNVLink technology was introduced starting with the Pascal architecture in 2016, aimed at overcoming the bandwidth limitations of the PCIe bus.
70%UnverifiedGB200 NVL72将NVLink互联从单节点内部扩展到了整个机架级别，机架内任意两块GPU之间都能通过NVLink直接通信
70%UnverifiedThe fifth generation of NVLink (featured in the Blackwell architecture) offers a single-link bidirectional bandwidth of up to 1.8 TB/s.
70%