Topology-Aware Scheduling
A scheduling strategy that accounts for the physical topology of underlying hardware. By understanding GPU interconnect hierarchies (intra-node, inter-node, and cross-rack), it assigns communication-intensive tasks to physically proximate hardware, reducing communication overhead and improving cluster throughput efficiency.
Timeline (last 90 days)
Topology-aware scheduling means that when placing tasks, the scheduler considers not only whether the quantity of resources is sufficient, but also the physical connection relationships among resources.
Topology-aware scheduling reads hardware topology information (such as affinity data provided by NVML and the node topology manager) to preferentially place communication-intensive tasks on resource combinations with higher interconnect bandwidth.
The core of topology-aware scheduling is enabling the scheduling system to understand the physical connection hierarchy between GPUs, so that tasks with intensive communication can be consolidated onto hardware that is physically closer, reducing cross-domain traffic.
In topology-aware scheduling practice, GPUs in a tensor-parallel group should be preferentially placed within the same node, pipeline stages can span nodes but should ideally be on the same rack, and data-parallel replicas can be distributed flexibly.
All Facts (4)
Topology-aware scheduling means that when placing tasks, the scheduler considers not only whether the quantity of resources is sufficient, but also the physical connection relationships among resources.
50%UnverifiedTopology-aware scheduling reads hardware topology information (such as affinity data provided by NVML and the node topology manager) to preferentially place communication-intensive tasks on resource combinations with higher interconnect bandwidth.
50%UnverifiedThe core of topology-aware scheduling is enabling the scheduling system to understand the physical connection hierarchy between GPUs, so that tasks with intensive communication can be consolidated onto hardware that is physically closer, reducing cross-domain traffic.
50%UnverifiedIn topology-aware scheduling practice, GPUs in a tensor-parallel group should be preferentially placed within the same node, pipeline stages can span nodes but should ideally be on the same rack, and data-parallel replicas can be distributed flexibly.
50%