Nsight Systems
A system-level performance analysis tool provided by NVIDIA that displays CPU and GPU activity in a timeline view. It is used to locate kernel launch overhead, synchronization waits, and CPU scheduling bottlenecks.
Timeline (last 90 days)
Nsight Systems profiling shows that at 7.3 tok/s each token takes about 137ms, of which the GPU is busy about 92ms and idle about 54ms, with the inter-launch gap only about 13ms, so CUDA graphs can only optimize those 13ms.
Using Nsight Systems in conjunction with Nsight Compute can pinpoint whether a performance bottleneck is memory-bound or compute-bound, enabling the selection of an optimization direction.
The number of GPU hardware performance counters is limited, and once dcgm-exporter occupies them, tools such as Nsight Systems and Nsight Compute cannot simultaneously use the same set of counters.
All Facts (4)
NCCL Inspector与nsys、Nsight Systems等传统profiling工具的核心差异在于实时性和持续监控能力，支持7×24小时运行
85%UnverifiedNsight Systems profiling shows that at 7.3 tok/s each token takes about 137ms, of which the GPU is busy about 92ms and idle about 54ms, with the inter-launch gap only about 13ms, so CUDA graphs can only optimize those 13ms.
50%UnverifiedUsing Nsight Systems in conjunction with Nsight Compute can pinpoint whether a performance bottleneck is memory-bound or compute-bound, enabling the selection of an optimization direction.
50%UnverifiedThe number of GPU hardware performance counters is limited, and once dcgm-exporter occupies them, tools such as Nsight Systems and Nsight Compute cannot simultaneously use the same set of counters.
50%