SM
The basic compute unit in Nvidia GPU architecture, containing multiple sub-partitions, each with its own instruction scheduler and execution units; in the Ampere architecture each SM contains four sub-partitions.
Timeline (last 90 days)
The Mega MoE fused operator makes communication warps and compute warps share the same batch of SMs, causing contention for registers, shared memory, and issue slots, with the synchronization domain covering the entire SM array, and each replay requires clearing the workspace and performing a cross-rank barrier.
With a batch size of 1, GPU parallel compute unit utilization is very low, because only one token participates in matrix operations at a time and a large number of SMs sit idle.
The instruction issue rate determines how many instructions an SM can dispatch to execution units per clock cycle.
On the measurement side, the gpu-unlock project provides a microbenchmark method for observing the issue behavior of specific SM sub-partitions under continuous FP16/BF16 instruction streams.
Each SM in the Ampere architecture contains four sub-partitions, and each sub-partition has 32 FP32 cores.
Instruction Issue Efficiency measures the proportion of clock cycles in which an SM successfully issues instructions.
The L1 cache is a high-speed cache private to each SM, while the L2 cache is shared by all SMs.
The Streaming Multiprocessor (SM) is the core compute unit of NVIDIA GPUs; each SM contains a number of CUDA cores, Tensor Cores, a register file, and shared memory, and is responsible for executing thread blocks in parallel.
A modern data center GPU (such as the H100) typically has tens to over a hundred SMs.
The core idea of Green Contexts is to perform a soft partitioning of GPU compute resources, allowing developers to explicitly specify how many SM resources a given portion of the workload uses.
4 more timeline events
All Facts (14)
The Mega MoE fused operator makes communication warps and compute warps share the same batch of SMs, causing contention for registers, shared memory, and issue slots, with the synchronization domain covering the entire SM array, and each replay requires clearing the workspace and performing a cross-rank barrier.
50%UnverifiedWith a batch size of 1, GPU parallel compute unit utilization is very low, because only one token participates in matrix operations at a time and a large number of SMs sit idle.
50%UnverifiedThe instruction issue rate determines how many instructions an SM can dispatch to execution units per clock cycle.
50%UnverifiedOn the measurement side, the gpu-unlock project provides a microbenchmark method for observing the issue behavior of specific SM sub-partitions under continuous FP16/BF16 instruction streams.
50%UnverifiedEach SM in the Ampere architecture contains four sub-partitions, and each sub-partition has 32 FP32 cores.
50%UnverifiedInstruction Issue Efficiency measures the proportion of clock cycles in which an SM successfully issues instructions.
50%UnverifiedThe L1 cache is a high-speed cache private to each SM, while the L2 cache is shared by all SMs.
50%UnverifiedThe Streaming Multiprocessor (SM) is the core compute unit of NVIDIA GPUs; each SM contains a number of CUDA cores, Tensor Cores, a register file, and shared memory, and is responsible for executing thread blocks in parallel.
50%UnverifiedA modern data center GPU (such as the H100) typically has tens to over a hundred SMs.
50%UnverifiedThe core idea of Green Contexts is to perform a soft partitioning of GPU compute resources, allowing developers to explicitly specify how many SM resources a given portion of the workload uses.
50%UnverifiedThe process by which thread blocks are dispatched to SMs is transparent and non-intervenable at the application layer; once a large kernel occupies all SMs, subsequently submitted small kernels can only queue and wait even if they have higher priority.
50%UnverifiedThe compute cores (SMs) inside the GPU access already-decrypted local protected memory and do not need to repeatedly perform software-level encryption and decryption before and after each computation.
50%UnverifiedSouth Korea's three major entertainment companies SM, JYP, and HYBE have transformed CDs into blind-box-style consumer products, drawing on Japan's AKB48 handshake ticket business model.
50%Unverified占用率（occupancy）衡量实际活跃warp数与SM支持的最大warp数之比，受每个线程使用的寄存器数量、每个线程块的共享内存大小和线程块配置方式影响
50%