WGMMA
An asynchronous matrix multiply-accumulate instruction introduced in NVIDIA's Hopper architecture, executed cooperatively by an entire warpgroup. It reads operands directly from shared memory and writes to tensor core accumulators, supporting larger matrix tiles and higher throughput.
Timeline (last 90 days)
WGMMA (Warpgroup Matrix Multiply-Accumulate) is an asynchronous matrix multiply-accumulate instruction introduced by the NVIDIA Hopper architecture, executed collaboratively by an entire warpgroup (usually 128 threads), which can read operands directly from shared memory and write to tensor core accumulators.
Compared with the previous generation Ampere's mma instruction, WGMMA supports larger matrix tiles and higher throughput, and can overlap in a pipeline with asynchronous copy mechanisms such as TMA (Tensor Memory Accelerator).
The Swap-AB strategy essentially swaps the roles of the two operands, enabling the kernel to achieve better data reuse and less synchronization overhead in Hopper's WGMMA pipeline, without changing mathematical equivalence.
On the Hopper architecture, general-purpose kernels in MoE inference struggle to saturate tensor core compute, because WGMMA has strict requirements on matrix layout and pipelining, and a mismatch in memory access patterns or data layout causes a noticeable drop in utilization.
All Facts (4)
WGMMA (Warpgroup Matrix Multiply-Accumulate) is an asynchronous matrix multiply-accumulate instruction introduced by the NVIDIA Hopper architecture, executed collaboratively by an entire warpgroup (usually 128 threads), which can read operands directly from shared memory and write to tensor core accumulators.
50%UnverifiedCompared with the previous generation Ampere's mma instruction, WGMMA supports larger matrix tiles and higher throughput, and can overlap in a pipeline with asynchronous copy mechanisms such as TMA (Tensor Memory Accelerator).
50%UnverifiedThe Swap-AB strategy essentially swaps the roles of the two operands, enabling the kernel to achieve better data reuse and less synchronization overhead in Hopper's WGMMA pipeline, without changing mathematical equivalence.
50%UnverifiedOn the Hopper architecture, general-purpose kernels in MoE inference struggle to saturate tensor core compute, because WGMMA has strict requirements on matrix layout and pipelining, and a mismatch in memory access patterns or data layout causes a noticeable drop in utilization.
50%