Swap-AB
A GEMM kernel optimization strategy that swaps the roles of the two operands in matrix multiplication. Using the equivalence (A×B)ᵀ = Bᵀ×Aᵀ, it allows the kernel to achieve better data reuse and fewer synchronization overheads in the Hopper WGMMA pipeline.
Timeline (last 90 days)
The AWS engineering team carried out kernel-level optimizations on Amazon SageMaker AI for a fused MoE model, including Swap-AB to improve WGMMA utilization, Triton backend routing to implement FP8 quantization, and eliminating repeated Triton JIT compilation.
The Swap-AB strategy essentially swaps the roles of the two operands, enabling the kernel to achieve better data reuse and less synchronization overhead in Hopper's WGMMA pipeline, without changing mathematical equivalence.
All Facts (2)
The AWS engineering team carried out kernel-level optimizations on Amazon SageMaker AI for a fused MoE model, including Swap-AB to improve WGMMA utilization, Triton backend routing to implement FP8 quantization, and eliminating repeated Triton JIT compilation.
50%UnverifiedThe Swap-AB strategy essentially swaps the roles of the two operands, enabling the kernel to achieve better data reuse and less synchronization overhead in Hopper's WGMMA pipeline, without changing mathematical equivalence.
50%