ProductPyTorch FlexAttention

FlexAttention

A programmable attention interface introduced in PyTorch 2.5 that allows users to customize attention score modification and masking logic through score_mod and mask_mod hooks, while automatically generating fused, efficient kernels at the underlying level, combining flexibility with hardware acceleration.

Timeline (last 90 days)

Oct 30

Compared with PyTorch SDPA, FlexAttention, and the SGLang attention kernel, CACPU achieves up to a 2.01x speedup.

Unverified50%

All Facts (1)

Unverified

Compared with PyTorch SDPA, FlexAttention, and the SGLang attention kernel, CACPU achieves up to a 2.01x speedup.

50%

Source Articles

CPU Can Efficiently Run Long-Context Attention Too: A Deep Dive into the CACPU KernelOct 30 Efficient Edge-aware Attention Network: A New Memory Optimization Approach for Graph Generative ModelsOct 21