DeepEP
An open-source MoE expert-parallel communication acceleration library released by DeepSeek, targeting the Ascend platform. It focuses on cross-device Dispatch/Combine communication challenges in Mixture-of-Experts models and supports the FP8 data type.
Core Facts
Timeline (last 90 days)
In DeepInfer, Ascend and NVIDIA are first-class citizens, with 30 TileLang operators on each side corresponding one-to-one, plus Ascend versions of DeepGEMM and DeepEP.
DeepEP is a library designed for MoE all-to-all communication, using NVLink within a single machine and RDMA across machines, supporting FP8 transmission, and the latest second version merges the two usage modes of high-throughput training/prefill and low-latency decoding into a single interface.
DeepEP corresponds to NCCL, handling communication between multiple GPUs; FlashMLA corresponds to FlashAttention, used for ultra-long text processing; TileKernels corresponds to CUTLASS; DeepSelect corresponds to data pipelines such as DALI.
DeepEP compresses Token data from BF16 to FP8 E4M3, halving the volume, and attaches a scale factor to each Token for restoration, which constitutes per-token quantization granularity.
With defer epilogue set, Dispatch is only responsible for data transfer, deferring the epilogue cleanup until event.current_stream.wait is called; the intermediate gap can be used for computing the shared expert, achieving overlap of communication and computation.
The DeepEP README validates numerical alignment of FP8 results with reference answers in the Dispatch stage, but does not cover the end-to-end impact on model accuracy.
The cross-card communication in the MoE layer consists of two symmetric operations: Dispatch and Combine, which together form one All-to-All communication.
DeepEP package installation only installs the host side; the kernel is compiled on-the-fly on first run and cached, then reused directly thereafter.
The six components have distinct roles: DeepGEMM accelerates general matrix operations, DeepEP handles large-scale cross-device communication, TileKernels provides common vector computation and memory access operators, FlashMLA provides sparse attention operators, and DeepSelect performs efficient data selection.
On September 30, 2025, DeepSeek open-sourced a batch of underlying infrastructure components, including operators, a programming language, compute libraries, and communication libraries
All Facts (10)
On September 30, 2025, DeepSeek open-sourced a batch of underlying infrastructure components, including operators, a programming language, compute libraries, and communication libraries
65%UnverifiedIn DeepInfer, Ascend and NVIDIA are first-class citizens, with 30 TileLang operators on each side corresponding one-to-one, plus Ascend versions of DeepGEMM and DeepEP.
50%UnverifiedDeepEP is a library designed for MoE all-to-all communication, using NVLink within a single machine and RDMA across machines, supporting FP8 transmission, and the latest second version merges the two usage modes of high-throughput training/prefill and low-latency decoding into a single interface.
50%UnverifiedDeepEP corresponds to NCCL, handling communication between multiple GPUs; FlashMLA corresponds to FlashAttention, used for ultra-long text processing; TileKernels corresponds to CUTLASS; DeepSelect corresponds to data pipelines such as DALI.
50%UnverifiedDeepEP compresses Token data from BF16 to FP8 E4M3, halving the volume, and attaches a scale factor to each Token for restoration, which constitutes per-token quantization granularity.
50%UnverifiedThe DeepEP README validates numerical alignment of FP8 results with reference answers in the Dispatch stage, but does not cover the end-to-end impact on model accuracy.
50%UnverifiedWith defer epilogue set, Dispatch is only responsible for data transfer, deferring the epilogue cleanup until event.current_stream.wait is called; the intermediate gap can be used for computing the shared expert, achieving overlap of communication and computation.
50%UnverifiedThe cross-card communication in the MoE layer consists of two symmetric operations: Dispatch and Combine, which together form one All-to-All communication.
50%UnverifiedDeepEP package installation only installs the host side; the kernel is compiled on-the-fly on first run and cached, then reused directly thereafter.
50%UnverifiedThe six components have distinct roles: DeepGEMM accelerates general matrix operations, DeepEP handles large-scale cross-device communication, TileKernels provides common vector computation and memory access operators, FlashMLA provides sparse attention operators, and DeepSelect performs efficient data selection.
50%