Prefill-Decode Disaggregation
Prefill-Decode Disaggregation (PD disaggregation) is a resource scheduling architecture for large language model inference systems. It separates the two computationally distinct phases of inference—the compute-intensive prefill phase and the memory-bandwidth-intensive decode phase—onto independent GPU nodes, avoiding mutual interference from co-located deployment. This enables targeted resource allocation and optimization for each phase, improving overall system throughput and inference efficiency.
Timeline (last 90 days)
In the traditional PD-disaggregated architecture, prefill nodes and decode nodes operate independently, and KV caches cannot be passed across rounds, so content computed in the previous round must be recomputed in the next.
All Facts (4)
KV Cache跨节点迁移是PD分离架构的关键技术挑战，通常要求100Gbps以上的网络互联
85%UnverifiedPD分离架构通过物理隔离消除Prefill对Decode的干扰，显著改善TTFT的P99延迟
85%UnverifiedPD分离架构允许不同阶段选用不同规格的GPU实例，并可独立扩缩Prefill或Decode节点数量
55%UnverifiedIn the traditional PD-disaggregated architecture, prefill nodes and decode nodes operate independently, and KV caches cannot be passed across rounds, so content computed in the previous round must be recomputed in the next.
50%