Verified75% confidenceFactExact time
ZeRO was proposed by the DeepSpeed team in 2020 and eliminates redundancy in data parallelism through three stages: ZeRO-1 shards optimizer states, ZeRO-2 shards gradients, and ZeRO-3 shards model parameters.
3
Sources
75%
Confidence
Long-term
Relevance
7/14/2026
First Seen
Sources
JAX主机卸载技术：突破LLM训练显存瓶颈的实战指南
Related Claims
Unverified微软DeepSpeed提出的ZeRO技术分为三个阶段：Stage-1分片优化器状态可将内存需求降低约4倍，Stage-2额外分片梯度，Stage-3进一步分片模型参数79% similarUnverifiedDeepSpeed's ZeRO optimizer can reduce the GPU memory required per device to 1/N of the original by sharding optimizer states, gradients, and model parameters.72% similarUnverifiedUnsloth adds DeepSpeed ZeRO-2 and ZeRO-3 support, including 4-bit QLoRA, full fine-tuning, DPO and CPU offload; the DDP + gradient checkpointing LoRA training issue that previously caused crashes has been fixed.71% similarUnverifiedZeRO-Infinity可将分片数据溢出至CPU内存乃至NVMe SSD63% similarUnverifiedZed added Grok 4.7 to the SuperGrok and xAI providers and set it as the recommended default model.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/503680API
curl https://kongchang.com/api/v1/knowledge/claims/503680MCP
get_claim(id=503680)