Unverified60% confidenceFactExact time
When using the Adam optimizer, 16 bytes of state information are required per parameter (4 bytes for FP32 parameters + 4 bytes for FP32 gradients + 4 bytes for first-order momentum + 4 bytes for second-order momentum), requiring at least approximately 24 GB of state information per GPU for a 1.5 billion parameter model.
2
Sources
60%
Confidence
Long-term
Relevance
8/26/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFull fine-tuning of a 70B model with the AdamW optimizer requires about 1,129 GB of VRAM (70.55 billion parameters × 16 bytes per parameter).80% similarVerifiedA 70B-parameter FP16 model requires approximately 140 GB of video memory, approximately 70 GB after INT8 quantization, and approximately 35–40 GB after INT4 quantization.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/802589API
curl https://kongchang.com/api/v1/knowledge/claims/802589MCP
get_claim(id=802589)