Unverified50% confidenceFactExact time
FP4-Local uses about 166 GiB of VRAM, and on two RTX PRO 6000 GPUs (192GB total) it can fit a 1.8M token FP8 KV cache; a dual DGX Spark setup can reach 2M tokens.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/18/2026
First Seen
Valid until: 1/16/2027
Sources
Related Entities
Related Claims
UnverifiedThe Qwen3.8-27B checkpoint occupies 28.75 GiB on disk, and after loading in vLLM the weights plus non-torch overhead are about 29.34 GiB, leaving 53.8 GiB on a 96 GB RTX Pro 6000 for the KV cache, equivalent to about 767,317 tokens.79% similarUnverifiedFP4-Local is not smaller than 4-bit GGUF: UD-IQ4_XS GGUF is 157 GB, while FP4-Local is about 166 GiB.75% similarUnverifiedOn NVIDIA L4, the bf16 model (6.32 GiB) has a theoretical memory-bandwidth ceiling of about 47 tokens/s, while the int4 model (2.85 GiB) has a theoretical ceiling of about 105 tokens/s, with measured peak reaching 131.9 tokens/s.74% similarUnverifiedOn an RTX Pro 6000 (96 GB VRAM), the official FP8 version combined with built-in MTP speculative decoding can reach approximately 120 tokens/second, with quality nearly indistinguishable from 16-bit.73% similarUnverifiedDeepSeek-V4.1-Flash has 763B parameters, BF16 weights are about 1.5 TB, the official FP4/FP8 checkpoint is 475.2 GiB, and coolbho3k's 3-bit EXL3 package compresses it to 309.3 GiB.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1064231API
curl https://kongchang.com/api/v1/knowledge/claims/1064231MCP
get_claim(id=1064231)