Expired50% confidenceFactExact time
In the replaced model, the tied embedding/lm_head matrix remains bf16, so each token actually reads about 1.263 GB of weights (plus about 0.06 GB of KV cache); calculated by actual bytes moved, 25.5 tok/s corresponds to about 34 GB/s, roughly 10% of measured bandwidth.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/12/2026
First Seen
This claim has expired. It is kept for the record and no longer describes the current state.
Valid until: 10/26/2026(expired)
Sources
Related Entities
Related Claims
UnverifiedWhen the model loads, it first allocates a full-precision shell of approximately 1.29 GB and then reads in 643 MB of full weights, with peak memory potentially reaching approximately 3,400 MB.77% similarUnverifiedMotrix's actual memory usage is in the range of 150–250 MB.75% similarUnverifiedIf all 36 layers of Spark X 2.5 tracked the full text, the KV cache overhead for tracking 1 million tokens would be approximately 155 GB, while the sparse design compresses this to approximately 39 GB74% similarUnverifiedQuantization can compress the storage size of a 70-billion-parameter model from approximately 140 GB in FP16 to approximately 35–40 GB in INT4, enabling it to run on consumer-grade hardware such as a single RTX 4090 with CPU memory offloading.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1006844API
curl https://kongchang.com/api/v1/knowledge/claims/1006844MCP
get_claim(id=1006844)