Unverified50% confidenceFactExact time
DeepEP compresses Token data from BF16 to FP8 E4M3, halving the volume, and attaches a scale factor to each Token for restoration, which constitutes per-token quantization granularity.
1
Sources
50%
Confidence
Long-term
Relevance
10/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedK8V4非对称量化将Key压缩到8位、Value压缩到4位，使128K上下文的KV Cache从16GB降到约6GB68% similarVerifiedQuantized KV caches (such as storing KV vectors in FP8 or INT4) can compress the cache size to 1/2 or even 1/4 of the original.68% similarUnverifiedWhen running 4-bit DeepSeek R1 on the M3 Ultra, a 1,000-token context generates approximately 19 tokens per second using 392GB, dropping to approximately 15 tokens per second using 410GB at a 16,000-token context.67% similarUnverifiedFP8 KV cache is a quantization technique that compresses the Key-Value cache in the attention mechanism from FP16 or BF16 to 8 bits, which can halve the GPU memory usage of the KV cache67% similarUnverifiedTowards AI ultimately chose to use DeepSeq V4 Flash with hybrid retrieval, with a memory strategy of retaining as much as possible and setting a 30K token compression threshold as a fallback.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/970622API
curl https://kongchang.com/api/v1/knowledge/claims/970622MCP
get_claim(id=970622)