Unverified50% confidenceFactExact time
主流BPE词表规模通常为32K-128K，全量成对相似度矩阵的内存占用可超过数十GB
1
Sources
50%
Confidence
Long-term
Relevance
7/7/2026
First Seen
Sources
分散损失：破解小语言模型嵌入坍缩的关键技术
hackernewshackernews7/3/2026
Related Claims
UnverifiedWhen models with hundreds of billions of parameters process a 128K context window, KV Cache can occupy tens of gigabytes of VRAM.71% similarUnverifiedLLaMA-2-70B's KV Cache for a single request at a context length of 128K can occupy more than 160 GB of VRAM.68% similarUnverifiedA unified memory pool of 128GB means that quantized models at the 70-billion-parameter scale or larger can be run locally.68% similarUnverifiedFor a typical 7B model with 32 layers and 32 attention heads, storing the KV Cache for each token requires approximately 0.5 MB of VRAM, so generating 1,000 tokens requires an additional ~500 MB of VRAM67% similarUnverifiedTaking LLaMA-2 70B as an example, when processing a 4096-token context with a batch size of 32, the KV Cache can consume over 80 GB of memory.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/199067API
curl https://kongchang.com/api/v1/knowledge/claims/199067MCP
get_claim(id=199067)