Unverified50% confidenceTradeoffExact time
Applying a block-diagonal Hadamard transform to the INT4 cache brings about a 0.20 increase in perplexity but in exchange reduces cache quantization error.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedDecoding from continuous representations to discrete tokens is an error-prone step; minor deviations in the embedding space can lead to decoding errors, and quantization errors are amplified as sequence length grows.66% similarUnverifiedMultiple studies show that when the context window expands from zero tokens to 256K tokens, the model's accuracy in solving problems actually declines.65% similarUnverifiedThe perplexity loss of Q4_K_M quantization compared to FP16 is typically within 1%–3%65% similarUnverifiedQuantization error originates from rounding deviations caused by precision truncation when mapping floating-point numbers (such as FP16 and BF16) to low-bit integers (such as INT8 and INT4).65% similarUnverifiedIn a program, when the original question text is used as the key, subtle differences such as spaces, line breaks, and full-width versus half-width punctuation are all treated as different questions, causing the cache hit rate to drop or duplicate processing64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1065916API
curl https://kongchang.com/api/v1/knowledge/claims/1065916MCP
get_claim(id=1065916)