Unverified50% confidenceTradeoffExact time
Entropy collapse is especially prominent in long Chain-of-Thought training, because reward signals are only given at the end of the reasoning chain and gradient signals at intermediate steps are extremely sparse.
1
Sources
50%
Confidence
Long-term
Relevance
9/21/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedLLM的幻觉问题源于其训练目标是最小化交叉熵损失而非优化命题真值，是与生成机制深度耦合的固有特性而非偶发缺陷72% similarUnverifiedNaïve post-training quantization causes individual weight errors to accumulate and amplify across multiple layers—a phenomenon known as compounding error—and at 2-bit or 1-bit precision the model structure collapses70% similarUnverifiedThe training objective of large language models is to minimize the prediction loss of the next token (cross-entropy loss), not to verify the factual accuracy of generated content.70% similarUnverified从信息论角度看，LLM的输出熵相对较低，倾向于收敛到训练数据的高概率区域69% similarUnverifiedDietterich (1998) pointed out that even when using cross-validation, the folds are not completely independent of one another because there is substantial overlap between training sets, which may cause inflation of the false positive rate.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/937680API
curl https://kongchang.com/api/v1/knowledge/claims/937680MCP
get_claim(id=937680)