Unverified50% confidenceFactExact time
Excess entropy is defined as the mutual information between the past and the future, measuring in bits the amount of information in the past that is useful for predicting the future.
1
Sources
50%
Confidence
Long-term
Relevance
10/7/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedCross-entropy loss is the most commonly used loss function in classification tasks, rooted in information theory, and measures the distance between the model's predicted probability distribution and the true distribution.70% similarUnverifiedThe concept of perplexity originates from entropy (Shannon Entropy) in information theory, and is essentially the average branching factor per prediction step of a language model.69% similarUnverifiedEntropy and KL divergence in information theory are important foundations for understanding loss function design and generative models (such as VAEs and diffusion models)68% similarUnverifiedThe recursive entropic risk measure applies the risk operator in a nested fashion at each time step, satisfies time consistency, and can therefore be directly combined with the Bellman equation.67% similarUnverified交叉熵恢复率通过将SAE重建激活替代原始激活输入到模型后续层比较交叉熵损失变化来计算，衡量SAE是否保留对下游计算重要的信息65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/983443API
curl https://kongchang.com/api/v1/knowledge/claims/983443MCP
get_claim(id=983443)