Unverified50% confidenceFactExact time
Effective rank is a metric of the "actually used dimensions" of a matrix, typically defined from the singular value distribution: normalizing the singular values into a probability distribution and computing the exponential of its entropy, or counting the number of directions needed to cover a given fraction of energy.
1
Sources
50%
Confidence
Long-term
Relevance
10/27/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe weight spectrum refers to the singular value distribution obtained by performing singular value decomposition on the weight matrix, reflecting the energy distribution of the weights in different directions, and can be used to assess the degree to which quantization damages the low-rank structure of the matrix.76% similarUnverifiedEffective branching factor is a concept in information theory and language modeling, referring to the number of nearly equally reasonable candidates at the next token position given the context, approximately characterized by the entropy of the distribution.66% similarPartially VerifiedIn information theory, entropy H(p) measures the uncertainty of a probability distribution p, and is equivalent to the average number of bits required to optimally encode information drawn from that distribution.63% similarUnverifiedRènyi divergence is a generalized family of divergences defined on probability distributions, controlled by a parameter α (α>0, α≠1); as α→1 it degenerates to the KL divergence, and as α→∞ it approaches the maximum divergence.62% similarUnverifiedA classification head is typically a simple fully connected layer with Softmax activation that maps a high-dimensional feature vector to a probability distribution over each category.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1102247API
curl https://kongchang.com/api/v1/knowledge/claims/1102247MCP
get_claim(id=1102247)