Unverified50% confidenceFactExact time
KLD (KL divergence) measures the difference between two probability distributions, with the formula D_KL(P||Q)=∑_x P(x)log(P(x)/Q(x)).
1
Sources
50%
Confidence
Long-term
Relevance
10/24/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedKL divergence is used to compare the difference in output probability distributions between the fine-tuned model and the original model given the same input; a larger value indicates greater deviation, and a KLD of 0.0244 means the fine-tuning has hardly changed the model's overall behavioral distribution.78% similarUnverifiedThe scheduling-averaged divergence first computes the KL divergence under each schedule and then takes the expectation, i.e., E_U D_KL(p||q_U).74% similarUnverifiedKLD measures relative entropy, i.e., the extra number of bits needed to encode the original model's output using the quantized model's distribution, so its unit is bits/token.71% similarUnverifiedKL divergence is defined as D_KL(P||Q)=∫log(dP/dQ)dP, and the reverse KL divergence swaps the positions of P and Q, i.e., D_KL(Q||P).70% similarUnverifiedKL divergence is an asymmetric metric measuring the difference between two probability distributions, and in reinforcement learning it is used to constrain the degree of deviation between the new and old policies (such as PPO's trust region).70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1094692API
curl https://kongchang.com/api/v1/knowledge/claims/1094692MCP
get_claim(id=1094692)