Unverified50% confidenceConceptExact time
Benchmark saturation refers to leading models' scores on old tasks generally approaching the ceiling and differentiation declining, causing the benchmark to lose its screening value.
1
Sources
50%
Confidence
Long-term
Relevance
10/28/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWhen all models approach a perfect score, the benchmark loses its discriminative power and rankings become noise.78% similarUnverifiedBenchmark saturation refers to the phenomenon where a benchmark loses its discriminative power when model performance approaches human-level performance or the upper limit of the test set.76% similarUnverifiedInstance normalization discards amplitude information, and amplitude may be a key discriminative feature in some classification tasks.75% similarVerified若用含测试集的全量数据计算均值和标准差，会导致预处理泄漏，使评估指标虚高，无法真实反映模型泛化能力75% similarUnverifiedSaturation in measurement theory refers to test scores losing the ability to discriminate between the subjects being measured; when all models' scores approach the ceiling or floor, the variance is extremely small.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1106586API
curl https://kongchang.com/api/v1/knowledge/claims/1106586MCP
get_claim(id=1106586)