Unverified50% confidenceOpinionExact time
Community skepticism holds that while deriving 60% of scores from private test sets can prevent leaderboard gaming and data contamination, it reduces the leaderboard's transparency and makes its credibility depend on the publisher's self-restraint.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/28/2026
First Seen
Valid until: 1/26/2027
Sources
Related Entities
Related Claims
UnverifiedAbout 60% of the index scores in v5 come from evaluations using private test sets or withheld answers, and external researchers cannot reproduce these results.73% similarUnverifiedAn insufficiently small reference corpus systematically underestimates true divergence, and the researchers introduce a source-split jackknife method that uses only reference data, which can reduce calibration error by 57% to 59%.69% similarUnverifiedUtilization data itself lacks a unified disclosure standard, and the calibers cited by different parties vary considerably.69% similarUnverifiedNiche suffixes, due to small sample sizes and relatively high abuse ratios, are often by default assigned lower trust scores or placed directly into manual review queues.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1106399API
curl https://kongchang.com/api/v1/knowledge/claims/1106399MCP
get_claim(id=1106399)