Verified75% confidenceDecision RuleExact time
In scenarios with extreme class imbalance (such as fraud detection), looking solely at accuracy as an evaluation metric is completely meaningless.
3
Sources
75%
Confidence
Long-term
Relevance
8/4/2026
First Seen
Sources
Related Claims
UnverifiedIf a model possesses deceptive capabilities, traditional misalignment detection and evaluation methods may fail entirely.79% similarUnverifiedAverage accuracy masks distributional skew; a judge that gets all easy cases right and is overconfident on hard cases can post a high average score while failing at critical points.77% similarUnverifiedSetting an absolute failure count ceiling for small-sample slices is more robust than setting a percentage threshold and is insensitive to sample size76% similarVerifiedWhen samples are severely imbalanced, Accuracy becomes misleading—a model that predicts everyone as healthy can still achieve an accuracy of over 99%.76% similarUnverifiedScoring with completion-style proxy metrics will classify failures that terminate cleanly but produce incorrect results as successes, which is a flaw in many current agent evaluations.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/683104API
curl https://kongchang.com/api/v1/knowledge/claims/683104MCP
get_claim(id=683104)