Unverified50% confidenceBenchmarkExact time
In tasks with propositional constraints, the proportion of models integrating candidates already verified as invalid is as high as 93%–100%.
1
Sources
50%
Confidence
Long-term
Relevance
9/8/2026
First Seen
Sources
Related Claims
UnverifiedBenchmark overfitting is a core problem in current large model evaluation; some models achieve pass rates above 90% on HumanEval but perform far below expectations in real codebases.71% similarVerifiedIn scenarios with extreme class imbalance, even an accuracy as high as 95% may indicate that the model is completely unable to identify the minority class.71% similarUnverifiedIn a fraud detection task with a positive-to-negative sample ratio of 1:99, a model that predicts all samples as 'non-fraud' can still achieve 99% accuracy.71% similarUnverified一个 90% 成功、10% 出错的工具调用系统几乎无法投入实际使用，因为单次错误会污染后续上下文造成级联失败70% similarUnverified讲者认为评估AI编程代理应关注80%、90%、99%成功率下的数据而非常被引用的50%成功率版本，因为99%才意味着可信赖并委托给模型70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/875708API
curl https://kongchang.com/api/v1/knowledge/claims/875708MCP
get_claim(id=875708)