Unverified50% confidenceFactExact time
HumanEval and MBPP primarily evaluate single-function-level code generation and have relatively limited discriminating power for agent-type tools.
1
Sources
50%
Confidence
Long-term
Relevance
9/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedHumanEval and MBPP primarily measure a model's code generation capability, while code review is an essentially different task, and the industry currently lacks standardized code review benchmarks.79% similarUnverified传统代码评测基准（如HumanEval、MBPP）主要测试模型在孤立函数级别的生成准确率76% similarUnverifiedHumanEval等传统代码评测基准通常只要求模型根据函数签名和文档字符串生成独立函数，本质上是受限的代码补全任务76% similarUnverifiedHumanEval, MBPP, and SWE-bench are standard benchmark datasets specifically targeting code generation.75% similarUnverified当前主流基准测试体系如HumanEval、MBPP等代码评估集往往关注平均性能，对分布式的场景特定退化现象敏感度不足72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/907388API
curl https://kongchang.com/api/v1/knowledge/claims/907388MCP
get_claim(id=907388)