Unverified60% confidenceFactExact time
HumanEval tests code generation capabilities by requiring models to generate correct Python code based on function signatures and docstrings.
2
Sources
60%
Confidence
Long-term
Relevance
8/6/2026
First Seen
Sources
Related Claims
VerifiedCode large language models are evaluated using executable benchmarks such as HumanEval and MBPP, meaning the generated code is directly executed and the output is checked for correctness.77% similarUnverifiedHumanEval等传统代码评测基准通常只要求模型根据函数签名和文档字符串生成独立函数，本质上是受限的代码补全任务77% similarUnverifiedHumanEval通过让模型根据函数签名和文档字符串生成完整函数实现来衡量编码准确率77% similarVerifiedHumanEval focuses on code generation capability, testing models' logical reasoning and code correctness with 164 programming problems.76% similarVerifiedHumanEval是由OpenAI于2021年发布的代码生成基准测试，包含164个手写的Python编程问题74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/692632API
curl https://kongchang.com/api/v1/knowledge/claims/692632MCP
get_claim(id=692632)