EvalPlus
A framework for augmented evaluation of code generation benchmarks such as HumanEval, increasing evaluation strictness through additional test cases.
Timeline (last 90 days)
HumanEval+ is an enhanced version of HumanEval by the EvalPlus team that expands the number of test cases and can detect code correctness more rigorously.
In the whatmodel.sh data, 14,436 results come from independent evaluations (ACL Anthology, arXiv, Epoch AI, LMArena, EvalPlus, EleutherAI harness, etc.) and 7,677 from lab self-reports, with independent evaluations accounting for about 77%.
All Facts (2)
HumanEval+ is an enhanced version of HumanEval by the EvalPlus team that expands the number of test cases and can detect code correctness more rigorously.
50%UnverifiedIn the whatmodel.sh data, 14,436 results come from independent evaluations (ACL Anthology, arXiv, Epoch AI, LMArena, EvalPlus, EleutherAI harness, etc.) and 7,677 from lab self-reports, with independent evaluations accounting for about 77%.
50%