Unverified50% confidenceBenchmarkExact time
Evaluated with llama.cpp on a Tesla T4, the HumanEval pass@1 retention rate is 96.7% (29/30).
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/30/2026
First Seen
Valid until: 1/28/2027
Sources
Related Entities
Related Claims
UnverifiedIn the HumanEval pass@1 comparison, phi-1-small (350M parameters) reached 45%, while Keera fine-tuned (97.5M parameters) reached 5.5%.76% similarUnverifiedCodex于2021年发布时在HumanEval基准上达到28.8%的pass@1得分，而GPT-4在同一基准上已超过67%72% similarUnverifiedEvaluated with llama.cpp on a Tesla T4, the MMLU subset retention rate is 92.9% (344/502).72% similarUnverifiedThe original Codex achieved a pass@1 score of 28.8% on the HumanEval benchmark, whereas using GPT-3 directly yielded only 0%.71% similarUnverifiedOn the T4 with --cpu-moe plus a 10000 MiB cache, the hit rate was 95.73% for small batches of up to 8 tokens and 59.91% for batches of 9 to 32 tokens.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1115186API
curl https://kongchang.com/api/v1/knowledge/claims/1115186MCP
get_claim(id=1115186)