Verified75% confidenceFactExact time
The pass@1 metric was originally borrowed from the code generation evaluation domain (such as the HumanEval benchmark) to measure the probability of a model generating correct code on the first attempt.
3
Sources
75%
Confidence
Long-term
Relevance
7/14/2026
First Seen
Sources
QuantaMind：本地验证AI Agent可靠性的开源测试工具
Related Claims
UnverifiedIn code generation evaluation, pass@1 measures the probability that the model passes the test with a single generation, while pass@10 allows the model to make 10 attempts and takes the best result74% similarPartially VerifiedThe pass@k family of metrics originates from the code generation evaluation field and was first systematically used in OpenAI's HumanEval paper.74% similarVerifiedThe pass@k metric measures the probability that, when the model generates k candidate code snippets, at least one of them passes all test cases.68% similarUnverifiedpass@k is a commonly used metric for measuring sampling quality in tasks such as code generation: k candidate answers are independently sampled for the same problem, and success is counted as long as any one of them passes verification.68% similarUnverifiedAnthropic is commercializing Code Review, essentially handing review over to the model as well: the model performs static analysis, logic verification and risk annotation on code produced by models, and humans only handle the flagged high-risk parts.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/507772API
curl https://kongchang.com/api/v1/knowledge/claims/507772MCP
get_claim(id=507772)