Unverified50% confidenceFactExact time
In code generation evaluation, pass@1 measures the probability that the model passes the test with a single generation, while pass@10 allows the model to make 10 attempts and takes the best result
1
Sources
50%
Confidence
Long-term
Relevance
9/8/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe pass@1 metric was originally borrowed from the code generation evaluation domain (such as the HumanEval benchmark) to measure the probability of a model generating correct code on the first attempt.74% similarUnverifiedThe pass@k metric measures the probability that, when the model generates k candidate code snippets, at least one of them passes all test cases.74% similarUnverifiedEach model configuration was tested under two reasoning intensities, HIGH and XH (Extra High), with each configuration run independently 10 times.65% similarUnverifiedThe most reliable way to determine whether a new model has gotten worse is to conduct an A/B comparison between the old and new versions on the same tasks with the same prompts.62% similarUnverified团队在项目期间经历四五次模型迭代，每次都发现模型能力越强从验证循环中获得的收益越大，对验证循环的需求是更持久的底层规律61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/875814API
curl https://kongchang.com/api/v1/knowledge/claims/875814MCP
get_claim(id=875814)