Verified75% confidenceFactExact time
The pass@k metric measures the probability that, when the model generates k candidate code snippets, at least one of them passes all test cases.
3
Sources
75%
Confidence
Long-term
Relevance
7/12/2026
First Seen
Sources
国内直连Codex配置教程：GPT Agent完整上手指南
bilibili无汁西瓜6/26/2026
Related Claims
Unverifiedpass@k is a commonly used metric for measuring sampling quality in tasks such as code generation: k candidate answers are independently sampled for the same problem, and success is counted as long as any one of them passes verification.79% similarUnverifiedIn code generation evaluation, pass@1 measures the probability that the model passes the test with a single generation, while pass@10 allows the model to make 10 attempts and takes the best result74% similarUnverifiedNumerical consistency testing can be systematized into a regression test suite that computes the maximum absolute error and KL divergence of per-token log probabilities from both sides after each engine version upgrade, with a tolerance threshold (e.g., KL < 1e-4) that blocks training when exceeded.74% similarUnverifiedThe pass@k metric was first systematically defined in OpenAI's Codex paper, measuring the probability that at least one of k independent samples for the same problem is correct.71% similarVerifiedSpeculative decoding submits the candidate sequence generated by the draft model to the target model as a whole for one-shot batch verification, requiring only a single forward pass to determine whether each candidate token is correct.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/488716API
curl https://kongchang.com/api/v1/knowledge/claims/488716MCP
get_claim(id=488716)