Unverified50% confidenceDecision RuleExact time
When interpreting benchmarks that use binary thresholds, one should distinguish between the two signals of threshold pass rate and dimension scores, to avoid misreading threshold luck as a capability gap.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe confidence threshold is a double-edged sword: lowering the threshold increases recall but decreases precision; raising the threshold increases precision but increases missed detections.73% similarUnverifiedOne approach to addressing the Goodhart problem is to construct a dual-objective evaluation that simultaneously tracks epistemic humility rate and uncertainty precision rate, replacing single-dimension optimization with an F1-style composite score.70% similarUnverifiedThe experiment used a pre-specified two-percentage-point margin for the coverage determination as the criterion for "substantial difference", in order to avoid treating Monte Carlo noise as a conclusion.69% similarUnverifiedThe comparison logic used for policy decisions blocks when score > threshold, so a score of exactly 0.50 is allowed through, consistent with Mistral's reference implementation.68% similarUnverifiedVector distance and retrieval thresholds are a trade-off between recall and accuracy: the looser the threshold (the larger the value), the more is recalled, but the higher the probability of irrelevant content being mixed in; tightening the threshold may miss answers that could have been reused.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1060014API
curl https://kongchang.com/api/v1/knowledge/claims/1060014MCP
get_claim(id=1060014)