Unverified50% confidenceSolutionExact time
Before running comparisons, several known-bad outputs should be fed to the scorer and all required to be rejected, to audit whether the scorer passes everything because its rules have been loosened.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedMandatory checks should be separated from scoring; no matter how high the score on optional check items, it cannot offset a failed mandatory check.78% similarUnverifiedIn the adjudication queue, there were cases showing one reviewer marking partial compliance as a pass while another marked it as a fail.75% similarUnverifiedThe probe includes a pair of fake judges as a self-check: one is set to give different scores to the harm version (should be flagged), and the other gives the same score to both versions (should not be flagged), otherwise the run report is invalid.73% similarUnverifiedBefore seeing evaluation scores, agree with the person accountable for results on launch thresholds in advance, including the passing score for typical cases and the types of failures that are unacceptable at any rate (mustNotFail cases).71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1066680API
curl https://kongchang.com/api/v1/knowledge/claims/1066680MCP
get_claim(id=1066680)