Unverified50% confidenceFactExact time
When an LLM serves as a reward judge, the determinism and reproducibility of rewards decrease, and the same answer may receive different scores under different prompts or sampling, posing challenges to training stability.
1
Sources
50%
Confidence
Long-term
Relevance
10/20/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe main risks of LLM-as-judge include optimistic bias from the reviewer and generator models sharing training data, the significant influence of prompt wording on scoring, and the reviewer model's over-preference for fluent text.77% similarUnverifiedThe reason LLM-as-Judge is effective is that evaluation is a less difficult task than generation; the judge LLM only needs to make judgments on constrained scoring dimensions, making its results more stable than generation.75% similarUnverifiedA refusal-to-answer mechanism triggers a degradation strategy—such as returning an uncertain response, requesting human review, or narrowing the scope of the answer—when it detects that the input is out of the training distribution or the model's confidence is below a threshold.75% similarUnverifiedLLM judge results are affected by factors such as temperature, prompt wording, and model version updates, causing the same set of test cases to potentially produce different conclusions when run at different times.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1073138API
curl https://kongchang.com/api/v1/knowledge/claims/1073138MCP
get_claim(id=1073138)