Unverified60% confidenceBenchmarkExact time
Research shows that the agreement rate between LLM judges and human scores is approximately 80%, approaching but not yet surpassing the level of human inter-annotator agreement (approximately 81%).
2
Sources
60%
Confidence
Long-term
Relevance
7/17/2026
First Seen
Sources
Related Claims
VerifiedThe LLM-as-Judge method can achieve over 80% agreement with human expert scoring.82% similarUnverifiedMultiple studies from 2024 indicate that the agreement rate between LLM judges and human experts is typically between 70% and 85%.78% similarUnverified研究表明人类评估者在判断LLM输出的事实准确性时，正确率往往只有50%-70%78% similarUnverified研究发现强大的LLM在评估开放式问答质量方面与人类评分者的一致性可达80%以上77% similarUnverified研究表明LLM裁判与人类专家的一致性在许多任务上可达80%以上76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/547417API
curl https://kongchang.com/api/v1/knowledge/claims/547417MCP
get_claim(id=547417)