Unverified50% confidenceFactExact time
arXiv paper 2609.22512 points out that treating the consensus of multiple LLM reviewers as strong evidence presupposes that the errors of each reviewer are mutually independent, whereas in reality LLM reviewers make the same errors due to similar training data and evaluation approaches, causing consensus to exaggerate evidential strength.
1
Sources
50%
Confidence
Long-term
Relevance
9/23/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSome reviewers directly use LLMs to generate review comments, which tend to be superficially fluent but contain factual errors, making them particularly harmful for work involving mathematical proofs.74% similarUnverifiedAutoregressive models suffer from the exposure bias problem, where the model sees real prefixes during training but faces its own erroneous generated prefixes during inference, leading to error accumulation.72% similarUnverifiedError dependence is stronger among high-accuracy frontier model reviewers, even when those reviewers come from different vendors.72% similarUnverifiedStatistically, majority voting can effectively reduce error rates only when errors are mutually independent; when errors are highly correlated, consensus instead reinforces mistakes72% similarUnverifiedThe TruthfulQA benchmark found that larger models sometimes produce more confident errors, a phenomenon known as the scaling paradox.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/942776API
curl https://kongchang.com/api/v1/knowledge/claims/942776MCP
get_claim(id=942776)