Unverified70% confidenceFactExact time
In 2021, NeurIPS conducted a famous experiment in which the same paper was submitted twice and received completely different review outcomes, revealing the high degree of randomness in the review process.
2
Sources
70%
Confidence
Long-term
Relevance
8/23/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedEmpirical studies such as the dual-track experiment at NIPS 2014 show that even the review processes of top conferences contain a considerable degree of randomness and subjectivity.78% similarUnverifiedFour replication experiments led by cognitive psychologists in 2021 failed to reproduce a reliable doorway effect, suggesting that its effect size may have been exaggerated or that it holds only under specific conditions.69% similarUnverifiedReviewing at top conferences is stochastic, and the same paper may receive vastly different evaluations in the hands of different reviewers.69% similarUnverifiedIn preliminary experiments, among 12 tasks selected from 5 NeurIPS 2025 benchmarks, 5 showed serious evaluation cheating, and the cheating behaviors can be categorized into four types: selective reporting, test set contamination, metric manipulation, and direct hardcoding.67% similarUnverifiedIn 2018, Henderson et al. published a paper noting that the experimental results of many RL papers are highly sensitive to minor differences in random seeds, hyperparameters, and code implementations.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/792250API
curl https://kongchang.com/api/v1/knowledge/claims/792250MCP
get_claim(id=792250)