AI Invades Academic Peer Review: Is the Review Process in Critical Condition?
A seasoned reviewer warns that AI-generated reviews and paper writing are eroding academic peer review and calls for a complete ban on AI writing.
Based on reviewing experiences at AAAI 2027 and ICLR 2027, a seasoned reviewer points out that AI-generated content has seriously polluted academic review. On the author side, most submissions show obvious AI writing traces such as marketing-style language, synonym stacking, and long sentences; on the reviewer side, about half of review comments are suspected to be entirely AI-generated or edited, showing language homogenization, uniform paragraph lengths, formatting misalignment (such as confused field names), and Unicode symbols. The reviewer suggests a complete ban on AI writing, including polishing, though this relies on an honor system. Community consensus is that AI reviews lack depth and author-side AI traces are identifiable, but opinions diverge on the feasibility of a full ban. The article stresses that the trust system of academic review faces severe challenges and requires serious discussion of countermeasures.
A seasoned reviewer issued a warning on Reddit: AI-generated review comments and paper writing are eroding the foundations of academic peer review. This reviewer has reviewed for top conferences such as NeurIPS, ICML, AAAI, and ICLR about 10 times since 2022, but recent reviewing experiences at AAAI 2027 and ICLR 2027 left him feeling an unprecedented crisis.
![reddit source: The Paper Reviewing Process is Now in Critical Condition [D]](/media/screenshots/source/48405_0.png)
Author Side: AI Writing Traces Hard to Ignore
Of 3 ICLR submissions, only 1 felt 100% human-written. The rest bore obvious AI traces: excessive use of descriptive titles and marketing-style language (e.g., "Theoretical predictions of performance, empirical evaluations confirm this"), as well as long, dense sentences—repeatedly replacing already-used terms with synonyms, which interrupts the reader's train of thought. These features cannot be definitively confirmed, but they are enough to make reviewers suspicious.
The appearance of marketing-style language in academic writing may be related to LLM training data. LLMs are trained on large amounts of web text (including marketing materials, blog posts, etc.) and tend to generate persuasive and engaging expressions rather than rigorous, neutral academic language. For example, the phrase "Theoretical predictions of performance, empirical evaluations confirm this" is common in marketing but appears redundant and imprecise in academic papers. In addition, LLMs often use synonym substitution to avoid repetition, but this can lead to inconsistent terminology and increase the reader's cognitive burden. Academic writing emphasizes precision and consistency, and the LLM's "synonym cycling" may undermine this principle. Another clue for identifying AI-generated papers is the excessive use of descriptive titles (e.g., "Unveiling the mechanism of..."), which are rare in human writing because human authors prefer concise, direct titles. Although these features are not conclusive evidence, combined with other signs (such as sudden shifts in language style), they can raise suspicion.
Reviewer Side: AI-Generated Review Comments Proliferate
This reviewer estimates that about half of the review comments he saw were entirely AI-generated or AI-edited. The language lacks variation, and paragraph lengths are almost uniform, as if the LLM was asked to provide 5 strengths, 5 weaknesses, and 5 questions. Some details exposed non-human traces: placing a "Questions to the Authors" field in the "Weaknesses" section (likely because the field name was changed to "Specific Rebuttal Points" this round), and filling "N/A" in the new field; overly precise additions of arXiv numbers or DOI links followed by an extra "(doi.org)"; use of smart quotes, Unicode symbols instead of LaTeX (e.g., the Unicode form of \tilde{d}), non-hyphen dashes, and even Unicode to represent minus signs. These review comments read as if written for public consumption rather than the way human reviewers, under time pressure, communicate to help authors understand.
The misuse of LLMs (large language models) such as the GPT series in academic review partly stems from the openness of conference review systems. Many top conferences use double-blind review, but reviewer identities are anonymous to authors, and author information is usually hidden from reviewers as well. However, reviewers may use external LLM tools to process review comments or even generate them directly. Typical features of LLM-generated text include: excessive use of certain connectives (e.g., "furthermore," "however"), highly consistent paragraph structures, and a lack of personalized tone variation. These features contrast with the concise, sometimes disjointed comments human reviewers write under time pressure. In addition, LLMs tend to generate "safe" criticism, avoiding overly sharp or specific negative evaluations, which can lead to review comments lacking depth and specificity. Clues for identifying AI-generated review comments also include: unnatural formatting errors (such as confusing field names), excessive use of Unicode characters (such as smart quotes, dashes), and uncommon symbol representations (such as using a Unicode minus sign instead of a hyphen). These traces often stem from how LLMs handle special characters when generating text.
Root Causes and Possible Ways Out
The reviewer admits he has no concrete solution either. The only suggestion he offers is: completely ban AI writing, not even allowing polishing of style and tone. Authors may feel that AI only changed the tone and style, but readers can sometimes quickly "sense" it. Before powerful LLMs appeared, we did not judge papers based on language. He knows this relies on an honor system, but believes that is acceptable. Allowing "polishing with AI" may create an expectation of polishing with AI.
In academic review, an honor system refers to relying on participants to voluntarily follow the rules rather than enforcing them through technical means. On the issue of AI use, an honor system means requiring reviewers and authors to voluntarily refrain from using AI tools, or at least disclose their use. However, the effectiveness of an honor system depends on community consensus and individual integrity. When AI tools are easily accessible and hard to detect, the honor system faces challenges. Some conferences have begun requiring authors to declare whether they used AI-assisted writing, but enforcement is limited. The proposal to completely ban AI writing faces practical difficulties: how to define "AI writing"? Should the use of tools for polishing, grammar checking, literature summarization, etc. be banned? In addition, detection technology for AI-generated text is still immature, with a high risk of misjudgment. Therefore, in addition to bans, alternative approaches need to be discussed, such as developing AI detection tools, strengthening oversight of the review process, and encouraging reviewers to provide more specific criticism.
Multi-Source Perspectives: Consensus and Divergence
According to the original Reddit post and community discussion, reviewers generally agree that AI-generated content is polluting the review process. Consensus includes: AI review comments are homogenized in language and lack depth; AI writing traces on the author side (such as marketing-style titles) can be identified; a complete ban on AI writing is a solution worth considering. Divergence lies in: some believe AI-assisted polishing is hard to avoid and beneficial, and a comprehensive ban may be unrealistic; others advocate strict prohibition to maintain review integrity. Unique insights from a single source include: smart quotes and Unicode symbols can serve as indirect evidence of AI generation; misalignment caused by field name changes (e.g., "Questions to the Authors" appearing in the "Weaknesses" section) is a typical flaw of AI review.
Conclusion
Academic review relies on trust and an honor system, but the intervention of AI is breaking this balance. The reviewer's warning is not alarmist—when both review comments and the papers themselves may be AI-generated, the fairness and quality of the review process will face severe challenges. It is time to seriously discuss how to respond to this crisis.
One-line Summary
A seasoned reviewer warns that AI-generated review comments and paper writing are eroding the foundations of academic peer review, and calls for a complete ban on AI writing.
Paragraph Summary
Based on reviewing experiences at AAAI 2027 and ICLR 2027, a seasoned reviewer points out that AI-generated content has seriously polluted the academic review process. On the author side, most submissions bear obvious AI writing traces, such as marketing-style language, synonym stacking, and long sentences; on the reviewer side, about half of review comments are suspected to be entirely AI-generated or edited, manifested as language homogenization, uniform paragraph lengths, formatting misalignment (such as confused field names), and the use of Unicode symbols and other non-human features. The reviewer believes the root cause lies in the misuse of AI tools and suggests a complete ban on AI writing, including polishing, although this relies on an honor system. In community discussions, the consensus is that AI review comments lack depth and AI traces on the author side are identifiable, but opinions diverge on the feasibility of a comprehensive ban. The article emphasizes that the trust system of academic review is facing severe challenges and that responses need to be seriously discussed.
Related articles
LLM Application Development: A High-Paying Career Guide for Ordinary People
LLM application development offers fresh graduates up to 35K/month, with SP offers reaching 600K. This article explains the core skills, how to break in, and how ordinary people can seize AI opportunities through learning by doing.
Say Goodbye to AI Influencer Faces: A Complete Guide to Creating Visual Assets for AI Comic Dramas from Scratch
Bilibili creator shares a full workflow for zero-basics AI comic drama visual assets: use Doubao Work skills with script-first, visual references, and real-person texture to escape AI influencer faces, then lock character and scene consistency with asset sheet layouts, weapon detail images, scene nine-grids, and cinematic color palettes.
The AI Production Pipeline Behind Great Videos: From Research to Agent Validation
AI video production isn't just a model race. This article breaks down a transferable four-step pipeline: researching professional workflows, organizing a skill checklist, configuring agents, and experimental validation—and explains how general-purpose LLMs like DeepSeek can replace paid tools to run the entire process.