ICLR27 Math Theory Paper Review Sparks Controversy: Theory Ignored, Experiments Under Attack

Introduction: A Discussion on Review Fairness

Recently, a researcher on Reddit shared their firsthand experience submitting to ICLR27, sparking widespread discussion about the review quality for math and statistics papers. The paper was in an applied theory style, with about 60-70% theoretical derivation and some experiments at the end. It ultimately received a 222 score in the probabilistic methods track (all three reviewers gave a score of 2). However, all three reviewers unanimously attacked the paper's novelty and demanded additional experiments, yet said nothing about the theoretical portion that was the core contribution. One reviewer even stated outright that they "did not understand this paper, nor why this task exists."

This case is not an isolated incident. The researcher used the accept-odds tool to track multiple similar submissions and found they received similar comments—no feedback on the mathematics whatsoever, with all opinions focused on the experimental section. Even more surprising, a paper containing 20 pages of proofs was commented on as follows: "Assuming the proofs are correct (which AI in 2026 can verify), these theorems only demonstrate trivial conclusions."

ICLR (International Conference on Learning Representations) is one of the top conferences in machine learning, known for open review (OpenReview) and its emphasis on representation learning. Its review process typically includes author submission, reviewer assignment, review scoring, author rebuttal, and AC adjudication. In recent years, ICLR submissions have surged (exceeding 10,000 in 2024), and the reviewer pool has expanded dramatically, leading to uneven review quality. The bidding phase refers to the mechanism by which reviewers express their willingness to review candidate papers before assignment, helping to assign papers to experts familiar with the field. Eliminating bidding may improve assignment efficiency but could weaken the match between reviewers and papers, with a particularly significant impact on highly theoretical papers.

Analysis of Typical Review Patterns

Theoretical Contributions Systematically Ignored

From the description, the review comments show a highly consistent tendency: focusing on deficiencies in experimental design while avoiding evaluation of theoretical depth. This may reflect a mismatch in reviewers' knowledge structures—when a paper involves deep mathematical derivation, reviewers without theoretical backgrounds tend to shift attention to the experimental aspects they are more familiar with.

In the above case, the reviewers demanded additional experiments but raised no specific questions about the novelty, correctness, or importance of the theory. This "avoiding the hard part" review approach prevents theoretical contributions from receiving due evaluation.

Questionable Reviewer Expertise Matching

The researcher speculated that ICLR's elimination of the bidding phase this year (a mechanism where reviewers actively select familiar fields) may have led to many highly theoretical papers being assigned to reviewers with purely computer science backgrounds. While this speculation cannot be confirmed, it does point to a possible systemic problem in the review assignment mechanism: when reviewers do not match the paper's field, review quality declines significantly.

The 222 score refers to the three scores in ICLR's review scoring system, typically ranging from 1-10 (1 being strong reject, 10 being strong accept). A 222 means all three reviewers gave a score of 2, which is a relatively low score, usually corresponding to a "reject" tendency. In ICLR's review system, the final decision is made by the Area Chair (AC) based on review comments and author rebuttal, but low-scoring papers have a lower probability of entering the discussion phase. The accept-odds tool is a third-party tool that predicts paper acceptance probability based on historical review data; authors can estimate final outcomes by inputting scores, review comments, and other features, commonly used for submission strategy analysis.

Applied theory style papers typically refer to work between pure theory and pure experimentation: they include rigorous mathematical modeling, theorem proofs, and complexity analysis, supplemented by experiments that verify theoretical conclusions or demonstrate practical effectiveness. The review challenge for such papers is that reviewers need both theoretical foundations and experimental judgment. If reviewers are only familiar with experimental methods, they tend to focus on surface issues such as experiment scale and baseline selection, and cannot evaluate the novelty of theorems, the difficulty of proof techniques, or the reasonableness of assumptions. This knowledge structure mismatch is one of the core reasons for the review predicament of theoretical papers.

Community Reactions and Common Predicament

The post resonated with many researchers. Several respondents said their theoretical papers submitted to ICLR encountered similar situations: review comments focused on surface issues such as experiment scale and baseline selection, with little in-depth discussion of core content such as proof approaches and assumption reasonableness. This patterned review feedback makes it difficult for authors to obtain valuable improvement directions from rebuttal.

As a detail, one comment noted that even when reviewers acknowledged "assuming the proofs are correct," they characterized the conclusions as merely "trivial." This evaluation approach avoids specific analysis of the difficulty of the proof techniques and is essentially an "evaluation without evaluation."

The Author's Dilemma: Rebuttal or Withdrawal?

At the end of the original post, the author raised a practical question: given such review comments, should one invest effort in a rebuttal or simply withdraw and submit elsewhere?

Strategically, the effectiveness of a rebuttal depends on whether reviewers are willing to re-examine theoretical contributions. If reviewers themselves lack relevant backgrounds, even detailed explanations of theorem value may struggle to change their entrenched judgments. However, withdrawal means giving up the opportunity at this conference and may delay publication progress.

In community discussion, some suggested that authors could politely point out in the rebuttal the misalignment between review comments and the paper's core contributions, and request Area Chair (AC) intervention, or suggest introducing better-matched reviewers. However, the success rate of this approach is uncertain.

Rebuttal is the phase in conference review where authors formally respond to review comments, typically lasting 1-2 weeks. Authors can rebut reviewers' concerns point by point, clarify misunderstandings, supplement experiments or theoretical explanations. In ICLR, reviewers can update scores after rebuttal, and the AC makes the final decision. However, the effectiveness of rebuttal depends heavily on reviewers' open-mindedness and professional background: if reviewers lack relevant field knowledge, even detailed author explanations may struggle to change their judgments. Withdrawal and resubmission means giving up the current conference opportunity and submitting the paper to another conference or journal, which may delay publication progress but can sometimes avoid unfair evaluation caused by reviewer mismatch.

Reflections on the Review Mechanism

This incident reflects the structural challenges facing the review system of top machine learning conferences. As submissions continue to grow, expansion of the reviewer pool is inevitable, but how to ensure precise matching between reviewers and paper fields remains an unresolved problem. Eliminating bidding may have been intended to improve assignment efficiency, but it may have sacrificed matching quality.

For theoretical papers, the review predicament is particularly prominent: the value of theoretical contributions often requires peer experts to accurately judge, while experimental sections are relatively easier to evaluate. When reviewers lack theoretical backgrounds, the review focus naturally slides toward experiments, causing theoretical innovation to be undervalued.

Conclusion

This ICLR27 sharing provides the academic community with a window into the review ecosystem. It reminds us that while pursuing review efficiency, we cannot neglect the professionalism and fairness of review. For authors facing mismatched review comments, in addition to actively communicating in the rebuttal, more prudent choices in submission strategy may also be needed.

(Note: The scores and review comments cited in this article are all from the original post description.)