ICLR Review Controversy: When a Reviewer "Doesn't Get" Your Paper, Should You Rebut?

A frustrating ICLR review scenario many authors face

A recent popular Reddit post titled "3321 ICLR, should I rebuttal? [R]" describes a typical review dilemma: of four reviewers, three understood the paper and grasped its core ideas, but one gave sharp criticism — the abstract was disorganized and hard to follow, the definitions were difficult to understand, and important explanations were missing. The poster was puzzled: why could the other three understand it, while this one "killer" reviewer couldn't?

This scenario is not uncommon in academic conference review, especially at top conferences like ICLR that use open review. Reviewers vary greatly in expertise, background, and level of effort. When most reviewers give positive or neutral evaluations and one gives negative feedback, authors often face a dilemma: rebut or not?

ICLR (International Conference on Learning Representations) is one of the top conferences in machine learning, known for its open review mechanism. Unlike most conferences that use double-blind review, ICLR's reviews, author rebuttals, and final decisions are all publicly available on the OpenReview platform for anyone to see. This mechanism aims to improve review transparency, but it also puts extra psychological pressure on authors — negative evaluations are publicly visible, and disagreements among reviewers are plain to see. ICLR review scores typically include an overall rating on a 1-10 scale, as well as sub-scores on dimensions such as technical quality, readability, and originality. When reviewer scores diverge significantly, the Area Chair (AC) must weigh the different opinions in the final decision, and the authors' rebuttal is an important basis for the AC's judgment.

Common reasons behind reviewer disagreement

The poster emphasized that "the other three reviewers could all understand it," which is indeed a strong signal that the paper's core ideas are not incomprehensible. But disagreements among reviewers often have deeper causes.

Differences in domain background: ICLR covers a broad range of machine learning topics, and reviewers may be assigned papers that are not perfectly aligned with their expertise. If a reviewer's research direction deviates from the paper, their familiarity with the terminology and definitions will differ, and their reading experience will naturally differ too.

Level of reading effort: Reviewers usually have to review multiple papers within a limited time, and the depth of their reading varies. One reviewer finding "the definitions hard to follow" may simply mean they didn't carefully work through the context, while other reviewers were willing to spend time understanding it.

Writing style preferences: Some reviewers have high standards for abstract structure and definitional clarity. Even if the paper's content is fine, they may criticize it for how it is expressed. The poster's mentions of "disorganized abstract" and "missing important explanations" fall into this category of subjective judgment.

The existence of "killer" reviewers: In open review, there are indeed some reviewers who tend to give harsh evaluations or are particularly picky about certain writing styles. The poster's use of "killer" to describe this reviewer reflects a feeling common among authors.

In academic conference review, the Area Chair (AC) plays a key role. The AC is usually a senior researcher responsible for coordinating the review process for papers within their scope, including assigning reviewers, overseeing review quality, moderating rebuttal discussions, and ultimately synthesizing reviews and author responses to recommend acceptance or rejection to the program committee. When serious disagreements arise among reviewers, the AC must judge which opinions are more credible — the consensus of the majority or the sharp criticism of an individual reviewer. ACs tend to focus on whether the criticism is specific, whether it has factual basis, and whether the authors' response is effective. Therefore, when authors respond to negative feedback clearly and systematically in their rebuttal, they are essentially helping the AC complete this judgment process.

The value and risks of rebuttal

Faced with this situation, authors need to weigh the pros and cons of rebuttal.

Potential value of rebuttal: If the negative feedback is indeed based on a misunderstanding, rebuttal can clarify the issue. For example, if a reviewer says "the definitions are hard to follow," the authors can provide clearer explanations in the rebuttal or point out the specific locations and context of the definitions in the paper. If the reviewer simply missed a certain explanation, the rebuttal can directly correct this information gap. In addition, rebuttal is also an opportunity to show the Area Chair (AC) the quality of the paper — even if this reviewer does not change their stance, the AC and other reviewers can see the authors' response.

Risks of rebuttal: If the response is handled poorly, it may escalate the conflict. The poster's tone carries emotion ("Why couldn't he???"). If the rebuttal comes across as confrontational, it will only make the reviewer cling more firmly to their negative evaluation. Academic rebuttals need to be calm, specific, and constructive, avoiding questioning the reviewer's competence or motives.

Another option: no rebuttal. If the other three reviewers' evaluations are positive enough and this reviewer's opinion clearly deviates from the consensus, the AC may give it less weight in the final decision. But this is not guaranteed, especially when reviewer scores differ greatly, as ACs tend to handle such cases cautiously.

Rebuttal (author response) is a formal stage in the academic conference review process. After reviewers submit their initial comments, authors have the opportunity to respond point by point to the review comments, clarifying misunderstandings, supplementing explanations, pointing out paper content that reviewers may have overlooked, or committing to revisions in the final version. Rebuttals usually have strict length limits (e.g., ICLR typically limits them to around 1000 words) and must be completed within a limited time. An effective rebuttal is not simply a defense but a strategic communication: it must address the reviewer's specific concerns while showing the AC the paper's substantive contributions. In many conferences, after the rebuttal, reviewers can choose to update their scores or maintain their original judgment, and the AC then synthesizes all information to make the final decision. Therefore, the quality of a rebuttal can sometimes directly affect the paper's acceptance outcome, especially when scores are borderline.

How to effectively respond to a reviewer who "doesn't get it"

If you decide to rebut, strategy matters more than emotion. Here are some actionable suggestions:

Confirm the facts first: Read the review comments carefully and distinguish factual errors (the reviewer missed content) from subjective preferences (writing style). For factual errors, directly point out the corresponding locations in the paper; for subjective preferences, you can acknowledge them and explain the direction of improvement, but there is no need to over-compromise.

Cite positive evaluations from other reviewers: The poster mentioned that the other three reviewers all understood the paper. This can be mentioned tactfully in the rebuttal, for example: "We thank Reviewer X for their suggestions on the abstract structure, and we also note that other reviewers found the paper's reasoning clear. We will further optimize the presentation in the revision to accommodate different reading habits." This approach both responds to the criticism and uses the consensus to reinforce the paper's readability.

Provide specific explanations rather than vague apologies: If the reviewer says the definitions are hard to follow, don't just reply "we will improve it." Instead, directly provide clearer definitions or examples. Let the reviewer see that you have taken their comments seriously rather than brushing them off.

Maintain a professional tone: Avoid phrasing like "you may not have read carefully." You can use "we realize this explanation may not be sufficient, and we add the following..." to guide the reviewer to re-examine the paper.

What community discussion reveals about review culture

This Reddit post resonated widely, showing that "one reviewer doesn't understand the paper" is a dilemma many authors have experienced. In conferences with open review, review comments are publicly visible, and authors' emotional reactions are more direct. But the advice that repeatedly appears in community discussions is: the core goal of rebuttal is not to "win" against the reviewer, but to help the AC make a more accurate judgment.

The poster's confusion — "why could the other three understand it but he couldn't" — itself already shows that the paper's readability is not a general problem. In this case, the focus of the rebuttal can be on clarifying misunderstandings and demonstrating the paper's substantive contributions, rather than dwelling on whether the reviewer is personally "qualified."

For authors facing similar situations, a practical criterion is: if the negative feedback focuses on presentation rather than methodology or experiments, and other reviewers have no similar criticism, then the cost-benefit ratio of rebuttal is usually favorable — because presentation issues can be resolved through clarification and revision and do not involve core flaws in the paper.

Conclusion

The ICLR review process is full of uncertainty, and reviewers' comprehension gaps are part of it. The poster's experience reminds us that rebuttal is not an emotional outlet but a strategic communication. When most reviewers recognize the paper, authors have the confidence to respond to individual negative comments; but how they respond determines the final effect. A calm, specific, and constructive rebuttal protects the paper's review outcome better than asking "why couldn't he understand it."

One-line Summary

When most reviewers understand the paper but one criticizes its presentation, authors should rebut calmly and specifically to help the AC make an accurate judgment.

Paragraph Summary

The article centers on a popular Reddit post about an ICLR review dilemma, discussing whether authors should rebut when three of four reviewers understand the paper and one sharply criticizes its presentation. It analyzes common causes of reviewer disagreement, including differences in domain background, reading effort, writing style preferences, and the existence of "killer" reviewers. It then weighs the value and risks of rebuttal: rebuttal can clarify misunderstandings and demonstrate paper quality, but an improper response may escalate conflict. The article proposes strategies for effectively responding to a reviewer who "doesn't get it": first distinguish factual errors from subjective preferences, tactfully cite positive evaluations from other reviewers, provide specific explanations rather than vague apologies, and maintain a professional tone. Finally, it points out that the core goal of rebuttal is to help the Area Chair make an accurate judgment, not to persuade the negative reviewer.