[KongchangAI]
· 1 min read· 927 words

Stop Using Majority Voting! Calibrated Models Preserve Annotator Disagreement and Improve Moderation Fairness at Low Cost

Stop Using Majority Voting! Calibrated Models Preserve Annotator Disagreement and Improve Moderation Fairness at Low Cost

Preserving annotator disagreement and using calibration instead of retraining improves distribution matching and group fairness in moderation models at low cost.

The article presents a study on cyberbullying severity annotation. It finds that only 4.1% of comments received identical labels from all 5 annotators, meaning majority voting discards massive disagreement information. Three findings: accuracy against majority-vote labels cannot distinguish competent models; calibrating existing models captures most of the improvement from predicting annotator distributions without retraining, validated on severity, abuse, and sarcasm datasets; and a shared distribution-predicting model significantly improves matching across demographic groups but cannot close inter-group performance gaps. Practical implications: on subjective tasks, supplement evaluation with distribution matching; calibration is a low-cost approach, but fairness requires more refined modeling.

In content moderation, training a model to identify the severity of cyberbullying typically relies on judgments from multiple human annotators. When annotators disagree, the most common approach is majority voting—compressing multiple labels into a single "consensus" label. This seems reasonable, but it discards the valuable information contained in annotator disagreement. A study on cyberbullying severity points out that the cost of this compression may be severely underestimated.

Calibrate, Don't Retrain research source screenshot

How Common Is Annotator Disagreement?

In the corpus used in the study, the proportion of comments where all 5 annotators gave exactly the same severity label was only 4.1%. In other words, over 95% of comments had some degree of annotator disagreement. Cyberbullying severity is itself a subjective judgment, and different annotators naturally have different definitions of "severe." Majority voting turns these 5 judgments into a single label, hiding not only the scale of disagreement but also the degree of evaluative difference among annotators.

What consequences does this information loss have for model training? The study offers three main findings.

Label Distribution is an alternative annotation representation to majority voting. For the same sample, instead of compressing to a single label, it records the frequency with which each class was chosen by annotators, forming a probability distribution. For example, if 5 annotators rate a comment's severity as 1, 2, 2, 3, 3, majority voting outputs 2, while the label distribution is represented as {1:0.2, 2:0.4, 3:0.4}. This representation preserves the scale and direction of disagreement, allowing the model to learn "the uncertainty of human judgment" rather than force-fitting a false consensus. In highly subjective tasks, label distributions more closely reflect the real annotation process and provide richer signals for evaluating models—the model should not only predict correctly but also predict like a real group of annotators.

Accuracy Cannot Distinguish Model Quality

The first finding directly targets a blind spot in traditional evaluation metrics: accuracy against majority-vote labels cannot distinguish between competent models. Two models with similar accuracy may differ greatly in their ability to match annotator judgment distributions. This means that selecting a model based on accuracy alone may yield one that appears to perform well but actually fails to reflect real annotation disagreement.

Predicting Distributions Requires Only Calibration, Not Retraining

The second finding is more practical: predicting the full annotator judgment distribution does not require much additional effort. Calibrating an existing model—adjusting its output probabilities after training—captures most of the improvement, without retraining. This conclusion was validated on three datasets covering severity, abuse, and sarcasm. On top of this, training with label distributions yields only marginal additional gains.

In other words, rather than retraining a complex model to learn annotation distributions, calibrating an existing model achieves comparable results at extremely low cost. For resource-constrained moderation teams, this is a pragmatic upgrade path.

Calibration refers to adjusting a model's output confidence so that it matches actual accuracy. An uncalibrated model may assign a probability of 0.9 to a prediction while its actual accuracy is only 0.7. Common calibration methods include Temperature Scaling, Platt Scaling, and Isotonic Regression, which typically fit a small number of parameters on a held-out set without changing the model's main weights. In the context of this study, calibration is used to "stretch" or "compress" the model's output class probabilities to better approximate the true distribution of annotator judgments. Since calibration involves very few parameters and requires no backpropagation retraining, its computational cost is minimal, making it particularly suitable for quickly adapting existing models to distribution prediction.

Can a Shared Model Close Group Gaps?

The third finding concerns annotator demographic information. The study examined a dataset containing annotator demographics and found that a single shared model has one advantage and one limitation.

The advantage is that models predicting judgment distributions significantly outperform models trained on majority labels in matching the judgments of each demographic group—both groups benefit. The limitation is that it cannot close the performance gap between groups, because a shared model cannot encode "which group is right on a given comment."

The core message of this research is that preserving annotator disagreement is a low-cost path toward fairer moderation, but it is not a complete solution. For content moderation systems, abandoning majority voting in favor of distribution prediction may be a starting point worth serious consideration.

Demographic Performance Gap refers to systematic differences in model performance across groups (e.g., gender, age, region). In subjective annotation tasks, this gap may stem from different judgment tendencies of the annotator groups themselves—one group may overall define "severe" more strictly. A shared model that learns the judgment distributions of all groups simultaneously can improve average matching across groups, but it cannot know at inference time "which group's standard this comment should be referenced against," making it difficult to eliminate relative gaps between groups. Closing the gap typically requires group-aware modeling designs, such as incorporating group conditions into the model, training adaptation heads for different groups, or penalizing the gap in the loss function.

Practical Implications

For teams building content moderation pipelines, this study offers actionable signals:

  • On highly subjective tasks (such as severity grading), accuracy metrics may mask substantive differences between models; evaluation should be supplemented with measures of how well models match annotator judgment distributions.
  • Calibrating existing models is a low-cost means of preserving disagreement information, requiring no training from scratch and suitable for rapid iteration on existing models.
  • Distribution prediction can improve matching across demographic groups, but if the goal is to close performance gaps between groups, more refined modeling designs are needed.
#annotator disagreement#label distribution#model calibration#content moderation#cyberbullying severity#demographic fairness#majority voting
Share:

Related articles