LLM-as-a-Judge

Mainstream approaches to agent evaluation include LLM-as-Judge, trajectory matching, and outcome verification.

The LLM-as-Judge method uses large language models such as GPT-4 as judges to automatically score the output quality of Agents.

LLM-as-a-Judge方法存在位置偏差、冗长偏差和自我偏好偏差等已知系统性偏差

AI system evaluation is divided into offline evaluation and online evaluation; offline evaluation can use a stronger model as a judge (LLM-as-a-Judge).

The LLM-as-a-Judge evaluation paradigm was first applied at scale by the LMSYS team in MT-Bench and Chatbot Arena.

LLM-as-a-judge模式即用一个能力更强的模型来评判另一个模型的输出质量，降低评测成本但引入评判模型自身的偏见问题

LLM as Judge was systematically proposed by Lianmin Zheng et al. in the MT-Bench paper (2023), with the core idea of using an independent large language model to score the output of another model as a substitute for human annotation.

The LLM-as-a-Judge paradigm was originally used to evaluate the output quality of generative AI and is now being applied to complex reasoning and judgment scenarios such as resume-job matching.

A single LLM judge exhibits position bias (a tendency to give higher scores to answers that appear first) and self-enhancement bias (a tendency to give higher scores to outputs in its own style).

Synonymous rephrasing can change LLMaJ evaluation results, and in some settings even reverse the relative rankings between models.

A user study with 10 human annotators shows that the consistency of LLMaJ judges under synonymous rephrasing drops 6-9 times as much as that of humans.

LLMaJ has been widely used in model leaderboards (such as the automated judging variant of Chatbot Arena), RLHF reward modeling, and academic benchmarks.

A study titled "Prompt Lottery in LLM-as-a-Judge: Measuring and Mitigating Sensitivity to Syntactic Rephrasing" reveals that semantically equivalent prompt variants with different wording cause LLM judges to give inconsistent verdicts, and the researchers named this phenomenon Prompt Lottery.

Through multi-prompt aggregation, emphasizing judgments that remain consistent across prompt variants can significantly improve the robustness of LLMaJ evaluation.

In model leaderboards, A/B tests, or academic comparisons, if only a single prompt is relied upon, rankings may flip due to minor wording changes.

The study calls on the community to re-examine LLMaJ evaluation protocols, and future work could explore more robust aggregation methods, techniques for automatically generating semantically equivalent prompt variants, and using human consistency as one of the metrics for evaluating LLMaJ quality.

"AI clone fault-finding" is an adversarial review model: the reviewer and the producer are played by different model instances, and the reviewer is forbidden from consulting the generation process and only looks at the diff, to avoid the "explanatory bias" of the generating side defending its own changes.

The study adopts a simulation-based evaluation method, uses real-world data, and combines LLM-as-a-Judge evaluation with human verification.

LLM judges exhibit position bias: in pairwise comparisons, judges tend to favor the answer in a certain position, usually the first; merely swapping the order of the candidate answers can flip the ranking (Wang et al.).

Having an LLM judge whether its own answers are correct is circular reasoning, because hallucinations and correct outputs may be structurally indistinguishable.