ROUGE

Mainstream approaches to agent evaluation include LLM-as-Judge, trajectory matching, and outcome verification.

BLEU is commonly used for machine translation evaluation, and ROUGE is widely applied to summarization tasks; both measure text quality by statistically calculating n-gram overlap

Unlike general summary benchmarks (such as ROUGE and BERTScore), reference-based evaluation metrics focus on whether information is correctly attributed rather than on the degree of lexical overlap.

The study's generation-side evaluation used DeepEval's claim-level metrics, with deepseek-v4-pro serving as the LLM judge, supplemented by traditional BLEU/ROUGE metrics.

Reference-based metrics (such as BLEU, ROUGE, and METEOR) score by computing n-gram overlap or semantic similarity, but they reward surface imitation and cannot determine whether the content is faithful to the input graph.

When evaluating summary quality, in addition to literal overlap metrics such as ROUGE, semantic completeness measures such as "condition retention rate" should also be introduced.

Automatic metrics such as BLEU and ROUGE require the existence of a high-quality reference translation, and many real-world scenarios do not have this condition.

Automatic metrics such as BLEU and ROUGE only do surface matching, cannot identify semantic errors, and cannot distinguish error severity.

ROUGE基于召回率统计n-gram重合度，BLEU基于精确率并加入长度惩罚，两者都只看字面重叠，对同义替换敏感。

BLEU and ROUGE are automated evaluation metrics based on reference answers, primarily used for translation or summarization tasks.

In the text domain, Perplexity or ROUGE scores are used to evaluate fluency and relevance

Prior to DoGBench, assessing the quality of AI-generated documentation mostly relied on subjective judgment or general-purpose text metrics such as BLEU and ROUGE.

ROUGE emphasizes recall and is commonly used for summarization evaluation.

General-purpose text metrics such as BLEU and ROUGE focus only on literal word matching and cannot measure whether users can actually complete tasks by following the documentation.

Existing generative AI evaluation methods rely on text-overlap metrics such as BLEU and ROUGE or LLM-as-judge, but have the limitation of being unable to capture narrative logic and visual layout when evaluating presentations.

Relying solely on aggregate metrics such as BLEU, ROUGE, or overall accuracy to evaluate context compression strategies is insufficient, because they measure average performance in typical cases rather than worst-case coverage guarantees.

In business conversation scenarios, optimization paths guided by BLEU, ROUGE, or human scoring may carry the risk of objective misalignment.

ROUGE quantifies summarization quality by computing the n-gram overlap between a system summary and a reference summary, with ROUGE-1, ROUGE-2, and ROUGE-L measuring unigram, bigram, and longest common subsequence overlap, respectively.

ROUGE metrics exhibit length bias: models that generate shorter summaries have an advantage in precision, models that generate longer summaries have an advantage in recall, and ROUGE cannot measure factual consistency.

Traditional metrics such as BLEU and ROUGE are less aligned with human judgment than LLM-as-a-judge when it comes to perceiving open-ended dimensions such as semantic quality and logical coherence.