GraphEcho
A benchmark specifically designed to evaluate evidence redundancy in LLM-based graph agents. It quantifies agent responses to path repetition versus substantive evidence content by controlling two dimensions: the number of paths and the source of evidence.
Timeline (last 90 days)
GraphEcho is a benchmark proposed in a paper on arXiv that tests whether graph agents mistakenly treat repeatedly encountered information as additional corroboration.
GraphEcho systematically varies two dimensions—the number of paths and the source of evidence—while keeping the evidence content fixed.
GraphEcho evaluates both agents' judgment conclusions and their active exploration behavior.
Controlled synthesis experiments found that agents' judgment-level bias is model-dependent, with no consistent pattern.
Redundant supporting paths increase the proportion of repeated traversals by all evaluated frozen agents.
The paper proposes a source-aware post-training method (PAPT) that makes agents aware of the provenance of evidence during training.
There is a gap between efficient exploration (reducing redundant paths) and effective evidence utilization—reducing repetition does not equate to making good use of evidence.
RAG systems also face the echo problem revealed by GraphEcho when multiple documents retrieved are substantially from the same source.
All Facts (8)
GraphEcho is a benchmark proposed in a paper on arXiv that tests whether graph agents mistakenly treat repeatedly encountered information as additional corroboration.
50%UnverifiedGraphEcho systematically varies two dimensions—the number of paths and the source of evidence—while keeping the evidence content fixed.
50%UnverifiedGraphEcho evaluates both agents' judgment conclusions and their active exploration behavior.
50%UnverifiedControlled synthesis experiments found that agents' judgment-level bias is model-dependent, with no consistent pattern.
50%UnverifiedRedundant supporting paths increase the proportion of repeated traversals by all evaluated frozen agents.
50%UnverifiedThe paper proposes a source-aware post-training method (PAPT) that makes agents aware of the provenance of evidence during training.
50%UnverifiedThere is a gap between efficient exploration (reducing redundant paths) and effective evidence utilization—reducing repetition does not equate to making good use of evidence.
50%UnverifiedRAG systems also face the echo problem revealed by GraphEcho when multiple documents retrieved are substantially from the same source.
50%