SmoothScore
A claim-level scoring procedure used to quantify the degree of "claim smoothing" produced by LLM research agents when synthesizing information from multiple sources. It evaluates how well conclusions are supported by evidence by extracting claims, clustering evidence, and scoring each claim.
Timeline (last 90 days)
The researchers propose SmoothScore to quantify the claim-smoothing phenomenon.
SmoothScore scores at the level of claims rather than the whole report or individual sentences.
The SmoothScore scoring process includes three steps: extracting key conclusive claims, finding and clustering relevant evidence, and scoring each claim based on the evidence.
Evidence clustering is a key step in the SmoothScore process; its role is to group heterogeneous evidence fragments by high-level aspect, so as to avoid duplicated evidence inflating a claim's credibility and to identify contradictions within the evidence.
Clustering quality directly affects the reliability of SmoothScore, so researchers usually need to combine semantic similarity with manual verification to build stable evidence clusters.
All Facts (5)
The researchers propose SmoothScore to quantify the claim-smoothing phenomenon.
50%UnverifiedSmoothScore scores at the level of claims rather than the whole report or individual sentences.
50%UnverifiedThe SmoothScore scoring process includes three steps: extracting key conclusive claims, finding and clustering relevant evidence, and scoring each claim based on the evidence.
50%UnverifiedEvidence clustering is a key step in the SmoothScore process; its role is to group heterogeneous evidence fragments by high-level aspect, so as to avoid duplicated evidence inflating a claim's credibility and to identify contradictions within the evidence.
50%UnverifiedClustering quality directly affects the reliability of SmoothScore, so researchers usually need to combine semantic similarity with manual verification to build stable evidence clusters.
50%