Unverified50% confidenceFactExact time
In RAG evaluation, about 85% of the run-to-run variance in the faithfulness metric comes from the generated answer, and almost all of the variance in answer relevancy comes from the answer; whereas 100% of the variance in context recall and context precision comes from the judge, because RAGAS does not read the generated answer when computing these two metrics, only comparing the retrieved chunks against the question and the reference answer.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe study found that the faithfulness score for counterfactual inputs was only 0.05 points lower than for factual inputs (on a 1–5 scale), a difference that is almost negligible.68% similarUnverifiedWhen only re-judging the same batch of saved answers, the standard deviation of the 18-question mean ranged from 0.021 (faithfulness) to 0.035 (precision); after regenerating answers, the standard deviation of faithfulness rose to 0.054, and relevancy rose from 0.002 to 0.028.68% similarUnverifiedOn a 50-sample evaluation set, an accuracy change from 74% to 76% corresponds to just one more correct sample, a difference that is statistically almost meaningless.65% similarUnverifiedThe paired 95% confidence interval for the strict score gain from evidence replacement is [8.77, 10.43].65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1065187API
curl https://kongchang.com/api/v1/knowledge/claims/1065187MCP
get_claim(id=1065187)