Hallucination Rate
A core metric for measuring the reliability of large language models, referring to the proportion of responses in which the model fabricates information that appears plausible but is untrue. A lower hallucination rate indicates that the model is more inclined to acknowledge uncertainty rather than force a fabrication.
Timeline (last 90 days)
Hallucination metrics differ in definition: the overall proportion of unsupported statements and the "fabricating instead of admitting it doesn't know" ratio are metrics of different dimensions, and the two can simultaneously show a combination of "overall improvement, boundary behavior worsening".
In everyday applications aimed at the general public, the ability to reduce hallucination rates and express uncertainty (such as "I'm not sure") is more important than simply pursuing low cost.
Lowering the hallucination rate often comes with a higher refusal rate or more conservative answers, so real-world deployment requires a trade-off between accuracy and coverage.
V7's research shows that the drop in hallucination rate comes from two aspects: the model judges knowledge boundaries more accurately, and the context graph filters out facts that should not be remembered during the data ingestion stage.
V7 advises technical decision-makers that when evaluating enterprise-grade AI solutions, they should focus on the model's actual tool-call error rate and hallucination rate on their own business graph, rather than only looking at model benchmark scores.
Independent benchmarks show that LLM hallucination rates are roughly between 15% and over 50%, depending on the task and model.
On structured tasks with clear retrieval context, the best models can push hallucination rates below about 2%.
The paper gives a lower bound: after pretraining, the hallucination rate is at least equal to the proportion of facts in the training set that appear only once.
Hallucination rate is a metric measuring a model's tendency to fabricate content when it does not know the answer, and it must be understood separately from the knowledge score.
A low hallucination rate reduces risk but cannot replace manual verification against the original text.
All Facts (11)
通过RAG加载外部知识库可以同时降低幻觉率和空答率，但在外部知识固定的情况下两者呈此消彼长的关系
80%UnverifiedHallucination metrics differ in definition: the overall proportion of unsupported statements and the "fabricating instead of admitting it doesn't know" ratio are metrics of different dimensions, and the two can simultaneously show a combination of "overall improvement, boundary behavior worsening".
50%UnverifiedIn everyday applications aimed at the general public, the ability to reduce hallucination rates and express uncertainty (such as "I'm not sure") is more important than simply pursuing low cost.
50%UnverifiedLowering the hallucination rate often comes with a higher refusal rate or more conservative answers, so real-world deployment requires a trade-off between accuracy and coverage.
50%UnverifiedV7's research shows that the drop in hallucination rate comes from two aspects: the model judges knowledge boundaries more accurately, and the context graph filters out facts that should not be remembered during the data ingestion stage.
50%UnverifiedV7 advises technical decision-makers that when evaluating enterprise-grade AI solutions, they should focus on the model's actual tool-call error rate and hallucination rate on their own business graph, rather than only looking at model benchmark scores.
50%UnverifiedOn structured tasks with clear retrieval context, the best models can push hallucination rates below about 2%.
50%UnverifiedIndependent benchmarks show that LLM hallucination rates are roughly between 15% and over 50%, depending on the task and model.
50%UnverifiedThe paper gives a lower bound: after pretraining, the hallucination rate is at least equal to the proportion of facts in the training set that appear only once.
50%UnverifiedHallucination rate is a metric measuring a model's tendency to fabricate content when it does not know the answer, and it must be understood separately from the knowledge score.
50%UnverifiedA low hallucination rate reduces risk but cannot replace manual verification against the original text.
50%