Unverified50% confidenceFactExact time
Evaluation methods that look only at the final answer (answer-only) count invalid execution processes as successes.
1
Sources
50%
Confidence
Long-term
Relevance
9/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedEvaluation metrics that only look at final answers or award partial credit may systematically overestimate AI system capabilities, and evaluations must examine the process rather than just the endpoint.74% similarUnverifiedThe study observed a clear lack of correlation between the correctness of answers and the validity of reasoning traces, with models frequently arriving at correct answers even when the reasoning traces are completely invalid.70% similarUnverified作者认为执行轨迹（trace）比自动评判结论更可靠，是判断Agent失败是否真实存在的关键依据68% similarUnverified纯写死规则的意图识别方案泛化能力差，用户换一种说法就识别失败66% similarUnverifiedThe most dangerous failure mode of LLM-driven agents is often not a crash or error, but confidently declaring a task complete when nothing has actually happened (hallucinatory completion).64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/898941API
curl https://kongchang.com/api/v1/knowledge/claims/898941MCP
get_claim(id=898941)