Unverified50% confidenceTradeoffExact time
The 82 tasks in OS World were not run under identical conditions—half of the tasks retained the original baseline scores, and the invested resources and number of attempts also differed—so the 7-percentage-point improvement cannot be entirely attributed to memory.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/17/2026
First Seen
Valid until: 12/16/2026
Sources
Related Entities
Related Claims
UnverifiedIn the Agency tool-calling test, both models scored 95 points, and the only failure for both was task No. 13 (USD to JPY foreign currency conversion).68% similarUnverifiedThe user pointed out that these models achieve high scores on benchmarks yet frequently fail at the simplest tasks.67% similarUnverifiedOSWorld 2.0中模型评估仅占总得分的11.53%，且没有任何任务的模型评估占比超过50%66% similarUnverified在一个396个文件的大型混合职责项目上测试时，召回率几乎跌到零66% similarUnverified微软研究院调查发现跨iOS/Android双平台维护的测试套件中，每次大版本UI改版平均会导致60%以上的测试用例失效65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/929316API
curl https://kongchang.com/api/v1/knowledge/claims/929316MCP
get_claim(id=929316)