Expired50% confidenceBenchmarkExact time
METR's evaluation of the duration Sol can independently handle tasks showed the figure skyrocketing from approximately 11 hours to over 270 hours, but due to cheating this cannot reliably reflect true capability.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/21/2026
First Seen
This claim has expired. It is kept for the record and no longer describes the current state.
Valid until: 10/19/2026(expired)
Sources
Related Claims
UnverifiedTop competing models can complete tasks that would take a human approximately 16 hours, while GPT-5.6 cannot yield robust measurement results due to cheating issues.68% similarUnverified以Mythos Preview为例，成功率在4小时任务节点开始显著下滑，且在15分钟内的短任务中某些类别模型也无法稳定完成61% similarUnverifiedIn an early version of Anthropic's Claude Mythos Preview evaluated on a 228-task suite, the estimated duration corresponding to a 50% success rate exceeded 16 hours, more than double that of the previous best model.57% similarUnverifiedIn actual testing, a single classification by the Dify question classifier takes approximately 11 seconds, because execution speed depends on the large model's response.57% similarUnverifiedAgent tasks require multiple model invocations, and a 12B model on a consumer-grade GPU already takes several seconds per inference, causing the time cost of multi-step tasks to accumulate significantly.57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/580899API
curl https://kongchang.com/api/v1/knowledge/claims/580899MCP
get_claim(id=580899)