Unverified50% confidenceFactExact time
SWE-Bench was proposed by a team at Princeton University and is divided into three subsets—Full (2294 tasks), Verified (500 tasks), and Pro Public—and directly comparing scores across subsets may be misleading.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSWE-Bench Verified是经人工验证的500道高质量子集，SWE-Bench Lite是300道较简单的子集67% similarUnverifiedBullet 在 SWE-bench Verified 基准测试中取得 95.8% 的成绩，位列前三，平均每个任务耗时 119 秒58% similarUnverifiedUnder a 15-minute time limit per problem, the proof agent achieved a successful proof rate of 34.9% (157/450) on StochBench.57% similarUnverifiedAccording to Partha Sarkar, when a CrewAI three-agent support team in hierarchical mode received a purely technical query, the manager invoked all three agents including the billing agent, the billing response covered up the correct technical answer, the run consumed 38 seconds and 15,759 tokens, produced about 200 tokens of output, and reported no errors; the root cause is that hierarchical mode does not enforce conditional branching or true delegation.57% similarUnverifiedOn the seven benchmarks, 14 tasks and 28 test splits of StructTokenBench, ProTok achieved an average rank of 1.4 among pure structure tokenizers, ranking first on 21 splits and entering the top two on 27 of the 28 splits.56% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1046048API
curl https://kongchang.com/api/v1/knowledge/claims/1046048MCP
get_claim(id=1046048)