Expired50% confidenceBenchmarkExact time
The pass rate of the new generation of models on real-world software engineering tasks such as SWE-bench has broken through the 50% mark for the first time.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/21/2026
First Seen
This claim has expired. It is kept for the record and no longer describes the current state.
Valid until: 10/19/2026(expired)
Sources
Related Claims
VerifiedCurrently, the resolution rate of top models on the SWE-bench Verified subset has exceeded 50%.81% similarUnverifiedWhen SWE-bench was first released, the pass rate of top models was below 2%; the pass rate of leading systems has now surpassed 50%.75% similarUnverifiedSWE-bench is a benchmark measuring models' ability to autonomously fix real GitHub Issues; early models had a resolution rate below 5%, and by 2025 mainstream products have surpassed 50%.71% similarUnverifiedOn SWE-bench, early models had a solve rate below 5%, while mainstream products in 2025 have surpassed 50%.69% similarUnverifiedAccording to public evaluations, the success rate of existing desktop agents in complex workflows is generally below 50%.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/582481API
curl https://kongchang.com/api/v1/knowledge/claims/582481MCP
get_claim(id=582481)