SWE Bench Pro
An extended version of SWE-bench for evaluating a model's ability to automatically fix code defects, expanding repository and language coverage and raising anti-contamination requirements.
Timeline (last 90 days)
The model's official benchmark scores are SWE Bench Pro 61.7, LiveCode Bench V6 90.3, and OS World 84.3.
On agentic coding benchmarks such as SWE Bench Pro, Qwen 3.8 reportedly surpasses Opus 4.6 Max on some tasks.
SWE Bench requires models to resolve real issues in actual GitHub repositories, and SWE Bench Pro is a higher-difficulty upgraded version of it.
On SWE Bench Pro, Glimmer narrowly beat Qwen with 51.2 vs. 50.2, but Qwen surpassed it on SWE Bench Verified (Qwen 77.2 vs. 76) and Terminal Bench (Qwen 60.7 vs. 51.7).
All Facts (4)
The model's official benchmark scores are SWE Bench Pro 61.7, LiveCode Bench V6 90.3, and OS World 84.3.
50%UnverifiedOn agentic coding benchmarks such as SWE Bench Pro, Qwen 3.8 reportedly surpasses Opus 4.6 Max on some tasks.
50%UnverifiedSWE Bench requires models to resolve real issues in actual GitHub repositories, and SWE Bench Pro is a higher-difficulty upgraded version of it.
50%UnverifiedOn SWE Bench Pro, Glimmer narrowly beat Qwen with 51.2 vs. 50.2, but Qwen surpassed it on SWE Bench Verified (Qwen 77.2 vs. 76) and Terminal Bench (Qwen 60.7 vs. 51.7).
50%