SWE-Marathon
A long-session benchmark for coding agents, designed to evaluate an AI system's ability to sustain continuous execution across software engineering tasks over extended interaction sessions.
Core Facts
Timeline (last 90 days)
Wonder Full scored 57.5 on the SWE-Marathon benchmark and reached 79.0 on DeepSWE 1.1.
In the SWE-Marathon evaluation, GPT-6 Astra and Claude Fable 5.1 used the Codex and Claude Code execution systems respectively, so this project is not a controlled comparison under a unified harness.
Muse Wonder Full's 57.5 score on SWE-Marathon used the xhigh reasoning setting.
In the SWE-Marathon benchmark, Muse Wonder scored 53.1 and Claude Opus 5.5 scored 46.3, a gap of 6.8 points.
In the API-available version of Muse Wonder, it beat Claude Opus 5.5 on the SWE-Marathon benchmark.
DeepSWE focuses on complex, deeply dependent engineering problems; SWE-Marathon emphasizes long-chain, multi-stage tasks, simulating scenarios of sustained work over long periods.
In the DeepSWE 1.1 benchmark, Muse Wonder Full achieved 79.0%, exceeding Claude Opus 5.5's 74.2% and GPT-6 Astra's 74.1%.
SWE-Marathon is an end-to-end outcome-based evaluation that gives the agent a software project spanning multiple hours and observes whether it can stay on course without drifting over a long horizon.
Meta uses FocusBench to evaluate focus and SWE-Marathon to test sustained execution on long-horizon software projects.
SWE-Marathon stretches tasks into dozens or even hundreds of steps of continuous editing, testing the agent's ability not to get lost in a long context.
6 more timeline events
All Facts (11)
A harness is a deterministic framework designed by software experts that manages state, executes build commands, runs linters, captures standard error output, and appends the logs back into the LLM context, prompting a statistical correction.
75%UnverifiedMuse Wonder Full's 57.5 score on SWE-Marathon used the xhigh reasoning setting.
50%UnverifiedIn the SWE-Marathon evaluation, GPT-6 Astra and Claude Fable 5.1 used the Codex and Claude Code execution systems respectively, so this project is not a controlled comparison under a unified harness.
50%UnverifiedWonder Full scored 57.5 on the SWE-Marathon benchmark and reached 79.0 on DeepSWE 1.1.
50%UnverifiedDeepSWE focuses on complex, deeply dependent engineering problems; SWE-Marathon emphasizes long-chain, multi-stage tasks, simulating scenarios of sustained work over long periods.
50%UnverifiedMeta uses FocusBench to evaluate focus and SWE-Marathon to test sustained execution on long-horizon software projects.
50%UnverifiedSWE-Marathon is an end-to-end outcome-based evaluation that gives the agent a software project spanning multiple hours and observes whether it can stay on course without drifting over a long horizon.
50%UnverifiedMeta notes in the model card that SWE-Marathon is not a controlled like-for-like comparison.
50%UnverifiedOn DeepSWE 1.1, Muse Wonder beats Claude Opus 5.5 78.8 to 74.2, leads 10.0 to 4.2 on the Harvey legal agent benchmark LAB-AA, and leads 53.1 to 46.3 on SWE-Marathon.
50%UnverifiedSWE-Marathon stretches tasks into dozens or even hundreds of steps of continuous editing, testing the agent's ability not to get lost in a long context.
50%UnverifiedTomasuLLM achieves a 1.27× matched-progress improvement across 18 sessions on SWE-Marathon.
50%