Terminal-Bench 2.0
A benchmark specifically designed to evaluate the ability of terminal agents to execute tasks in real command-line environments. It covers typical terminal workflows such as file operations, software installation, script writing, and system configuration.
Core Facts
Timeline (last 90 days)
On the Terminal-Bench 2.0 benchmark, the same model Opus 4.6 scores from 60.45% to 82.02% under different harnesses (a gap of about 22 percentage points).
LangChain's Deep Agents, on GPT-5.2-Codex, raised its Terminal-Bench 2.0 score from 52.8% to 66.5% solely by restructuring the harness.
On the Terminal-Bench 2.0 benchmark, the same model GPT-5.5 scores from 66.07% to 88.09% under different harnesses (a gap of about 22 percentage points).
On Terminal-Bench 2.0, Qwen3.5-35B-A3B scores 37.1% with the previous method and 48.3% with EditAct.
Terminal-Bench 4.0 is an evaluation suite for command-line agents, requiring models to complete multi-step operations in real terminal environments, testing coherence and self-correction ability in long-horizon execution.
ReTerm is built on Qwen3.5-9B, and the resulting ReTerm-9B achieves 34.8% Pass@1 on Terminal-Bench 2.0, an improvement of 14.6 percentage points over the base model.
TomasuLLM achieves a 1.35× speedup on 28 tasks from Terminal-Bench 2.0.
On Terminal-Bench 2.0, one team moved from outside the top 30 into the top 5 solely by changing the Harness without swapping the Model.
All Facts (8)
Terminal-Bench 4.0 is an evaluation suite for command-line agents, requiring models to complete multi-step operations in real terminal environments, testing coherence and self-correction ability in long-horizon execution.
65%UnverifiedOn the Terminal-Bench 2.0 benchmark, the same model Opus 4.6 scores from 60.45% to 82.02% under different harnesses (a gap of about 22 percentage points).
50%UnverifiedLangChain's Deep Agents, on GPT-5.2-Codex, raised its Terminal-Bench 2.0 score from 52.8% to 66.5% solely by restructuring the harness.
50%UnverifiedOn the Terminal-Bench 2.0 benchmark, the same model GPT-5.5 scores from 66.07% to 88.09% under different harnesses (a gap of about 22 percentage points).
50%UnverifiedOn Terminal-Bench 2.0, Qwen3.5-35B-A3B scores 37.1% with the previous method and 48.3% with EditAct.
50%UnverifiedReTerm is built on Qwen3.5-9B, and the resulting ReTerm-9B achieves 34.8% Pass@1 on Terminal-Bench 2.0, an improvement of 14.6 percentage points over the base model.
50%UnverifiedTomasuLLM achieves a 1.35× speedup on 28 tasks from Terminal-Bench 2.0.
50%UnverifiedOn Terminal-Bench 2.0, one team moved from outside the top 30 into the top 5 solely by changing the Harness without swapping the Model.
50%