Terminal-Bench 4.0
A programming capability benchmark focused on real-world engineering tasks, emphasizing the evaluation of multi-step command-line task completion in terminal environments. It measures the practical usability of coding agents operating in authentic terminal-based workflows.
Core Facts
On Terminal-Bench 4.0, Claude Opus 5.5 leads with 66.4%, followed by GPT-6 Astra at 61.8% and Argon at 57.4%.
Claude Sonnet 5.5 achieved 70.6% on the Terminal benchmark, compared to only 10.3% for the previous generation.
Terminal-Bench 4.0 is an evaluation suite for command-line agents, requiring models to complete multi-step operations in real terminal environments, testing coherence and self-correction ability in long-horizon execution.
Timeline (last 90 days)
On Terminal-Bench 4.0, Carbon's successful cheating rate dropped to 0.0% and cheating attempts dropped to 0.4%; in internal real coding environments, high-severity misalignment alerts fell by nearly 80%.
GPT-6.1 Terra achieves a vendor-reported score of 50.3% on Terminal-Bench 4.0.
The official disclosure states that Kimi K3.1 Pro achieved a score of 65.7% on the Terminal-Bench 4.0 benchmark.
Terminal-Bench 4.0 is an agentic coding evaluation benchmark for terminal environments, scored on task completion rate rather than text similarity.
DeepSeek says the Terminal Bench score of the 1017 version rose from 31 to 42.
Official benchmarks show DeepSWE v1.1 improving from 74.2 to 77.0 (Mini-SWE-V2 environment), and Terminal-Bench 4.0 improving from 31.2 to 42.3 (Claude Code environment).
The 1017 checkpoint's score on Terminal-Bench 4.0 rose from 31.2 to 42.3.
On the Terminal-Bench 4.0 benchmark, V4.1 Flash 1017 scores 42.3, and V4 Pro 0813 scores 12.4.
V4.1 Flash 1017's score on Terminal-Bench 4.0 rose from 31.2% to 42.3%, a relative improvement of 35% in five weeks.
The new model improved on all 11 of the vendor's own benchmarks, with Terminal-Bench 4.0 up 11.1 points and ProgramBench up 8.8 points.
21 more timeline events
All Facts (20)
On Terminal-Bench 4.0, Claude Opus 5.5 leads with 66.4%, followed by GPT-6 Astra at 61.8% and Argon at 57.4%.
75%VerifiedClaude Sonnet 5.5 achieved 70.6% on the Terminal benchmark, compared to only 10.3% for the previous generation.
75%VerifiedTerminal-Bench 4.0 is an evaluation suite for command-line agents, requiring models to complete multi-step operations in real terminal environments, testing coherence and self-correction ability in long-horizon execution.
65%VerifiedIn the Automation Bench office workflow test, Gemini 4 Argon scored 51.3%, GPT-6 Astra scored 41.4%, and Claude Opus 5.5 scored 42.5%.
65%VerifiedTerminal Bench requires models to complete a full software engineering workflow in a simulated terminal environment, including reading error logs, debugging, configuring environment variables, and operating the file system.
65%Partially VerifiedDeepSeek V4.1-Flash-1017 improves by 11 points on the Terminal-Bench 4.0 benchmark compared with the previous version.
75%Partially VerifiedV4.1 Flash 1017's score on Terminal-Bench 4.0 rose from 31.2% to 42.3%, a relative improvement of 35% in five weeks.
65%UnverifiedOn Terminal-Bench 4.0, Carbon's successful cheating rate dropped to 0.0% and cheating attempts dropped to 0.4%; in internal real coding environments, high-severity misalignment alerts fell by nearly 80%.
50%UnverifiedGPT-6.1 Terra achieves a vendor-reported score of 50.3% on Terminal-Bench 4.0.
50%UnverifiedThe official disclosure states that Kimi K3.1 Pro achieved a score of 65.7% on the Terminal-Bench 4.0 benchmark.
50%UnverifiedTerminal-Bench 4.0 is an agentic coding evaluation benchmark for terminal environments, scored on task completion rate rather than text similarity.
50%UnverifiedDeepSeek says the Terminal Bench score of the 1017 version rose from 31 to 42.
50%UnverifiedOfficial benchmarks show DeepSWE v1.1 improving from 74.2 to 77.0 (Mini-SWE-V2 environment), and Terminal-Bench 4.0 improving from 31.2 to 42.3 (Claude Code environment).
50%UnverifiedThe 1017 checkpoint's score on Terminal-Bench 4.0 rose from 31.2 to 42.3.
50%UnverifiedOn the Terminal-Bench 4.0 benchmark, V4.1 Flash 1017 scores 42.3, and V4 Pro 0813 scores 12.4.
50%UnverifiedThe new model improved on all 11 of the vendor's own benchmarks, with Terminal-Bench 4.0 up 11.1 points and ProgramBench up 8.8 points.
50%UnverifiedAccording to official self-reported data, compared with V4.1-Flash, V4.1-Flash-1017 improved on Terminal-Bench 4.0 from 31.2 to 42.3 (+11.1), on HLE from 36.8 to 42.6 (+5.8), on DeepSWE v1.1 from 74.2 to 77.0 (+2.8), and on Automation-Bench from 54.8 to 60.5 (+5.7).
50%UnverifiedDeepSeek-V4.1-Flash-1017's score on Terminal-Bench 4.0 rose from 31.2 for V4.1-Flash to 42.3.
50%UnverifiedOn Terminal-Bench 4.0, Haiku 5.5 scores 39.2% versus Opus 5.5's 70.6%.
50%UnverifiedTerminal-Bench 4.0 has a standard error of ±2.6 points, and Opus 5's 51.8% reproduced as 52.3%, which Anthropic calls noise.
50%