Terminal Bench 4.0
A benchmark test suite designed to evaluate AI model capabilities in executing programming tasks within terminal environments. It focuses on command-line operations, script writing, code debugging, and automated workflow performance in realistic development scenarios.
Core Facts
Terminal-Bench is a benchmark specifically designed to evaluate AI models' ability to execute complex tasks in command-line environments, covering scenarios such as file operations, script writing, and system debugging.
On Terminal Bench 4.0, Opus 5.5 scored 66.4%, Astra scored 57.9%, Fable 5.1 scored 55.8%, and the previous generation Opus 5 scored 52.3%.
DeepSeek V4.1-Flash-1017 improves by 11 points on the Terminal-Bench 4.0 benchmark compared with the previous version.
Timeline (last 90 days)
In the Terminal Bench 4.0 terminal engineering test, Carbon scored 65.2%, ahead of Opus 5.5's 59.6%.
Gemini 4 Carbon's cheating success rate on Terminal Bench was 0.0%.
On Terminal-Bench 4.0, Carbon's successful cheating rate dropped to 0.0% and cheating attempts dropped to 0.4%; in internal real coding environments, high-severity misalignment alerts fell by nearly 80%.
On Terminal Bench 4.0, Kimi K3.1 Pro trails Claude Opus 5.5 by less than 1 point while leading GPT-6 Astra.
V4.1 Flash 1017's score on Terminal-Bench 4.0 rose from 31.2% to 42.3%, a relative improvement of 35% in five weeks.
DeepSeek V4.1 Flash 1017 rose from 31.2 to 42.3 on Terminal Bench 4.0, from 20.2 to 42.3 on Program Bench, from 31.8 to 35.4 on Agent's Last Exam, from 64.6 to 72.1 on Apex Bench, from 78.9 to 83.0 on Chartography, and from 89.6 to 92.5 on Baby Vision.
The Terminal Bench 4.0 score rose from 31.2 to 42.3, an improvement of 11.1 points.
DeepSeek V4.1 Flash 1017's score on Terminal Bench 4.0 rose from 31.2% to 42.3%, surpassing GPT 5.6 Sol's 39.9%.
DeepSeek V4.1-Flash-1017 improves by 11 points on the Terminal-Bench 4.0 benchmark compared with the previous version.
Gemini 4 Argon lost to Cloud Opus 5.5 in Terminal Bench 4.0, and lost to GPT-6 Astra in the Frontier SWB suite.
13 more timeline events
All Facts (20)
Terminal-Bench is a benchmark specifically designed to evaluate AI models' ability to execute complex tasks in command-line environments, covering scenarios such as file operations, script writing, and system debugging.
90%VerifiedOn Terminal Bench 4.0, Opus 5.5 scored 66.4%, Astra scored 57.9%, Fable 5.1 scored 55.8%, and the previous generation Opus 5 scored 52.3%.
65%Partially VerifiedDeepSeek V4.1-Flash-1017 improves by 11 points on the Terminal-Bench 4.0 benchmark compared with the previous version.
75%Partially VerifiedV4.1 Flash 1017's score on Terminal-Bench 4.0 rose from 31.2% to 42.3%, a relative improvement of 35% in five weeks.
65%UnverifiedGemini 4 Carbon's cheating success rate on Terminal Bench was 0.0%.
60%UnverifiedOn Terminal Bench, mid-tier Sonnet 5.5 scores 28.8% at approximately $0.83 per task, while Sonnet 5's highest tier only reaches 10.3% at approximately $11.6 per task
60%UnverifiedIn the Terminal Bench 4.0 highest-accuracy test, Fable 5.1 achieved an accuracy of 55.8% at a cost of $19.50, while Astra achieved an accuracy of 56.7% at a cost of $10.35.
60%UnverifiedTerminal-Bench 2.1 is an iterative version of the benchmark.
60%UnverifiedIn the Terminal Bench 4.0 terminal engineering test, Carbon scored 65.2%, ahead of Opus 5.5's 59.6%.
50%UnverifiedOn Terminal-Bench 4.0, Carbon's successful cheating rate dropped to 0.0% and cheating attempts dropped to 0.4%; in internal real coding environments, high-severity misalignment alerts fell by nearly 80%.
50%UnverifiedOn Terminal Bench 4.0, Kimi K3.1 Pro trails Claude Opus 5.5 by less than 1 point while leading GPT-6 Astra.
50%UnverifiedDeepSeek V4.1 Flash 1017 rose from 31.2 to 42.3 on Terminal Bench 4.0, from 20.2 to 42.3 on Program Bench, from 31.8 to 35.4 on Agent's Last Exam, from 64.6 to 72.1 on Apex Bench, from 78.9 to 83.0 on Chartography, and from 89.6 to 92.5 on Baby Vision.
50%UnverifiedThe Terminal Bench 4.0 score rose from 31.2 to 42.3, an improvement of 11.1 points.
50%UnverifiedDeepSeek V4.1 Flash 1017's score on Terminal Bench 4.0 rose from 31.2% to 42.3%, surpassing GPT 5.6 Sol's 39.9%.
50%UnverifiedGemini 4 Argon lost to Cloud Opus 5.5 in Terminal Bench 4.0, and lost to GPT-6 Astra in the Frontier SWB suite.
50%UnverifiedHaiku 5.5's score on Terminal Bench 4.0 rose from 0% in the previous generation to 39.2%.
50%UnverifiedOpus 5.5 outperformed Astra and Fable 5.1 on the Terminal Bench 4.0 test.
50%UnverifiedClaude Opus 5.5 scores approximately 66.4 on Terminal Bench.
50%UnverifiedUnder the highest difficulty settings of Terminal Bench 4.0, Sonnet 5.5 outperforms Opus 5.5.
50%UnverifiedOn Terminal Bench 4.0, Astra costs approximately $10.35 and Fable 5.1 approximately $19.50.
50%