AutomationBench
An evaluation benchmark designed to assess the ability of AI agents to execute automation tasks, measuring model performance across automated workflow scenarios.
Core Facts
Timeline (last 90 days)
On AutomationBench and VitaBench, SpecAgent recovers 44% to 75% of the quality gap between the two models, while reducing latency and memory overhead.
GLM-5.3-Flash scores 48.8 on AutomationBench, ahead of Claude Opus 4.8's 41.0 and GPT-5.6 Terra's 37.2, at a per-task cost of about $0.045.
Mistral Large 4 scores 59.9% on AutomationBench (657 business processes) and achieves 1393 Elo on AA-Briefcase.
OpenAI claims that GPT-6.1 Sol leads its predecessor by 4.8 points on the AutomationBench automated testing benchmark and scores 2.2 points higher than Anthropic's Opus 5.5.
On DeepSuite, GPT-6.1 Sol leads Sonnet 5.5 by approximately 4.2 points, while Sonnet wins on AutomationBench by a margin of 8.7 points.
Argon ranked first on Zapier's AutomationBench with a score of 51.3%.
The per-task cost of GPT-6 Sol High on Automation Bench is only one-tenth that of Opus 5.
V4.1 Flash took first place on the DeepSWE, CyberDream, and AutomationBench agent benchmarks.
All Facts (6)
The per-task cost of GPT-6 Sol High on Automation Bench is only one-tenth that of Opus 5.
65%UnverifiedOn AutomationBench and VitaBench, SpecAgent recovers 44% to 75% of the quality gap between the two models, while reducing latency and memory overhead.
50%UnverifiedOpenAI claims that GPT-6.1 Sol leads its predecessor by 4.8 points on the AutomationBench automated testing benchmark and scores 2.2 points higher than Anthropic's Opus 5.5.
50%UnverifiedOn DeepSuite, GPT-6.1 Sol leads Sonnet 5.5 by approximately 4.2 points, while Sonnet wins on AutomationBench by a margin of 8.7 points.
50%UnverifiedArgon ranked first on Zapier's AutomationBench with a score of 51.3%.
50%UnverifiedV4.1 Flash took first place on the DeepSWE, CyberDream, and AutomationBench agent benchmarks.
50%