Agents' Last Exam
A challenging public evaluation benchmark targeting the comprehensive capabilities of AI agents, assessing models on planning, execution, and correction in long-horizon, multi-step autonomous task scenarios.
Core Facts
Timeline (last 90 days)
Under the medium-intensity setting of the Agents' Last Exam benchmark, GPT-6.1 Terra costs $0.29 per completed task, while GPT-6 Astra costs $4.82.
Under the medium-intensity setting of the Agents' Last Exam benchmark, GPT-6.1 Terra's score is on par with GPT-6 Astra.
On the four benchmarks ProgramBench, NL2Repo-Bench, Automation-Bench and Agents' Last Exam, V4.1 Flash 1017 scores 29.1, 68.2, 60.5 and 35.4 respectively, all higher than V4 Pro 0813's 15.5, 61.5, 43.2 and 25.7.
Tev-27B, trained only on ALE-related data, can nonetheless generalize to Terminal-Bench, suggesting that it has learned some domain-agnostic long-horizon task execution capability rather than merely memorizing specific task solutions.
Tev-27B was trained entirely with PPRS on the computational and mathematical sciences domains of ALE (Agents' Last Exam), matching GPT-6 Astra and Claude Opus 5 on the Pass@5 metric of ALE-CM.
GLM-5.3 achieved a state-of-the-art score among open-source models on the Agents' Last Exam benchmark.
All Facts (6)
Agent's Last Exam is a high-difficulty benchmark designed specifically to evaluate the autonomous agent capabilities of large models, requiring continuous invocation of external tools, maintenance of long-context memory, and self-correction across multi-step tasks.
65%UnverifiedUnder the medium-intensity setting of the Agents' Last Exam benchmark, GPT-6.1 Terra costs $0.29 per completed task, while GPT-6 Astra costs $4.82.
50%UnverifiedUnder the medium-intensity setting of the Agents' Last Exam benchmark, GPT-6.1 Terra's score is on par with GPT-6 Astra.
50%UnverifiedOn the four benchmarks ProgramBench, NL2Repo-Bench, Automation-Bench and Agents' Last Exam, V4.1 Flash 1017 scores 29.1, 68.2, 60.5 and 35.4 respectively, all higher than V4 Pro 0813's 15.5, 61.5, 43.2 and 25.7.
50%UnverifiedTev-27B, trained only on ALE-related data, can nonetheless generalize to Terminal-Bench, suggesting that it has learned some domain-agnostic long-horizon task execution capability rather than merely memorizing specific task solutions.
50%UnverifiedGLM-5.3 achieved a state-of-the-art score among open-source models on the Agents' Last Exam benchmark.
50%