HLE
A high-difficulty benchmark designed to evaluate AI model performance on expert-level academic tasks, assessing both the breadth and depth of a model's world knowledge.
Core Facts
Timeline (last 90 days)
According to the official DeepSeek model card, the 1017 version's score on HLE without tools rose from 36.8 to 42.6.
GAIA, WebWalker, HLE and DeepSearchQA evaluate different capabilities: GAIA targets tool calling for general AI assistants, WebWalker focuses on web navigation and multi-hop retrieval, HLE is a high-difficulty closed-book knowledge reasoning exam, and DeepSearchQA evaluates accuracy on multi-hop deep retrieval question answering.
On the text-only HLE subset, V4.1 Flash 1017 scores 44.0, surpassing V4 Pro 0813's 42.7.
Dory's score on the AA-Omniscience benchmark barely changes from 1 loop to 4 loops (10.03, 10.08, 10.03), and it trails Qwen3.5-4B on both GPQA Diamond and HLE.
According to official self-reported data, compared with V4.1-Flash, V4.1-Flash-1017 improved on Terminal-Bench 4.0 from 31.2 to 42.3 (+11.1), on HLE from 36.8 to 42.6 (+5.8), on DeepSWE v1.1 from 74.2 to 77.0 (+2.8), and on Automation-Bench from 54.8 to 60.5 (+5.7).
DeepSeek V4.1 Flash 1017's score on the HLE benchmark rose from 36.8% to 42.6%.
On the tool-free HLE benchmark, Claude Opus 5 scores 56.3 and DeepSeek-V4.1-Flash-1017 scores 42.6.
GLM-5.3-Flash scores 63.4 on DeepSWE v1.1, a 17.2-point improvement over GLM-5.2's 46.2, and 55.3 on HLE.
High-Level Emulation (HLE) re-implements hardware functionality at a higher level of abstraction, whereas Low-Level Emulation (LLE) accurately simulates the behavior of every circuit in the original hardware on a cycle-by-cycle basis.
The Qwen4 preview version scored only 35.9 on the HLE reasoning leaderboard, losing to a competitor's enhanced version that scored 40.
9 more timeline events
All Facts (19)
DeepSeek V4.1 Flash scores lower than the previous version on GPQA, HLE, and other benchmarks.
75%UnverifiedQwen3.8-27B trails Claude Opus 4.6 on Terminal Bench, GPQA Diamond, and HLE benchmarks (73 vs 78.2, 89.2 vs 91.3, and 30.8 vs 40, respectively).
60%UnverifiedAccording to the official DeepSeek model card, the 1017 version's score on HLE without tools rose from 36.8 to 42.6.
50%UnverifiedGAIA, WebWalker, HLE and DeepSearchQA evaluate different capabilities: GAIA targets tool calling for general AI assistants, WebWalker focuses on web navigation and multi-hop retrieval, HLE is a high-difficulty closed-book knowledge reasoning exam, and DeepSearchQA evaluates accuracy on multi-hop deep retrieval question answering.
50%UnverifiedOn the text-only HLE subset, V4.1 Flash 1017 scores 44.0, surpassing V4 Pro 0813's 42.7.
50%UnverifiedDory's score on the AA-Omniscience benchmark barely changes from 1 loop to 4 loops (10.03, 10.08, 10.03), and it trails Qwen3.5-4B on both GPQA Diamond and HLE.
50%UnverifiedAccording to official self-reported data, compared with V4.1-Flash, V4.1-Flash-1017 improved on Terminal-Bench 4.0 from 31.2 to 42.3 (+11.1), on HLE from 36.8 to 42.6 (+5.8), on DeepSWE v1.1 from 74.2 to 77.0 (+2.8), and on Automation-Bench from 54.8 to 60.5 (+5.7).
50%UnverifiedDeepSeek V4.1 Flash 1017's score on the HLE benchmark rose from 36.8% to 42.6%.
50%UnverifiedOn the tool-free HLE benchmark, Claude Opus 5 scores 56.3 and DeepSeek-V4.1-Flash-1017 scores 42.6.
50%UnverifiedHigh-Level Emulation (HLE) re-implements hardware functionality at a higher level of abstraction, whereas Low-Level Emulation (LLE) accurately simulates the behavior of every circuit in the original hardware on a cycle-by-cycle basis.
50%UnverifiedThe Qwen4 preview version scored only 35.9 on the HLE reasoning leaderboard, losing to a competitor's enhanced version that scored 40.
50%UnverifiedAREX-2 achieved the following scores on deep research tasks: BrowseComp 84.0, HLE 52.6, GAIA 92.2, and DeepSearchQA 93.8
50%UnverifiedAccording to official statements, V4.1 Flash surpasses GPT-5.6 SOAR and Opus 5 on multiple benchmarks including Terminal Bench 2.1, DeepSWE 1.1, CyberGym, HLE, Automation Bench, and Agent's Last Exam.
50%UnverifiedOn pure reasoning tasks, Qwen3 27B scores only 30.8 on HLE, notably behind Opus's score of 40.0.
50%UnverifiedOn the HLE benchmark, COMED improved GPT-5.5's performance from 23.1% to 28.1%.
50%UnverifiedHLE (Humanity's Last Exam) is an extremely high-difficulty comprehensive knowledge test constructed by Scale AI in collaboration with numerous domain experts.
50%UnverifiedWith context management enabled, Iris-mini achieves 82.2%, 84.8%, 86.9%, and 52.3% on BrowseComp, BrowseComp-ZH, DeepSearchQA, and HLE respectively.
50%UnverifiedHLE (Humanity's Last Exam) is an extremely high-difficulty comprehensive reasoning benchmark jointly launched by Scale AI and multiple academic experts, covering dozens of disciplines including mathematics, physics, biology, philosophy, and law.
50%UnverifiedHLE（Humanity's Last Exam）是由Scale AI与学术机构联合开发的超高难度综合评估基准
50%