GPQA Diamond
GPQA Diamond is a graduate-level question-answering benchmark and the most difficult subset within the GPQA (Graduate-Level Google-Proof Q&A) series. The benchmark covers natural science disciplines including biology, chemistry, and physics, with questions designed by domain experts. It is intended to evaluate the performance of large language models on high-difficulty professional reasoning tasks and serves as an important reference standard for measuring scientific understanding in frontier AI models.
Timeline (last 90 days)
DeepSeek has not published V4.1 Flash 1017's score on GPQA Diamond; Pro 0813 scores 92.4 on this item, and the September version of Flash scores 90.9.
GPQA Diamond is the hardest and most strictly filtered subset of GPQA, with 198 questions written by PhD-level question writers in physics, chemistry, and biology, designed so that even non-experts allowed to use search engines would struggle to answer correctly in a short time.
GPQA Diamond leaderboard top five: Darwin-180B-RSI 94.4% (180B), Kimi-K3 93.5% (2.8T), Darwin-397B-ZTC 93.4% (404B), Ornith-1.5-397B 92.8% (403B), Qwen3.8-2.4T-A95B 92.6% (2.4T).
As of October 19, 2026, Darwin-180B-RSI ranks first on the AIhoster leaderboard for MMLU-Pro at 88.1% and first for GPQA Diamond at 94.4%.
Dory's score on the AA-Omniscience benchmark barely changes from 1 loop to 4 loops (10.03, 10.08, 10.03), and it trails Qwen3.5-4B on both GPQA Diamond and HLE.
The GPQA Diamond subset has 198 questions in total, and at a 95% confidence interval the standard error of a single sample is about 2 percentage points, so a difference of 0.7 points is within the noise range.
NVIDIA's FP4 version of Qwen3.8 achieves 91.5% accuracy on GPQA Diamond, versus 92.2% for the BF16 baseline, a difference of 0.7 percentage points.
GPQA Diamond, MMLU-Pro, Humanity's Last Exam, SWE-bench Verified, Terminal-Bench, MMMU-Pro, LEXam, and Open ASR Leaderboard are all official benchmarks marked by Hugging Face as official benchmark.
On the GPQA Diamond scientific reasoning benchmark, the RSI inference engine ranks 1st among models under 500B parameters.
VIDRAFT achieved 93.43% in the sub-500B parameter category on GPQA Diamond.
10 more timeline events
All Facts (17)
GPQA Diamond测试题目来自生物学、化学、物理学等领域的博士级别知识，领域专家平均正确率约65%
85%UnverifiedGPQA Diamond由纽约大学研究团队构建，人类专家（博士生）的平均正确率约为65%，非专业人类约为34%
85%UnverifiedQwen3.8-27B trails Claude Opus 4.6 on Terminal Bench, GPQA Diamond, and HLE benchmarks (73 vs 78.2, 89.2 vs 91.3, and 30.8 vs 40, respectively).
60%UnverifiedDeepSeek has not published V4.1 Flash 1017's score on GPQA Diamond; Pro 0813 scores 92.4 on this item, and the September version of Flash scores 90.9.
50%UnverifiedGPQA Diamond leaderboard top five: Darwin-180B-RSI 94.4% (180B), Kimi-K3 93.5% (2.8T), Darwin-397B-ZTC 93.4% (404B), Ornith-1.5-397B 92.8% (403B), Qwen3.8-2.4T-A95B 92.6% (2.4T).
50%UnverifiedGPQA Diamond is the hardest and most strictly filtered subset of GPQA, with 198 questions written by PhD-level question writers in physics, chemistry, and biology, designed so that even non-experts allowed to use search engines would struggle to answer correctly in a short time.
50%UnverifiedAs of October 19, 2026, Darwin-180B-RSI ranks first on the AIhoster leaderboard for MMLU-Pro at 88.1% and first for GPQA Diamond at 94.4%.
50%UnverifiedDory's score on the AA-Omniscience benchmark barely changes from 1 loop to 4 loops (10.03, 10.08, 10.03), and it trails Qwen3.5-4B on both GPQA Diamond and HLE.
50%UnverifiedNVIDIA's FP4 version of Qwen3.8 achieves 91.5% accuracy on GPQA Diamond, versus 92.2% for the BF16 baseline, a difference of 0.7 percentage points.
50%UnverifiedThe GPQA Diamond subset has 198 questions in total, and at a 95% confidence interval the standard error of a single sample is about 2 percentage points, so a difference of 0.7 points is within the noise range.
50%UnverifiedGPQA Diamond, MMLU-Pro, Humanity's Last Exam, SWE-bench Verified, Terminal-Bench, MMMU-Pro, LEXam, and Open ASR Leaderboard are all official benchmarks marked by Hugging Face as official benchmark.
50%UnverifiedGPQA Diamond is a multiple-choice question set targeting doctoral-level scientific knowledge, covering biology, chemistry, and physics, with domain experts averaging approximately 65% accuracy.
50%Unverified在全部 38 项基准中，Aplomb 1 在 8 项上拿到 5.3B 以内模型最高分，包括 MMLU-Pro、GPQA Diamond 和 BBH
50%UnverifiedThe IQ3S tier scores 92.93 on GPQA Diamond (vs. BF16 baseline 91.92), 86.86 on LiveCodeBench V6 (vs. 87.43), with a three-task average of 93.26 vs. 93.12, and a file size less than one quarter of BF16 (approximately 22%).
50%UnverifiedThe evaluation covered five benchmarks: MMLU-Pro, GPQA Diamond, GSM8K, German MGSM, and IFEval.
50%UnverifiedGPQA Diamond is a benchmark testing graduate-level scientific reasoning ability, with ordinary human performance at approximately 35% and top closed-source models in the 60–75% range.
50%UnverifiedThe NVFP4 version of Qwen3-27B shows only marginal loss compared to uncompressed FP16 on the MMLU Pro and GPQA Diamond benchmarks.
50%