GPQA-Diamond
A graduate-level science question-answering benchmark subset whose questions are adversarially filtered to prevent search or memorization shortcuts, used to measure deep reasoning capability of models.
Timeline (last 90 days)
J-Comm reaches 94.80% on MATH500, 54.55% on GPQA-Diamond, 73.33% on AIME2024+2025, and a 62.93% success rate on ALFWorld.
GPQA-Diamond is a graduate-level subset of science questions that are adversarially filtered to prevent search or memorization shortcuts.
In the GPQA-Diamond xhigh test, the base model Qwen3.8-27B achieves 88.4% accuracy consuming 6,642 median tokens, while Swift maintains 88.3% accuracy with tokens reduced to 2,771
All Facts (3)
J-Comm reaches 94.80% on MATH500, 54.55% on GPQA-Diamond, 73.33% on AIME2024+2025, and a 62.93% success rate on ALFWorld.
50%UnverifiedGPQA-Diamond is a graduate-level subset of science questions that are adversarially filtered to prevent search or memorization shortcuts.
50%UnverifiedIn the GPQA-Diamond xhigh test, the base model Qwen3.8-27B achieves 88.4% accuracy consuming 6,642 median tokens, while Swift maintains 88.3% accuracy with tokens reduced to 2,771
50%