BenchmarkMATH500

MATH500

An evaluation benchmark comprising 500 mathematics competition problems, commonly used to assess the mathematical reasoning capabilities of large language models across a range of difficulty levels and problem types.

Timeline (last 90 days)

Oct 26

J-Comm reaches 94.80% on MATH500, 54.55% on GPQA-Diamond, 73.33% on AIME2024+2025, and a 62.93% success rate on ALFWorld.

Unverified50%

Oct 2

Experiments were conducted on Qwen2.5-7B-Instruct using Math-Shepherd as the PRM, tested on the MATH500 dataset (n=500, three random seeds).

Unverified50%

All Facts (2)

Unverified

J-Comm reaches 94.80% on MATH500, 54.55% on GPQA-Diamond, 73.33% on AIME2024+2025, and a 62.93% success rate on ALFWorld.

50%Unverified

Experiments were conducted on Qwen2.5-7B-Instruct using Math-Shepherd as the PRM, tested on the MATH500 dataset (n=500, three random seeds).

50%

Source Articles

J-Comm: A New Framework for Jacobian-Guided Latent Communication in LLM AgentsOct 26 Why Does PRM-Pruned Fragment Grafting Fail? An Empirical Analysis Across Three Reasoning ModelsOct 2