arXivLean
A formal mathematics benchmark built on Lean 4 and Mathlib, designed to evaluate the ability of LLMs to formalize theorem proofs from arXiv papers.
Timeline (last 90 days)
Lean 4 and its accompanying mathematics library Mathlib have become the de facto standard for formalized mathematics, and benchmarks such as arXivLean and PutnamBench are built on top of them.
Using Claude Opus 5, Maestro autonomously discovered proofs for 34 of 48 theorems in the June 2026 version of arXivLean, whereas the native agent managed only 19.
arXivLean is a benchmark obtained by extracting theorems from arXiv mathematics papers and formalizing them, and it belongs to research-level mathematics.
All Facts (3)
Lean 4 and its accompanying mathematics library Mathlib have become the de facto standard for formalized mathematics, and benchmarks such as arXivLean and PutnamBench are built on top of them.
50%UnverifiedUsing Claude Opus 5, Maestro autonomously discovered proofs for 34 of 48 theorems in the June 2026 version of arXivLean, whereas the native agent managed only 19.
50%UnverifiedarXivLean is a benchmark obtained by extracting theorems from arXiv mathematics papers and formalizing them, and it belongs to research-level mathematics.
50%