Harvey LAB
An agent benchmark in the legal AI domain that simulates real lawyer workflows, requiring models to perform end-to-end tasks such as multi-step reasoning, document retrieval, clause comparison and structured output. It uses an all-pass scoring criterion.
Timeline (last 90 days)
The GLM-Legal model launched by Z.ai achieved a 19.2% all-pass rate on the Harvey LAB legal Agent benchmark, surpassing Claude Fable 5 and Kimi K3.
Harvey LAB (Legal Agent Benchmark) simulates Agent tasks from real lawyer workflows, requiring the model to have comprehensive capabilities such as multi-step reasoning, document retrieval, clause comparison, and structured output.
Harvey LAB adopts an all-pass criterion, where the model must be correct in all sub-steps of a task; any error in any step means it does not count.
The running cost of each test case on Harvey LAB is about $0.15.
GLM-Legal's 19.2% all-pass rate on Harvey LAB means that on most tasks the model still needs a human lawyer as a fallback.
Harvey LAB comes from within the industry rather than an academic institution, and its task design is closer to the real workflows of law firms.
All Facts (3)
Harvey LAB (Legal Agent Benchmark) simulates Agent tasks from real lawyer workflows, requiring the model to have comprehensive capabilities such as multi-step reasoning, document retrieval, clause comparison, and structured output.
50%UnverifiedHarvey LAB adopts an all-pass criterion, where the model must be correct in all sub-steps of a task; any error in any step means it does not count.
50%UnverifiedHarvey LAB comes from within the industry rather than an academic institution, and its task design is closer to the real workflows of law firms.
50%