MBPP+
An enhanced version of the MBPP benchmark used to evaluate the Python code generation capabilities of large language models. It features more rigorous test cases and broader functional coverage compared to the original benchmark.
Timeline (last 90 days)
Paper evaluations show that, on RTX 3090 and Jetson Orin, BitMTP achieves up to 2.0x five-task average throughput improvement, with the MBPP+ single task reaching 3.3x.
Saluki scored 78.0 on MBPP+, 67.5 on multi-step causal reasoning, and 80.0 on AIME, retaining 82%–85% of the original model's mathematical reasoning capability.
All Facts (2)
Paper evaluations show that, on RTX 3090 and Jetson Orin, BitMTP achieves up to 2.0x five-task average throughput improvement, with the MBPP+ single task reaching 3.3x.
50%UnverifiedSaluki scored 78.0 on MBPP+, 67.5 on multi-step causal reasoning, and 80.0 on AIME, retaining 82%–85% of the original model's mathematical reasoning capability.
50%