MLE-Bench
A competition-style benchmark for machine learning engineering tasks that evaluates the performance of AI systems on autonomous modeling tasks using medal rate as the primary metric.
Core Facts
Timeline (last 90 days)
MLE-Bench was proposed by OpenAI and collects real machine learning competition problems from Kaggle, requiring an agent to run the entire pipeline from data exploration and feature engineering to training and submission.
Another study by Apple and EPFL found that the dedicated MLE harness also brought no large score differences on MLE-bench.
MLE-Bench was proposed by OpenAI, is collected from real Kaggle competitions, and requires agents to complete the full pipeline of data loading, feature engineering, model training and prediction submission within a limited time.
Medal rate refers to the proportion of agent submissions that reach the bronze medal threshold of that competition or above, directly benchmarked against the relative performance of human contestants.
AIBuildAI-2.5 ranked first on MLE-Bench, achieving a medal rate of 73.3%.
MLE-Bench is a standardized evaluation framework proposed by OpenAI that adapts historical Kaggle competition tasks into an automatically scorable benchmark with gold, silver, and bronze medal tiers.
All Facts (5)
Another study by Apple and EPFL found that the dedicated MLE harness also brought no large score differences on MLE-bench.
65%UnverifiedMLE-Bench was proposed by OpenAI and collects real machine learning competition problems from Kaggle, requiring an agent to run the entire pipeline from data exploration and feature engineering to training and submission.
50%UnverifiedMLE-Bench was proposed by OpenAI, is collected from real Kaggle competitions, and requires agents to complete the full pipeline of data loading, feature engineering, model training and prediction submission within a limited time.
50%UnverifiedMedal rate refers to the proportion of agent submissions that reach the bronze medal threshold of that competition or above, directly benchmarked against the relative performance of human contestants.
50%UnverifiedMLE-Bench is a standardized evaluation framework proposed by OpenAI that adapts historical Kaggle competition tasks into an automatically scorable benchmark with gold, silver, and bronze medal tiers.
50%