Benchmark
DeepSweep
A large-model evaluation benchmark in the software engineering domain, designed to measure model performance on automated software engineering tasks.
Timeline (last 90 days)
Sep 29
GPT-6 Sol achieved 68.8% on the software engineering benchmark DeepSweep, approximately 1.1 percentage points behind top-tier flagship models, with costs reduced by about 80%.
Unverified50%