AlpacaEval 2.0
A widely used instruction-following evaluation benchmark containing 805 diverse user instructions. It employs an LLM as an automatic judge to compute win rates and introduces a length-controlled (LC) win rate to mitigate length bias.
Timeline (last 90 days)
On the AlpacaEval 2.0 benchmark (with GPT-4o as the judge), after three rounds of DNO-Prct iteration, the length-controlled (LC) win rate of Llama 3.1 8B Instruct against GPT-4o rose from 31.7% to 45.3%.
On the AlpacaEval 2.0 benchmark (with GPT-4o as the judge), after three rounds of DNO-Prct iteration, the LC win rate of Mistral 7B Instruct v0.1 against GPT-4o rose from 6.8% to 18.4%.
AlpacaEval 2.0 is an instruction-following benchmark containing 805 diverse user instructions that uses an LLM as an automatic judge and introduces length-controlled (LC) win rate to control for length bias.
All Facts (3)
On the AlpacaEval 2.0 benchmark (with GPT-4o as the judge), after three rounds of DNO-Prct iteration, the length-controlled (LC) win rate of Llama 3.1 8B Instruct against GPT-4o rose from 31.7% to 45.3%.
50%UnverifiedOn the AlpacaEval 2.0 benchmark (with GPT-4o as the judge), after three rounds of DNO-Prct iteration, the LC win rate of Mistral 7B Instruct v0.1 against GPT-4o rose from 6.8% to 18.4%.
50%UnverifiedAlpacaEval 2.0 is an instruction-following benchmark containing 805 diverse user instructions that uses an LLM as an automatic judge and introduces length-controlled (LC) win rate to control for length bias.
50%