LMSYS

Chatbot Arena, operated by LMSYS, ranks models through blind evaluation votes by real users on anonymous models, using an ELO rating system derived from chess.

Chatbot Arena is a crowdsourced live evaluation platform operated by LMSYS, where users converse with two anonymous models simultaneously and vote for the better one.

Arena (Chatbot Arena) is an AI model evaluation platform created by the LMSYS team at UC Berkeley, which uses a blind battle mechanism to generate an ELO score leaderboard.

LMSys Chatbot Arena was created by the LMSYS organization at UC Berkeley and has collected more than one million human votes since its launch in 2023.

LM Arena (formerly Chatbot Arena) is an anonymous head-to-head large model evaluation platform developed by a team at the University of California, Berkeley.

LM Arena (formerly Chatbot Arena), maintained by UC Berkeley's LMSYS team, uses an Elo rating system to measure model capabilities through user blind-test comparative voting.

The LLM-as-a-Judge evaluation paradigm was first applied at scale by the LMSYS team in MT-Bench and Chatbot Arena.

Chatbot Arena is maintained by LMSYS Org and uses the ELO rating system.

Chatbot Arena was introduced by a research team at UC Berkeley in 2023 and is one of the earliest platforms to implement large-scale LLM battle evaluation

WebDev Arena draws on the human preference voting mechanism of Chatbot Arena, ranking participants by ELO score through blind-test voting.

Arena-type platforms use a pairwise blind-testing mechanism in which users vote for the better response without knowing the identity of the models.

Web Dev Arena is a specialized evaluation platform launched by LMSYS that uses a blind-test battle mechanism where users vote based on visual results and rankings are determined through the Elo rating system.

LMSYS's RouteLLM paper describes a router as a policy that dynamically switches between a strong model and a weak model to optimize the balance between cost and quality.

RadixArk/Qwen3.8-Flash-Next-NVFP4 mentioned in the LMSYS blog is an SGLang checkpoint that uses post-training quantization, quantizing only the routed experts, calibrated with 128 CNN/DailyMail articles truncated to 512 tokens, validated on GB300 and B300, and reported to reach 540 tok/s at batch size 1 with B200 TP4 plus MTP.

Chatbot Arena (LMSYS) partially mitigates the bias limitations of LLM-as-a-judge through large-scale blind human comparisons, but its cost makes it difficult to integrate into developers' daily workflows

Chatbot Arena (LMSYS) uses real-user blind-test voting, having two anonymous models answer the same question, with results presented as ELO scores.

User votes in Chatbot Arena may be influenced by non-capability factors such as response style and Markdown formatting preferences, and LMSYS has introduced controlled variable analysis and bootstrap confidence interval estimation to improve ranking reliability.

SGLang is developed by LMSYS, focuses on structured output and complex workflows, and introduces the RadixAttention mechanism, which can automatically detect and reuse the common prefix of prompts

The LLM-as-a-Judge method was originally popularized by LMSYS in its Chatbot Arena.

Scale AI's SEAL leaderboard and LMSYS's Chatbot Arena adopt closed evaluation strategies; Chatbot Arena evaluates models through an Elo scoring mechanism based on blind-test votes from real users.