Judgment Model
A class of AI model architecture that does not generate long-form text but instead outputs probabilities or classification scores for specific questions. Designed for decision-making scenarios requiring downstream automated processing, these models complement large language models in hybrid workflows.
Timeline (last 90 days)
The ABACUS framework improves multi-step reasoning by composing reusable programs to freeze the judgment model's decisions, and the model parameters are not updated throughout.
Traditional judgment models output the final answer directly, skipping intermediate reasoning steps, and have limited accuracy on complex tasks such as BIG-Bench Hard.
ABACUS uses a generative teacher to design judgment patterns and composition programs offline, while the judgment model solves local semantic problems online, and the saved programs compose decisions into the final answer.
The ABACUS team instantiated the framework with Kev: the teacher model plans reasoning steps and generates program code, and the frozen judgment model acts as a semantic executor that scores candidate answers for each step.
Kev is a judgment-model-based reasoning framework/interface that allows invoking a frozen model to score candidate answers via natural language or structured instructions, without relying on the model to directly generate a complete answer.
The judgment model is optimized specifically for rapid classification and confidence scoring, outputting only probability values, and has advantages in high-frequency, low-latency scenarios such as content moderation, risk assessment, and real-time routing.
All Facts (6)
The ABACUS framework improves multi-step reasoning by composing reusable programs to freeze the judgment model's decisions, and the model parameters are not updated throughout.
50%UnverifiedTraditional judgment models output the final answer directly, skipping intermediate reasoning steps, and have limited accuracy on complex tasks such as BIG-Bench Hard.
50%UnverifiedABACUS uses a generative teacher to design judgment patterns and composition programs offline, while the judgment model solves local semantic problems online, and the saved programs compose decisions into the final answer.
50%UnverifiedThe ABACUS team instantiated the framework with Kev: the teacher model plans reasoning steps and generates program code, and the frozen judgment model acts as a semantic executor that scores candidate answers for each step.
50%UnverifiedKev is a judgment-model-based reasoning framework/interface that allows invoking a frozen model to score candidate answers via natural language or structured instructions, without relying on the model to directly generate a complete answer.
50%UnverifiedThe judgment model is optimized specifically for rapid classification and confidence scoring, outputting only probability values, and has advantages in high-frequency, low-latency scenarios such as content moderation, risk assessment, and real-time routing.
50%