AI Evaluation

AI testing requires simultaneously fixing the version under test and the evaluation version; only by fixing one and varying the other does a meaningful horizontal comparison become possible.

AI evaluation targets are divided into three layers—large language model evaluation, knowledge base (RAG) evaluation, and agent (Agent) evaluation—forming a progressive relationship of increasing complexity and knowledge depth.

The AI field commonly uses continuous values from 0 to 1 for scoring, and can also use the two discrete values 0 and 1 to represent the traditional fail and perfect score.

AI testing introduces the concept of Evaluation, which typically uses continuous values from 0 to 1 for scoring, where evaluation is responsible for scoring and testing is responsible for setting the passing threshold.

AI evaluation bears a high degree of similarity to performance testing: neither focuses particularly on the correctness of a single result, but instead identifies problems and draws conclusions through a set of data and multiple rounds of comparison.

AI evaluation is not about judging right or wrong, but about expressing quality through a scoring mechanism.

AI systems reason probabilistically, meaning the same input may produce different outputs, and the reasoning process is typically a black box.

For claimed breakthroughs in decrypting long-unsolved ciphers, the cryptographic community typically requires public disclosure of methodology and verifiable results.

The core transformation in AI testing lies in moving from 'assertions' (Test) to 'evaluation' (Evaluation), which is an internationally accepted practice.

AI testing cannot simply judge right or wrong, but should shift toward evaluation (scoring), because large model outputs are probabilistic and diverse.

Evaluation results are increasingly becoming the basis for regulatory and procurement decisions.

During his time at Meta, Patel was mainly responsible for AI evaluation and reinforcement learning.

Evaluation produces a continuous score (0 to 1), which can be discretized into pass or fail, and launching an AI product requires making a decision based on a threshold of the evaluation score.

In AI evaluation, forced binary choice can expose leanings that models may hide during free generation; however, this method measures the model's output behavior under specific prompts rather than the model's "internal beliefs," and the results are highly sensitive to prompt wording, option order and context.

Hugging Face chief policy officer Irene Solaiman pointed out that AI evaluation methods are quite complex and still maturing, and the industry still lacks consensus on "what counts as a good evaluation."

Solaiman says the AI evaluation field as a whole is currently at an early stage, with substantial room for subjective judgment in benchmark design, dataset construction and the choice of scoring criteria.

Evaluators need to distinguish nominal oversight from effective oversight: the former requires only drawing a human node on a process diagram, while the latter requires verifying whether humans can substantively change outcomes under real workloads, information conditions, and organizational incentives.

Next-generation AI evaluation needs to go beyond the model level and assess the entire system around the model, including aspects such as data, design, predictive validity, fairness, interpretability, authority, oversight, reversibility, and monitoring.

In AI evaluation, 'auditable' means that the inputs, outputs, intermediate states, and system interventions of every decision are fully recorded, and the records can be independently verified.

In order to test model performance in real-world scenarios, the evaluation process often requires the model to access the live internet, for example to retrieve information, call online APIs, or execute multi-step web operations.