Three Stages of AI Large Model Testing: The Journey from Assertions to Evaluation

Over the past six months, significant changes have taken place in the field of AI testing. According to a Bilibili creator's sharing, major tech companies like Tencent, Alibaba, and ByteDance are investing heavily in AI testing, while DeepSeek^Ad's shift from model research to engineering research has jointly driven a consensus: using AI merely to assist testing is already a thing of the past. The industry is entering a new stage where AI leads testing, and ultimately moves toward testing the AI applications themselves.

Three Layers of the Testing System: From Using AI to Testing AI

In the course, the creator systematically outlined three testing systems in the AI era, which have now evolved from vague ideas into increasingly clear frameworks.

Using AI for Testing: The Bygone Era of Efficiency Assistance

Early discussions focused on leveraging AI to assist testing and improve efficiency. But according to the creator's observations, over the past six months, leading domestic AI companies—Tencent, Alibaba, ByteDance, and DeepSeek—have already "turned the page" on this stage. AI is no longer just an auxiliary tool; it has begun to take on more core testing responsibilities.

Testing with AI: Humans and Machines Each Playing Their Part

What major companies are currently pushing is "AI-led testing." The creator explains that this no longer means testers completely dominate all the work; rather, most testing tasks—from requirements analysis, test case design, and system exploration to code writing and automated regression—can be handed over to AI. Humans and AI each have their roles, but AI's share has increased significantly.

Testing AI: Evaluating the AI Application Itself

The third layer is testing AI itself, i.e., evaluating software systems that internally use AI features. The creator gives examples: companies like China Mobile already have AI customer service, and various AI fortune-telling, AI divination, and video generation applications are emerging endlessly. The testing approaches and processes for these AI applications are completely different from traditional software testing and require a separate methodology.

These three systems are not a choose-one-of-three relationship but a progressive one. The creator points out that testing an agent includes testing the knowledge base and the large model, with complexity increasing level by level.

Why Must Testing Become Evaluation?

Traditional software testing relies on assertions—asserting that 1 equals 1, or asserting status codes, returned data, and field contents. Outputs are basically consistent, and assertions are stable and reliable. But AI large models break this determinism.

Probabilistic Nature: An Inherent Property of Large Models

The creator explains the source of probabilistic nature from two angles. At the principle level, most current mainstream large models adopt or reference the Transformer architecture, whose output is essentially a probabilistic choice and inherently uncertain. The same question may yield different answers. At the result level, even if the output is a definite number "2," it can be expressed in many forms—Chinese uppercase, Chinese lowercase, English, Arabic numerals, binary, decimal, hexadecimal—and the style may also differ.

More critically, any tiny change in model parameters, prompts, or context can lead to huge changes in results. Using equality assertions to judge "right or wrong" is clearly no longer applicable.

Transformer is a neural network architecture proposed by the Google team in the 2017 paper "Attention Is All You Need." Its core is the self-attention mechanism, which can process dependencies between any positions in a sequence in parallel, replacing the step-by-step recursive structure of earlier RNN/LSTM. Mainstream large models such as GPT, BERT, LLaMA, and DeepSeek are all based on or borrow from this architecture. When generating each token, the model computes a probability distribution over all candidate words in the vocabulary and then samples output according to that distribution, so the same input under different sampling strategies (such as temperature, top-p) may yield different results. This is precisely the technical root of what the article calls "output is essentially a probabilistic choice," and the fundamental reason why assertion-based testing is difficult to apply.

Evaluation: Expressing Good or Bad with Scores

The creator uses a vivid analogy: judging a person cannot simply be "good person" or "bad person"—such judgment is biased. Judgments about complex matters—such as IQ, cancer, tumors, depression—are often evaluated through scores. The college entrance examination, as a recognized fair selection method, also reflects differences through scores.

Therefore, AI testing adopts a scoring approach, using continuous values from 0 to 1 to express "how good" or "how bad." But evaluation does not completely replace testing. The creator emphasizes that after evaluation gives a score, a passing line still needs to be set to determine whether the software can go live or be released. Scores can be discretized into 0 and 1, corresponding to pass or fail in traditional assertions.

Similarities Between AI Evaluation and Performance Testing

For testers unfamiliar with AI evaluation, the creator suggests understanding it by analogy with performance testing. Neither particularly cares about right or wrong; instead, they discover problems through a set of data under specific conditions. Performance testing requires at least two rounds and comparisons to file bugs; AI evaluation likewise requires horizontal comparison of different models, different prompts, and different thresholds.

Those who have done system performance testing can map the concepts and ideas of AI evaluation onto performance testing, making it much smoother to understand.

Version Pinning: A Special Requirement of AI Evaluation

Traditional testing has a concept of versions—which version was tested, which version the report is responsible for—everyone understands this. But version pinning in AI evaluation is stricter, requiring two versions to be pinned.

System Under Test Version

Includes what model was used, what parameters were passed, what prompts were used, and what context was added. Every change means the tested version has changed. The creator gives an example: testing DeepSeek today and switching to GPT tomorrow means the version is wrong, and results need to be re-evaluated.

Evaluation Version

Includes what dataset was used, what assertions, and what thresholds. If the dataset or threshold changes, the results will also differ.

The creator points out that only by pinning one version and replacing the other does horizontal comparison have value. If both versions change simultaneously, the result is a completely new independent evaluation with no comparative significance.

What to Evaluate? A Three-Layer Progressive Path

Many students care about what metrics AI evaluation should assess, such as recall and accuracy. But the creator reminds us that the evaluation object must first be determined—different objects have different purposes and different metrics.

First Layer: Large Language Models

Evaluate the correctness, completeness, safety, and stability of content generated by large models. For large model companies this is mandatory; for ordinary companies, evaluation can be used for model selection, prompt optimization, and validation of fine-tuned models.

Second Layer: Knowledge Base

The principle of a knowledge base is to first retrieve correct materials, then answer questions based on those materials. Evaluation must cover both the retrieval and generation stages, focusing on whether retrieval is accurate and whether generated results are supported by reference materials—i.e., the faithfulness issue.

Knowledge base question answering typically adopts the RAG (Retrieval-Augmented Generation) paradigm: documents are first split into chunks and vectorized into a vector database; when a user asks a question, the most relevant chunks are retrieved first and then inserted as context into the prompt for the large model to generate an answer. Therefore its quality is determined jointly by retrieval and generation—the retrieval stage focuses on recall, hit rate, and ranking accuracy, while the generation stage focuses on whether the answer is supported by retrieved content. The "faithfulness" mentioned in the article measures whether every claim in the generated answer can be found in the retrieved materials, and is one of the core metrics of RAG evaluation. Other common metrics alongside it include answer relevancy and context precision.

Third Layer: Agents

Agents involve task planning, tool use, and task advancement, making them the most complex evaluation objects. The creator predicts that travel agents, legal agents, and medical agents will become widespread in more important settings, and evaluation methods will need to develop accordingly.

The learning path is clear: first test large models, then knowledge bases, and finally agents. The creator emphasizes that the theoretical framework is more important than hands-on tool practice—practical operation can be glanced at, but directional content is the foundation for interviews and actual work.

An agent refers to a system that uses a large model as its decision-making core, capable of autonomously decomposing tasks, calling external tools, and iteratively advancing based on execution results. Its typical structure includes: a planning module (decomposing user goals into subtasks), a tool-calling module (calling external capabilities such as search, databases, and code executors via protocols like Function Calling or MCP), and a memory module (storing short-term dialogue and long-term knowledge). Unlike traditional single-turn Q&A, an agent's output is a trajectory containing multiple decision steps; evaluation must consider not only whether the final task is completed but also whether intermediate steps are reasonable, whether tool calls are correct, and whether there are redundant or dangerous operations. Common evaluation dimensions include task success rate, step efficiency, tool-call accuracy, and trajectory-level LLM-as-a-Judge scoring.

Tool Selection: Promptfoo and RAGAS

The creator lists various AI testing tools and focuses on two. The first is a PyTest-style evaluation tool, suitable for those with PyTest automation experience. The second originated from RAG evaluation and can now also test agents and models.

The final recommendation is Promptfoo. The creator gives several reasons: it supports both CLI and UI driving methods; CLI means it can be invoked by AI itself, and a future scenario of "AI evaluating AI" may emerge. It also supports batch invocation and Python customization, with strong extensibility. More importantly, Promptfoo has been acquired by OpenAI, and its excellence has been recognized.

The creator also points out that tool selection should depend on the scenario: for academic research on metrics, specialized tools can be chosen; for online agent production monitoring, monitoring tools can be chosen; for most testers, evaluation is an acceptance scenario to determine whether a product can go live, and Promptfoo is more suitable.

Promptfoo is an open-source evaluation tool for prompts and LLM applications. It declares test cases, models or prompts to compare, and assertion types through YAML configuration files, can run in batches from the command line and generate visual comparison reports, and supports custom Python/JavaScript assertions and CI integration. RAGAS (Retrieval Augmented Generation Assessment) initially focused on automated evaluation of RAG pipelines, providing a metric framework for computing faithfulness, answer relevancy, context recall, and other metrics without manual reference annotations, and later expanded to evaluate agents and general models. The two have different positioning: Promptfoo leans more toward engineering-oriented prompt comparison and regression acceptance, while RAGAS leans more toward academic measurement of metric systems; in practice they are often used together.

Conclusion

From assertions to evaluation is not just a change in terminology but a shift in mindset. The probabilistic nature of AI determines that testing must move from "right-or-wrong judgment" to "degree scoring," while retaining the acceptance step of pass-line determination. The three evaluation objects—large models, knowledge bases, and agents—form a clear learning path, while version pinning and tool selection are key guarantees for practical implementation.

One-line Summary

AI testing is shifting from assertions to scoring-based evaluation, progressing through three layers: large models, knowledge bases, and agents.

Paragraph Summary

The article points out that AI testing has moved past "using AI to assist testing" and entered a three-layer progressive system of AI-led testing and ultimately evaluating AI applications themselves. Due to the sampling mechanism of the Transformer architecture and the diversity of output forms, large models are inherently probabilistic, making traditional equality assertions ineffective; continuous scores from 0 to 1 must be used to express quality, while a pass line is still needed for release acceptance. The author analogizes AI evaluation to performance testing, emphasizing that both rely on multi-round comparison rather than right-or-wrong judgment, and proposes that two dimensions must be pinned—the system under test version (model, parameters, prompts, context) and the evaluation version (dataset, assertions, thresholds)—with only one replaced at a time for horizontal comparison to be meaningful. Evaluation objects fall into three layers: large language models, knowledge bases, and agents, with increasing complexity; the recommended tool is Promptfoo, which is CLI-driven, supports Python extension, and has been acquired by OpenAI, suitable for most product acceptance scenarios.