AI Testing Is Outdated? The 2026 Shift Toward AI Evaluation and AI-Led Testing

Over the past six months, AI adoption in software testing has accelerated far beyond expectations. Half a year ago, everyone was still talking about "using AI to assist testing and boost efficiency." Today, several leading Chinese tech giants—Tencent, Alibaba, ByteDance—as well as DeepSeek, which has moved from model research to engineering—have pushed the conversation to "AI-led testing." Testing professionals now face not a tool upgrade, but a reconstruction of the entire methodology.

From "Testing with AI" to "Testing AI": A Clear Three-Layer System

According to a course breakdown by a Bilibili creator, the testing system in the AI era can be divided into three layers: testing with AI, testing alongside AI, and testing AI itself.

• Testing with AI: Using AI to assist humans in completing testing tasks and improve efficiency. This was a hot topic six months ago, but is now considered "past tense."
• Testing alongside AI: Humans and AI each play their roles, with most testing work—requirements analysis, test case design, system exploration, code writing, automated regression—handed over to AI. This is "AI-led testing."
• Testing AI itself: As more and more software integrates AI capabilities (AI customer service, AI fortune-telling, AI video generation, etc.), these AI applications themselves need to be tested—and the testing approach is completely different from traditional software.

The creator points out that the first two layers have already been surpassed by leading domestic AI companies, and the third layer—"testing AI itself"—is becoming the new focus. These three layers are not a three-way fork in the road but a progressive relationship: first master evaluation of large models, then knowledge bases, and finally agents.

Why "Evaluation" Instead of "Assertion"?

Traditional software testing relies on assertions: pass or fail, with no third outcome. But the output of large AI models is inherently probabilistic. The creator explains this from two angles:

1. At the principle level: Most mainstream large models use the Transformer architecture, where probabilistic choices occur during output generation, making them inherently non-deterministic.
2. At the result level: Even when the result is deterministic (e.g., outputting the number 2), the form of expression varies widely—Chinese numerals, English, Arabic numerals, different bases, different styles—making it hard to cover with equality assertions.

Therefore, AI testing cannot simply judge right or wrong; it must judge "how good, how much better, how much worse." This is evaluation—expressing capability through scoring. The creator draws an analogy to the college entrance exam: the fairest selection is reflected through scores, and complex domains such as IQ, tumors, and depression are also commonly assessed through scoring. Evaluation does not entirely replace testing; the testing step still exists, essentially setting a passing threshold for the score and providing a clear conclusion on "whether it can ship." Scores can be continuous values from 0 to 1, or 0 and 1 to indicate pass or fail.

Transformer Architecture and Output Probabilistic Nature

Transformer is the foundational architecture of current mainstream large language models (such as GPT, LLaMA, DeepSeek^Ad, etc.). Its core is the self-attention mechanism, which can process input sequences in parallel and capture long-distance dependencies. When generating text, the model typically uses an autoregressive approach, predicting the probability distribution of the next word one by one based on the preceding context, and then selecting a specific output through sampling strategies (such as greedy search, beam search, temperature sampling, Top-p sampling, etc.). This probabilistic selection means the same input may produce different outputs—even with model parameters and input completely fixed, the output may vary due to random seeds or sampling strategies. In addition, the model may express the same semantics in multiple ways (e.g., "2", "二", "two", "0b10"), making it difficult for exact-match assertions to cover all cases. Therefore, AI testing needs to shift from "right/wrong judgment" to "quality evaluation," quantifying the quality of outputs through scoring or grading.

Version Pinning: A Special Requirement of AI Evaluation

Traditional testing only requires pinning the version number of the system under test, but AI evaluation requires pinning two versions:

• Tested version: Which model was used, what parameters were passed, what prompts were used, what context was added—any change in any of these means a version change.
• Evaluation version: What dataset was used, what assertions, what thresholds—these also need to be pinned.

The creator emphasizes that replacing only one version allows for horizontal comparison (e.g., using the same evaluation method with different models to compare model quality); if both versions change simultaneously, the results have no comparative value and need to be reinterpreted.

Three Layers of Evaluation Targets: Large Models, Knowledge Bases, Agents

AI evaluation first requires clarifying the evaluation target—different targets have different purposes and metrics:

• Large Language Models (LLMs): Evaluate generation correctness, completeness, safety, and stability. Use cases include model selection (which is smarter and faster), prompt optimization, and validation of fine-tuned small models.
• Knowledge Bases (RAG): Retrieve correct materials first, then generate answers. Need to evaluate retrieval accuracy, recall rate, and the faithfulness of generated results to reference materials (moderate level).
• Agents: Involve task planning, tool use, and task progression—the highest complexity. Agents often contain large models and knowledge bases internally, so evaluation is progressive.

The creator recommends a learning path of: large models → knowledge bases → agents, from simple to complex.

RAG (Retrieval-Augmented Generation) and Key Evaluation Points

RAG (Retrieval-Augmented Generation) is a technical paradigm that combines external knowledge base retrieval with large model generation. Its process typically includes: user asks a question → retriever recalls relevant document fragments from the knowledge base → generator synthesizes an answer based on the recalled content. RAG can mitigate large model hallucination issues and support private knowledge updates. Evaluating a RAG system requires attention to three dimensions: retrieval quality (e.g., recall, accuracy, MRR), generation quality (e.g., relevance of answers to questions, information completeness), and faithfulness (whether generated content is faithful to retrieved reference materials, avoiding model fabrication). Common tools include RAGAS, TruLens, etc., which provide automated metrics (such as faithfulness, answer relevancy, context precision) to quantify these dimensions. For testing professionals, understanding RAG evaluation points helps in designing test plans for knowledge base applications.

Tool Selection: Why Promptfoo Is Recommended

The course lists several AI evaluation tools, with a strong recommendation for Promptfoo. Reasons include:

• Supports command-line and configuration file-driven operation, meaning AI can call it itself—the creator calls this a potential "AI evaluating AI" scenario.
• Human evaluation remains the current gold standard; Promptfoo supports writing test case assertions in configuration files, lowering the barrier to entry.
• Supports batch invocation and Python custom extensions, with strong extensibility.
• Has been acquired by OpenAI, indirectly confirming its tool value.

The creator also mentions the advantages of tools like RAGAS for knowledge base evaluation, but believes that for most testing professionals, Promptfoo is more suitable for acceptance scenarios. There is no single answer to technology selection—it can be flexibly chosen based on actual company needs.

Promptfoo's Core Capabilities and Usage Patterns

Promptfoo is an open-source LLM evaluation tool that supports defining test cases, assertions, and evaluation metrics through YAML configuration files. Its core capabilities include: batch invocation of different models or prompt versions, built-in assertion types (such as contains, regex, similarity, model grading), support for custom Python assertion functions, and generation of visual reports. It can be run directly via command line (CLI), making it easy to integrate into CI/CD pipelines, and also supports programmatic invocation, providing a foundation for "AI evaluating AI." After being acquired by OpenAI, its ecosystem and stability have been further enhanced. For testing teams, Promptfoo lowers the evaluation barrier, especially suitable for quickly building evaluation pipelines in scenarios such as model selection, prompt iteration, and regression testing.

Takeaways for Testing Professionals

The creator repeatedly emphasizes that theoretical direction is more important than hands-on details—hands-on practice can be picked up at a glance, but if the direction is wrong, even the best tools are useless. For testing professionals, three mindset shifts are needed:

1. From assertion thinking to scoring thinking;
2. From single-version management to dual-version pinning;
3. From functional testing perspective to layered evaluation perspective.

The industry is changing, and learning cannot stop. The creator's own shift from Python projects to JS is a case in point. AI evaluation is not the future tense—it's the present continuous.

One-line Summary

AI testing is shifting from "testing with AI" to "testing AI itself," requiring evaluation to replace assertions and layered evaluation of large models, knowledge bases, and agents.

Paragraph Summary

The article points out that AI adoption in software testing has advanced from "using AI to assist testing" to "AI-led testing," forming a progressive three-layer system: testing with AI, testing alongside AI, and testing AI itself. Because large model outputs are probabilistic, traditional assertions fail, and scoring-based evaluation is needed while retaining testing to set passing thresholds. AI evaluation requires pinning both the tested version and the evaluation version to ensure single-variable comparison. Evaluation targets are divided into three layers: large models, knowledge bases (RAG), and agents, with increasing complexity. For tools, Promptfoo is recommended for its CLI support, configuration file-driven operation, AI-callable nature, and acquisition by OpenAI. Finally, it emphasizes that testing professionals need to complete three mindset shifts: from assertion to scoring, from single-version to dual-version, and from functional testing to layered evaluation.