No Verifier Needed: How Test-Time Self-Play Breaks Through LLM Reasoning Bottlenecks

Test-Time Scaling is an important means of improving large language models' performance on difficult tasks, with the core idea being to invest additional computational resources at deployment time to optimize outputs. However, mainstream methods generally rely on a verifier to assess the correctness of candidate answers and thereby select or refine results. But in many practical scenarios, a strong verifier for the target problem is unavailable. A recent paper proposes the Test-Time Self-Play (TTSP) method, which bypasses this limitation through asymmetric self-play and offers a new approach to test-time scaling without verifiers.

Verifier Dependence: The Invisible Ceiling of Test-Time Scaling

Existing test-time scaling methods can be broadly divided into two categories: one samples multiple candidate solutions and uses a verifier to score and select the best (such as Best-of-N), and the other uses verifier-provided fine-grained feedback to iteratively correct answers (such as self-refinement). Both paradigms assume the existence of a reliable verifier capable of accurately judging the correctness of candidate solutions.

The problem is that verifiers are expensive to build and difficult to obtain in many specialized domains (such as complex code generation and mathematical proofs). When the verifier signal is weak or even completely absent, the gains from these methods drop sharply. The authors point out that in the limiting case with no verifier, verifier-dependent methods almost fail. This constitutes a key bottleneck for bringing test-time scaling into practical use.

Best-of-N is the most basic baseline method in test-time scaling: independently sample N candidate answers to the same problem, then select one as the final output. The selection can be by majority voting or by taking the highest score from a verifier or reward model. When a reliable verifier is lacking, majority voting can only rely on consistency among answers; once the problem is difficult and the model's outputs are highly dispersed, the voting signal fails. This is exactly why Best-of-N's gains decay rapidly in verifier-free settings.

Self-Refine has the model critique its own generated answers and iteratively revise them. It usually requires the model itself or an external model to provide actionable feedback signals, and therefore also implicitly assumes that "there exists a judge capable of determining right from wrong or at least pointing out problems." When the model lacks the ability to recognize its own errors, self-refinement easily falls into error accumulation or spinning in place.

TTSP's Core Mechanism: Replacing External Scoring with Self-Play

TTSP's design is inspired by Asymmetric Self-Play. Its basic framework has the same model play two roles: Proposer and Solver. The proposer is responsible for generating challenging problems or subtasks, and the solver attempts to solve these problems. The two confront each other and co-evolve through iterations.

Unlike traditional self-play, TTSP's asymmetry is reflected in role division and difficulty control—the problems generated by the proposer are calibrated to ensure moderate difficulty, so that they challenge the solver without being completely beyond its ability. Through this internal adversarial loop, the model can achieve self-improvement without an external verifier. The paper emphasizes that the method operates at test time and does not rely on additional training data or human annotations.

Asymmetric Self-Play was first systematized in reinforcement learning and multi-agent game research. Its typical setup is to have two agents with asymmetric capabilities or information compete: one is responsible for posing problems or setting goals, and the other is responsible for achieving the goals, with each side receiving its own learning signal based on how difficult task completion is. Unlike symmetric self-play (such as self-play in board games), the asymmetric setting allows difficulty to adjust automatically—after the solver's ability improves, the proposer naturally needs to pose harder problems, thereby forming an automatic curriculum. TTSP brings this idea into the inference stage of large language models, using the large model itself to play the proposer and solver respectively, thereby constructing an endogenous, difficulty-adaptive training signal without any external annotations or reward model.

Experimental Results: The Weaker the Verifier, the More Obvious the Advantage

The paper evaluates TTSP on a difficulty-calibrated subset of LiveCodeBench v6. The results show that TTSP is the best-performing method under all tested verifier strengths. More importantly, as verifier strength decreases, the gap between TTSP and the strongest baseline widens. In the completely verifier-free setting, TTSP achieves an absolute improvement of 35% on the average@8 metric.

This trend is consistent with the method's design intent: when the verifier signal is reliable, traditional methods can still maintain some effectiveness; once the verifier is absent, TTSP's self-play mechanism becomes the main driver. However, the paper also points out the method's boundary condition—when the base policy's coverage of correct answers is close to zero, TTSP's gains no longer hold. This means the model itself needs to have some latent capability for self-play to effectively elicit it.

LiveCodeBench is an evaluation benchmark for competition-level code generation, characterized by problems with timestamps that can be updated regularly to reduce inflated scores caused by training data contamination. v6 is a version of this benchmark. Since competitive programming problems often have strict test cases or standard answers, it can be used to construct verifiers with controllable strength—for example, using complete test cases as a strong verifier, using partial or noised test results as a weak verifier, or even using no test signal at all. The paper artificially adjusts verifier strength on this benchmark to systematically observe how different methods perform as verifier quality declines, which is also the experimental basis for its conclusion that "the weaker the verifier, the greater TTSP's advantage."

Implications and Limitations: Self-Play Is Not a Panacea

TTSP's value lies in broadening the applicable scenarios of test-time scaling, allowing domains lacking verifiers to also enjoy the benefits of additional computation. For tasks where verifiers are difficult to build, such as code generation and complex reasoning, this method provides a practical alternative. At the same time, it also provides empirical support for the fundamental question of "whether models can improve themselves without supervised signals."

But it should be noted that TTSP is not a panacea. The base policy's initial coverage capability is a prerequisite for the method to work; if the model is completely "insensitive" to the target problem, the self-play loop will find it difficult to create correct solutions out of thin air. In addition, the paper validates the method only on a code generation benchmark, and its generalization ability on other task types remains to be further studied. Future work could explore how to combine it with lightweight verification signals, as well as its adaptability in more domains.

One-line Summary

TTSP uses asymmetric self-play to achieve verifier-free test-time scaling, with more obvious advantages under weak verifiers.

Paragraph Summary

This article introduces a test-time scaling method called Test-Time Self-Play (TTSP), aimed at addressing mainstream methods' strong dependence on verifiers. Existing paradigms such as Best-of-N selection and self-refinement all assume the existence of a reliable verifier, while in domains such as complex code generation and mathematical proofs verifiers are difficult to construct, causing the gains of these methods to drop sharply. TTSP draws on the idea of asymmetric self-play, having the same model play both a proposer responsible for posing problems and a solver responsible for solving them, and achieves self-improvement without an external verifier through a difficulty-calibrated internal adversarial loop. Experiments on a difficulty-calibrated subset of LiveCodeBench v6 show that TTSP is optimal under all verifier strengths, achieving an absolute improvement of 35% on average@8 without a verifier, and the weaker the verifier, the more obvious the advantage. However, the method's premise is that the base policy has some coverage of correct answers, and it has so far been validated only on code generation tasks.