ReSI-Math: Mining the Frontier of Model Recoverability to Boost Reasoning Efficiently

Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective means of improving the mathematical reasoning abilities of large language models, but a fundamental question remains unresolved: when a model cannot solve a problem, does it truly lack the corresponding capability? The ReSI-Math framework answers this in the negative—some failures are not a lack of capability, but rather a capability that has not been fully elicited.

Core Finding: Failure Does Not Equal Incapability

The research reveals that direct decoding failure does not necessarily mean the model lacks the ability to solve the problem. By exploring the model's own reasoning trajectories, some failure cases can be "recovered," thereby exposing a recoverable capability frontier. This is a policy-dependent region: within this region, answers cannot be reliably generated through direct decoding, but they can be elicited through trajectory-level search.

This finding challenges the assumption of traditional RLVR—that failed samples can only serve as negative examples or require a stronger external model to generate positive examples. ReSI-Math demonstrates that the model itself harbors untapped problem-solving potential.

Reinforcement Learning with Verifiable Rewards (RLVR) has been a mainstream paradigm in recent years for improving the reasoning abilities of large language models. Its basic process is: given a task with a definite answer, such as a math problem, the model samples multiple reasoning trajectories, and a rule-based verifier (such as answer matching) automatically judges correctness; then policy optimization is performed with correct trajectories as positive examples and incorrect trajectories as negative examples. Typical representatives include training schemes based on algorithms such as GRPO. The key advantage of RLVR is that reward signals can be obtained automatically without manual annotation, making it particularly suitable for verifiable domains such as mathematics and code. However, traditional approaches assume by default that "a verifier judging an answer wrong means insufficient capability," simply discarding failed samples as negative examples or relying on stronger external models to generate positive examples, which both wastes the model's own potential and limits the efficiency of training data utilization.

The ReSI-Math Framework: From Recovering Capabilities to Efficient Supervision

ReSI-Math transforms recoverable capabilities into data-efficient RLVR supervision signals. Its core process consists of two steps:

• Self-judging-guided evolutionary search: Under a fixed policy, the model's own judging ability is used to identify recoverable failure cases, and candidate trajectories are generated through evolutionary search.
• Adaptive frontier problem selection: As the policy iterates, difficult problems located at the capability frontier are dynamically selected for serialized RLVR training, ensuring that the training data always matches the model's current capabilities.

This design avoids the inefficiency of random sampling, ensuring that every supervision signal is used on problems that are "just within reach."

Evolutionary search draws on the mutation-selection mechanism in biological evolution: it maintains a population of candidate trajectories, generates new trajectories through crossover (exchanging reasoning segments between two trajectories) and mutation (rewriting or continuing part of the steps), and then a judge or verifier scores and selects better individuals to enter the next generation. In ReSI-Math, it is used to start from failed trajectories and explore correct reasoning paths that the model itself might generate but that are difficult to sample through direct decoding. Working in tandem with it, self-judging refers to letting the model act as a judge to evaluate the reasoning quality or recoverability of candidate trajectories, thereby screening out failure cases worth continuing to search without introducing a stronger external model, reducing search costs.

Experimental Validation: Trajectory Crossover and Serialized Training

On the Qwen3-VL-8B model, crossing correct and incorrect trajectories produced significant results: from 621 attempts, 272 correct descendant trajectories were obtained after verifier confirmation, accounting for 43.8%. This means that nearly half of the failure cases can be recovered into correct solutions through trajectory-level search.

Subsequently, after three serialized one-shot RLVR stages, the model's average score on MATH500 and WE-MATH increased from 78.71 to 80.16. Under a matched single-problem RLVR setting, random selection reached only 53.31, whereas frontier-guided Q1 selection reached 54.16, validating the effectiveness of the frontier-guided strategy.

Implications and Outlook

The value of ReSI-Math lies not only in performance improvement, but also in the new perspective it provides: failed samples are not garbage, but a gold mine waiting to be tapped. Through trajectory-level search and adaptive selection, the model can self-recover and strengthen its reasoning abilities. This idea is expected to generalize to reasoning tasks beyond mathematics, opening a new path for efficient RLVR training.

For practitioners, ReSI-Math suggests that when constructing training data, rather than blindly increasing data volume, it is better to focus on those "recoverable failures" at the capability frontier, using less data to leverage greater improvement.

Background Supplement

Serialized RLVR training refers to dividing training into multiple sequential stages, each using only a subset of problems that matches the current policy's capability. As model capability improves, the difficulty frontier of problems shifts upward, and subsequent stages switch to harder problems. Compared with training on all difficulty levels mixed together at once, this approach keeps supervision signals always within the "just within reach" range, avoiding wasting compute on overly easy samples and providing ineffective gradients from overly hard samples; it is a concrete implementation of the curriculum learning idea in RLVR.

One-line Summary

ReSI-Math demonstrates that model failure is not a lack of capability; recoverable failures can be transformed into efficient RLVR supervision through trajectory search.

Paragraph Summary

The article introduces the ReSI-Math framework, which challenges the default assumption in Reinforcement Learning with Verifiable Rewards (RLVR) that "failure means incapability." The research points out that some math problems that fail direct decoding can still be recovered through the model's own trajectory-level search, forming a policy-dependent "recoverable capability frontier." ReSI-Math uses self-judging-guided evolutionary search to identify recoverable failure cases and adaptively selects difficult problems at the current capability frontier for serialized RLVR training. On Qwen3-VL-8B, crossing correct and incorrect trajectories recovered 272 correct descendants from 621 attempts (43.8%); after three serialized RLVR stages, the average score on MATH500 and WE-MATH increased from 78.71 to 80.16, and frontier-guided selection also outperformed random selection (54.16 vs. 53.31), validating the effectiveness of the data-efficient training strategy.