Curriculum Learning

In sparse-reward scenarios, agents often need techniques such as curriculum learning, reward shaping, or intrinsic motivation-based exploration to converge effectively.

Curriculum Learning was systematically proposed by Bengio et al. in 2009, with the core idea of mimicking the human learning pattern of progressing 'from easy to hard'.

Reinforcement learning acquires locomotion skills through massive trial-and-error in simulation environments, with reward functions that include metrics such as energy efficiency, driving robot movements toward being effortless and fluid, thereby producing human-like characteristics.

Curriculum Learning was proposed by Bengio et al. in 2009; its core idea is to progressively transition from simple samples to complex ones, which can accelerate model convergence and improve final performance

Curriculum learning is an effective means of improving the training success rate: start with simple scenarios and gradually increase the difficulty, avoiding the situation where the agent cannot obtain an effective reward signal in the early stage because the task is too difficult.

Common improvement approaches include curriculum learning (gradually increasing task difficulty) and domain randomization (randomizing physical parameters to improve policy robustness).

In Agent reinforcement learning, curriculum learning introduces scorers in stages: first only the pass rate of the most core unit tests, then format and length constraints once stable, and finally product-level metrics such as quality and time consumption.

Training a single policy typically relies on large-scale parallel simulation and curriculum learning in reinforcement learning, gradually increasing terrain difficulty.

In curriculum learning within RLVR, the signal is most effective when the success rate at each level falls between 20% and 80%.

The strategy of slowly fading out the rule-based tower failed; training collapsed immediately once the assistance disappeared.

Curriculum learning is necessary for sparse-reward cooperative tasks; directly fading out the auxiliary rule-based policy is less robust than staged replacement.

The solution is staged training: first train the booster paired with a rule-based tower, then freeze the booster and train the tower agent from scratch, and finally fine-tune the booster to work with the learned tower.

BATE implements data-driven adaptive curriculum learning: it dynamically allocates sampling probabilities and local exploration incentives according to each atomic subtask's current mastery level.

The authors adopted a curriculum learning schedule: initially only a penalty term for joint position initialization was used, and as training iterations progressed the domain randomization range was gradually expanded and the reward/penalty thresholds were tightened.

Increasing the entropy coefficient after curriculum learning reaches its upper limit is a "exploit first, then explore" strategy: low entropy in the early stage quickly masters the basic gait, while high entropy in the later stage helps the policy find more robust solutions within the disturbance range it has adapted to.

Unlike previous methods that mitigate compounding errors through curriculum learning, prefix replay, or teacher intervention, R2S directly corrects the scoring signal itself and requires no environment rewards throughout.

Common approaches to alleviating the state sparsity of PDF environments include introducing a retrieval module to narrow the candidate range, using curriculum learning to transition from short documents to long documents, and designing intermediate rewards.

HLR-LLM is trained with a curriculum learning strategy and introduces chain-of-thought supervision in the final stage, specifically targeting multi-step spatial reasoning tasks.

The competence frontier refers to the task difficulty range where the policy's current success rate is neither close to 0 nor close to 1, typically corresponding to success rates roughly between 20% and 80%.

REMORY adopts a curriculum learning strategy, restricting N to 8 tokens in early epochs and gradually relaxing it to 64 in later epochs.