EnvSched
An online scheduler for agent reinforcement learning that organizes environment-task pairs through a dual relation graph, combining exploitation and coverage value to predict reward variance and thereby optimize the informativeness and diversity of training batches under a limited rollout budget.
Timeline (last 90 days)
EnvSched is an online scheduler that learns from rollout history for environment-level data management in agent reinforcement learning.
EnvSched organizes environment-task pairs as nodes in a dual-relation graph, containing intra-environment edges and cross-environment edges.
EnvSched predicts within-task reward variance by propagating historical rollout statistics along the dual-relation graph.
EnvSched introduces environment quotas and a similarity penalty to prevent batch concentration and redundancy, maintaining diversity along the environment and task dimensions.
EnvSched requires no additional pilot rollouts to warm up the scheduler.
The paper conducted RL experiments on Qwen3-8B, and after 40 training steps, EnvSched achieved an average score of 48.81 on the three benchmarks BFCL-v3, tau-Bench, and ACEBench-Agent.
After 40 training steps, EnvSched's average score exceeded the highest score reached by training with random task selection for 120 steps.
Under the same training budget, EnvSched improves downstream performance across different model scales.
The effectiveness of EnvSched depends on the reliability of historical rollout statistics, and variance predictions on the graph may be insufficiently accurate in the early stage of training when data is sparse.
All Facts (9)
EnvSched is an online scheduler that learns from rollout history for environment-level data management in agent reinforcement learning.
50%UnverifiedEnvSched organizes environment-task pairs as nodes in a dual-relation graph, containing intra-environment edges and cross-environment edges.
50%UnverifiedEnvSched predicts within-task reward variance by propagating historical rollout statistics along the dual-relation graph.
50%UnverifiedEnvSched introduces environment quotas and a similarity penalty to prevent batch concentration and redundancy, maintaining diversity along the environment and task dimensions.
50%UnverifiedEnvSched requires no additional pilot rollouts to warm up the scheduler.
50%UnverifiedThe paper conducted RL experiments on Qwen3-8B, and after 40 training steps, EnvSched achieved an average score of 48.81 on the three benchmarks BFCL-v3, tau-Bench, and ACEBench-Agent.
50%UnverifiedAfter 40 training steps, EnvSched's average score exceeded the highest score reached by training with random task selection for 120 steps.
50%UnverifiedUnder the same training budget, EnvSched improves downstream performance across different model scales.
50%UnverifiedThe effectiveness of EnvSched depends on the reliability of historical rollout statistics, and variance predictions on the graph may be insufficiently accurate in the early stage of training when data is sparse.
50%