Hierarchical Reinforcement Learning
Hierarchical reinforcement learning decomposes a policy into multiple levels: higher levels output abstract intentions or subgoals, while lower levels handle concrete execution. This structure mitigates the problems of an excessively large action space and sparse exploration in long-horizon tasks.
Timeline (last 90 days)
Putting a planning model inside the simulation loop is similar to an AI Director in games or a high-level policy in hierarchical reinforcement learning: the high level outputs events or subgoals at a lower frequency, and the low-level world model renders frames at a high frequency.
Hierarchical reinforcement learning splits the policy into multiple layers, with the upper layer outputting abstract intents or subgoals and the lower layer handling concrete execution, used to alleviate the problems of an overly large action space and sparse exploration in long-horizon tasks, but each layer needs its own credit assignment signal, making training less stable than a single-layer policy.
All Facts (2)
Putting a planning model inside the simulation loop is similar to an AI Director in games or a high-level policy in hierarchical reinforcement learning: the high level outputs events or subgoals at a lower frequency, and the low-level world model renders frames at a high frequency.
50%UnverifiedHierarchical reinforcement learning splits the policy into multiple layers, with the upper layer outputting abstract intents or subgoals and the lower layer handling concrete execution, used to alleviate the problems of an overly large action space and sparse exploration in long-horizon tasks, but each layer needs its own credit assignment signal, making training less stable than a single-layer policy.
50%