Policy Gradient
A class of algorithms in reinforcement learning that directly optimize policy parameters, estimating gradients to maximize expected return. In large-model settings, policy gradient methods commonly face the problem of high variance.
Core Facts
Timeline (last 90 days)
Policy gradient methods directly parameterize the policy π(a|s) and maximize expected return through gradient ascent, naturally supporting continuous action spaces and stochastic policies, with representative algorithms including REINFORCE and PPO.
Theoretical last-iterate convergence guarantees for Q-Learning have long lagged behind averaging-iterate analyses, as well as analogous results for methods such as policy gradient.
Policy gradient methods (such as PPO) rely on a discount factor γ to exponentially decay future rewards, with a reward t steps away from the current moment retaining only a weight of γ^t.
策略梯度定理表明期望回报对策略参数的梯度可表示为 ∇J(θ) = E[∇log π_θ(a|s) · A(s,a)]，其中A是优势函数
All Facts (5)
PPO通过引入裁剪目标函数将新旧策略概率比值限制在[1-ε, 1+ε]区间，替代TRPO的KL散度约束，简化了实现复杂度
65%UnverifiedPolicy gradient methods directly parameterize the policy π(a|s) and maximize expected return through gradient ascent, naturally supporting continuous action spaces and stochastic policies, with representative algorithms including REINFORCE and PPO.
50%UnverifiedTheoretical last-iterate convergence guarantees for Q-Learning have long lagged behind averaging-iterate analyses, as well as analogous results for methods such as policy gradient.
50%UnverifiedPolicy gradient methods (such as PPO) rely on a discount factor γ to exponentially decay future rewards, with a reward t steps away from the current moment retaining only a weight of γ^t.
50%Unverified策略梯度定理表明期望回报对策略参数的梯度可表示为 ∇J(θ) = E[∇log π_θ(a|s) · A(s,a)]，其中A是优势函数
50%