Verified65% confidenceFactExact time
TRPO was proposed by John Schulman et al. in 2015, formalizing policy updates as a constrained optimization problem: maximizing a surrogate objective function while constraining the KL divergence between the new and old policies to not exceed a threshold.
3
Sources
65%
Confidence
Long-term
Relevance
8/5/2026
First Seen
Sources
Related Claims
UnverifiedTRPO guarantees monotonic improvement by constraining the KL divergence of policy updates, but requires computing second-order derivatives (the Fisher information matrix), making its implementation more complex than PPO.70% similarVerifiedPPO通过引入裁剪目标函数将新旧策略概率比值限制在[1-ε, 1+ε]区间，替代TRPO的KL散度约束，简化了实现复杂度69% similarUnverifiedPPO was proposed by Schulman et al. in 2017 and constrains the policy update step size by clipping the importance sampling ratio.69% similarUnverifiedPPO constrains policy updates to a neighborhood of the old policy by clipping the importance sampling ratio, approximately achieving the stability guarantees of on-policy training.62% similarUnverifiedPolicy iteration alternates between policy evaluation and policy improvement, and in a finite MDP it necessarily converges to the optimal policy within a finite number of steps.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/691006API
curl https://kongchang.com/api/v1/knowledge/claims/691006MCP
get_claim(id=691006)