Verified80% confidenceFactExact time
PPO (Proximal Policy Optimization) is a policy gradient algorithm proposed by OpenAI in 2017 that limits the magnitude of each policy update through a clipped objective function.
6
Sources
80%
Confidence
Long-term
Relevance
8/26/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedProximal Policy Optimization (PPO) is the primary policy gradient algorithm tool for robot reinforcement learning.77% similarUnverifiedPPO 需要同时维护策略模型、参考模型、奖励模型和价值模型，对 70B 参数模型意味着需同时加载约四倍模型参数，显存开销巨大65% similarUnverifiedGRPO (Group Relative Policy Optimization) is an improved variant of PPO that requires no additional value network training; instead, it calculates the reward signal by sampling a group of responses to the same question and using intra-group relative performance as a baseline.64% similarUnverified策略梯度方法本质上是on-policy的，因为梯度估计中的期望是关于当前策略分布的63% similarUnverifiedPPO in RLHF scenarios requires simultaneously maintaining four neural networks: the policy model, reference model, reward model, and value model.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/805065API
curl https://kongchang.com/api/v1/knowledge/claims/805065MCP
get_claim(id=805065)