Unverified50% confidenceFactExact time
PPO is an on-policy algorithm and can only use data collected by the current policy for updates, so data produced by old policies is discarded; off-policy algorithms such as DDPG and SAC can use historical data in the experience replay buffer and have higher sample efficiency.
1
Sources
50%
Confidence
Long-term
Relevance
10/28/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe on-policy nature of PPO means that old samples become invalid after each update, so it places high demands on the throughput of the simulation environment.76% similarUnverifiedIn online policy distillation, old response distributions shift after each policy update, and continued use introduces off-policy bias.66% similarUnverifiedCommon countermeasures for the off-policy problem include importance sampling correction, setting data expiration thresholds, and adopting algorithms such as GRPO.66% similarUnverifiedThe key innovation of GRPO compared to the traditional PPO algorithm is that it replaces the independent value network by sampling multiple outputs for the same problem and computing relative scores within the group, thereby reducing memory usage.65% similarUnverifiedMAQS-Way is positioned as the pathway layer between the policy layer and the audit layer, comprising 8 principles (P1–P8), a data model based on OMPS, lifecycle management, and measurable verification means (LoCoMo, LongMemEval, deletion lattice, negative tests).63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1105720API
curl https://kongchang.com/api/v1/knowledge/claims/1105720MCP
get_claim(id=1105720)