Unverified50% confidenceFactExact time
When fine-tuning precipitation diffusion models with reinforcement learning, improper reward design can lead to mode collapse (reduced generation diversity) or reward hacking (optimizing only the score at the expense of physical consistency), and training requires extensive sampling, resulting in high computational cost.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn such tasks, reinforcement learning usually means having the model generate multiple candidate answers to the same question, then reinforcing high-scoring outputs and suppressing low-scoring ones based on a reward signal; crude reward design can induce score-gaming behavior.78% similarUnverifiedTraining stability is crucial in reinforcement learning, because policy updates, sparse rewards, and environment randomness all cause curve fluctuations, and some self-distillation methods may exhibit a sudden drop in reward, entropy collapse (the policy converging prematurely to a single behavior), or abnormal gradients midway through training.77% similarUnverifiedExternal reward models typically require a large amount of human preference data for training, and once the distribution drifts they may reward outputs that are 'seemingly plausible but actually failed', the so-called reward hacking.77% similarUnverifiedAdversarial reward design draws on the idea of generative adversarial networks (GANs), using two or more mutually constraining scorers to prevent a single reward from being easily hacked; the cost is higher training cost and greater variance in the reward signal, and if the scorers share the same bias, the adversarial effect is greatly reduced.76% similarUnverifiedTypical reinforcement learning algorithms include PPO and SAC, and the core difficulty lies in reward design—rewards that are too sparse lead to slow learning, while rewards that are too specific make it easy for the policy to exploit loopholes.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1113886API
curl https://kongchang.com/api/v1/knowledge/claims/1113886MCP
get_claim(id=1113886)