Unverified50% confidenceOpinionExact time
The reward signals of alignment training often favor answers that are "comprehensive, exhaustive, and non-offensive," which systematically encourages models to pile up qualifiers, list multiple possibilities, and avoid making clear judgments, forming defensive writing.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/24/2026
First Seen
Valid until: 1/22/2027
Sources
Related Entities
Related Claims
UnverifiedIn such tasks, reinforcement learning usually means having the model generate multiple candidate answers to the same question, then reinforcing high-scoring outputs and suppressing low-scoring ones based on a reward signal; crude reward design can induce score-gaming behavior.78% similarUnverifiedSatisDive adopts a training-free strategy that takes effect at inference time, introducing batch-relative reward truncation to distinguish between low-reward and high-reward candidates.77% similarUnverifiedThe effect of the matched-prompt penalty does not stem from the quality of the prompt itself, but is caused by the interaction between the training prompt and the evaluation prompt.77% similarUnverifiedDense rewards provide continuous feedback at each step or state, which can efficiently guide policy learning, but may also over-constrain the policy near the reference trajectory.76% similarUnverifiedIn multi-agent reinforcement learning, when opponents continuously improve, the training signal automatically remains in a difficulty range that is just challenging enough, avoiding the test-cramming phenomenon of fixed datasets.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1094413API
curl https://kongchang.com/api/v1/knowledge/claims/1094413MCP
get_claim(id=1094413)