Unverified50% confidenceCautionExact time
The preference data that RLHF relies on has systematic flaws such as low inter-annotator agreement, position bias, and length bias.
1
Sources
50%
Confidence
Long-term
Relevance
9/6/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedRRF (Reciprocal Rank Fusion) uses only the ranking position information from each retrieval path (formula RRF(d)=Σ1/(k+rank_i(d)), where k is typically set to 60), is insensitive to score distribution, and bypasses the cross-modal score alignment problem64% similarUnverified离线RL的核心难题是如何避免对数据集未覆盖的状态-动作对产生过于乐观的价值估计（分布外外推问题）64% similarUnverifiedRRF does not require normalization of raw scores from different retrievers and relies only on rank information, but when multiple retrievers systematically make the same type of error, RRF only amplifies the mistakes63% similarUnverifiedHNSW可能陷入局部最优漏掉近邻，IVF在文件夹边界可能漏检，PQ用编号代替真实值带来精度损失63% similarUnverifiedRLHF alignment has a 'one-size-fits-all' problem, where the same safety standards are applied to all scenarios, making it impossible to distinguish between malicious requests and legitimate creative writing needs.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/866643API
curl https://kongchang.com/api/v1/knowledge/claims/866643MCP
get_claim(id=866643)