#RLHF
3 related articles
Differentially Private Sparse Reward Estimation: A New Breakthrough in RLHF Privacy Protection
New RLHF privacy research proposes a differentially private sparse reward estimation framework, achieving a better privacy-utility trade-off under sparsity assumptions, with theoretical error bounds significantly outperforming dense methods, validated on sentiment generation and safety alignment tasks.
CleanRM: Cracking the RLHF Noisy Reward Modeling Challenge with a Few Clean Feedback
CleanRM introduces a few clean feedback combined with optimal transport to calibrate per-sample noise ratios, solving the bias in reward models caused by non-uniform noise in RLHF preference data, with theoretical guarantees of unbiased estimation and improved downstream alignment.
Deep DivesThe Three Stages of AI Training: Pre-training, Post-training, and Alignment Explained
Explore the three key stages of LLM training: pre-training learns language, post-training learns tasks, alignment learns boundaries. Understand why AI hallucinates.