Reinforcement Learning
Reinforcement Learning is a major branch of machine learning in which an agent learns an optimal decision-making policy through continuous interaction with an environment. At each state, the agent takes an action and adjusts its behavior based on reward or penalty signals from the environment, with the objective of maximizing long-term cumulative reward. It does not rely on labeled data, is applicable to sequential decision problems, and is widely used in game playing, robotics, autonomous driving, and resource scheduling.
Core Facts
Machine learning is typically divided into three major paradigms: supervised learning, unsupervised learning, and reinforcement learning.
DQN is an algorithm proposed by DeepMind in 2013 that for the first time combined deep neural networks with Q-learning reinforcement learning, enabling AI to learn to play 49 Atari games.
The pre-training process of large language models requires thousands of high-end GPUs running in parallel for months, with costs typically ranging from tens of millions to hundreds of millions of dollars.
Timeline (last 90 days)
强化学习中的记忆机制面临两个问题：注意力机制需要对整个记忆库进行评分，成本高昂；且检索到记忆带来回报时，信用分配需跨越长序列。
When fine-tuning precipitation diffusion models with reinforcement learning, improper reward design can lead to mode collapse (reduced generation diversity) or reward hacking (optimizing only the score at the expense of physical consistency), and training requires extensive sampling, resulting in high computational cost.
FitzRoy is a precipitation forecasting system that fine-tunes a conditional discrete diffusion model via reinforcement learning, and the study claims it is the first to simultaneously achieve leading forecast skill and physical realism across the continental United States.
Experiments on three real-world crisis datasets show that GRID outperforms RL and MCMC baselines in scenario diversity, realism, and tail-risk coverage.
In reinforcement learning, agents maximize cumulative reward through trial and error; in large-language-model-based agents, the objective function is implicit in prompts and training preferences.
In actual products, user input is not directly used as training data for instruction fine-tuning; behavior calibration is mainly accomplished through reinforcement learning.
Using reinforcement learning for quantum circuit generation is not an entirely new idea, but focusing on diversity rather than optimization is an entry point of practical value.
Adding diversity rewards to reinforcement learning (such as a penalty term based on circuit structure distance or entropy regularization) can encourage exploration of different structures.
Reinforcement learning (RL) is significantly higher in complexity and cost than supervised fine-tuning (SFT), but its ceiling is also higher.
The Noise Steering study proposes jointly controlling yaw angle and power setpoints via reinforcement learning, which can substantially reduce power loss while satisfying noise constraints.
40 more timeline events
All Facts (20)
Machine learning is typically divided into three major paradigms: supervised learning, unsupervised learning, and reinforcement learning.
90%VerifiedDQN is an algorithm proposed by DeepMind in 2013 that for the first time combined deep neural networks with Q-learning reinforcement learning, enabling AI to learn to play 49 Atari games.
90%VerifiedThe pre-training process of large language models requires thousands of high-end GPUs running in parallel for months, with costs typically ranging from tens of millions to hundreds of millions of dollars.
90%Verified奖励函数设计不当会导致奖励欺骗（Reward Hacking）现象，即智能体找到在奖励函数上得分高但实际无意义甚至有害的行为路径
85%VerifiedThe concept of world models originates from cognitive science and reinforcement learning, strongly promoted by Yann LeCun and others, referring to an agent's internal representation of the external environment.
80%VerifiedReinforcement learning learns optimal policies through continuous interaction between an agent and its environment; its mathematical foundation is the Markov Decision Process (MDP), which consists of a state space, action space, transition function, reward function, and discount factor.
80%Verified大语言模型的预训练阶段本质上属于自监督学习，是一种特殊的非监督学习，模型通过预测下一个词来从海量文本中自动学习语言规律
80%VerifiedMachine learning is divided into three major categories: supervised learning, unsupervised learning, and semi-supervised learning.
80%VerifiedLearning-based methods have stronger performance when data is sufficient, but at the cost of robustness under domain shift.
75%VerifiedReward shaping refers to adding extra, denser guiding reward signals on top of the original sparse reward to help agents learn faster and more correctly.
75%VerifiedEach RL training step requires running inference (generating rollouts) and backpropagation (updating weights) simultaneously, making the compute requirements far higher than ordinary fine-tuning.
75%VerifiedThe full-stack GPU paradigm keeps environment state, dynamics simulation, policy network forward passes, and gradient computation entirely within video memory, eliminating the data transfer bottleneck between CPU and GPU.
75%VerifiedIn sparse-reward scenarios, agents often need techniques such as curriculum learning, reward shaping, or intrinsic motivation-based exploration to converge effectively.
75%VerifiedIn reinforcement learning training, the model's goal is to maximize the score given by the reward function.
75%VerifiedRewards in real-world tasks are often extremely sparse and delayed, causing long-horizon credit assignment problems that make it difficult for agents to determine which action led to the final outcome.
75%VerifiedWhen language models are repeatedly trained on data generated by themselves or similar models, systematic degradation occurs, with the diversity of model outputs decreasing with each generation and the tail information of the distribution gradually being lost.
75%VerifiedAI misalignment behavior is fundamentally a direct product of current mainstream training paradigms, particularly reinforcement learning.
75%VerifiedReinforcement learning training is the key stage in which a model transitions from general language capability to precise task execution, further optimizing reasoning, planning, and instruction-following through reinforcement learning after large-scale pre-training.
75%VerifiedDeepMind's AlphaGo learned Go strategies by playing millions of games against itself.
75%VerifiedA Markov Decision Process (MDP) is mathematically defined by a five-tuple (S, A, P, R, γ).
75%